Develop Reference

Can I create dynamically a table for a serveless Azure CosmosDB (Azure Table)?

i'm making a migration from Table Storage to Cosmos DB.
I've created a serveless Cosmos DB (Table Azure)
When i execute the below code
CloudTable table = _client.GetTableReference(tableName)
await table.CreateAsync();
i'm getting an error :
Reading or replacing offers is not supported for serverless
accounts.\r\nActivityId: 46c760ee-fb3f-400e-a3fc-819bec68b82b,
Microsoft.Azure.Documents.Common/2.14.0, Windows/10.0.19042
documentdb-netcore-sdk/2.11.2"}
To make a test I've created an other Azure CosmosDB with this time "Provisioned throughput" instead of "Serveless" and it works.
Any idea?

Serverless table creation with the .NET Tables SDK works in REST mode only
You can try the below code,
TableClientConfiguration config = new TableClientConfiguration();
config.UseRestExecutorForCosmosEndpoint = true;
CloudTableClient tableClient = storageAccount.CreateCloudTableClient(config);
Console.WriteLine("Create a Table for the demo");
// Create a table client for interacting with the table service
CloudTable table = tableClient.GetTableReference(tableName);
if (table.CreateIfNotExists())
{
Console.WriteLine("Created Table named: {0}", tableName);
}
else
{
Console.WriteLine("Table {0} already exists", tableName);
}
Here is the reference

Unable to save variable to table storage from Azure QnA chat bot C#

I am trying to save a single variable "userinput" into my Azure table storage, the variable populates with the first message the user send to the chat bot, when the user sends "no" to the bot.
This is my current code for uploading the variable "userinput" into my table storage:
private void storeuserinput(Activity activity)
{
var uid = activity.From.Id;
var uname = activity.From.Name;
if (activity.Text?.ToLower().ToString() == "no" || activity.Text?.ToLower().ToString() == "NO" || activity.Text?.ToLower().ToString() == "No" || activity.Text?.ToLower().ToString() == "Nope" || activity.Text?.ToLower().ToString() == "nope")
{
var userinput = firstmessage;
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the table client.
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
// Retrieve a reference to the table.
CloudTable table = tableClient.GetTableReference("NoSolutionFound");
// Create a new customer entity.
unansweredQuestion input = new unansweredQuestion();
input.Query = userinput;
// Create the TableOperation object that inserts the customer entity.
TableOperation insertOperation = TableOperation.Insert(input);
// Execute the insert operation.
table.Execute(insertOperation);
}
//extract other data from "activity" object
//your code logic here
//store data in your table storage
//Note: specifcial scenario of user send attachment
}
public class unansweredQuestion : TableEntity
{
public unansweredQuestion() { } // the parameter-less constructor must be provided
public string Query { get; set; }
}
My storage table remains empty when my code is run and two errors appear inthe output box:
Microsoft.WindowsAzure.ServiceRuntime Error: 102 : Role environment .
FAILED TO INITIALIZE. hr: -2147024894
Getting "StorageConnectionString" from ServiceRuntime: FAIL.
Exception thrown: 'Microsoft.WindowsAzure.Storage.StorageException' in Microsoft.WindowsAzure.Storage.dll

read image path from file share and store in table storage azure

I'm able to upload the image in the Azure file share using below code.
CloudStorageAccount cloudStorageAccount = ConnectionString.GetConnectionString();
CloudFileClient cloudFileClient = cloudStorageAccount.CreateCloudFileClient();
CloudFileShare fileShare = cloudFileClient.GetShareReference("sampleimage");
if (await fileShare.CreateIfNotExistsAsync())
{
await fileShare.SetPermissionsAsync(
new FileSharePermissions
{
});
}
//fileShare.CreateIfNotExists();
string imageName = Guid.NewGuid().ToString() + "-" + Path.GetExtension(imageToUpload.FileName);
CloudFile cloudFile = fileShare.GetRootDirectoryReference().GetFileReference(imageName);
cloudFile.Properties.ContentType = imageToUpload.ContentType;
await cloudFile.UploadFromStreamAsync(imageToUpload.InputStream);
imageFullPath = cloudFile.Uri.ToString();
}
catch (Exception ex)
{
}
return imageFullPath;
Here is how I'm trying to read the file path: [Before insertinginto a table]
public class ReadFileSharePath
{
string Path = null;
public string ReadFilePath()
{
try
{
CloudStorageAccount cloudStorageAccount = ConnectionString.GetConnectionString();
CloudFileClient cloudFileClient = cloudStorageAccount.CreateCloudFileClient();
CloudFileShare fileShare = cloudFileClient.GetShareReference("sampleimage");
if (fileShare.Exists())
{
CloudFileDirectory rootdir = fileShare.GetRootDirectoryReference();
CloudFileDirectory sampleDir = rootdir.GetDirectoryReference("sampleimage");
if (sampleDir.Exists())
{
// Get a reference to the file we created previously.
CloudFile file = sampleDir.GetFileReference("90e94676-492d-4c3c-beb2-1d8d48044e4e-.jpg");
// Ensure that the file exists.
if (file.Exists())
{
// Write the contents of the file to the console window.
//Console.WriteLine(file.DownloadTextAsync().Result);
Path = file.DownloadTextAsync().Result.ToString();
}
}
}
}
catch (Exception)
{
throw;
}
return Path;
}
}
However, this if condition
if (sampleDir.Exists())
is getting failed.And, the control is not entering into loop.
I would like to store the path of file share in the Azure table storage. I would like to get partition key and row key. How to achieve this ? any link or suggestion would help? Thanks.

As #Gaurav said, after you return imageFullPath with above code, you could use following code to store path in table storage.
void SavePath(string fullpath)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the table client.
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
// Create the CloudTable object that represents the "people" table.
CloudTable table = tableClient.GetTableReference("people");
// Create a new customer entity.
CustomerEntity customer1 = new CustomerEntity("joey", "cai");
customer1.path = fullpath;
// Create the TableOperation object that inserts the customer entity.
TableOperation insertOperation = TableOperation.Insert(customer1);
// Execute the insert operation.
table.Execute(insertOperation);
}
public class CustomerEntity : TableEntity
{
public CustomerEntity(string lastName, string firstName)
{
this.PartitionKey = lastName;
this.RowKey = firstName;
}
public CustomerEntity() { }
public string path { get; set; }
}
Note: The fullpath is the imageFullPath that you have returned.
Edit:
Entities map to C# objects by using a custom class derived from TableEntity. To add an entity to a table, create a class that defines the properties of your entity.
The above code defines an entity class that uses the customer's first name as the row key and last name as the partition key. Together, an entity's partition and row key uniquely identify it in the table. Entities to be stored in tables must be of a supported type, for example derived from the TableEntity class.
And the above code shows the creation of the CloudTable object and then a CustomerEntity object. To prepare the operation, a TableOperation object is created to insert the customer entity into the table. Finally, the operation is executed by calling CloudTable.Execute.
For more details, you could refer to this article.
Update:
As I understand the Row and Partition key is unique hence error.
So, when you insert the second entity into the table, it use the same Partition key and Row key. So you could keep the partition key the same and change the row key value.
Change the following code to the above:
CustomerEntity customer1 = new CustomerEntity("joey", "cai"+Guid.NewGuid());
customer1.path = fullpath;
TableOperation insertOperation = TableOperation.Insert(customer1);
table.Execute(insertOperation);

Retrieving Chat Bot conversation data in Azure

A brief background first: The Bot framework stores the conversation data within a storage either Azure tables/Cosmos DB (Azure tables in my case). For each conversation there is an entry made into the Azure table with the timestamp, userid, conversation messages and other details.
I am trying to retrieve the conversation details from Azure table storage using custom code in C# as shown below.
Microsoft.WindowsAzure.Storage.CloudStorageAccount storageAccount = Microsoft.WindowsAzure.Storage.CloudStorageAccount.Parse(connectionString);
Microsoft.WindowsAzure.Storage.Table.CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("botdata");
TableQuery<DynamicTableEntity> projectionQuery = new TableQuery<DynamicTableEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "directline:user"));
var dataRow = table.ExecuteQuery(projectionQuery).Where(q => q.RowKey == "souvik").FirstOrDefault();
var conversation = Encoding.UTF8.GetString(dataRow.Properties["Data"].BinaryValue);
Console.WriteLine(conversation);
I am getting some gibberish text, something like this- \u001f�\b\0\0\0\0\0\u0004\0��\u0005\0C���\u0002\0\0\0 when I retrieve the data stored in Binary format in the Azure table storage. The data looks something like as shown below in the Azure storage explorer-
I have tried and tested for each conversation and I have seen that each time I type and send a data to the bot and get a response, new records are getting added to the Azure table with correct timestamp. Is it the correct way of retrieving the data or am I doing something wrong? Let me know if I can add more details.
Update-1:
I have referred this- How to retrieve Saved Conversation Data in Azure (Tablelogger) but did not get much help my issue.
Update-2
Based on Fei's answer below, I just noticed when I use the PartitionKey as directline:user or directline:conversation I get the deserialized values as {} but if I use directline:private I get somthing like this-
{"ResumptionContext":{"locale":"en-US","isTrustedServiceUrl":true},
"DialogState":"H4sIAAAAAAAEAO1a328iVRRmYGYKbTfbdZtNzLq7GM26DWS2FGi7MVUptUrcVi1VYxrSvcCFjh1mmrl3quyjb74YEx+N/hW+uzFxE/8KX9fsH6HnzAxQ7A8GhHpZoemBDs
O953znO+d+99KQFAqF/oIHPuNjNgzm8y29YlvMqnFt3YJfRzeq1E7GP6U20y1zLa2llrQlLZOM5x2DOzZdM6nDbWIk4x85ZUOvfECbu9YhhTtT5Vp6NbtMqunlDE1nFZznF+nM8bWCyaltEoNpm
3oZpvKeHqX29s6+f0MnhlVnJz7nXdkl7DAZH1kMpVIEglAYJ5VDtWaTBmXKl0TnLCLLT8LFJuO0oeUtw6AVDjMx7T1qUluvaA91xs+P5lT0hU0cW8DwRzo4CYhOESAlhv6YIMaJIrdsWjAhJ2aF
rjd3m0e06Ni2VSecJj4sfwGp2KE1alN4Gxl+ObPgbwzJEpPRKGBk/HPCkmFUoLqvA4ZM2Wf6YxrdP/aGBaR/DdpfhMVur4TUiUZjqs8i5I8iXwpxsUPLeEfkjs/ST0ydN7sGfN/CWV16T4GRFN
/FWBTMFDodkvxL8pOxz4YbZwzNNJgrYRlj7gUOMlTeIJzE3DsQ0NkcY7RRNprbEKcUkqLqDNz0NPBqiPBskgrksikgSphsdXak6kFy6YXzXAHzW1DgPoPFWWzcYoibEutLGwlcLHKD2Icyai
JZvpyV9odwH1x4tCQaask9v5t4oSXjDVaxbEMvd0bPaIv403vw8soKyVayy6kH6QxdXH3gpcTtYVfRzGFzxkbmdrRraF7CS1dbPfw6cnFuAmpvUGWbMi6blFZlOTBYO/CZ/yNYiYD4bAOcbcb
OIxefw/60n0+j9Jg6JoZD9/dD0bAvRhSk+tNxp3T32HnLNCkubFphizJG6jQHu4lj0Bv/9KJ95xC7/An2B8ZVVPYLhOtAhXIDWf4nFMpz/zBHwqZ+rf3Ws5NvXW/JqJdDfeyWth3DcMtCPNHR
0VDz/axbopJxlK3Y3Zdw0FMqJ3adcrVB+YFVjcg9t3s/nUOUi3BK7B445iHg7HhiddSLzH1/AhCH/sGKtgVbLmoXzJp13hb2JppX0NxCCt3oZ50QlUKC9LPB2fb7ENgmCAgDE/M2mjto4ti2b
7ba9quTJndxK+isB1jWojcuXJNV251K/qaHr55HuRr3vhi5lG3ja1jEtwYgMZJVxsOurpOvWN4gjLmvinrdJOjWdPvV0rQ3JG755/wz8Zxdh6BNziR8hCLRK74jeNNeSX0d50GeuzlX745BztU
3wMmvI604DmxKqrpZ19AP5lrwIOjBqa/K+o+4VIojbPcCn9D6X8gNMNM5zWSQBnSGYwVAEfzqyhhMeQrePJ4nGYbLUTc1C+o9SMQz5eJE9JsJwZrtaL+LGYxCYwLRcPk50pYSiO54gj4t3W4piY
WJ1h2azOuIDpRsL5B+PalPfu5PnwgSQVvKJFBHo5hGOaMmu3WD9mKlTb0P8fw4kRjDbeFBkxtQfSxCjr5XJ+pjoj4uQX0I0piCCxUFt54CbYkHP876tkcQ+QOim4mHlnXkXRH2sDSFZglNGsxsBs
zax2YOYur4lQrql+kYhoJr8njKisHpcLfbryJpHBlU83DUdiyLe4FnBs1SFs0ymhVUPKmW3l8Vq546khkZJVqNoO6dzkE3rdAqNbncR9LCLfyldEtsvglmxseWNc2KpznXBIxbfQuc+k66QJTET8
RxztrbXrm8F1B5nH7Fe65zOfyvIVI26KnTnAX1bXDrj//erZFi72/Qs62CfWdsm2OntrEOAtYO3BnC+pFWWmWzDmbe93wHKlE/plU3wWr+Xy896kZwz9R33UxcxL6zvBwdDQNvRBbUTdxkhMVxXRSKesU28zeNVmRMBTMAAA=="}
I have a feeling that the DialogState data in the JSON above needs to be decrypted?
For every message I type and the response I receive, there are three records inserted into the Azure table each with a different PartitionKey which are- directline:private, directline:user and directline:conversation.

Binary data in Azure Table Storage is stored as Base64 encoded string. What you would need to do is convert this string to bytes first and then get the string from those bytes.
Something like:
var conversation = Encoding.UTF8.GetString(Convert.FromBase64String(dataRow.Properties["Data"].BinaryValue));

If you’d like to use WindowsAzure.Storage client library to retrieve entities from table storage and extract data from Data property, you can refer to the following code.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName={your_account_name};AccountKey={your_account_key};EndpointSuffix=core.windows.net");
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("botdata");
TableQuery<MessageEntity> query = new TableQuery<MessageEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "emulator:user"));
foreach (MessageEntity entity in table.ExecuteQuery(query))
{
string mydata = "";
using (var msi = new MemoryStream(entity.Data))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(msi, CompressionMode.Decompress))
{
gs.CopyTo(mso);
}
mydata = Encoding.UTF8.GetString(mso.ToArray());
}
object data = JsonConvert.DeserializeObject(mydata);
//.....
}
MessageEntity:
public class MessageEntity : TableEntity
{
public MessageEntity(string pk, string rk)
{
this.PartitionKey = pk;
this.RowKey = rk;
}
public MessageEntity() { }
public string BotId { get; set; }
public string ChannelId { get; set; }
public string ConversationId { get; set; }
public byte[] Data { get; set; }
public string UserId { get; set; }
}
Test result:
Note:
In source code, you can find the Data property defined as byte[] in the BotDataEntity class.
In Jason Sowers's reply, he has shared the information about serialize and deserialize of the Data property.

I am assuming you are using the botbuilder-azure package
don't do this in a dialog because you can just access the same data in the context object. it just happens to be where I wrote the code
It's actually as simple as this:
private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
{
var activity = await result as Activity;
IBotDataStore<BotData> table = new TableBotDataStore(ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);
//use the type of state data you need
var userData = await table.LoadAsync(Address.FromActivity(activity), BotStoreType.BotUserData, CancellationToken.None );
var privateConvoData = await table.LoadAsync(Address.FromActivity(activity), BotStoreType.BotPrivateConversationData, CancellationToken.None );
var convoData = await table.LoadAsync(Address.FromActivity(activity), BotStoreType.BotConversationData, CancellationToken.None);
//in this case I am just replying with the data, but do what you need with it here
var reply = activity.CreateReply(userData.Data.ToString());
var reply2 = activity.CreateReply(privateConvoData.Data.ToString());
var reply3 = activity.CreateReply(convoData.Data.ToString());
await context.PostAsync(reply);
await context.PostAsync(reply2);
await context.PostAsync(reply3);
context.Wait(MessageReceivedAsync);
}
If you notice in the source for botbuilder-azure there is a serialize and deserialize method. Where this happens:
private byte[] Serialize(object data)
{
using (var cmpStream = new MemoryStream())
using (var stream = new GZipStream(cmpStream, CompressionMode.Compress))
using (var streamWriter = new StreamWriter(stream))
{
var serializedJSon = JsonConvert.SerializeObject(data, serializationSettings);
streamWriter.Write(serializedJSon);
streamWriter.Close();
stream.Close();
return cmpStream.ToArray();
}
}
So the data you need is compressed into the "jibberish" you were seeing. When accessing the data via the LoadAsync method it is also being decompressed like here:
private object Deserialize(byte[] bytes)
{
using (var stream = new MemoryStream(bytes))
using (var gz = new GZipStream(stream, CompressionMode.Decompress))
using (var streamReader = new StreamReader(gz))
{
return JsonConvert.DeserializeObject(streamReader.ReadToEnd());
}
}
deserialize is called inside the LoadAsync method in the return statement return new BotData(entity.ETag, entity.GetData()); It its the GetData() method like below:
internal ObjectT GetData<ObjectT>()
{
return ((JObject)Deserialize(this.Data)).ToObject<ObjectT>();
}

finding out all the entities in a table C#

I am using Microsoft Azure and I am trying to find out all the entity(s) that I have in my table. Unfortunately, I do not know the specifics of these entity(s). I've read through http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-tables/ and it appears most of the entity's they know the specifics of them. For example, they know that it will be a person.
Is there a way to go through all the entity's of my table without knowing the specifics of them?
The reason I am wanting to do this is because I eventually want to find out how much memory my tables are using and I assume I will first need to go through each entity to find how much memory is used for it. Here's the code I have so far:
static double CalculateTableMemoryUsage()
{
double memory = 0;
try
{
var storageAccount = Microsoft.WindowsAzure.Storage.CloudStorageAccount.Parse("UseDevelopmentStorage=true");
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("mytable");
table.CreateIfNotExists();
//I've successfully created the table. Any idea how I can look
// through the entity(s) of that table though?
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
return memory;
}

Is there a way to go through all the entity's of my table without
knowing the specifics of them?
Yes. Storage client library has something called DynamicTableEntity so you can fetch entities as DynamicTableEntity from table. As they name suggests, you don't really need to know the schema in case of a dynamic table entity. Here's a sample code to do so:
static void ListAllEntities()
{
var storageAccount = new CloudStorageAccount(new StorageCredentials(StorageAccount, StorageAccountKey), true);
var tableClient = storageAccount.CreateCloudTableClient();
var table = tableClient.GetTableReference("mytable");
List<DynamicTableEntity> entities = new List<DynamicTableEntity>();
TableContinuationToken token = null;
do
{
var result = table.ExecuteQuerySegmented(new TableQuery(), token);
token = result.ContinuationToken;
entities.AddRange(result.Results);
} while (token != null);
Console.WriteLine("Total Entities Fetched: " + entities.Count);
}
For calculating the size of an entity, you may find this blog post useful: http://blogs.msdn.com/b/avkashchauhan/archive/2011/11/30/how-the-size-of-an-entity-is-caclulated-in-windows-azure-table-storage.aspx