I am using Microsoft Azure and I am trying to find out all the entity(s) that I have in my table. Unfortunately, I do not know the specifics of these entity(s). I've read through http://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-tables/ and it appears most of the entity's they know the specifics of them. For example, they know that it will be a person.
Is there a way to go through all the entity's of my table without knowing the specifics of them?
The reason I am wanting to do this is because I eventually want to find out how much memory my tables are using and I assume I will first need to go through each entity to find how much memory is used for it. Here's the code I have so far:
static double CalculateTableMemoryUsage()
{
double memory = 0;
try
{
var storageAccount = Microsoft.WindowsAzure.Storage.CloudStorageAccount.Parse("UseDevelopmentStorage=true");
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("mytable");
table.CreateIfNotExists();
//I've successfully created the table. Any idea how I can look
// through the entity(s) of that table though?
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
return memory;
}
Is there a way to go through all the entity's of my table without
knowing the specifics of them?
Yes. Storage client library has something called DynamicTableEntity so you can fetch entities as DynamicTableEntity from table. As they name suggests, you don't really need to know the schema in case of a dynamic table entity. Here's a sample code to do so:
static void ListAllEntities()
{
var storageAccount = new CloudStorageAccount(new StorageCredentials(StorageAccount, StorageAccountKey), true);
var tableClient = storageAccount.CreateCloudTableClient();
var table = tableClient.GetTableReference("mytable");
List<DynamicTableEntity> entities = new List<DynamicTableEntity>();
TableContinuationToken token = null;
do
{
var result = table.ExecuteQuerySegmented(new TableQuery(), token);
token = result.ContinuationToken;
entities.AddRange(result.Results);
} while (token != null);
Console.WriteLine("Total Entities Fetched: " + entities.Count);
}
For calculating the size of an entity, you may find this blog post useful: http://blogs.msdn.com/b/avkashchauhan/archive/2011/11/30/how-the-size-of-an-entity-is-caclulated-in-windows-azure-table-storage.aspx
Related
I created a simple text file in NotePad, it is a .txt file. I want to push this data to an azure table, but I'm not sure how to go about doing this in c#. Does anyone know how I can push some sample data from my computer to an azure table? Thanks!
There are three basic steps for doing this:
Read the text file into memory.
We can do that with a single line of code:
string text = System.IO.File.ReadAllText(#"C:\Users\Public\TestFolder\WriteText.txt");
Authenticate to Azure Tables.
You will need to get the relevant nuget packages for Azure Storage. You could write the code without the SDK, but I don't recommend it.
CloudStorageAccount storageAccount = new CloudStorageAccount(
new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials(
"<name>", "<account-key>"), true);
// Create the table client.
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
// Get a reference to a table named "textTable"
CloudTable textTable = tableClient.GetTableReference("textTable");
Write to Azure Tables.
We need to make a class that defines the data structure we are uploading into storage. Every entity must have a row key and a partition key.
public class TextEntity : TableEntity
{
public TextEntity(string partitionKey, string rowKey)
{
this.PartitionKey = partitionKey;
this.RowKey = rowKey;
}
public TextEntity() { }
public string Text { get; set; }
}
We can then use that class to create the object that we will then upload to storage.
var tableEntry = new TextEntry("partitionKey", "rowKey");
tableEntry.Text = text;
TableOperation insertOperation = TableOperation.Insert(tableEntry);
// Execute the insert operation.
await textTable.ExecuteAsync(insertOperation);
At the momment I'm using Cosmos DB fixed 10 gb where I save all my data in same collection. I want to continue this way. So do I just need to create an unlimited collection. When I do that it ask for a partion key which I don't quite understand what it is for. I have tried to read about it but didn't get much clever. Hope you guys can help me.
Today I create documents like this. Should it be done in another way if using unlimited collection. Thinking about should i declare the partion key somewhere?:
protected async Task<bool> CreateDocumentAsync(Resource document)
{
var collectionUri = UriFactory.CreateDocumentCollectionUri(_db.Options.Value.DatabaseName, _db.Options.Value.CollectionName);
ResourceResponse<Document> result = null;
for (int i = 0; i < MaxRetryCount; i++)
{
try
{
result = await _db.Client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(_db.Options.Value.DatabaseName, _db.Options.Value.CollectionName), document);
break;
}
catch (DocumentClientException dex) when (dex.StatusCode.HasValue && (int)dex.StatusCode.Value == 429)
{
_logger.LogWarning($"");
await Task.Delay(dex.RetryAfter);
}
}
if (result == null)
return false;
int statusCode = (int)result.StatusCode;
return statusCode >= 200 && statusCode < 300;
}
The partition key value can be specified by providing the RequestOptions object with the PartitionKey value set.
Your create document line would become:
var requestOptions = new RequestOptions{
PartitionKey = new PartitionKey("yourPartitionKeyValueHere");
}
result = await _db.Client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(_db.Options.Value.DatabaseName, _db.Options.Value.CollectionName), document, requestOptions);
However it does should like you don't understand CosmosDB partitioning which means you can get in really deep pitfall.
The partition key definition should be a property that most of the times if not always, is known to you when you are doing querying or reading of documents.
I highly recommend you watch this video. It will greatly help you understand partitioning: https://azure.microsoft.com/en-us/resources/videos/azure-documentdb-elastic-scale-partitioning/
On a side note, I can see several mistakes in the way you are using the CosmosDB SDK which I cannot fix in a single answer. For example, the CosmosDB SDK has already logic to handle the retry policy logic on the document client level via the RetryOptions object so you don't need to write any custom code. I would suggest you read more of the CosmosDB documentation.
I'm able to upload the image in the Azure file share using below code.
CloudStorageAccount cloudStorageAccount = ConnectionString.GetConnectionString();
CloudFileClient cloudFileClient = cloudStorageAccount.CreateCloudFileClient();
CloudFileShare fileShare = cloudFileClient.GetShareReference("sampleimage");
if (await fileShare.CreateIfNotExistsAsync())
{
await fileShare.SetPermissionsAsync(
new FileSharePermissions
{
});
}
//fileShare.CreateIfNotExists();
string imageName = Guid.NewGuid().ToString() + "-" + Path.GetExtension(imageToUpload.FileName);
CloudFile cloudFile = fileShare.GetRootDirectoryReference().GetFileReference(imageName);
cloudFile.Properties.ContentType = imageToUpload.ContentType;
await cloudFile.UploadFromStreamAsync(imageToUpload.InputStream);
imageFullPath = cloudFile.Uri.ToString();
}
catch (Exception ex)
{
}
return imageFullPath;
Here is how I'm trying to read the file path: [Before insertinginto a table]
public class ReadFileSharePath
{
string Path = null;
public string ReadFilePath()
{
try
{
CloudStorageAccount cloudStorageAccount = ConnectionString.GetConnectionString();
CloudFileClient cloudFileClient = cloudStorageAccount.CreateCloudFileClient();
CloudFileShare fileShare = cloudFileClient.GetShareReference("sampleimage");
if (fileShare.Exists())
{
CloudFileDirectory rootdir = fileShare.GetRootDirectoryReference();
CloudFileDirectory sampleDir = rootdir.GetDirectoryReference("sampleimage");
if (sampleDir.Exists())
{
// Get a reference to the file we created previously.
CloudFile file = sampleDir.GetFileReference("90e94676-492d-4c3c-beb2-1d8d48044e4e-.jpg");
// Ensure that the file exists.
if (file.Exists())
{
// Write the contents of the file to the console window.
//Console.WriteLine(file.DownloadTextAsync().Result);
Path = file.DownloadTextAsync().Result.ToString();
}
}
}
}
catch (Exception)
{
throw;
}
return Path;
}
}
However, this if condition
if (sampleDir.Exists())
is getting failed.And, the control is not entering into loop.
I would like to store the path of file share in the Azure table storage. I would like to get partition key and row key. How to achieve this ? any link or suggestion would help? Thanks.
As #Gaurav said, after you return imageFullPath with above code, you could use following code to store path in table storage.
void SavePath(string fullpath)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the table client.
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
// Create the CloudTable object that represents the "people" table.
CloudTable table = tableClient.GetTableReference("people");
// Create a new customer entity.
CustomerEntity customer1 = new CustomerEntity("joey", "cai");
customer1.path = fullpath;
// Create the TableOperation object that inserts the customer entity.
TableOperation insertOperation = TableOperation.Insert(customer1);
// Execute the insert operation.
table.Execute(insertOperation);
}
public class CustomerEntity : TableEntity
{
public CustomerEntity(string lastName, string firstName)
{
this.PartitionKey = lastName;
this.RowKey = firstName;
}
public CustomerEntity() { }
public string path { get; set; }
}
Note: The fullpath is the imageFullPath that you have returned.
Edit:
Entities map to C# objects by using a custom class derived from TableEntity. To add an entity to a table, create a class that defines the properties of your entity.
The above code defines an entity class that uses the customer's first name as the row key and last name as the partition key. Together, an entity's partition and row key uniquely identify it in the table. Entities to be stored in tables must be of a supported type, for example derived from the TableEntity class.
And the above code shows the creation of the CloudTable object and then a CustomerEntity object. To prepare the operation, a TableOperation object is created to insert the customer entity into the table. Finally, the operation is executed by calling CloudTable.Execute.
For more details, you could refer to this article.
Update:
As I understand the Row and Partition key is unique hence error.
So, when you insert the second entity into the table, it use the same Partition key and Row key. So you could keep the partition key the same and change the row key value.
Change the following code to the above:
CustomerEntity customer1 = new CustomerEntity("joey", "cai"+Guid.NewGuid());
customer1.path = fullpath;
TableOperation insertOperation = TableOperation.Insert(customer1);
table.Execute(insertOperation);
I'll start by asking am I right in thinking that in the image below:
the 'TABLE=CLOASEUCDBA.T_BASIC_POLICY' is not part of the connection string? in fact it is the source table name?
I'm looking to alter this to another linked table on the same database. The connection string should there be the same and the name that appears in ACCESS should be the same. The only difference should be under the hood it is actually referencing another table and of course if you open the table it will contain different fields and data.
my code for far to do this is:
var dbe = new DBEngine();
Database db = dbe.OpenDatabase(#"C:\Users\xxxx\Documents\Test.accdb");
foreach (TableDef tbd in db.TableDefs)
{
if (tbd.Name.Contains("CLOASEUCDBA_T_BASIC_POLICY"))
{
tbd.SourceTableName = "CLOASEUCDBA_T_BILLING_INFORMATION";
}
}
db.Close();
However I'm getting a big fat COMException "Cannot set this property once the object is part of a collection.". I'm not sure exactly why and all the examples I can find online are all written in VB/VBA and I only have very very limited exposure to this. Any help is appreciated.
EDIT:
I have tried to go a different route with no futher success using the code:
if (tbd.Name.Contains("CLOASEUCDBA_T_BASIC_POLICY"))
{
var newtable = db.CreateTableDef("this is a new table");
newtable.Name = "new table";
newtable.Connect = tbd.Connect;
newtable.SourceTableName = "CLOASEUCDBA_T_BILLING_INFORMATION";
db.TableDefs.Append(newtable);
//tbd.SourceTableName = "CLOASEUCDBA_T_BILLING_INFORMATION";
}
In this case I get the error "ODBC--call failed."
Since we're not allowed to change the SourceTableName of a TableDef object that already exists in the TableDefs collection we need to create a new TableDef object, .Delete the old one, and then .Append the new one:
// This code requires the following COM reference in your project:
//
// Microsoft Office 14.0 Access Database Engine Object Library
//
// and the declaration
//
// using Microsoft.Office.Interop.Access.Dao;
//
// at the top of the class file
string tableDefName = "CLOASEUCDBA_T_BASIC_POLICY";
var dbe = new DBEngine();
Database db = dbe.OpenDatabase(#"C:\Users\xxxx\Documents\Test.accdb");
var tbdOld = db.TableDefs[tableDefName];
var tbdNew = db.CreateTableDef(tableDefName);
tbdNew.Connect = tbdOld.Connect;
tbdNew.SourceTableName = "CLOASEUCDBA_T_BILLING_INFORMATION";
db.TableDefs.Delete(tableDefName); // remove the old TableDef ...
db.TableDefs.Append(tbdNew); // ... and append the new one
db.Close();
i keep getting this error, "The current value 'String.Empty' type is not compatible with the expected 'System.Boolean' type", when i try to loop through a bunch of entities from an Azure table, i am only new to using Azure so this could be something very easy, the error that i am getting.
my code :
private void registerButton_Click(object sender, RoutedEventArgs e)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("StorageConnectionString"));
// Create the table client
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
// Get the data service context
TableServiceContext serviceContext = tableClient.GetDataServiceContext();
// Create a new customer entity
user = new UserDetailsEntity();
//Setting the fields of the new userEntity
user.username = usernameText.Text;
user.password = passwordText.Text;
user.subscriptionID = subText.Text;
user.subscriptionName = subscriptionNameText.Text;
user.thumbprint = thumbprintText.Text;
user.email = emailText.Text;
user.phoneNumber = "3530" + numberText.Text;
int rowCount = 1;
CloudTableQuery<UserDetailsEntity> Query = (from en in serviceContext.CreateQuery<UserDetailsEntity>("userdetails")
select en).AsTableServiceQuery<UserDetailsEntity>();
//error occurs in the next line
foreach (UserDetailsEntity ent in Query)
{
rowCount++;
}
user.RowKey = rowCount.ToString();
// Add the new customer to the people table
serviceContext.AddObject("userdetails", user);
// Submit the operation to the table service
serviceContext.SaveChangesWithRetries();
//Set the variables so they can be retrieved when the next screen loads
Application.Current.Properties["username"] = usernameText.Text;
Application.Current.Properties["password"] = passwordText.Text;
Window1 userHome = new Window1();
this.Close(); //to close Password window
userHome.Show(); //to show Main form
}
Without more code, I cannot tell you exactly where the problem is, however the exception is fairly explanatory. You are trying to set a boolean property to a value of a string.
If the error is occurring in your foreach as you noted in the code comment, then I would check how your UserDetailsEntity object is set up. There is probably a property that is set up as a boolean, yet your data is coming back as a String.Empty. The reason you are getting this in your foreach is because your LINQ query is of type IQueryable, so it will not actually execute and fill your objects until you actually access the data (by your foreach)*. So, you could put breakpoints in your UserDetailsEntity properties to see which one it is if this is not blatent from looking at the code.
*Keep in mind that this is the N+1 problem, where you are making a call to the database on each iteration of your loop. You can resolve this by calling .ToList() to eager load all of your data at once into your query...if this is a problem for you, that is.