tl;dr: The .NET integration is returning partial objects, with some properties read as null. How can I get the full object with all properties?
I'm currently using C#.NET to read and modify Data Factory objects, using Microsoft.Azure.Management.DataFactories.
I can apparently successfully get most of my object data with a call like this:
var datasets = client.Datasets.List(resourceGroupName, dataFactoryName).Datasets;
While this gives me all of my Dataset objects, in all of the datasets, certain properties are simply left out. Here are some screenshots showing this:
This is what the Dataset is defined as, shown in Azure:
This is what I'm given by programmatically retrieving the Dataset object as JSON:
Similarly, here's the object as-is in memory at a breakpoint in my application:
As you can see, it's missing virtually all of the "Properties", with values being replaced with null.
How can I get the full, unadulterated object in my C# application?
Listing datasets will give you a "summary" of each dataset; as you observed, this means that certain details are not returned (e.g. typeProperties and structure).
After using the List method to get all datasets in the data factory, you can use the Microsoft.Azure.Management.DataFactories.DatasetsOperationExtensions.Get() method to get the full definition of each. Of course, you could simply call Get() if you already know the name of the dataset you are looking for.
Related
Suppose i have multiple entities that inherit from TableEntity.
i want to use a generic class , like Cachemanager , and write to azure storage different T
This link below mentions TableEntityAdapter
https://learn.microsoft.com/en-us/dotnet/api/microsoft.windowsazure.storage.table.tableentityadapter-1.-ctor?view=azure-dotnet
I am looking for a Code examples.
Thanks,Peter
I wrote TableEntityAdapter class so I will try to provide usage example, though I see question is asked a month ago, hopefully it is still usefull.
Firstly have a look at how it is tested by the unit tests by the SDK:
https://github.com/Azure/azure-storage-net/blob/master/Test/ClassLibraryCommon/Table/TableOperationUnitTests.cs
search for TableEntityAdapter to find relevant tests.. Note that unit tests of course do not write to actual table Storage service, they simulate what would happen when you read and write in terms of api calls, but they should give you a good idea.
Your original ShapeEntity object does not even need to implement ITableEntity interface. It also can contain complex nested properties. TableEntityAdapter class supports these scenarios.
Below is a simplified sample code for TableEntityAdapter:
To write your custom .Net object to Table Storage:
// 1. ShapeEntity is the custom .Net object that we want to write and read from Table Storage.
ShapeEntity shapeEntity = new ShapeEntity(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "square", 4, 4);
OperationContext operationContext = new OperationContext();
// 2. Instantiate a TableEntityAdapter object, passing in the custom object to its constructor. Internally this instance will keep a reference to our custom ShapeEntity object.
TableEntityAdapter<ShapeEntity> writeToTableStorage = new TableEntityAdapter<ShapeEntity>(shapeEntity, partitionKey, rowKey);
// 3. Now you can write the writeToTableStorage object to Table Storage just like any other object which inherits from ITableEntity interface. The TableAdapter generic class handles the boiler plate code and hand shaking between your custom .Net object and table storage sdk / service.
To read your custom object from Table Storage:
// 1. Read your entity back from table storage using the same partition / row key you specified (see step 2 above). You can use the Retrieve<T> table operation, specify T as TableEntityAdapter<ShapeEntity>
// 2. This should return you the TableEntityAdapter<ShapeEntity> object that you wrote to TableStorage at step 3 above.
// 3. To access the original ShapeEntity object, just refer to the OriginalEntity property of the returned TableEntityAdapter<ShapeEntity> object.
Your original ShapeEntity object does not even need to implement ITableEntity interface. It also can contain complex nested properties. TableEntityAdapter class supports these scenarios.
Note: TableEntityAdapter api does not support objects with collection type properties ie. List, Array, IEnumerable, ICollection etc..
If your objects contain these types of properties (directly under the root or somewhere in their object graph) then you should consider using the original Nuget package that I wrote which supports collection type property types as well.
ObjectFlattenerRecomposer Api version 2.0 that supports Collection, Enumerable type properties:
https://www.nuget.org/packages/ObjectFlattenerRecomposer/
and recently uploaded .Net Core version:
https://www.nuget.org/packages/ObjectFlattenerRecomposer.Core/1.0.0/
I'm using v2.0 of the API via the C# dll. But this problem also happens when I pass a Query String to the v2.0 API via https://rally1.rallydev.com/slm/doc/webservice/
I'm querying at the Artifact level because I need both Defects and Stories. I tried to see what kind of query string the Rally front end is using, and it passes custom fields and built-in fields to the artifact query. I am doing the same thing, but am not finding any luck getting it to work.
I need to be able to filter out the released items from my query. Furthermore, I also need to sort by the custom c_ReleaseType field as well as the built-in DragAndDropRank field. I'm guessing this is a problem because those built-in fields are not actually on the Artifact object, but why would the custom fields work? They're not on the Artifact object either. It might just be a problem I'm not able to guess at hidden in the API. If I can query these objects based on custom fields, I would expect the ability would exist to query them by built-in fields as well, even if those fields don't exist on the Ancestor object.
For the sake of the example, I am leaving out a bunch of the setup code... and only leaving in the code that causes the issues.
var request = new Request("Artifact");
request.Order = "DragAndDropRank";
//"Could not read: could not read all instances of class com.f4tech.slm.domain.Artifact"
When I comment the Order by DragAndDropRank line, it works.
var request = new Request("Artifact");
request.Query = (new Query("c_SomeCustomField", Query.Operator.Equals, "somevalue").
And(new Query("Release", Query.Operator.Equals, "null")));
//"Could not read: could not read all instances of class com.f4tech.slm.domain.Artifact"
When I take the Release part out of the query, it works.
var request = new Request("Artifact");
request.Query = (((new Query("TypeDefOid", Query.Operator.Equals, "someID").
And(new Query("c_SomeCustomField", Query.Operator.Equals, "somevalue"))).
And(new Query("DirectChildrenCount", Query.Operator.Equals, "0"))));
//"Could not read: could not read all instances of class com.f4tech.slm.domain.Artifact"
When I take the DirectChildrenCount part out of the query, it works.
Here's an example of the problem demonstrated by an API call.
https://rally1.rallydev.com/slm/webservice/v2.0/artifact?query=(c_KanbanState%20%3D%20%22Backlog%22)&order=DragAndDropRank&start=1&pagesize=20
When I remove the Order by DragAndDropRank querystring, it works.
I think most of your trouble is due to the fact that in order to use the Artifact endpoint you need to specify a types parameter so it knows which artifact sub classes to include.
Simply adding that to your example WSAPI query above causes it to return successfully:
https://rally1.rallydev.com/slm/webservice/v2.0/artifact?query=(c_KanbanState = "Backlog")&order=DragAndDropRank&start=1&pagesize=20&types=hierarchicalrequirement,defect
However I'm not tally sure if the C# API allows you to encode additional custom parameters onto the request...
Your question already contains the answer.
UserStory (HierarchicalRequirement in WS API) and Defect inherit some of their fields from Artifact, e.g. FormattedID, Name, Description, LastUpdateDate, etc. You may use those fields in the context of Artifact type.
The fields that you are trying to access on Artifact object do not exist on it. They exist on a child level, e.g. DragAndDropRank, Release, Iteration. It is not possible to use those fields in the context of Artifact type.
Parent objects don't have access to attributes specific to child object.
Artifact is an abstract type.
If you need to filter by Release, you need to make two separate requests - one for stories, the other for defects.
I load types dynamically through reflection, instantiate the classes, fill them with data and then save them to RavenDB. They all implement an interface IEntity.
In the RavenDB UI, I can see the classes correctly displayed and the meta data has the correct CLR type.
Now I want to get the data back out, change it and then save it.
What I'd like to do
I have the System.Type that matches the entity in RavenDB's CLR meta data, assuming that's called myType, I would like to do this:
session.Query(myType).ToList(); // session is IDocumentSession
But you can't do that, as RavenDB queries like so:
session.Query<T>();
I don't have the generic type T at compile time, I'm loading that dynamically.
Solution 1 - the Big Document
The first way I tried (for a proof of concept) was to wrap all the entities in a single data object and save that:
public class Data {
List<IEntity> Entities = new List<IEntity>();
}
Assuming the session is opened/closed properly and that I have an id:
var myDataObject = session.Load<Data>(Id);
myDataObject.Entities.First(); // or whatever query
As my types are all dynamic, specified in a dynamically loaded DLL, I need my custom json deserializer to perform the object creation. I specify that in the answer here.
I would rather not do this as loading the entire document in production would not be efficient.
## Possible solution 2 ##
I understand that Lucene can be used to query the type meta data and get the data out as a dynamic type. I can then do a horrible cast and make the changes.
Update after #Tung-Chau
Thank you to Tung, unfortunately neither of the solutions work. Let me explain why:
I am storing the entities with:
session.Store(myDataObject);
In the database, that will produce a document with the name of myDataObject.GetType().Name. If you do:
var myDataObject = session.Load<IEntity>(Id);
Then it won't find the document because it is not saved with the name IEntity, it is saved with the name of the dynamic type.
Now the Lucene solution doesn't work either but for a slightly more complex reason. When Lucene finds the type (which it does), Raven passes it to the custom JsonDeserialiser I mentioned. The custom Json Deserialiser does not have access to the meta data, so it does not know which type to reflect.
Is there a better way to retrieve data when you know the type but not at compile time?
Thank you.
If you have an ID (or IDs):
var myDataObject = session.Load<IEntity>(Id);
//change myDataObject. Use myDataObject.GetType() as you want
//....
session.SaveChange();
For query, LuceneQuery is suitable
var tag = documentStore.Conventions.GetTypeTagName(typeof(YourDataType));
var myDataObjects = session.Advanced
.LuceneQuery<IEntity, RavenDocumentsByEntityName>()
.WhereEquals("Tag", tag)
.AndAlso()
//....
this is related to a post I made yesterday, but havent been able to resolve: ASP.Net Web API showing correctly in VS but giving HTTP500
I think I need to simplify what I'm trying to do, and work up from there.
Can anyone please point me to an example of using the asp.net Web API (I'm using VS 2012 Express RC), to return JSON from a parent/child model?
eg: (pseudo Json):
Parent: Mark
..Child: Tom
..Child: Adam
..Child: Becki
Parent: Terry
..Child: Sophie
..Child: robert
I can get it to return data from one table, but not from a linked table.
Thanks for any help,
Mark
There are two ways to go away with this.
Create a new template class, loop through the list fetched using the EF and assign the values to the properties defined in the template class. This would give you the accurate results from one to multiple tables if any. Finally return the list to the json call.
While fetching the list from the EF, create a new anonymous type and select your desired columns. For this your webmethod would have the return type as IEnumerable
Cheers!
After looking at your original post my guess is that you have circular references in your objects. This post makes reference to using Json.Net which will give you more control over what is being returned to the client.
Your other option is to remove the foreign key reference tblCustomerBooking from the tblRental object (see below).
This may allow you to return the JSON objects and test that circular references are the issue.
[ForeignKey("customer_id")]
public virtual tblCustomerBooking tblCustomerBooking { get; set; }
I do suggest using Json.NET if you're planning on returning your Domain (i.e. Entity Objects) as this will avoid all circular references, and allow you keep your two-way object relationships.
My personal preference is to using DTO's and map your Domain objects to these DTO's, allowing you to have more control over what the client sees (as seeing the 'tbl' prefix in a object name isn't good practice)
I want to store list of parameters (that will define how document is going to be generated on the web page) in data base.
There is a number of item (or document) types, each type has a different set of parameters that vary (each type has it's own parameters).
Is it a good idea to store all parameters (key-value) as JSON in table's column?
Otherwise I would have to create Parameter Table for every Type and column for every parameter (10-30 params for every type).
A note: I am not going to search by parameters or something like that.
I will load the the JSON string (if I'll choose JSON), serialize it to Object and apply them on document as usual.
Since you have no requirement of searching by parameters, To me Json seems to be more robust because you will have Object ready with information when you Deserialize it. where as if you store it in columns and table you will have to initialize class members yourself. It will also have performance benifit as there will be only one column to fetch based on your document type.
Conclusion go with Json data in Database
Sounds like you should have a look at http://sisodb.com. However, it does support querying, but that is something you could turn-off and only rely on GetById.