Develop Reference

Parsing JSON Using Newtonsoft.Json Without Knowing the Structure

I'm working on a project that involves automating API calls using a Swagger Definition. I download the swagger.json file. The structure of the JSON Object I need to parse is not consistent. When parsing paths, there are a list of objects, then within that they have the methods that can be used for that specific path. I can retrieve just the path using various string methods but my question was, is there a good way to parse json if the JSON is structured in such a way that it does not have a firm key? Here is an example of what I mean:
{"/user": {
"post": {
"tags": [
"user"
],
"summary": "Create user",
"description": "This can only be done by the logged in user.",
"operationId": "createUser",
"consumes": [
"application/json"
],
"produces": [
"application/json",
"application/xml"
],
"parameters": [
{
"in": "body",
"name": "body",
"description": "Created user object",
"required": true,
"schema": {
"$ref": "#/definitions/User"
}
}
],
"responses": {
"default": {
"description": "successful operation"
}
}
}
}
If I wanted to just parse that path and retrieve the method object how could I go about that considering sometimes the object will be "post" or sometimes it will be "get", "put", etc depending on what is allowable for the path.
JObject jsonResp = swaggerDownload();
JObject paths = (JObject)jsonResp["paths"];
foreach (var i in paths)
{
string pathToString = i.ToString();
var shaveSomethings = pathToString.Substring(1, something.Length - 2);
var pathAndJson = shaveSomethings.Split(new[] { ',' }, 2);
string correctJsonStructure = "{\"" + pathAndJson[0] + "\":" + pathAndJson[1] + "}";
JObject bd = JObject.Parse(correctJsonStructure);
//dynamic pathsTest = JsonConvert.DeserializeObject<dynamic>(correctJsonStructure);
//JObject result = JsonConvert.DeserializeObject<JObject>(correctJsonStructure);
//Console.WriteLine(bd["/user"]);
}

The swagger.json file should have full definition of each entity that endpoints return. You can follow How to create Rest API client to get a working client.
I've dealt with an API where responses didn't always match the definition. I saved all responses to a store/log first and then would try to de-serialize JSON. In case of an exception I would go back to store/log and see what was different and update my code to accommodate for the change. After few iterations there were no new changes and the ordeal was over.
Hope that helps.

Elasticsearch DeleteByQuery not working, getting 400 bad request

I have the following Nest query to delete all the matching documents, quite straight forward but I am getting 400 bad request on it.
var client = new ElasticClient();
var request = new DeleteByQueryRequest<Type>("my-index")
{
Query = new QueryContainer(
new TermQuery
{
Field = "versionId",
Value = "ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"
}
)
};
var response = client.DeleteByQuery(request);
Assert.IsTrue(response.IsValid);
Thanks for any help.
---------------Update---------------
Request Body
{"query":{"term":{"versionId":{"value":"ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"}}}}
Response Body
{"took":0,"timed_out":false,"_indices":{"_all":{"found":0,"deleted":0,"missing":0,"failed":0}},"failures":[]}
Query in Sense plugin:
GET /my-index/type/_search
{
"query": {
"match": {
"versionId": "ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"
}
}
}
Query Response:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 116,
"max_score": 2.1220484,
"hits": []
...
}}
---------------NEST QUERY--------------
DELETE http://localhost:9200/my-index/component/_query?pretty=true
{
"query": {
"term": {
"versionId": {
"value": "ea8e517b-c2e3-4dfe-8e49-edc8bda67bad"
}
}
}
}
Status: 200
{
"took" : 0,
"timed_out" : false,
"_indices" : {
"_all" : {
"found" : 0,
"deleted" : 0,
"missing" : 0,
"failed" : 0
}
},
"failures" : [ ]
}

It sounds like you may be using Elasticsearch 2.x in conjunction with NEST 2.x. As part of Elasticsearch 2.0, Delete by query was moved out of Elasticsearch core and into a separate plugin that needs to be installed. You can install the plugin using the following command within the Elasticsearch bin directory
bin/plugin install delete-by-query
Starting up the node again, Delete by query should now work as expected.
If you ever need to get more details about why a request has failed, you can inspect the .DebugInformation on the response to get the audit trail for the request.

Which JSON adapter enables Ember to propagate all necessary requests to the server to maintain model consistence?

I am using Andy Crum's EmberDataModelMaker.
Having punched in the following two classes
// app/models/server-item.js
export default DS.Model.extend({
hostName: DS.attr('string'),
syncServers: DS.hasMany('string'),
subscribers: DS.hasMany('string'),
mailHost: DS.attr('string'),
mailHostLogin: DS.hasMany('credentials')
});
// app/models/credentials.js
export default DS.Model.extend({
user: DS.attr('string'),
password: DS.attr('string'),
server: DS.belongsTo('serverItem')
});
It's showing the following three different expected JSON formats (a very nice feature btw.):
DS.RESTAdapter
"serverItems": [
{
"id": 1,
"hostName": "foo",
"syncServers": [
<stringids>
],
"subscribers": [
<stringids>
],
"mailHost": "foo",
"mailHostLogin": [
<Credentialsids>
]
}
],
"credentials": [
{
"id": 1,
"user": "foo",
"password": "foo",
"server": <ServerItemid>
}
]
DS.ActiveModelAdapter
"serverItems": [
{
"id": 1,
"host_name": "foo",
"sync_server_ids": [
<stringids>
],
"subscriber_ids": [
<stringids>
],
"mail_host": "foo",
"mail_host_login_ids": [
<Credentialsids>
]
}
],
"credentials": [
{
"id": 1,
"user": "foo",
"password": "foo",
"server_id": <ServerItemid>
}
]
DS.JSONAPIAdapter
{
"data": {
"type": "server-items",
"id": "1",
"attributes": {
"HostName": "foo",
"MailHost": "foo",
},
"relationships": {
"SyncServers": {
"data": {
"type": "SyncServers",
"id": <SyncServersid>
}
},
"Subscribers": {
"data": {
"type": "Subscribers",
"id": <Subscribersid>
}
},
"MailHostLogin": {
"data": {
"type": "MailHostLogin",
"id": <MailHostLoginid>
}
}
},
"included": [
{
<sideloadedrelationships>
]
}
}
}
{
"data": {
"type": "credentials",
"id": "1",
"attributes": {
"User": "foo",
"Password": "foo",
},
"relationships": {
"Server": {
"data": {
"type": "Server",
"id": <Serverid>
}
}
},
"included": [
{
<sideloadedrelationships>
]
}
}
}
I am going to implement (or rather change) some WebServices on the Server side (using C#, ASP.NET Web API). Currently, the WebService already creates a result that is pretty similar to the format expected with DS.RESTAdapter - obviously, it would be ideal if I could use it without compromising the Data Integrity - can I?
If yes, would it empower Ember Data to send all the requests necessary to maintain the data consistency on the server? Meaning, would the client send a DELETE request to the server not only for the ServerItem but also for the Credentials item that is referenced via the mailHostLogin property when the user wants to delete a ServerItem?
If not: are both of the other two adapters fulfilling the above mentioned consistency requirement? Which of the other two should I implement - any experiences/recommendations out there?

You should choose whichever Adapter closest fits your API data structure as a basis(sounds like DS.RESTAdapter in this case). You can extend the adapters and serializers that are a closest fit to make any necessary adjustments(this can be done both application wide or on a per model basis).
However, I don't think that the Ember Data model relationships(i.e. belongsTo and hasMany) are binding in such a way that will automatically result in the "data consistency" you are looking for. If your application requirements are to delete all associated Credentials records when a ServerItem is deleted, I would recommend doing that server side when handling the DELETE ServerItem API request. That would result in better performance(1 HTTP call instead of 2 or N depending if credentials can be deleted in bulk) and be much less error prone due to potential network or other failure of calls to delete Credentials after a ServerItem is deleted.
After a successful ServerItem delete, you could loop through it's credentials and unload the records from the client side store to keep it in sync with the new state on the server. Something like:
serverItemCredentials.forEach(function(id) {
if (this.store.recordIsLoaded('credential', id)) {
this.store.unloadRecord(this.store.peekRecord('credential', id));
}
});

How to manage Facebook api response data in C# asp.net?

How to convert facebook api response in user readable HTML format?
I call graph api
https://graph.facebook.com/me/feed?access_token=<token>
below is my response data from API.
{
"data": [
{
"id": "100000626589435_240877109276507",
"from": {
"name": "Abhi Patel",
"id": "100000626589435"
},
"type": "status",
"created_time": "2011-08-02T10:36:17+0000",
"updated_time": "2011-08-02T10:36:17+0000"
},
{
"id": "100000626589435_240760105954874",
"from": {
"name": "Abhi Patel",
"id": "100000626589435"
},
"type": "status",
"created_time": "2011-08-02T03:02:21+0000",
"updated_time": "2011-08-02T03:02:21+0000"
},
{
"id": "100000626589435_223775454320006",
"from": {
"name": "Abhi Patel",
"id": "100000626589435"
},
"picture": "http://profile.ak.fbcdn.net/hprofile-ak-snc4/274314_100000898272591_5481895_q.jpg",
"link": "http://www.facebook.com/?ref=nf_fr",
"icon": "http://static.ak.fbcdn.net/images/icons/?8:",
"type": "link",
"created_time": "2011-06-28T18:56:44+0000",
"updated_time": "2011-06-28T18:56:44+0000"
}
],
"paging": {
"previous": "<previous link>",
"next": "<next link>"
}
}
also want paging in facebook response data,
I want 20 records from facebook api response. How to manage this things..

Facebook returns raw JSON data. There are no style elements to it. It's up to you to present the data returned in the format you choose. Imagine if Facebook returned HTML and style elements. That wouldn't work very well for desktop applications or mobile devices. Instead, they just give you the raw data, and you design the HTML elements, or the WPF Views, or whatever to show the data you want to show.
By returning the raw data, you can also store it locally in a database for your own queries, or your own applications purposes.
Edited to add: You can parse out the objects by accessing the JSON elements directly, or you can deserialize the result to C# objects.
Console.WriteLine(response.data[0].from.name);
As for paging, you need to parse out the Paging elements. The Facebook C# SDK returns dynamic objects, so you can do something like
string next = response.paging.next;
string prev = response.paging.prev;
And then just make a request to each URL to fetch the data you want.

Use JSON.net and convert into the XML then it would be easy to manage for you.

WCF Data Services Expand behaves unexpectedly when certain entity page sizes are set

While getting our WCF Data Service ready for production we encountered an issue with the behaviour of the expand operator when paging is enabled.
With paging disabled, expand works as expected. But when I enable paging on any of the expanded entity sets, no matter what the page sizes, the expanded entities appear to page with a size of 1.
[UPDATE]
In the absence of any further input from here or the MSDN forums I've created a bug on Connect. Maybe someone over the wall will get to the bottom of it!
For example, supposed I have the following simple model:
It's running on a generated SQL database with some sample data:
INSERT INTO [dbo].[Towns] (Name) VALUES ('Berlin');
INSERT INTO [dbo].[Towns] (Name) VALUES ('Rome');
INSERT INTO [dbo].[Towns] (Name) VALUES ('Paris');
INSERT INTO [dbo].[Gentlemen] (Id, Name) VALUES (1, 'Johnny');
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Frieda', 'Berlin', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Adelita', 'Berlin', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Milla', 'Berlin', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Georgine', 'Paris', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Nannette', 'Paris', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Verona', 'Rome', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Gavriella', 'Rome', 1);
The Data Service is straightforward (note that here paging is disabled):
namespace TestWCFDataService
{
public class TestWCFDataService : DataService<TestModel.TestModelContainer>
{
// This method is called only once to initialize service-wide policies.
public static void InitializeService(DataServiceConfiguration config)
{
config.SetEntitySetAccessRule("Ladies", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("Gentlemen", EntitySetRights.AllRead);
config.SetEntitySetAccessRule("Towns", EntitySetRights.AllRead);
//config.SetEntitySetPageSize("Ladies", 10);
//config.SetEntitySetPageSize("Gentlemen", 10);
//config.SetEntitySetPageSize("Towns", 10);
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
}
}
}
Now, my user wants to find every Lady whose Town is "Berlin" and also who their Gentleman is.
The query in question is:
http://localhost:62946/TestWCFDataService.svc/Towns('Berlin')?$expand=Ladies/Gentleman
When I run this query (JSON because the Atom version is gigantic) I get the expected output; a town with three ladies, all of whom have Johnny as their gentleman.
var result = {
"d": {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Towns('Berlin')", "type": "TestModel.Town"
}, "Name": "Berlin", "Ladies": [
{
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)", "type": "TestModel.Lady"
}, "Id": 1, "Name": "Frieda", "Gentleman": {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
}, "Id": 1, "Name": "Johnny", "Ladies": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
}
}
}, "Town": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)/Town"
}
}
}, {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(2)", "type": "TestModel.Lady"
}, "Id": 2, "Name": "Adelita", "Gentleman": {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
}, "Id": 1, "Name": "Johnny", "Ladies": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
}
}
}, "Town": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(2)/Town"
}
}
}, {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(3)", "type": "TestModel.Lady"
}, "Id": 3, "Name": "Milla", "Gentleman": {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
}, "Id": 1, "Name": "Johnny", "Ladies": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
}
}
}, "Town": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(3)/Town"
}
}
}
]
}
}
There are going to be many Towns eventually so I enable paging for Town.
...
config.SetEntitySetPageSize("Towns", 10);
...
The query continues to function as expected. But there are also going to be a lot of Ladies and Gentlemen so I want to be able to limit the number of results that are returned:
...
config.SetEntitySetPageSize("Ladies", 10);
config.SetEntitySetPageSize("Gentlemen", 10);
...
But when I set a page size on either the Ladies entity set or the Gentlemen entity set (or both) the results of my query change unexpectedly:
var result = {
"d": {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Towns('Berlin')", "type": "TestModel.Town"
}, "Name": "Berlin", "Ladies": {
"results": [
{
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)", "type": "TestModel.Lady"
}, "Id": 1, "Name": "Frieda", "Gentleman": {
"__metadata": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
}, "Id": 1, "Name": "Johnny", "Ladies": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
}
}
}, "Town": {
"__deferred": {
"uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)/Town"
}
}
}
]
}
}
}
The expand only includes one of the Lady objects (although at least her Gentleman is included). It doesn't matter how large the page size is set to, the query still only returns one object in the expanded collection.
It also does not matter whether or not the page size is set on one or both of the expanded entities, as long as one of them has a page size set then only one of the Lady objects will be eagerly loaded.
This behaviour smells buggy to me, as according to the OData Specification:
"A URI with a $expand System Query Option indicates that Entries associated with the Entry or Collection of Entries identified by the Resource Path section of the URI must be represented inline (i.e. eagerly loaded)."
Am I misreading the spec? Should I have expected this behaviour? I just want to be able to limit the page size of the entity sets when accessed directly but also have them eagerly loadable.
Is it a bug in WCF Data Services? (or my code? or my brain?)
[EDIT]
More info: the documentation for WCF Data Services states that:
"Also, when paging is enabled in the data service, you must explicitly load subsequent data pages from the service."
But I can't find an explanation of why the page size for the related entity sets seems to default to 1 no matter what page size is specified.
[EDIT]
Yet more info: the version in question is on .NET 4 version 4.0.30319 with System.Data.Services version 4.0.0.0. It's the version that comes in the box with Visual Studio 2010 (with SP1 installed).
[EDIT]
A sample solution showing the behaviour is now up in a github repository. It's got paging turned on in the InitializeService method and a DB creation script that also adds some sample data so that we're on the same page.

It took a few months but this is apparently going to be fixed in the next version:
Posted by Microsoft on 12/15/2011 at 8:08 AM
Thank you for reporting this issue. We have confirmed that a bug in the Entity Framework is
causing this issue. The fix required changes to the core Entity Framework components that ship
in the .NET Framework. We have fixed the bug, the fix will be included in the next release of
the .NET Framework.

What version of WCF Data Services are you using?
There is a bug that I found relating to using Expand with server-driven paging in .NET Framework 4, but I thought that it only affected entities with compound keys and when using the OrderBy option, neither of which seem to apply here.
Still it definitely sounds like a bug.
Have you tried using Atom instead of JSON, and if so are the entities in the expand still missing?

Your query:
localhost:62946/TestWCFDataService.svc/Towns('Berlin')?$expand=Ladies/Gentleman
doesn't expand Ladies, only the Gentelman.
The query should look like:
localhost:62946/TestWCFDataService.svc/Towns('Berlin')?$expand=Ladies,Ladies/Gentleman.
Hope this helps!
Monica Frintu