I have dynamic keys and values that I get from db and then parse with Newtonsoft Json.NET but I don't know how can I serve them as static ones.
Example
This is what I have
{
"Id": 1,
"IsPublic": false,
"Notes": "",
"Values": [
{
"Key": "1",
"Value": "12.02.1991"
}
]
}
This is what I want
{
"Id": 1,
"IsPublic": false,
"Notes": "",
"Values": [
{
"1": "12.02.1991"
}
]
}
What I have tried
I tried to do it manually inside my query itself but it didn't work since it's trying to assign the value.
return _db.Archives.Single(x => x.Id == id).Batches.SelectMany(x => x.Items).Select(item => new
{
item.Id,
item.IsPublic,
item.Notes,
Values = item.ArchiveFieldValues.Select(value => new
{
/*
This works just fine
Key = value.ArchiveField.Key,
Value = value.Value
*/
// This is what I tried but it does not work
value.ArchiveField.Key = value.Value
})
}).AsQueryable();
First off, it's complex enough that you probably want to pull it out into it's own function.
You can use an ExpandoObject as an object that can have properties dynamically added and removed to it. Just cast it to an IDictionary (it implements that interface explicitly) and add the pairs. You can type the result as a dynamic or ExpandoObject based on whichever you prefer.
//I know this isn't the real type of your input;
//modify the parameter to be of the actual type of your collection of pairs
//TODO come up with better name for this function
public static dynamic Foo(IEnumerable<KeyValuePair<string,string>> pairs)
{
IDictionary<string, object> result = new ExpandoObject();
foreach (var pair in pairs)
result.Add(pair.Key, pair.Value);
return result;
}
Your query can then be modified to have:
Values = Foo(item.ArchiveFieldValues),
Also note that the query provider most likely won't be able to do anything with that translation, so you'll probably need to throw in an AsEnumerable before the select so that this projection is done in linq to objects.
Related
I am trying the convert my json string into dictionary but could not successful.
{
"healths": [
{
"serviceName": "UserMgt",
"subService": [
{
"subServiceName": "Base",
"status": 10
},
{
"subServiceName": "Url",
"description": "Url is bad",
"status": 10
},
{
"subServiceName": "Configuration",
"status": 2
}
]
},
{
"serviceName": "ConfigurationMgt",
"subService": [
{
"subServiceName": "url",
"description": "urlis OK!",
"status": 2
},
{
"subServiceName": "Configuration Db",
"status": 2
}
]
}
}...and so son
So, as you see we have a service name, and based on that we have subserviceName then again service name and subservicename.Basiclly i want to convert this into dictionary as a key-value pair so that based on the key(service) we have values subservice(again a key value pair).
I tried many things but does not get a proper result.
like for example using newtonsoft nuget.
JObject jsonObj = JObject.Parse(formattedJson);
Dictionary<string, object> dictObj = jsonObj.ToObject<Dictionary<string, object>>();
foreach(var kv in dictObj)
{
Console.WriteLine(kv.Key + "::::" + kv.Value);
}
In this scenario I am getting output like key as a healths and every thing as a value.
Can anyone please help me regarding this.The Json and c# is new for me.
Thanks In Advance.
Assuming that you are guaranteed to have the described structure, you can deserialize to JObject and use LINQ's ToDictionary to process it:
var dict = JsonConvert.DeserializeObject<JObject>(json)["healths"]
.ToDictionary(
s => s["serviceName"].ToString(),
s => s["subService"]
.ToDictionary(ss => ss["subServiceName"].ToString(), ss => ss["status"].ToString()));
Where you can change ss => ss["status"].ToString() to select what is actually needed as subService value.
I have a JSON string like below:
{
"MetaData": {
"ResourcesUsed": 1
},
"Result": [
{
"locations": [
{
"country": "Papua New Guinea",
"city": "Jacquinot Bay",
"locTypeAttributes": {
"localDate": "2018-10-08T04:21:00-07:00",
"utcDate": "2018-10-08T04:21:00-07:00",
},
"point": {
"coordinates": [
151.52,
-5.6
],
"type": "Point"
}
},{
"country": "Papua New Guinea2",
"city": "Jacquinot Bay2",
"locTypeAttributes": {
"localDate": "2018-10-08T04:21:00-07:00",
"utcDate": "2018-10-02T04:21:00-07:00",
},
"point": {
"coordinates": [
151.52,
-5.6
],
"type": "Point"
}
}
]
}
]
}
I converted it to a JSON object using Newtonsoft. What I want to do is to sort the locations array(s) inside the Result array by the utcDate field nested in each locations item. I found the following thread: C# Sort JSON string keys. However, I could not still implement it since I have arrays inside my object, while that question deals purely with sorting objects inside objects alphabetically by property name.
Here is a piece of code that I wrote so far:
public string GenerateJson()
{
var model = (JObject)JsonConvert.DeserializeObject(data);
Sort(model);
}
private void Sort(JObject jObj)
{
var props = jObj["Result"][0]["locations"].ToList();
foreach (var prop in props)
{
prop.Remove();
}
foreach (var prop in props.OrderBy(p => p.Name))
{
jObj.Add(prop);
if (prop.Value is JObject)
Sort((JObject)prop.Value);
if (prop.Value is JArray)
{
Int32 iCount = prop.Value.Count();
for (Int32 iIterator = 0; iIterator < iCount; iIterator++)
if (prop.Value[iIterator] is JObject)
Sort((JObject)prop.Value[iIterator]);
}
}
}
You can sort each individual "Result[*].locations" array using LINQ as follows:
// Load the JSON without parsing or converting any dates.
var model = JsonConvert.DeserializeObject<JObject>(data, new JsonSerializerSettings{ DateParseHandling = DateParseHandling.None });
// Construct a serializer that converts all DateTime values to UTC
var serializer = JsonSerializer.CreateDefault(new JsonSerializerSettings{ DateTimeZoneHandling = DateTimeZoneHandling.Utc });
foreach (var locations in model.SelectTokens("Result[*].locations").OfType<JArray>())
{
// Then sort the locations by utcDate converting the value to UTC at this time.
var query = from location in locations
let utcDate = location.SelectToken("locTypeAttributes.utcDate").ToObject<DateTime>(serializer)
orderby utcDate
select location;
locations.ReplaceAll(query.ToList());
}
Notes:
The JSON is initially loaded using DateParseHandling.None to prevent the "localDate" and "utcDate" strings from being prematurely interpreted as DateTime objects with a uniform DateTime.Kind.
(For a discussion of how Json.NET interprets strings that look like dates, see Serializing Dates in JSON.)
We then iterate through all "locations" arrays using SelectTokens("Result[*].locations") where [*] is the JSONPath wildcard character, selecting all entries in the "Results" array.
We then order each "locations" array by deserializing the nested locTypeAttributes.utcDate to a UTC date, then ordering using LINQ.
Finally the array is updated using JArray.ReplaceAll().
If any locTypeAttributes.utcDate property is missing, an exception will be thrown. You could instead deserialize to DateTime? if that is a possibility.
Working sample .Net fiddle here.
The following code returns the List as Key-Value Pair what is available in the Database using Dapper. The KeyValue Pair is being called from the WebApi Controller and being served as Json, so that list can be used by the Javascript script for a drop down box.
public static IEnumerable<KeyValuePair<int, string>> KeyValueList()
{
const string sql = #"SELECT Key ,
Value
FROM Constants";
var result = DataStoreReader.Query(ConnectionHelper.GetConnectionString, sql)
.Select(item =>
new KeyValuePair<int, string>(item.Key, item.Value));
return result;
}
The above code returns the following result as Json.
{
"results": [
{
"key": 1,
"value": "Value - 1"
},
{
"key": 2,
"value": "Value - 2"
}
]
}
But, I need this result, notice the first object as null must be automatically added. However, my database doesn't have the null record and it is not possible to add.
{
"results": [
{
"key": null,
"value": "Select"
},
{
"key": 1,
"value": "Value - 1"
},
{
"key": 2,
"value": "Value - 2"
}
]
}
You can modify your key parameter type to int?, and:
public static IEnumerable<KeyValuePair<int?, string>> KeyValueList()
{
const string sql = #"SELECT Key ,
Value
FROM Constants";
var result = DataStoreReader.Query(ConnectionHelper.GetConnectionString, sql)
.Select(item => new KeyValuePair<int?, string>(item.Key, item.Value))
.ToList();
//Insert the first/default element
result.Insert(0, new KeyValuePair<int?, string>(null, "Select"));
return result;
}
With the help of you all, I managed to achieve it. Here is the solution I have got so far.
Converted the int to int? to accept null.
Converted the Dapper result to .ToList().
Inserting the object row manually to the List with null and constant value.
Code:
public static IEnumerable<KeyValuePair<int?, string>> KeyValueList()
{
const string selectValue = "Select";
const string sql = #"SELECT Key ,
Value
FROM Constants";
var result = DataStoreReader.Query(ConnectionHelper.GetConnectionString, sql)
.Select(item =>
new KeyValuePair<int?, string>(item.Key, item.Value)).ToList();
result.Insert(0, new KeyValuePair<int?, string>(null, selectValue));
return result;
}
Background
What I'm Trying to Do
I have a list of vehicles.
I have an API (WebAPI v2) that takes in a list of filters for a make and models
a filter consists of 1 make and 0 or more models. (e.g. "Honda" and ["Civic", "Accord"])
If a filter is passed in with a make and no models, I want it to match all models for that make.
If a filter is passed in with a make and models, I want it to make only those models for that make.
The Filter Object I'm using
public class MakeModelFilter : IMakeModelFilter
{
public string Make { get; set; }
public List<string> Models { get; set; }
}
What the entire API Call Looks Like
{
"MakeModelFilters": [
{"Make": "BMW", "Models": ["X3", "X5"]}
],
"TypeFilter": [],
"GenericColorFilter": [],
"FeaturesFilter": [],
"MaxMileage" : 100000,
"PriceRange": {"Min": 1, "Max": 1000000},
"SearchText": ""
}
The portion I'm concerned with is the MakeAndModelFilters list (the rest works as designed currently).
How I'm currently obtaining search results:
var vehicles = _esClient.Search<Vehicle>(s => s
.From(0).Size(10000)
.Query(q => q
.Filtered(fq => fq
.Filter(ff => ff
.Bool(b => b
.Must(m=> m.And(
m.Or(makeModelFilterList.ToArray()),
m.Or(featureFilters.ToArray()),
m.Or(typeFilters.ToArray()),
priceRangeFilter,
mileageFilter))
)
)
.Query(qq => qq
.QueryString(qs => qs.Query(criteria.SearchText))
)
)
)
);
The Problem
No matter how I structure the filter, it seems to filter out all documents -- not in our best interest. :) Something in my boolean logic is wrong.
Where I think the problem lies
The list of make and model filters that I or together is generated by this method:
private List<FilterContainer> GenerateMakeModelFilter(List<MakeModelFilter> makeModelFilters)
{
var filterList = new List<FilterContainer>();
foreach (var filter in makeModelFilters)
{
filterList.Add(GenerateMakeModelFilter(filter));
}
return filterList;
}
This method calls the individual method to generate a bool for each make/model filter I have.
What I think the problem method is
The below method, as far as I'm aware, does the following:
If no make is passed in, throw exception
If only a make is passed in, return a bool for only that make.
If a make and models are passed in, return an a bool of the make filter + an or of all the model terms. e.g. Make:BMW AND (model:X3 OR model:X5)
Code is below:
private FilterContainer GenerateMakeModelFilter(MakeModelFilter makeModelFilter)
{
if (string.IsNullOrWhiteSpace(makeModelFilter.Make)) { throw new ArgumentNullException(nameof(makeModelFilter));}
var makeFilter = new TermFilter { Field = Property.Path<Vehicle>(it => it.Make), Value = makeModelFilter.Make };
var boolMake = new BoolFilter { Must = new List<FilterContainer> { makeFilter } };
var modelFilters = GenerateFilterList(Property.Path<Vehicle>(it => it.Model), makeModelFilter.Models);
if (!modelFilters.Any())
{
// If it has a make but no model, generate boolFilter make only.
return boolMake;
}
var orModels = new OrFilter {Filters = modelFilters};
var boolModels = new BoolFilter {Must = new List<FilterContainer> {orModels}};
var boolMakeAndModels = new AndFilter {Filters = new List<FilterContainer> {boolMake, boolModels}};
return new BoolFilter {Must = new List<FilterContainer> {boolMakeAndModels}};
}
FYI, GenerateFilterList just creates a list of Term filters and returns the list.
FYI: Generated ElasticSearch JSON
This might be a clue to where I'm going wrong (though it's huge). I've just been staring at it so long that I can't see it I think.
{
"from": 0,
"size": 10000,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"and": {
"filters": [
{
"or": {
"filters": [
{
"bool": {
"must": [
{
"and": {
"filters": [
{
"bool": {
"must": [
{
"term": {
"make": "BMW"
}
}
]
}
},
{
"bool": {
"must": [
{
"or": {
"filters": [
{
"term": {
"model": "x3"
}
},
{
"term": {
"model": "x5"
}
}
]
}
}
]
}
}
]
}
}
]
}
}
]
}
},
{ },
{ },
{
"range": {
"sellingPriceUSD": {
"lte": "1000000",
"gte": "1"
}
}
},
{
"range": {
"miles": {
"lte": "100000"
}
}
}
]
}
}
]
}
}
}
}
}
Refactor 1: Move more Towards Bitwise operations
Per Martijn's answer and Zachary's post that he references, I've updated my GenerateFilterList to return a concatenated filterContainer:
private FilterContainer GenerateFilterList(PropertyPathMarker path, List<string> filter)
{
if (filter == null || filter.Count <= 0){ return null; }
FilterContainer returnFilter = null;
foreach (var aFilter in filter)
{
returnFilter |= new TermFilter {Field = path, Value = aFilter.ToLowerInvariant()};
}
return returnFilter;
}
And then for my GenerateMakeModelFilter, I perform an "and" against the "model filters", which should be a bitwise or based on the above code:
private FilterContainer GenerateMakeModelFilter(MakeModelFilter makeModelFilter)
{
if (string.IsNullOrWhiteSpace(makeModelFilter.Make)) { throw new ArgumentNullException(nameof(makeModelFilter)); }
var makeFilter = new TermFilter { Field = Property.Path<Vehicle>(it => it.Make), Value = makeModelFilter.Make };
var modelFilters = GenerateFilterList(Property.Path<Vehicle>(it => it.Model), makeModelFilter.Models);
return makeFilter && modelFilters;
}
This shortens the part that retrieves the query:
QueryContainer textQuery = new QueryStringQuery() {Query = criteria.SearchText };
FilterContainer boolFilter = makeModelFilter || featureFilter || typeFilter || priceRangeFilter || mileageFilter;
var vehicles = _esClient.Search<Vehicle>(s => s
.From(0).Size(10000) //TODO: Extract this into a constant or setting in case the inventory grows to 10k+. This prevents it from paging.
.Query(q => q
.Filtered(fq => fq
.Filter(filter => filter.Bool(bf => bf.Must(boolFilter)))
.Query(qq => textQuery)
)
)
);
return vehicles.Documents.ToList<IVehicle>();
...but I still have no documents returned. What the heck am I missing? If I have a Make of Honda with Models of "Civic" and "Accord", and a make of "BMW" with no models, I should receive all vehicles with honda + civic || honda + accord || bmw + (any model). I'll keep at it.
And,or, & not filters might not be doing what you want. They are a special filter construct that performs better when combining filters that do not operate on bitsets. Must read on this topic:
https://www.elastic.co/blog/all-about-elasticsearch-filter-bitsets
Knowing when to use and/or/not filters vs bool filters can be quite confusing and with Elasticsearch 2.0 you can use the bool filter in ALL contexts and it will know how to best execute the filters/queries in its clauses. No more need for you to hint!
Further more although the bool filter/query is named bool it does a unary bool whereas you might expect it to be a binary bool.
This is why the bool clauses are must/should/must_not vs and/or/not.
In NEST if you use the && || ! operators combined with parenthesis we will compose one or many bool queries so that it acts in the binary bool fashion you write it down in C#.
e.g:
.Query(q=>q
(q.Term("language", "php")
&& !q.Term("name", "Elastica")
)
||
q.Term("name", "NEST")
)
If you need a more dynamic list you can use the assignment operators != and &=:
private FilterContainer GenerateMakeModelFilter(List<MakeModelFilter> makeModelFilters)
{
FilterContainer filter = null;
foreach (var filter in makeModelFilters)
{
filter |= GenerateMakeModelFilter(filter);
}
return filter;
}
Similarly if you refactor GenerateMakeModelFilter to take advantage of the C# boolean operator overloads you'll end up with an easier to read and debug query. Both in terms of C# as well as the query that gets send to Elasticsearch.
Our documentation goes into it in some more detail http://nest.azurewebsites.net/nest/writing-queries.html
UPDATE
Awesome refactor! Now we can focus on mappings in elasticsearch. When you index a json property it goes through an analysis chain which takes the single string and tries to make 1 or more terms out of it that are going to be stored in lucene's inverted index.
By default elasticsearch will analyze all string fields using the standard analyzer
In your case BMW will go through the standard analyzer which splits on whitespace (Unicode standard annex #29 to be exact) and lowercases it.
So the term in the inverted index is bmw. In elasticsearch some queries are also analyzed at query time so a e.g a match query for BMW is also analyzed and transformed to bmw before consulting the inverted index and thus will find documents no matter the casing of BMW at query time.
The term query/filter that you are using is not analyzed at query time so it will try to find BMW in the inverted index where the inverted index only has bmw. This is great if you only want exact term matches. If you set up your mapping so that a field is not analyzed you could for instance do exact matches on New York without worrying its actually stored as two separate terms new and york and inadvertently also get results from New New York
I have a C# class which contains a property of Dictionary
I have a web-page which contains a list of items i need to cast into this dictionary.
My web-site will send the list up to my C# MVC application as JSON and then JsonConvert.Deserialise the JSON into my Dictionary object.
JsonConvert.Deserialise is expecting JSON in the following format:
"MyVariableName":{
"Item 1":true,
"Item 2":true
}
I need to know how i can construct this object in JavaScript.
So far, i have tried this without luck:
var items = [];
var v = $('#Items :input');
$.each(v, function(key, val) {
items.push({
key: val.value,
value: val.checked
});
});
JSON.stringify(v, null, 2);
But this returns a json converted value of:
"MyVariableName": [
{
"key": "Item 1",
"value": true
},
{
"key": "Item 2",
"value": true
}]
Which in turn does not de-serialize to my dictionary.
Thanks
Don't make an array; make an object:
var items = {};
$('#Items :input').each(function(i, val) {
items[val.value] = val.checked;
});
You have to use javascript serialization
One more thing you have different value int key, value pair like string and Boolean type, so you have to use Dictionary type.
And JavaScriptSerializerobject you will get System.Web.Script.Serialization name space of System.Web.Extensions.dll, v4.0.30319 assembly.
var jSerializer = new JavaScriptSerializer();
var newList= jSerializer.Deserialize<Dictionary<string,object>>(newData);