I'm using highcharts to show some data...however i'm having issues due to my query not returning the data in the correct format for some charts.
Year Key Value
2011 Key1 5
2011 Key2 10
2012 Key1 12
2012 Key2 8
2013 Key1 3
Here's part of my query:
XAxisList = db.Datas.Where(w => w.CategoryID == measurementID).Select(x => x.Year).Distinct(),
Data = from d in db.Datas
where d.CategoryID == measurementID
group d by d.DataKey into g
select new
{
Key = g.Key,
Values = g.Select(s => s.Value)
}
Here is what my linq query returns:
"XAxisList": [
"2011",
"2012",
"2013"
],
"Data": [
{
"Key": "Key1",
"Values": [
"5",
"12"
"3"
]
},
{
"Key": "Key2",
"Values": [
"10",
"8"
]
}
]
How would I add a 'null' array value in the Key2 values so it would look like below?
"XAxisList": [
"2011",
"2012",
"2013"
],
"Data": [
{
"Key": "Key1",
"Values": [
"5",
"12"
"3"
]
},
{
"Key": "Key2",
"Values": [
"10",
"8",
**null**
]
}
]
Thanks in advance for your help!
This simply cannot be done, based on your dataset. If your dataset contained the following record
2013 Key2 null
then there wouldn't have been any issue.
You could understand, why this cannot be done, if you think about the following:
group d by d.DataKey into g
What are you doing there?
You are grouping your records based of the DataKey property. Since there are two different keys, key1 and key2, there would be two groups. The group of key1 would have three records and the group of key2 would have two records.
Furthermore, I don't see any way that you could overcome this. As I can understand by your data, you have some yearly values for some keys and you want to use them for creating a chart.
Now there is missing a value for the key2 and the year 2013. So you think that if you could add a null at the end of the corresponding sequence everything would be ok.
{
"Key": "Key2",
"Values": [
"10",
"8",
"null"
]
}
Have you thinked about what would be the situation of missing a value for 2012? It would be exactly the same ! So adding a null at the end of the Values array wouldn't make any sense.
If the Year values would be specific or in a specific range I could come up with an idea, but I am not sure that this is the case.
I'd do the modification for reporting or charts after the query, or it'll get extremely messy:
XAxisList = db.Datas.Where(w => w.CategoryID == measurementID).Select(x => x.Year).Distinct(),
Data = //your query
int largest = Data.Values.Max(v => v.Count);
foreach(var d in Data)
{
while(d.Values.Count < largest)
d.Values.Add(null);
}
Or similar. You'll probably need to make your Values a list with ToList to get the Add method.
Related
I have below code :
[
{
"OrderId": "Order1",
"filterOrder": [ "ABC", "XYZ" ],
"Details": [
{
"id": 1,
"value": 100,
"filterDetails": [ "Apples", "Oranges" ]
},
{
"id": 2,
"value": 200,
"filterDetails": [ "Banana", "Blank" ]
}
]
},
{
"OrderId": "Order2",
"filterOrder": [ "PQR", "Blank" ],
"Details": [
{
"id": 1,
"value": 100,
"filterDetails": [ "Apples", "Peaches" ]
},
{
"id": 2,
"value": 200,
"filterDetails": [ "Banana", "Mango" ]
}
]
}
]
C# code:
string i = GetJsonText();
var lst = JsonConvert.DeserializeObject<List<Root>>(i);
My requirement here is to remove all those objects from the response where the filters are mentioned as "Blank".
So I have used the below code to remove the blanks :
lst.RemoveAll(x => x.filterOrder.Contains("Blank"));
lst.ForEach(fe => {fe.Details.RemoveAll(r=> r.filterDetails.Contains("Blank"));});
This is working perfectly fine. But now the requirement changed to remove objects based on multiple strings and not single string. Means I'll have something like
string removeCriteria ="Blank,Blank1,Blank2"
I convert this into a list of strings like this :
List<String> removeList = removeCriteria.Split(",").ToList();
Now I have use removeList in the above code instead of hardcoded "Blank". What can I try to resolve this?
No sure if I am missing something here but isn't this a fairly simple ForEeach loop?
string removeCriteria ="Blank,Blank1,Blank2"
List<String> removeList = removeCriteria.Split(",").ToList();
ForEach (String toRemove in removeList)
{
lst.RemoveAll(x => x.filterOrder.Contains(toRemove));
lst.ForEach(fe => {fe.Details.RemoveAll(r=>r.filterDetails.Contains(toRemove));});
}
For a little better performance, especialy if each parent has alot of child records, I would suggest to change MattBaran answer a little
string removeCriteria ="Blank,Blank1,Blank2"
List<String> removeList = removeCriteria.Split(",").ToList();
//at first remove all parents with children
foreach (var toRemove in removeList)
{
lst.RemoveAll(x => x.filterOrder.Contains(toRemove));
}
//after this check the children of remaining records
foreach (var toRemove in removeList)
{
lst.ForEach(fe => {fe.Details.RemoveAll (r=>r.filterDetails.Contains (toRemove));});
}
I'm using Azure Search. I have a model with a property with this attributes
[IsRetrievable(true), IsSearchable, IsSortable, Analyzer("standardasciifolding.lucene")]
public string Title { get; set; }
I want the search to be accent insensitive. Although it is working when searching/filtering, it is not working when sorting the results. So, If I have words that start with an accent and I sort alphabetically, those results appear at the end of the list.
I verified your use case by creating an index with Id and a Title field that uses the standardasciifolding.lucene analyzer. I then submitted the 4 sample records via the REST API:
{
"value": [
{
"#search.action": "mergeOrUpload",
"Id": "1",
"Title" : "øks"
},
{
"#search.action": "mergeOrUpload",
"Id": "2",
"Title": "aks"
},
{
"#search.action": "mergeOrUpload",
"Id": "3",
"Title": "áks"
},
{
"#search.action": "mergeOrUpload",
"Id": "4",
"Title": "oks"
}
]}
I then ran a query with $orderby specified. I used Postman with variables wrapped in double curly braces. Replace with relevant values for your environment.
https://{{SEARCH_SVC}}.{{DNS_SUFFIX}}/indexes/{{INDEX_NAME}}/docs?search=*&$count=true&$select=Id,Title&searchMode=all&queryType=full&api-version={{API-VERSION}}&$orderby=Title asc
The results were returned as
{
"#odata.context": "https://<my-search-service>.search.windows.net/indexes('dg-test-65224345')/$metadata#docs(*)",
"#odata.count": 4,
"value": [
{
"#search.score": 1.0,
"Id": "2",
"Title": "aks"
},
{
"#search.score": 1.0,
"Id": "4",
"Title": "oks"
},
{
"#search.score": 1.0,
"Id": "3",
"Title": "áks"
},
{
"#search.score": 1.0,
"Id": "1",
"Title": "øks"
}
]
}
So, the sort order is indeed a, o, á, ø which confirms what you find. The order is inversed if I change to $orderby=Title desc. Thus, the sorting appears to be done by the original value and not the normalized value. We can check how the analyzer works, by posting a sample title to the analyzer with a POST request to
https://{{SEARCH_SVC}}.{{DNS_SUFFIX}}/indexes/{{INDEX_NAME}}/docs?search=*&$count=true&$select=Id,Title&searchMode=all&queryType=full&api-version={{API-VERSION}}&$orderby=Title asc
{ "text": "øks", "analyzer": "standardasciifolding.lucene" }
Which produces the following tokens
{
"#odata.context": "https://<my-search-service>.search.windows.net/$metadata#Microsoft.Azure.Search.V2020_06_30_Preview.AnalyzeResult",
"tokens": [
{
"token": "oks",
"startOffset": 0,
"endOffset": 3,
"position": 0
},
{
"token": "øks",
"startOffset": 0,
"endOffset": 3,
"position": 0
}
]
}
You could try to define a custom analyzer which produces a normalized version, but I am not sure it will work. For example, the sorting does not appear to support case-insensitive sorting which would be related to this use case where multiple characters should be sorted as if they were a normalized version. E.g. a and A cannot be sorted as the same character according to this user voice entry (feel free to vote for it).
WORKAROUND
The best workaround I can think of is to process the data yourself. Let Title contain the original title, and then create another field called TitleNormalized where you store the normalized version. In your application you would then query with $orderby on the TitleNormalized field.
There is a new feature that allows you to enable normalization while filtering. Please check out the Text normalization for case-insensitive filtering, faceting and sorting feature that's in Preview.
You can update your index to use this "normalizer" feature for the fields in which you'd like case-insensitive order-by operations.
You don't need a separate field TitleNormalized anymore. You can add "normalizer": "standard" to the Title field, and $orderBy=Title will sort in the order you'd like, ignoring casing and accents.
The "standard" normalizer is pre-defined. If you'd like other filters to be applied, please look at predefined and custom normalizers
"index": {
"name": "your-name-here",
"fields": [
{"name": "Title", "type": "Edm.String", "filterable": true, "sortable": true, "facetable": false, "searchable": true, "normalizer":"standard"}
]
}
I am querying a CosmosDB in such a way that I am getting a string in and ned to return some data out through a C# WEB API, the query that works for me is as below
SELECT *
FROM c IN jongel.OriginalData.base.sales.variants
WHERE c.globalTradeItemNumber.globalTradeItemNumberType[0].GTIN = '1111111111111'
The problem is that I have to know the ARRAY INDEX for the globalTradeItemNumberType ARRAY, [0] in this example, for it to work but it is not always 0, it could be any number from 0-9 basically and I cannot figure out how to rewrite the query so that it works regardless of the index where the matching data is found?
How can I rewrite this query so that I do not need to know the ARRAY INDEX beforehand?
--- EDIT ---
A sample document shortened to only include the needed parts
{
"id": "635af816-8db7-49c6-8284-ab85116b499b",
"brand": "XXX",
"IntegrationSource": "XXX",
"DocumentType": "Item",
"ItemInformationType": "",
"ItemLevel": "Article",
"ItemNo": "0562788040",
"UpdatedDate": "1/1/2020 4:00:01 AM",
"UpdatedDateUtc": "2020-01-01T04:00:01.82Z",
"UpdatedBy": "XXX",
"OriginalData": {
"corporateBrandId": "2",
"productId": "0562788",
"articleId": "0562788040",
"season": "201910",
"base": {
"sales": {
"SAPArticleNumber": "562788040190",
"simpleColour": {
"simpleColourId": "99",
"simpleColourDescription": "Green",
"translatedColourDescription": [
{
"languageCode": "sr",
"simpleColourDescription": "Zeleno"
},
{
"languageCode": "zh-Hans",
"simpleColourDescription": "绿色"
},
{
"languageCode": "vi-VN",
"simpleColourDescription": "Xanh la cay"
}
]
},
"variants": [
{
"variantId": "0562788040001",
"variantNumber": "562788040190001",
"variantDescription": "YYYYYYYYY, XXS",
"sizeScaleAndCode": "176-001",
"netWeight": 0.491,
"unitsOfMeasure": {
"unitsOfMeasureType": [
{
"alternativeUOM_ISO": "PCE",
"length": 320,
"width": 290,
"height": 31,
"unitOfDimension": "MM",
"volume": 2876.8,
"volumeUnit": "CCM",
"weightUnit": "KG"
}
]
},
"globalTradeItemNumber": {
"globalTradeItemNumberType": [
{
"GTIN": "1111111111111",
"GTINCategory": "Z3"
},
{
"GTIN": "2222222222222",
"GTINCategory": "Z3"
},
{
"GTIN": "3333333333333",
"GTINCategory": "IE"
}
]
}
}
]
}
}
}
}
I tried the following query based on suggested answer below but it did not work
SELECT *
FROM c
WHERE ARRAY_CONTAINS(c.OriginalData.base.sales.variants.globalTradeItemNumber.globalTradeItemNumberType, {GTIN:"1111111111111"}, true)
I guess the above fails because variants part of the tree is also an array?
NOTE: the variants array can hold several objects so its not always index[0]
You could try using the ARRAY_CONTAINS function.
SELECT *
FROM c IN jongel.OriginalData.base.sales.variants
WHERE ARRAY_CONTAINS(c.globalTradeItemNumber.globalTradeItemNumberType, {GTIN:"1111111111111"}, true)
This will allow the query to search all items in the array for a matching GTIN value.
https://learn.microsoft.com/en-us/azure/cosmos-db/sql-query-array-contains
My data in Cosmos DB is like this:
{
id:1,
key:"USA",
states:["New York","New Jersey", "Ohio", "Florida" ]
}
I want to check if a state exists in the document. Ex: "California". If it exists, then I want to return true otherwise false. Can someone help me with the query that fetches me a boolean value.
You should be able to use ARRAY_CONTAINS(). Typically, you'd use it in your WHERE clause, but you can also just return the boolean directly. In your example, you'd do something like:
SELECT ARRAY_CONTAINS(c.states, "California")
FROM c
This results in something like:
[
{
"id": "1",
"key": "USA",
"$1": true
},
{
"id": "2",
"key": "USA",
"$1": false
}
]
You can also return other properties along with the boolean:
SELECT c.id, c.key, ARRAY_CONTAINS(c.states, "California")
FROM c
And if you want to make sure you have the array defined for properties you check:
SELECT c.id, c.key, ARRAY_CONTAINS(c.states, "California")
FROM c
WHERE IS_DEFINED(c.states)
Lastly: You can alias the boolean, so that it returns an actual property name instead of something like $1:
SELECT c.id, c.key,
ARRAY_CONTAINS(c.states, "California") AS ContainsCalifornia
FROM c
WHERE IS_DEFINED(c.states)
And you'd get something back like:
{
"id": "1",
"key": "USA",
"ContainsCalifornia": true
},
{
"id": "2",
"key": "USA",
"ContainsCalifornia": false
}
I am using following code for searching the articleid and control fields. it will hold the 2 fields values. But I can't access these two fields values.
HERE search<> is dynamic.
var searchrange = _client.Search<dynamic>(s => s
.Indices("kb_v2").Types("kb")
.From(0).Size(10)
.Fields("articleid","control")
.Query(q => q
.Range(r =>r
.OnField("articleid")
.Greater("2")
.Lower("5"))));
can you explain How to get the this two fields values..
Since Elasticsearch 1.0 fields are always returned as a Dictionary<string, object[]> on hits to access these in NEST you can use:
foreach (var doc in queryResults.FieldSelections)
{
var articleIds = doc.FieldValues<int[]>("articleid");
}
See this PR for more details on the syntax.
The search response (ISearchResponse type) has a FieldSelections property which holds the results and details. With the older version of Nest, one had to loop over the Hits property to find the value of each field.
"hits": [
{
"_index": "kb_v2",
"_type": "kb",
"_id": "3565178",
"_score": 1,
"fields": {
"articleid": [
"3"
]
}
},
{
"_index": "kb_v2",
"_type": "kb",
"_id": "3932480",
"_score": 1,
"fields": {
"articleid": [
"4"
]
}
}]
More on how to use the FieldSelections in ElastichSearch.net client is mentioned in this Unit test here