Develop Reference

Filtering or removing array objects from the DTO class object based on an another list in .net

I have below code :
[
{
"OrderId": "Order1",
"filterOrder": [ "ABC", "XYZ" ],
"Details": [
{
"id": 1,
"value": 100,
"filterDetails": [ "Apples", "Oranges" ]
},
{
"id": 2,
"value": 200,
"filterDetails": [ "Banana", "Blank" ]
}
]
},
{
"OrderId": "Order2",
"filterOrder": [ "PQR", "Blank" ],
"Details": [
{
"id": 1,
"value": 100,
"filterDetails": [ "Apples", "Peaches" ]
},
{
"id": 2,
"value": 200,
"filterDetails": [ "Banana", "Mango" ]
}
]
}
]
C# code:
string i = GetJsonText();
var lst = JsonConvert.DeserializeObject<List<Root>>(i);
My requirement here is to remove all those objects from the response where the filters are mentioned as "Blank".
So I have used the below code to remove the blanks :
lst.RemoveAll(x => x.filterOrder.Contains("Blank"));
lst.ForEach(fe => {fe.Details.RemoveAll(r=> r.filterDetails.Contains("Blank"));});
This is working perfectly fine. But now the requirement changed to remove objects based on multiple strings and not single string. Means I'll have something like
string removeCriteria ="Blank,Blank1,Blank2"
I convert this into a list of strings like this :
List<String> removeList = removeCriteria.Split(",").ToList();
Now I have use removeList in the above code instead of hardcoded "Blank". What can I try to resolve this?

No sure if I am missing something here but isn't this a fairly simple ForEeach loop?
string removeCriteria ="Blank,Blank1,Blank2"
List<String> removeList = removeCriteria.Split(",").ToList();
ForEach (String toRemove in removeList)
{
lst.RemoveAll(x => x.filterOrder.Contains(toRemove));
lst.ForEach(fe => {fe.Details.RemoveAll(r=>r.filterDetails.Contains(toRemove));});
}

For a little better performance, especialy if each parent has alot of child records, I would suggest to change MattBaran answer a little
string removeCriteria ="Blank,Blank1,Blank2"
List<String> removeList = removeCriteria.Split(",").ToList();
//at first remove all parents with children
foreach (var toRemove in removeList)
{
lst.RemoveAll(x => x.filterOrder.Contains(toRemove));
}
//after this check the children of remaining records
foreach (var toRemove in removeList)
{
lst.ForEach(fe => {fe.Details.RemoveAll (r=>r.filterDetails.Contains (toRemove));});
}

Azure Search - Accent insensitive analyzer not working when sorting

I'm using Azure Search. I have a model with a property with this attributes
[IsRetrievable(true), IsSearchable, IsSortable, Analyzer("standardasciifolding.lucene")]
public string Title { get; set; }
I want the search to be accent insensitive. Although it is working when searching/filtering, it is not working when sorting the results. So, If I have words that start with an accent and I sort alphabetically, those results appear at the end of the list.

I verified your use case by creating an index with Id and a Title field that uses the standardasciifolding.lucene analyzer. I then submitted the 4 sample records via the REST API:
{
"value": [
{
"#search.action": "mergeOrUpload",
"Id": "1",
"Title" : "øks"
},
{
"#search.action": "mergeOrUpload",
"Id": "2",
"Title": "aks"
},
{
"#search.action": "mergeOrUpload",
"Id": "3",
"Title": "áks"
},
{
"#search.action": "mergeOrUpload",
"Id": "4",
"Title": "oks"
}
]}
I then ran a query with $orderby specified. I used Postman with variables wrapped in double curly braces. Replace with relevant values for your environment.
https://{{SEARCH_SVC}}.{{DNS_SUFFIX}}/indexes/{{INDEX_NAME}}/docs?search=*&$count=true&$select=Id,Title&searchMode=all&queryType=full&api-version={{API-VERSION}}&$orderby=Title asc
The results were returned as
{
"#odata.context": "https://<my-search-service>.search.windows.net/indexes('dg-test-65224345')/$metadata#docs(*)",
"#odata.count": 4,
"value": [
{
"#search.score": 1.0,
"Id": "2",
"Title": "aks"
},
{
"#search.score": 1.0,
"Id": "4",
"Title": "oks"
},
{
"#search.score": 1.0,
"Id": "3",
"Title": "áks"
},
{
"#search.score": 1.0,
"Id": "1",
"Title": "øks"
}
]
}
So, the sort order is indeed a, o, á, ø which confirms what you find. The order is inversed if I change to $orderby=Title desc. Thus, the sorting appears to be done by the original value and not the normalized value. We can check how the analyzer works, by posting a sample title to the analyzer with a POST request to
https://{{SEARCH_SVC}}.{{DNS_SUFFIX}}/indexes/{{INDEX_NAME}}/docs?search=*&$count=true&$select=Id,Title&searchMode=all&queryType=full&api-version={{API-VERSION}}&$orderby=Title asc
{ "text": "øks", "analyzer": "standardasciifolding.lucene" }
Which produces the following tokens
{
"#odata.context": "https://<my-search-service>.search.windows.net/$metadata#Microsoft.Azure.Search.V2020_06_30_Preview.AnalyzeResult",
"tokens": [
{
"token": "oks",
"startOffset": 0,
"endOffset": 3,
"position": 0
},
{
"token": "øks",
"startOffset": 0,
"endOffset": 3,
"position": 0
}
]
}
You could try to define a custom analyzer which produces a normalized version, but I am not sure it will work. For example, the sorting does not appear to support case-insensitive sorting which would be related to this use case where multiple characters should be sorted as if they were a normalized version. E.g. a and A cannot be sorted as the same character according to this user voice entry (feel free to vote for it).
WORKAROUND
The best workaround I can think of is to process the data yourself. Let Title contain the original title, and then create another field called TitleNormalized where you store the normalized version. In your application you would then query with $orderby on the TitleNormalized field.

There is a new feature that allows you to enable normalization while filtering. Please check out the Text normalization for case-insensitive filtering, faceting and sorting feature that's in Preview.
You can update your index to use this "normalizer" feature for the fields in which you'd like case-insensitive order-by operations.
You don't need a separate field TitleNormalized anymore. You can add "normalizer": "standard" to the Title field, and $orderBy=Title will sort in the order you'd like, ignoring casing and accents.
The "standard" normalizer is pre-defined. If you'd like other filters to be applied, please look at predefined and custom normalizers
"index": {
"name": "your-name-here",
"fields": [
{"name": "Title", "type": "Edm.String", "filterable": true, "sortable": true, "facetable": false, "searchable": true, "normalizer":"standard"}
]
}

Mongodb Lookup with sorting and grouping in C#

I have the following db config:
db={
"order": [
{
"id": 1,
"version": 1
},
{
"id": 1,
"version": 2
},
{
"id": 2,
"version": 1
},
{
"id": 2,
"version": 2
}
],
"orderDetail": [
{
"orderId": 1,
"orderDate": new Date("2020-01-18T16:00:00Z")
},
{
"orderId": 1,
"orderDate": new Date("2020-01-11T16:00:00Z")
},
{
"orderId": 1,
"orderDate": new Date("2020-01-12T16:00:00Z")
}
]
}
I'm using the fluent interface to perform a Lookup joining the orderDetails to the order collection (as shown in this post). Now that I have the join in place what's the best method to:
Sort the joined array such that the details are sorted by orderDate
Group the Orders (by OrderID) and sort by version to select the latest (largest Version #)
The workaround I implemented for #1 involves sorting the list after performing the lookup, but that's only because I wasn't able to apply a sort to the "as" of collection as part of the Lookup.
If anyone has any ideas, I'd appreciate it. Thanks!

If you are using MongoDB v3.6 or higher, you can use the $lookup with uncorrelated subqueries to use the inner pipelines to archive what you want.
Join Conditions and Uncorrelated Sub-queries
Since you didn't provide what collections or fields you are using, I will give a generic example:
db.customers.aggregate([
{
$lookup: {
from: "orders",
let: { customer_id: "$_id" },
pipeline: [
{ $match: { $expr: { $eq: [ "$customer_id", "$$customer_id" ] } } },
{ $sort: { orderDate: -1 } }
],
as: "orders"
}
}
]);
I hope that gives you a way to get where you want. =]

LINQ query return

I'm using highcharts to show some data...however i'm having issues due to my query not returning the data in the correct format for some charts.
Year Key Value
2011 Key1 5
2011 Key2 10
2012 Key1 12
2012 Key2 8
2013 Key1 3
Here's part of my query:
XAxisList = db.Datas.Where(w => w.CategoryID == measurementID).Select(x => x.Year).Distinct(),
Data = from d in db.Datas
where d.CategoryID == measurementID
group d by d.DataKey into g
select new
{
Key = g.Key,
Values = g.Select(s => s.Value)
}
Here is what my linq query returns:
"XAxisList": [
"2011",
"2012",
"2013"
],
"Data": [
{
"Key": "Key1",
"Values": [
"5",
"12"
"3"
]
},
{
"Key": "Key2",
"Values": [
"10",
"8"
]
}
]
How would I add a 'null' array value in the Key2 values so it would look like below?
"XAxisList": [
"2011",
"2012",
"2013"
],
"Data": [
{
"Key": "Key1",
"Values": [
"5",
"12"
"3"
]
},
{
"Key": "Key2",
"Values": [
"10",
"8",
**null**
]
}
]
Thanks in advance for your help!

This simply cannot be done, based on your dataset. If your dataset contained the following record
2013 Key2 null
then there wouldn't have been any issue.
You could understand, why this cannot be done, if you think about the following:
group d by d.DataKey into g
What are you doing there?
You are grouping your records based of the DataKey property. Since there are two different keys, key1 and key2, there would be two groups. The group of key1 would have three records and the group of key2 would have two records.
Furthermore, I don't see any way that you could overcome this. As I can understand by your data, you have some yearly values for some keys and you want to use them for creating a chart.
Now there is missing a value for the key2 and the year 2013. So you think that if you could add a null at the end of the corresponding sequence everything would be ok.
{
"Key": "Key2",
"Values": [
"10",
"8",
"null"
]
}
Have you thinked about what would be the situation of missing a value for 2012? It would be exactly the same ! So adding a null at the end of the Values array wouldn't make any sense.
If the Year values would be specific or in a specific range I could come up with an idea, but I am not sure that this is the case.

I'd do the modification for reporting or charts after the query, or it'll get extremely messy:
XAxisList = db.Datas.Where(w => w.CategoryID == measurementID).Select(x => x.Year).Distinct(),
Data = //your query
int largest = Data.Values.Max(v => v.Count);
foreach(var d in Data)
{
while(d.Values.Count < largest)
d.Values.Add(null);
}
Or similar. You'll probably need to make your Values a list with ToList to get the Add method.

how to select nested documents in mongodb?

Lets say I have a nested documents with this stucture:
{
"_id": "a125",
"Language": null,
"Name": "Some name",
"Related": [{
"_id": "b125",
"Status": 0,
}, {
"_id": "b126",
"Status": 1,
}]
}
is it possible using c# drivers to select "Related" model where id is b126 and at the same time to get parent document id (a125)?
As I imagine outcome should look like this:
{
"_id": "a125",
"Related": {
"_id": "b126",
"Status": 1,
}
}

You can use dot notation with the positional $ projection operator to find the matching document and only include the matching Related element.
In the shell:
db.test.findOne({'Related._id': 'b125'}, {'Related.$': 1})
To do this in C#:
var filter = Builders<BsonDocument>.Filter.Eq("Related._id", "b125");
var projection = Builders<BsonDocument>.Projection.Include("Related.$");
var result = await collection.Find(filter).Project(projection).FirstAsync();

You should use dot notation for your purpose. Your query will look like this:
{"Related._id": "b126"}
This will bring you all the documents, with all the fields including your parent _id, where there is a document element in the Related array, which has a field _id with value "b126"