dynamic fields sorting in elasticsearch and.net - c#

I am trying to sort records based on dynamic field names sent to the search API.
the d19FilterCriteria object gives me the field name(SortOn) and the order(SortOrder) for sorting. I have used a sort descriptor for this purpose.
var sortDescriptor = new SortDescriptor<MPANStatus>();
// If Field Name is Dynamic
if (d19FilterCriteria.SortOrder == "asc")
{
sortDescriptor.Field(d19FilterCriteria.SortOn, Nest.SortOrder.Ascending);
}
else if (d19FilterCriteria.SortOrder == "desc")
{
sortDescriptor.Field(d19FilterCriteria.SortOn, Nest.SortOrder.Descending);
}
var result = await _elasticClient.SearchAsync<MPANStatus>(s => s
.Index("ampower-mpanstatusindex")
.Skip(skip)
.Size(pageSize)
.Sort(sort =>
{
sort = sortDescriptor;
return sort;
})
While debugging the sort descriptor shows me an object that has a valid value for Name and order
This query returns empty list for this code. Could I know what the issue here is?

You can have list of sort order based on your inputs:
List<ISort> sdBookSortOrder = new List<ISort>();
SortOrder oSortOrder = SortOrder.Ascending; //SortOrder.Descending;
sdBookSortOrder.Add(new FieldSort { Field = <sField1>, Order = oSortOrder });
sdBookSortOrder.Add(new FieldSort { Field = <sField2>, Order = oSortOrder });
And, you can use above sort order collection while sending search request - as shown below:
ISearchRequest searchRequest = new SearchRequest(SearchEngine.IndexName)
{
From = iFrom,
Size = iSize,
Query = query,
Sort = oSortOrder,
};
I hope, it will solve your problem.
Regards,
Nikunj

My guess is since you are using the dynamic mapping, the string will be considered as text field and sub field keyword
use field.keyword instead of just field

The better approach to work with dynamic sorting fields using NEST, it's using SortDescriptor
SortDescriptor<dynamic> sort = GetSorting(request.Sorting);
ISearchResponse result = elasticClient.Search<YourType>(s => s
.Index("your-index")
.Sort(s => sort));
Use this method to add a list of sorting
// method to generate the Sorting
public SortDescriptor<dynamic> GetSorting(List<Sort>? sorting)
{
SortDescriptor<dynamic> sortDescriptor = new();
if (sorting != null)
{
foreach (Sort sort in sorting)
sortDescriptor.Field(sort.Field, getOrder(sort.Order));
}
return sortDescriptor;
}
class "Sort" just to store Field and Order values
public string Field { get; set; }
public string Order { get; set; }

Related

Merge data from two arrays or something else

How to combine Id from the list I get from file /test.json and id from list ourOrders[i].id?
Or if there is another way?
private RegionModel FilterByOurOrders(RegionModel region, List<OurOrderModel> ourOrders, MarketSettings market, bool byOurOrders)
{
var result = new RegionModel
{
updatedTs = region.updatedTs,
orders = new List<OrderModel>(region.orders.Count)
};
var json = File.ReadAllText("/test.json");
var otherBotOrders = JsonSerializer.Deserialize<OrdersTimesModel>(json);
OtherBotOrders = new Dictionary<string, OrderTimesInfoModel>();
foreach (var otherBotOrder in otherBotOrders.OrdersTimesInfo)
{
//OtherBotOrders.Add(otherBotOrder.Id, otherBotOrder);
BotController.WriteLine($"{otherBotOrder.Id}"); //Output ID orders to the console works
}
foreach (var order in region.orders)
{
if (ConvertToDecimal(order.price) < 1 || !byOurOrders)
{
int i = 0;
var isOurOrder = false;
while (i < ourOrders.Count && !isOurOrder)
{
if (ourOrders[i].id.Equals(order.id, StringComparison.InvariantCultureIgnoreCase))
{
isOurOrder = true;
}
++i;
}
if (!isOurOrder)
{
result.orders.Add(order);
}
}
}
return result;
}
OrdersTimesModel Looks like that:
public class OrdersTimesModel
{
public List<OrderTimesInfoModel> OrdersTimesInfo { get; set; }
}
test.json:
{"OrdersTimesInfo":[{"Id":"1"},{"Id":"2"}]}
Added:
I'll try to clarify the question:
There are three lists with ID:
First (all orders): region.orders, as order.id
Second (our orders): ourOrders, as ourOrders[i].id in a while loop
Third (our orders 2): from the /test.json file, as an array {"Orders":[{"Id":"12345..."...},{"Id":"12345..." ...}...]}
There is a foreach in which there is a while, where the First (all orders) list and the Second (our orders) list are compared. If the id's match, then these are our orders: isOurOrder = true;
Accordingly, those orders that isOurOrder = false; will be added to the result: result.orders.Add(order)
I need:
So that if (ourOrders[i].id.Equals(order.id, StringComparison.InvariantCultureIgnoreCase)) would include more Id's from the Third (our orders 2) list.
Or any other way to do it?
You should be able to completely avoid writing loops if you use LINQ (there will be loops running in the background, but it's way easier to read)
You can access some documentation here: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/linq/introduction-to-linq-queries
and you have some pretty cool extension methods for arrays: https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable?view=net-6.0 (these are great to get your code easy to read)
Solution
unsing System.Linq;
private RegionModel FilterByOurOrders(RegionModel region, List<OurOrderModel> ourOrders, MarketSettings market, bool byOurOrders)
{
var result = new RegionModel
{
updatedTs = region.updatedTs,
orders = new List<OrderModel>(region.orders.Count)
};
var json = File.ReadAllText("/test.json");
var otherBotOrders = JsonSerializer.Deserialize<OrdersTimesModel>(json);
// This line should get you an array containing
// JUST the ids in the JSON file
var idsFromJsonFile = otherBotOrders.Select(x => x.Id);
// Here you'll get an array with the ids for your orders
var idsFromOurOrders = ourOrders.Select(x => x.id);
// Union will only take unique values,
// so you avoid repetition.
var mergedArrays = idsFromJsonFile.Union(idsFromOurOrders);
// Now we just need to query the region orders
// We'll get every element that has an id contained in the arrays we created earlier
var filteredRegionOrders = region.orders.Where(x => !mergedArrays.Contains(x.id));
result.orders.AddRange(filteredRegionOrders );
return result;
}
You can add conditions to any of those actions (like checking for order price or the boolean flag you get as a parameter), and of course you can do it without assigning so many variables, I did it that way just to make it easier to explain.

Sorting List Array based on an index of array

I want to sort a List Array on the basis of an array item.
I have a List Array of Strings as below:
List<String>[] MyProjects = new List<String>[20];
Through a loop, I have added five strings
(Id, Name, StartDate, EndDate, Status)
to each of the 20 projects from another detailed List source.
for(int i = 0; i<20; i++){
MyProjects[i].Add(DetailedProjectList.Id.ToString());
MyProjects[i].Add(DetailedProjectList.Name);
MyProjects[i].Add(DetailedProjectList.StartDate);
MyProjects[i].Add(DetailedProjectList.EndDate);
MyProjects[i].Add(DetailedProjectList.Status)}
The Status values are
"Slow", "Normal", "Fast", "Suspended" and "" for unknown status.
Based on Status, I want to sort MyProject List Array.
What I have done is that I have created another List as below
List<string> sortProjectsBy = new List<string>(){"Slow", "Normal", "Fast", "", "Suspended"};
I tried as below to sort, however unsuccessful.
MyProjects = MyProjects.OrderBy(x => sortProjectsBy.IndexOf(4));
Can anyone hint in the right direction. Thanks.
I suggest you to create class Project and then add all the fields inside it you need. It's much nicer and scalable in the future. Then create a List or an Array of projects and use the OrderBy() function to sort based on the field you want.
List<Project> projects = new List<>();
// Fill the list...
projects.OrderBy(project => project.Status);
The field Status has to be a primitive type or needs to implement the interface IComparable in order for the sorting to work. I suggest you add an enum for Status with int values.
First consider maybe to use Enum for status and put it in a different file lite (utils or something) - better to work like that.
enum Status {"Slow"=1, "Normal", "Fast", "", "Suspend"}
Now about the filtering you want to achieve do it like this (you need to tell which attribute of x you are referring to. In this case is status)
MyProjects = MyProjects.OrderBy(x => x.status == enum.Suspend);
Read about enums :
https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/enum
Read about lambda expressions :
https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/statements-expressions-operators/lambda-expressions
First of all, storing project details as List is not adivisable. You need to create a Custom Class to represent them.
For example,
public class DetailedProjectList
{
public string Name {get;set;}
public eStatus Status {get;set;}
// rest of properties
}
Then You can use
var result = MyProjects.OrderBy(x=> sortProjectsBy.IndexOf(x.Status));
For example
List<string> sortProjectsBy = new List<string>(){"Slow", "Normal", "Fast", "", "Suspended"};
var MyProjects= new List<DetailedProjectList>{
new DetailedProjectList{Name="abc1", Status="Fast"},
new DetailedProjectList{Name="abc2", Status="Normal"},
new DetailedProjectList{Name="abc3", Status="Slow"},
};
var result = MyProjects.OrderBy(x=> sortProjectsBy.IndexOf(x.Status));
Output
abc3 Slow
abc2 Normal
abc1 Fast
A better approach thought would be to use Enum to represent Status.
public enum eStatus
{
Slow,
Normal,
Fast,
Unknown,
Suspended
}
Then your code can be simplified as
var MyProjects= new List<DetailedProjectList>{
new DetailedProjectList{Name="abc1", Status=eStatus.Fast},
new DetailedProjectList{Name="abc2", Status=eStatus.Normal},
new DetailedProjectList{Name="abc3", Status=eStatus.Slow},
};
var result = MyProjects.OrderBy(x=> x.Status);
Ok so you have a collection of 20 items. Based on them you need to create a list of strings(20 DetailedProjectList items).
What you can do to solve your problem is to SORT YOUR COLLECTION before you create your list of strings. In this way your list of strings will be sorted.
But your code is not optimal at all. So you should concider optimization on many levels.
Lets say you have ProjectDetail class as follow:
private class ProjectDetail
{
public int Id {get;set;}
public string Name {get;set;}
DateTime StartDate {get;set;} = DateTime.Now;
DateTime EndDate {get;set;} = DateTime.Now;
public string Status {get;set;}
public string toString => $"{Id} - {Name} - {StartDate} - {EndDate} - {Status}";
}
Notice that I have added a toString attribute to make things easier, and I also have added default values.
Then your program could be like:
static void Main(string[] args)
{
var projectDetails = MockProjectItems();
Console.WriteLine("Before sortig:");
foreach (var item in projectDetails)
{
Console.WriteLine(item.toString);
}
var myProjects = projectDetails.OrderBy(p => p.Status).Select(p => p.toString);
Console.WriteLine("\n\nAfter sorting:");
foreach (var item in myProjects)
{
Console.WriteLine(item);
}
}
where the helper method is
private static List<ProjectDetail> MockProjectItems()
{
var items = new List<ProjectDetail>(20);
for(int i = 0; i < 20 ; i += 4){
items.Add(new ProjectDetail{Id = i, Name = "RandomName "+i, Status = "Slow"});
items.Add(new ProjectDetail{Id = i+1, Name = "RandomName "+(i+1), Status = "Normal"});
items.Add(new ProjectDetail{Id = i+2, Name = "RandomName "+(i+2), Status = "Fast"});
items.Add(new ProjectDetail{Id = i+3, Name = "RandomName "+(i+3), Status = "Suspended"});
}
return items;
}
Then your program should print the following:

Update collection from DbSet object via Linq

i know it is not complicated but i struggle with it.
I have IList<Material> collection
public class Material
{
public string Number { get; set; }
public decimal? Value { get; set; }
}
materials = new List<Material>();
materials.Add(new Material { Number = 111 });
materials.Add(new Material { Number = 222 });
And i have DbSet<Material> collection
with columns Number and ValueColumn
I need to update IList<Material> Value property based on DbSet<Material> collection but with following conditions
Only one query request into database
The returned data from database has to be limited by Number identifier (do not load whole database table into memory)
I tried following (based on my previous question)
Working solution 1, but download whole table into memory (monitored in sql server profiler).
var result = (
from db_m in db.Material
join m in model.Materials
on db_m.Number.ToString() equals m.Number
select new
{
db_m.Number,
db_m.Value
}
).ToList();
model.Materials.ToList().ForEach(m => m.Value= result.SingleOrDefault(db_m => db_m.Number.ToString() == m.Number).Value);
Working solution 2, but it execute query for each item in the collection.
model.Materials.ToList().ForEach(m => m.Value= db.Material.FirstOrDefault(db_m => db_m.Number.ToString() == m.Number).Value);
Incompletely solution, where i tried to use contains method
// I am trying to get new filtered collection from database, which i will iterate after.
var result = db.Material
.Where(x=>
// here is the reasonable error: cannot convert int into Material class, but i do not know how to solve this.
model.Materials.Contains(x.Number)
)
.Select(material => new Material { Number = material.Number.ToString(), Value = material.Value});
Any idea ? For me it is much easier to execute stored procedure with comma separated id values as a parameter and get the data directly, but i want to master linq too.
I'd do something like this without trying to get too cute :
var numbersToFilterby = model.Materials.Select(m => m.Number).ToArray();
...
var result = from db_m in db.Material where numbersToFilterBy.Contains(db_m.Number) select new { ... }

LINQ - Deserialize JSON column and filter

How to deserialize/serialize a property with JSON string array value and then filter (using where clause) in LINQ inside a lambda expression?
void Main()
{
var regionList = new List<Row>() {
new Row { RegionJsonList = "[\"QLD\",\"NSW\"]" },
new Row { RegionJsonList = "[\"TAZ\",\"SA\"]" },
new Row { RegionJsonList = "[\"QLD\",\"VIC\"]" }
};
var filterRegionList = new List<string>() {
"QLD", "NSW"
};
var queryable = regionList.AsQueryable();
// this is obviously wrong, i just want to find the Row that contains one on filterRegionList
var result = queryable.Where(r => JsonConvert.DeserializeObject<string[]>(r.RegionJsonList).Contains(filterRegionList));
result.Count().Dump(); // should be 2
}
class Row
{
public string RegionJsonList { get;set; }
}
Following would work:
var result =
filterRegionList.Aggregate(regionList,(current,filter) =>
current.Where( r => r.RegionJsonList.Contains(filter)).ToList())
Aggregating the filterRegionList and regionList and thus applying filters for the final result. I did not find a requirement to Deserialize the RegionJsonList, since this would work as is, but you may add that part in case you are keen.
Also we are applying And filter via aggregation, it checks for the rows which contains both the filters, and thus provide the result, you may modify filter to achieve more number of rows, like following will select two entries from original regionList
var filterRegionList = new List<string>() { "QLD" };
To filter for rows that contain at least one of the entries from filterRegionList, you can use Enumerable.Intersect and check for non-empty intersections:
var resultAny = queryable.Where(r => JsonConvert.DeserializeObject<string[]>(r.RegionJsonList).Intersect(filterRegionList).Any());
To filter for rows that contain all of the entries from filterRegionList, you can use Enumerable.Except to remove the row's entries from the filter list. If everything gets removed, it's a match:
var resultAll = queryable.Where(r => !filterRegionList.Except(JsonConvert.DeserializeObject<string[]>(r.RegionJsonList)).Any());
(It wasn't entirely clear from your question which you wanted.)

Cannot implicitly convert type '.List<AnonymousType#1>' to '.List<WebApplication2.Customer>'

In the following code that returns a list:
public List<Customer> GeAllCust()
{
var results = db.Customers
.Select(x => new { x.CustName, x.CustEmail, x.CustAddress, x.CustContactNo })
.ToList()
return results;
}
I get an error reporting that C# can't convert the list:
Error: Cannot implicitly convert type System.Collections.Generic.List<AnonymousType#1> to System.Collections.Generic.List<WebApplication2.Customer>
Why is that?
Here's a screenshot showing some additional information that Visual Studio provides in a tooltip for the error:
Is it right way to return some columns instead of whole table....?
public object GeAllCust()
{
var results = db.Customers.Select(x => new { x.CustName, x.CustEmail, x.CustAddress, x.CustContactNo }).ToList();
return results;
}
When you look the code:
x => new { ... }
This creates a new anonymous type. If you don't need to pull back only a particular set of columns, you can just do the following:
return db.Customers.ToList();
This assumes that Customers is an IEnumerable<Customer>, which should match up with what you are trying to return.
Edit
You have noted that you only want to return a certain subset of columns. If you want any sort of compiler help when coding this, you need to make a custom class to hold the values:
public class CustomerMinInfo
{
public string Name { get; set; }
public string Email { get; set; }
public string Address { get; set; }
public int? ContactNumber { get; set; }
}
Then change your function to the following:
public List<CustomerMinInfo> GetAllCust()
{
var results = db.Customers.Select(x => new CustomerMinInfo()
{
Name = x.CustName,
Email = x.Email,
Address = x.Address,
ContactNumber = x.CustContactNo
})
.ToList();
return results;
}
This will work, however, you will lose all relationship to the database context. This means if you update the returned values, it will not stick it back into the database.
Also, just to repeat my comment, returning more columns (with the exception of byte arrays) does not necessarily mean longer execution time. Returning a lot of rows means more execution time. Your function is returning every single customer in the database, which when your system grows, will start to hang your program, even with the reduced amount of columns.
You are selecting to an anonymous type, which is not a Customer.
If you want to do (sort of) this, you can write it like this:
return db.Customers.Select(x => new Customer { Name = x.CustName, Email = x.CustEmail, Address = x.CustAddress, ContactNo = x.ContactNo }).ToList();
This assumes the properties on your Customer object are what I called them.
** EDIT ** Per your comment,
If you want to return a subset of the table, you can do one of two things:
Return the translated form of Customer as I specified above, or:
Create a new class for your business layer that only has only those four fields, and change your method to return a List<ShrunkenCustomer> (assuming ShunkenCustomer is the name that you choose for your new class.)
GetAllCust() is supposed to return a List of Customer, Select New will create a list of Anonymous Types, you need to return a list of Customer from your query.
try:
var results = db.Customers.Select( new Customer{CustName = x.CustName}).ToList(); //include other fields
I guess Customer is a class you have defined yourself?
The my suggestion would be to do something like the following:
var results = db.Customers.Select(x => new Customer(x.Custname, x.CustEmail, x.CustAddress, x.CustContactNo)).ToList();
The reason is that you are trying to return a list of Customer but the results from your link is an anonymous class containing those four values.
This would of course require that you have a constructor that takes those four values.
Basically whatever u got in var type, loop on that and store it in list<> object then loop and achieve ur target.Here I m posting code for Master details.
List obj = new List();
var orderlist = (from a in db.Order_Master
join b in db.UserAccounts on a.User_Id equals b.Id into abc
from b in abc.DefaultIfEmpty()
select new
{
Order_Id = a.Order_Id,
User_Name = b.FirstName,
Order_Date = a.Order_Date,
Tot_Qty = a.Tot_Qty,
Tot_Price = a.Tot_Price,
Order_Status = a.Order_Status,
Payment_Mode = a.Payment_Mode,
Address_Id = a.Address_Id
});
List<MasterOrder> ob = new List<MasterOrder>();
foreach (var item in orderlist)
{
MasterOrder clr = new MasterOrder();
clr.Order_Id = item.Order_Id;
clr.User_Name = item.User_Name;
clr.Order_Date = item.Order_Date;
clr.Tot_Qty = item.Tot_Qty;
clr.Tot_Price = item.Tot_Price;
clr.Order_Status = item.Order_Status;
clr.Payment_Mode = item.Payment_Mode;
clr.Address_Id = item.Address_Id;
ob.Add(clr);
}
using(ecom_storeEntities en=new ecom_storeEntities())
{
var Masterlist = en.Order_Master.OrderByDescending(a => a.Order_Id).ToList();
foreach (var i in ob)
{
var Child = en.Order_Child.Where(a => a.Order_Id==i.Order_Id).ToList();
obj.Add(new OrderMasterChild
{
Master = i,
Childs = Child
});
}
}

Categories

Resources