Develop Reference

Dynamic lists in Linq Query - solr - Sitecore

So here is the summary:
Sitecore - SOLR index query
I have items that I am trying to retrieve using a set of sites that can vary.
I have a query:
query = query.Where(x => x.Language == this.ItemLanguage)
.Where(x => x.Templates.Contains(new Guid("94c1f3e5ac174a319cc5bbb942fe80c6")));
this will return all of the items correctly.
What I need is to add a dynamic list call for this query.
Something like:
.Where(x => x.Site.Any(y => siteNames.Contains(y)));
I have tried adding this line of code and I get an error:
System.ArgumentException: 'Argument must be array'
(some details)
Items returned (x) have a field "Site" of List<string>
siteNames is a List<String> of variable site names
The following code works in other places:
.Where(x => x.Site.Contains("somesite"));
Is there a way to manage a dynamic list or will I need to manually generate this expression based on the number of items in the siteNames list?
.Where(x => x.Site.Contains("dynamic") || x.Site.Contains("dynamic")
|| x.Site.Contains("dynamic") || x.Site.Contains("dynamic")
|| and so on);
Here is the full code example:
using (var context = Sitecore.ContentSearch.ContentSearchManager.GetIndex(AccelConstants.SEARCH_INDEX).CreateSearchContext())
{
IQueryable<SearchResultModel> query = context.GetQueryable<LMSSearchResultModel>();
SearchResults<SearchResultModel> results = null;
List<string> siteNames = new List<string>();
siteNames = this.SiteNames;
// Define the base search
query = query.Where(x => x.Language == this.ItemLanguage)
.Where(x => x.Templates.Contains(new Guid("94c1f3e5ac174a319cc5bbb942fe80c6")))
.Where(x => x.Site.Any(p => siteNames.Contains(p)));
// Execute the query
results = query.GetResults();
}
the site field is a solr field of Site_SM which outputs like this:
The reason the name "Site" is used is that Sites is also a field.
This is not code that I have control over so I am working with what I have.
"site_sm":["login",
"admin",
"service",
"covid19",
"scheduler",
"system",
"publisher"],
The search results model simply converts computed solr fields to c#
public class SearchResultModel : SearchResultItem
{
[IndexField("_templates")]
public List<Guid> Templates { get; set; }
[IndexField("site_sm")]
public List<string> Site { get; set; }
}

Use Predicate Builder to create a dinamic OR (I just use it to fix this issue).
Sitecore have a implementation for this is:
//using Sitecore.ContentSearch.Linq.Utilities
private IQueryable<LMSSearchResultModel> LimitSearchBySite(IQueryable<LMSSearchResultModel> query, IEnumerable<string> sites)
{
if (sites != null && sites.Any())
{
// https://www.albahari.com/nutshell/predicatebuilder.aspx
var predicate = PredicateBuilder.False<StoreLocatorResult>();
foreach (string s in site)
predicate = predicate.Or(p => p.Site.Contains(s));
return query.Where(predicate);
}
return query;
}
if you are using Solr and C# directly the best way is to use this site as reference:
https://www.albahari.com/nutshell/predicatebuilder.aspx
At the end is the same development I did but you need to add the PredicateBuilder class to your project
So this will be what you need to replace:
using (var context = Sitecore.ContentSearch.ContentSearchManager.GetIndex(AccelConstants.SEARCH_INDEX).CreateSearchContext())
{
IQueryable<SearchResultModel> query = context.GetQueryable<LMSSearchResultModel>();
SearchResults<SearchResultModel> results = null;
List<string> siteNames = new List<string>();
siteNames = this.SiteNames;
// Define the base search
query = query.Where(x => x.Language == this.ItemLanguage)
.Where(x => x.Templates.Contains(new Guid("94c1f3e5ac174a319cc5bbb942fe80c6")));
query = LimitSearchBySite(query, siteNames);
// Execute the query
results = query.GetResults();
}

OK, so here is a rather hacked way of making my search work.
using (var context = Sitecore.ContentSearch.ContentSearchManager.GetIndex(AccelConstants.SEARCH_INDEX).CreateSearchContext())
{
IQueryable<SearchResultModel> query = context.GetQueryable<LMSSearchResultModel>();
SearchResults<SearchResultModel> results = null;
List<string> siteNames = new List<string>();
siteNames = this.SiteNames;
// Define the base search
// remove site filtering from query
query = query.Where(x => x.Language == this.ItemLanguage)
.Where(x => x.Templates.Contains(new Guid("94c1f3e5ac174a319cc5bbb942fe80c6")));
// Execute the query
results = query.GetResults();
// Get the results
foreach (var hit in results.Hits)
{
//move site filter code to here
if (hit.Document != null && hit.Document.Site.Any(p => siteNames.Contains(p)))
{
// Add the model to the results.
}
else
{
}
}
}
By moving the site filter code to after the query, the Site List<string> is instantiated and has value at the time of the call. Which seems to have been my issue.
This then allows me to filter by the sites in the siteNames List<string> and get my final result set.
I don't know if this is the best way of making this section of code work but it does work.

I ran across this exact problem and found this StackOverflow answer searching for the error. It turns out you have to take the error literally and change the List<string> Site to a string[] Site in order to use .Any() in a predicate.
public class SearchResultModel : SearchResultItem
{
[IndexField("_templates")]
public List<Guid> Templates { get; set; }
[IndexField("site_sm")]
public string[] Site { get; set; }
}

MongoDb c# driver find item in array by field value

i found the way to check is the value contains in simple array :
var filter = Builders<Post>.Filter.AnyEq(x => x.Tags, "mongodb");
But how to find a complex item with many fields by a concrete field ?
I found the way to write it via the dot notation approach with BsonDocument builder, but how can i do it with typed lambda notations ?
upd
i think it some kind of
builderInst.AnyIn(p => p.ComplexCollection.Select(ml => ml.Id), mlIds)
but can't check right now, is anyone could help ?

There is ElemMatch
var filter = Builders<Post>.Filter.ElemMatch(x => x.Tags, x => x.Name == "test");
var res = await collection.Find(filter).ToListAsync()

Here's an example that returns a single complex item from an array (using MongoDB.Driver v2.5.0):
Simple Data Model
public class Zoo
{
public List<Animal> Animals { get; set; }
}
public class Animal
{
public string Name { get; set; }
}
Option 1 (Aggregation)
public Animal FindAnimalInZoo(string animalName)
{
var zooWithAnimalFilter = Builders<Zoo>.Filter
.ElemMatch(z => z.Animals, a => a.Name == animalName);
return _db.GetCollection<Zoo>("zoos").Aggregate()
.Match(zooWithAnimalFilter)
.Project<Animal>(
Builders<Zoo>.Projection.Expression<Animal>(z =>
z.Animals.FirstOrDefault(a => a.Name == animalName)))
.FirstOrDefault(); // or .ToList() to return multiple
}
Option 2 (Filter & Linq) This was about 5x slower for me
public Animal FindAnimalInZoo(string animalName)
{
// Same as above
var zooWithAnimalFilter = Builders<Zoo>.Filter
.ElemMatch(z => z.Animals, a => a.Name == animalName);
var zooWithAnimal = _db.GetCollection<Zoo>("zoos")
.Find(zooWithAnimalFilter)
.FirstOrDefault();
return zooWithAnimal.Animals.FirstOrDefault(a => a.Name == animalName);
}

You need the $elemMatch operator. You could use Builders<T>.Filter.ElemMatch or an Any expression:
Find(x => x.Tags.Any(t => t.Name == "test")).ToListAsync()
http://mongodb.github.io/mongo-csharp-driver/2.0/reference/driver/expressions/#elemmatch

As of the 2.4.2 release of the C# drivers, the IFindFluent interface can be used for querying on array element. ElemMatch cannot be used on an array of strings directly, whereas the find interface will work on either simple or complex types (e.g. 'Tags.Name') and is strongly typed.
FilterDefinitionBuilder<Post> tcBuilder = Builders<Post>.Filter;
FilterDefinition<Post> tcFilter = tcBuilder.Eq("Tags","mongodb") & tcBuilder.Eq("Tags","asp.net");
...
await myCollection.FindAsync(tcFilter);
Linq driver uses the aggregation framework, but for a query with no aggregation operators a find is faster.
Note that this has been broken in previous versions of the driver so the answer was not available at the time of original posting.

Linq Method Error IOrderedQueryable

I have a database with a specific id with recorded Time's, I need help on trying to figure out time gap's between an ID's time's e.g 13:05:15 and 13:05:45 though if the time gap is over 10/15 seconds it needs to be recorded so it can be used in say a text file/other data etc. I previously asked a similar question on here, here is what my code looks like so far:
This class is used to manipulate data through the linq var being queried/looped
public class Result
{
public bool LongerThan10Seconds { get; set; }
public int Id { get; set; }
public DateTime CompletionTime { get; set; }
}
This is the foor loop within a separate class which was my original idea
using (var data = new ProjectEntities())
{
Result lastResult = null;
List<Result> dataResults = new List<Result>();
foreach(var subResult in data.Status.Select(x => x.ID).Distinct().Select(Id => data.Status.Where(x => x.ID == Id).OrderBy(x => x.Time)))
{
if (lastResult != null)
{
if (subResult.CompletionTime.Subtract(lastResult.CompletionTime).Seconds > 10)
dataResults.Add(subResult);
}
lastResult = subResult;
}
Which I got the error:
Linq.IOrderedQueryAble does not contain a definition for 'CompletionTime' and no Extension method 'CompletionTime' accepting a first argument of type 'System.Linq.IOrderedQueryable.
I changed the for loop to use an object of the manipulation class
foreach(Result subResult in data.AssetStatusHistories.Select(x => x.ID).Distinct().SelectMany(Id => data.AssetStatusHistories.Where(x => x.ID == Id).OrderBy(x => x.TimeStamp)))
{
if (lastResult != null)
{
if (subResult.CompletionTime.Subtract(lastResult.CompletionTime).Seconds > 10)
{
vehicleResults.Add(subResult);
}
}
lastResult = subResult;
}
Though now I get the error: Cannot convert type 'Project.Status' to 'Project.Result'
Does anyone possibly have a solution to get around this I have looked through a few resources but haven't been able to find anything of helps also even on Microsoft's Linq Forum. Any help is much appreciated ! :)

Try adding .ToList() to the end of your LINQ statement, after OrderBy:
var results = data.Status.Select(x => x.ID).Distinct()
.Select(Id => data.Status.Where(x => x.ID == Id)
.OrderBy(x => x.Time)
.ToList();
foreach(var subResult in results))
{
...
}
Also, I think you could modify your LINQ to do a GroupBy of the ID column, but that's something you could do research on if you wish. (Tutorial)

Your linq query (in the second try) will return an IEnumerable of whatever is the element type of data.AssetStatusHistories. I assume this is some kind of IEnumerable<Project.Status>, so you're in fact trying to assign Project.Status objects to an iterator variable (subResult) of type Result, which is why you're getting the error Cannot convert type 'Project.Status' to 'Project.Result'.
So your problem is not really in the linq query, you just need a way to convert your Project.Status objects to Project.Result objects.

Flattening a loop with lookups into a single linq expression

In Type member support in LINQ-to-Entities? I was attempting to declare a class property to be queried in LINQ which ran into some issues. Here I will lay out the code inside the implementation in hopes of some help for converting it to a query.
I have a class Quiz which contains a collection of Questions, each of which is classified according to a QuestionLevel... I need to determine whether a quiz is "open" or "closed", which is accomplished via an outer join on the question levels and a count of the questions in each level, as compared with a table of maximum values. Here's the code, verbatim:
public partial class Quiz
{
public bool IsClosed
{
get
{
// if quiz has no questions, it's open
if (this.Questions.Count() == 0) return false;
// get a new handle to the EF container to do a query for max values
using (EFContainer db = new EFContainer())
{
// we get a dictionary of LevelName/number
Dictionary<string, int> max = db.Registry
.Where(x => x.Domain == "Quiz")
.ToDictionary(x => x.Key, x => Convert.ToInt32(x.Value));
// count the number of questions in each level, comparing to the maxima
// if any of them are less, the quiz is "open"
foreach (QuestionLevel ql in db.QuestionLevels)
{
if (this.Questions.Where(x => x.Level == ql).Count() < max["Q:Max:" + ql.Name])
return false;
}
}
// the quiz is closed
return true;
}
}
}
so here's my not-yet-working attempt at it:
public static IQueryable<Quiz> WhereIsOpen(this IQueryable<Quiz> query)
{
EFContainer db = new EFContainer();
return from ql in db.QuestionLevels
join q in query on ql equals q.Questions.Select(x => x.Level)
into qs
from q in qs.DefaultIfEmpty()
where q.Questions.Count() < db.Registry
.Where(x => x.Domain == "Quiz")
.Where(x => x.Key == "Q:Max" + ql.Name)
.Select(x => Convert.ToInt32(x.Value))
select q;
}
it fails on account on the join, complaining:
The type of one of the expressions in the join clause is incorrect.
The type inference failed in the call to 'GroupJoin'
I'm still trying to figure that out.
* update I *
ah. silly me.
join q in query on ql equals q.Questions.Select(x => x.Level).Single()
one more roadblock:
The specified LINQ expression contains references to queries that are
associated with different contexts.
this is because of the new container I create for the maximum lookups; so I thought to re-factor like this:
public static IQueryable<Quiz> WhereIsOpen(this IQueryable<Quiz> query)
{
EFContainer db = new EFContainer();
IEnumerable<QuestionLevel> QuestionLevels = db.QuestionLevels.ToList();
Dictionary<string, int> max = db.Registry
.Where(x => x.Domain == "Quiz")
.ToDictionary(x => x.Key, x => Convert.ToInt32(x.Value));
return from ql in QuestionLevels
join q in query on ql equals q.Questions.Select(x => x.Level).Single()
into qs
from q in qs.DefaultIfEmpty()
where q.Questions.Count() < max["Q:Max:" + ql.Name]
select q;
}
but I can't get the expression to compile... it needs me to cast QuestionLevels to an IQueryable (but casting doesn't work, producing runtime exceptions).
* update II *
I found a solution to the casting problem but now I'm back to the "different contexts" exception. grr...
return from ql in QuestionLevels.AsQueryable()
* update (Kirk's suggestion) *
so I now have this, which compiles but generates a run-time exception:
public static IQueryable<Quiz> WhereIsOpen(this IQueryable<Quiz> query)
{
EFContainer db = new EFContainer();
IEnumerable<string> QuestionLevels = db.QuestionLevels.Select(x => x.Name).ToList();
Dictionary<string, int> max = db.Registry
.Where(x => x.Domain == "Quiz")
.ToDictionary(x => x.Key, x => Convert.ToInt32(x.Value));
return from ql in QuestionLevels.AsQueryable()
join q in query on ql equals q.Questions.Select(x => x.Level.Name).Single()
into qs
from q in qs.DefaultIfEmpty()
where q.Questions.Count() < max["Q:Max:" + ql]
select q;
}
which I then call like this:
List<Product> p = db.Quizes.WhereIsOpen().Select(x => x.Component.Product).ToList();
with the resulting exception:
This method supports the LINQ to Entities infrastructure and is not
intended to be used directly from your code.

The issues you're coming across are common when you couple your database objects to your domain objects. It's for this exact reason that it's good to have a separate set of classes that represent your domain and a separate set of classes that represent your database and are used for database CRUD. Overlap in properties is to be expected, but this approach offers more control of your application and decouples your database from your business logic.
The idea that a quiz is closed belongs to your domain (the business logic). Your DAL (data access layer) should be responsible for joining all the necessary tables so that when you return a Quiz, all the information needed to determine whether or not it's closed is available. Your domain/service/business layer should then create the domain object with the IsClosed property properly populated so that in your UI layer (MVC) you can easily access it.
I see that you're access the database context directly, I'd warn against that and encourage you to look into using DI/IoC framework (Ninject is great), however, I'm going to access the database context directly also
Use this class in your views/controllers:
public class QuizDomainObject
{
public int Id {get; set;}
public bool IsClosed {get; set;}
// all other properties
}
Controller:
public class QuizController : Controller
{
public ActionResult View(int id)
{
// using a DI/IoC container is the
// preferred method instead of
// manually creating a service
var quizService = new QuizService();
QuizDomainObject quiz = quizService.GetQuiz(id);
return View(quiz);
}
}
Service/business layer:
public class QuizService
{
public QuizDomainObject GetQuiz(int id)
{
// using a DI/IoC container is the
// preferred method instead of
// access the datacontext directly
using (EFContainer db = new EFContainer())
{
Dictionary<string, int> max = db.Registry
.Where(x => x.Domain == "Quiz")
.ToDictionary(x => x.Key, x => Convert.ToInt32(x.Value));
var quiz = from q in db.Quizes
where q.Id equals id
select new QuizDomainObject()
{
Id = q.Id,
// all other propeties,
// I'm still unclear about the structure of your
// database and how it interlates, you'll need
// to figure out the query correctly here
IsClosed = from q in ....
};
return quiz;
}
}
}

Re: your comment
The join to QuestionLevels is making it think there are two contexts... but really there shouldn't be because the QuestionLevels should contain in-memory objects
I believe that if you join on simple types rather than objects you'll avoid this problem. The following might work for you:
return from ql in QuestionLevels
join q in query
on ql.LevelId equals q.Questions.Select(x => x.Level).Single().LevelId
into qs
(and if this doesn't work then construct some anonymous types and join on the Id)
The problem is that joining on the Level objects causes EF to do some under-the-covers magic - find the objects in the database and perform a join there. If you tell it to join on a simple type then it should send the values to the database for a SELECT, retrieve the objects and stitch them together back in your application layer.

Dynamic where clause in LINQ to Objects

I know there are a lot of examples of this on the web, but I can't seem to get this to work.
Let me try to set this up, I have a list of custom objects that I need to have limited on a range of values.
I have a sort variable that changes based on some action on the UI, and I need to process the object differently based on that.
Here is my object:
MyObject.ID - Just an identifier
MyObject.Cost - The cost of the object.
MyObject.Name - The name of the object.
Now I need to filter this based on a range in the cost, so I will have something similar to this, considering that I could be limiting by Either of my bottom two properties.
var product = from mo in myobject
where mo.Cost <= 10000
or
var product = from mo in myobject
where mo.Name equals strName
Now I have the dynamic linq in my project, but I'm not figuring out how to get it to actually work, as when I do some of the examples I am only getting:
Func<Tsourse>bool> predicate
as an option.
Update:
I am trying to find a solution that helps me Objectify my code, as right now it is a lot of copy and paste for my linq queries.
Update 2:
Is there an obvious performance difference between:
var product = from mo in myobject
... a few joins ...
where mo.Cost <= 10000
and
var product = (from mo in myobject
... a few joins ...)
.AsQueryable()
.Where("Cost > 1000")

Maybe not directly answering your question, but DynamicQuery is unnecessary here. You can write this query as:
public IEnumerable<MyObject> GetMyObjects(int? maxCost, string name)
{
var query = context.MyObjects;
if (maxCost != null)
{
query = query.Where(mo => mo.Cost <= (int)maxCost);
}
if (!string.IsNullOrEmpty(name))
{
query = query.Where(mo => mo.Name == name);
}
return query;
}
If the conditions are mutually exclusive then just change the second if into an else if.
I use this pattern all the time. What "Dynamic Query" really means is combining pure SQL with Linq; it doesn't really help you that much with generating conditions on the fly.

using System.Linq;
var products = mo.Where(x => x.Name == "xyz");
var products = mo.Where(x => x.Cost <= 1000);
var products = mo.Where(x => x.Name == "xyz" || x.Cost <= 1000);

Read this great post on DLINQ by ScottGu
Dynamic LINQ (Part 1: Using the LINQ Dynamic Query Library)
You would need something like
var product = myobject.Where("Cost <= 10000");
var product = myobject.Where("Name = #0", strName);
If you downloaded the samples you need to find the Dynamic.cs file in the sample. You need to copy this file into your project and then add
using System.Linq.Dynamic; to the class you are trying to use Dynamic Linq in.
EDIT: To answer your edit. Yes, there is of course a performance difference. If you know the variations of filters beforehand then I would suggest writing them out without using DLINQ.
You can create your own Extension Method like so.
public static class FilterExtensions
{
public static IEnumerable<T> AddFilter<T,T1>(this IEnumerable<T> list, Func<T,T1, bool> filter, T1 argument )
{
return list.Where(foo => filter(foo, argument) );
}
}
Then create your filter methods.
public bool FilterById(Foo obj, int id)
{
return obj.id == id;
}
public bool FilterByName(Foo obj, string name)
{
return obj.name == name;
}
Now you can use this on an IEnumerable<Foo> very easily.
List<Foo> foos = new List<Foo>();
foos.Add(new Foo() { id = 1, name = "test" });
foos.Add(new Foo() { id = 1, name = "test1" });
foos.Add(new Foo() { id = 2, name = "test2" });
//Example 1
//get all Foos's by Id == 1
var list1 = foos.AddFilter(FilterById, 1);
//Example 2
//get all Foo's by name == "test1"
var list2 = foos.AddFilter(FilterByName, "test1");
//Example 3
//get all Foo's by Id and Name
var list1 = foos.AddFilter(FilterById, 1).AddFilter(FilterByName, "test1");