Develop Reference

AzureCosmos DB - Implementing a query with .Contains() functionality over an array of search terms

I'm querying the Azure Cosmos DB and need to apply a filter based on an array of Search terms.
Something like below:
searchTerms = searchTerms.Where(s => s != null).Select(f => f.ToLower()).ToArray();
query = query.Where(uci => searchTerms.Any(f => uci.User.FirstName.ToLower().Contains(f))
|| searchTerms.Any(f => uci.User.LastName.ToLower().Contains(f))
|| searchTerms.Any(f => uci.User.Company.Name.ToLower().Contains(f)));
However the query fails with an error "Input is not of type IDocumentQuery"
So, "Any" is not supported by Microsoft.Azure.Cosmos.
The alternative was to use the sql query instead of using linq. SQL has 2 options "Array_contains" or "Contains" but that doesn't work because
Array_contains does a "=" check on the array elements rather than substring check
eg. SELECT ARRAY_CONTAINS(["apples", "strawberries", "bananas"], "apples") AS b1
Contains works on a single string
eg. SELECT CONTAINS("abc", "ab") AS c1
cosmos array_contains
This link speaks about the "like" keyword, however that also works on a single string.
The closest solution that I could come up with is below
searchTerms = searchTerms.Where(s => s != null).Select(f => f.ToLower()).ToArray();
foreach (string term in searchTerms) {
query = query.Where(uci => uci.User.FirstName.ToLower().Contains(term)
|| uci.User.LastName.ToLower().Contains(term)
|| uci.User.Company.Name.ToLower().Contains(term));
}
This will add a "where" clause for each search term which is more of a hack than a solution.
Has anyone run across such situation? Any optimal suggestions?

Need to format complex foreach and IF statements to better formatted LINQ expression

I have a Foreach Statement as shown below
foreach (var fieldMappingOption in collectionHelper.FieldMappingOptions
.Where(fmo => fmo.IsRequired && !fmo.IsCalculated
&& !fmo.FieldDefinition.Equals( MMPConstants.FieldDefinitions.FieldValue)
&& (implicitParents || anyParentMappings
|| fmo.ContainerType == collectionHelper.SelectedOption.ContainerType)))
{
if (!collectionHelper.FieldMappingHelpers
.Any(fmh => fmh.SelectedOption.Equals(fieldMappingOption)))
{
requiredMissing = true;
var message = String.Format(
"The MMP column {0} is required and therefore must be mapped to a {1} column.",
fieldMappingOption.Label, session.ImportSource.CollectionLabel);
session.ErrorMessages.Add(message);
}
}
Can I break the above complex foreach and IF statements into better formatted LINQ expression. Also, performance wise which will be better. Please suggest.

Re : Change the Foreach to a Linq statement
Well, you could convert the two for loops into a LINQ Select, and since inside the loop, you've only one branch with an additional predicate, you could combine the predicate into the outer loop, something like so:
var missingFieldMappingOptions = collectionHelper.FieldMappingOptions
.Where(fmo => fmo.IsRequired && !fmo.IsCalculated
&& !fmo.FieldDefinition.Equals( MMPConstants.FieldDefinitions.FieldValue)
&& (implicitParents || anyParentMappings
|| fmo.ContainerType == collectionHelper.SelectedOption.ContainerType))
&& !collectionHelper.FieldMappingHelpers
.Any(fmh => fmh.SelectedOption.Equals(fmo)))
.Select(fmo =>
$"The MMP column {fmo.Label} is required and therefore" +
$" must be mapped to a {session.ImportSource.CollectionLabel} column.");
var requiredMissing = missingFieldMappingOptions.Any();
session.ErrorMessages.AddRange(missingFieldMappingOptions)
However, even LINQ can't make the filter clauses in the .Where disappear, so the LINQ Select is hardly more readable than the for loop, and isn't really any more performant either (there may be some marginal benefit to setting the requiredMissing flag and adding to session.ErrorMessages in one bulk chunk.
Performance
From a performance perspective, the below is problematic as it will be O(N log N) when combined in the outer for loop (fortunately .Any() will return early if a match is found, otherwise it would be as bad as N^2):
if (!collectionHelper
.FieldMappingHelpers.Any(fmh => fmh.SelectedOption.Equals(fieldMappingOption)))
Does FieldMappingOption have a unique key? If so, then suggest add a Dictionary<Key, FieldMappingOption> to collectionHelper and then use .ContainsKey(key) which approaches O(1), e.g.
!collectionHelper
.SelectedFieldMappingOptions.ContainsKey(fieldMappingOption.SomeKey)
Even if there isn't a unique key, you could use a decent HashCode on FieldMappingOption and key by that to get a similar effect, although you'll need to consider what happens in the event of a hash collision.
Readability
The Where predicate in the outer for loop is arguably messy and could use some refactoring (for readability, if not for performance).
IMO most of the where clauses could be moved into FieldMappingOption as a meta property, e.g. wrap up
fmo.IsRequired
&& !fmo.IsCalculated
&& !fmo.FieldDefinition.Equals(MMPConstants.FieldDefinitions.FieldValue)
into a property, e.g. fmo.MustBeValidated etc.
You could squeeze minor performance by ensuring the predicate returns false as soon as possible, by rearranging the && clauses which are most likely to fail first, but wouldn't do so if it impacts the flow of readability of the code.

How can I issue a LINQ query to a SQL Server to check if a column value starts with a word?

I have this already
if (options.English != null) query = query
.Where(w => w.English.Contains(options.English));
What I would like to do is to extend this (maybe with another if clause) to make it so that if:
a user enters ^abc then my query would check if the word starts with "abc".
a user abc then it would check if the column contains "abc"
I am using a SQL Server back-end database. Could anyone give me a suggestion as to how I could implement this functionality.

Assuming that you're using Entity Framework, you can use the StartsWith() and EndsWith() methods, to achieve the same results as Contains() except only at the beginning or the end of a string. It will generate the code for you.
Then simply create conditional statements in your code, in order to determine which one of the methods you should use.
Some word of advice:
There might be a bug with EF Core, in which it turns StartsWith("string") into LIKE "string%" which might yield incorrect results with strings, containing wildcard characters such as "_".
So I'd advise you to use plain SQL with EF Core, and given that you're using SQL Server as a DBMS, query like that:
if (searchText.StartsWith("^"))
{
var result = query.FromSql($"SELECT something FROM table WHERE PATINDEX({searchText.Substring(1)}, something) = 1");
}
else
{
var result = query.FromSql($"SELECT * FROM table WHERE PATINDEX({searchText.Substring(1)}, something ) <> 0");
}
With PATINDEX() you will get correct results even if your pattern string contains wildcard characters - escaping potential bugs with relying on StartsWith() and EndsWith() to generate proper SQL code.
But that's only for EF Core, EF 6 works like a charm the way other people answered :)

You can put that choice in a conditional statement:
IQueryable<Whatever> query = ...;
if (searchText.StartsWith("^"))
{
query = query.Where(w => w.English.StartsWith(searchText.Substring(1)));
}
else
{
query = query.Where(w => w.English.Contains(searchText));
}
You can also do the same comparison inline, but that'll generate very ugly SQL, if it even works:
query = query.Where(w =>
searchText.StartsWith("^")
? w.English.StartsWith(searchText.Substring(1))
: w.English.Contains(searchText));
Do note that you generally don't want to search text using SQL, as that results in a pretty poor user experience. Take a look at full-text indexing.

if (options.English != null)
{
bool englishStartsWith = options.English.StartsWith("^");
if(englishStartsWith)
{
query = query.Where(w => w.English.StartsWith(options.English.Substring(1)));
}
else
{
query = query.Where(w => w.English.Contains(options.English));
}
}

Linq Where clause not returning what I expect when performing String.Contains(String) on a null string

I scratched my head for one hour on this yesterday with no results but sweat.
string SearchTag = "";
Extension.getDBService<MyClass>().FindAll(i => <true condition>);
This returned me all my MyClass DB records as I would expect.
string SearchTag = "";
Extension.getDBService<MyClass>().FindAll(i => <true condition> && i.TAG.ToLower().Trim().Contains(SearchTag.ToLower().Trim()));
This returned a 0 Count collection!! I do not understand this.
string SearchTag = "e";
Extension.getDBService<MyClass>().FindAll(i => <true condition> && i.TAG.ToLower().Trim().Contains(SearchTag.ToLower().Trim()));
This returns a collection containing all MyClass DB records again. This is normal as i.TAG always contains "e".
Why do I get a 0 members collection with the second expression?
"string".Contains("") should always be true right?
PS: Extension.getDBService() is a call to a DBContext by the way.
Thx for your assistance.

Interestingly, the way you wrote the LINQ query generates SQL CHARINDEX(...) > 0 criteria which returns false for empty string.
However, if you remove (move outside the query) the ToLower().Trim() part of the SearchTag variable
SearchTag = SearchTag.ToLower().Trim();
and use
i.TAG.ToLower().Trim().Contains(SearchTag)
inside the LINQ query, then the generated SQL criteria is LIKE operator and works as expected.
Just another example that LINQ to Entities is not like LINQ to Objects.

How to use two conditions in Linq lambda which has different where clause

I want to query my item in table Items, where the last update of each item must be less than 91 days old (from last update till now) and the quantity > 0.
This is my code in the Model:
public IList<Item> GetAllProducts()
{
var ien_item = from i in this.DataContext.Items
orderby i.LastUpdated descending
select i;
return ien_item.ToList().Where(
s =>
HelperClasses.HelperClass.IsLastUpdate(s.LastUpdated.Value) == true
&&
(s => s.Quantity) > 0
)
.ToList();
}
Anyone can solve it? Thanks.

We don't really know what's not working here. EDIT: Merlyn spotted it; your lambda syntax is messed up. There's more to do here though.
However, I'd have thought you'd want this:
public IList<Item> GetAllProducts()
{
var lastUpdateLimit = DateTime.UtcNow.Date.AddDays(-91);
var query = from item in DataContext.Items
where item.Quantity > 0 && item.LastUpdated >= lastUpdateLimit
orderby item.LastUpdated descending
select item;
return query.ToList();
}
Note that this is able to do all the querying at the database side instead of fetching all the items and filtering at the client side. It does assume that HelperClasses.HelperClass.IsLastUpdate is simple though, and basically equivalent to the filter I've got above.
(One additional point to note is that by evaluating UtcNow.Date once, the result will be consistent for all items - whereas if your code evaluates "today" on every call to IsLastUpdate, some values in the query may end up being filtered against a different date to other values, due to time progressing while the query is evaluating.)
EDIT: If you really need to use HelperClasses.HelperClass.IsLastUpdate then I'd suggest:
public IList<Item> GetAllProducts()
{
var query = from item in DataContext.Items
where item.Quantity > 0
orderby item.LastUpdated descending
select item;
return query.AsEnumerable()
.Where(s => HelperClass.IsLastUpdate(s.LastUpdated.Value))
.ToList();
}
... then at least the quantity filter is performed at the database side, and you're not creating a complete buffered list before you need to (note the single call to ToList).

The problem is your lambda syntax. You're trying to define a second lambda while in the middle of defining a first lambda. While this is possible to do, and useful in some contexts, it is sort of an advanced scenario, and probably won't be useful to you until you know you need it.
Right now, you don't need it. Unless you know you need it, you don't need it :)
So -
Instead of what you've written:
.Where(
s =>
HelperClasses.HelperClass.IsLastUpdate(s.LastUpdated.Value) == true
&& (s => s.Quantity) > 0
)
Write this instead:
.Where(
s =>
HelperClasses.HelperClass.IsLastUpdate(s.LastUpdated.Value) == true
&& s.Quantity > 0 // Notice I got rid of the extra lambda here
)
If you're morbidly curious:
The compile error you got is because you didn't define your second lambda correctly. It redefined a variable you'd already used (s), and you were trying to check if a lambda was greater than zero. That makes no sense. You can only compare the result of a lambda to some value. It's like calling a function. You don't compare functions to numbers - you compare the result you get when calling a function to a number.

Easy ...
public IList<Item> GetAllProducts()
{
var ien_item =
from i in DataContext.Items
where
HelperClasses.HelperClass.IsLastUpdate(i.LastUpdated.Value)
&& s.Quantity > 0
orderby i.LastUpdated descending
select i;
return ien_item.ToList();
}
Linq to SQL: Methods are not allowed (linq is not magic and can not convert C# methods to TSQL)
http://msdn.microsoft.com/en-us/library/bb425822.aspx
Linq to Object: while looking the same, it is much more powerful than linq to SQL... but can not query SQL databases :)
http://msdn.microsoft.com/en-us/library/bb397919.aspx
Linq to XML: same as linq to Object, with xml object
http://msdn.microsoft.com/en-us/library/bb387098.aspx
Linq to Dataset: not the same as Linq to SQL !
http://msdn.microsoft.com/en-us/library/bb386977.aspx
Other linq providers:
http://en.wikipedia.org/wiki/Language_Integrated_Query