Split a string in a linq select expression - c#

The geoStartLoc holds the string in this format "54.5,44.5". I am trying to split and store the results in lat and longitude. I am getting the following error
LINQ to Entities does not recognize the method 'System.String[] Split(Char[])' method, and this method cannot be translated into a store expression. (select statement)
var data = from UberTrip in db.UberTrips
group UberTrip by new { UberTrip.startLoc, UberTrip.geoStartLoc }
into startLocGroup
select new LocationGroup() {
startLocation = startLocGroup.Key.startLoc,
latitude = startLocGroup.Key.geoStartLoc.Split(',').Count().ToString(),
//longitude= startLocGroup.Key.
countTrips = startLocGroup.Count()
};

This is a schema problem in your database. It's bad to ever store delimited values in a column. You should really have two columns: one for geoLatitude, and one for geoLongitude.
But since you probably can't make that change on your own, what you'll need to do is make sure the data is pulled down to your program's code and then split it there. Right now, linq is trying to take the expression tree created by this code and convert it to an SQL query, and it can't do it for the Split() call because not all supported database targets have an analogous Split method available. You need to save that part for after the data is loaded to memory in your program.
To accomplish this, just retrieve the full geoStartLoc string (into an anonymous type if you have to), use .ToList() to force the query compile and retrieve all the data, and then use a .Select() to convert to your LocationGroup objects.

Whenever you get this error message you are performing a query AsQueryable instead of AsEnumerable.
The main difference is that AsQueryable will usually be performed in another process like a database, while AsEnumerable will be performed in-memory in your process.
An IEnumerable object knows how to create an Enumerator. An Enumerator, knows how to do two things: "Give me the first element of your sequence", and "Give me the next element of your sequence (or null if there is no next element).
If your query is AsEnumerable it knows can use all classes and data from your process to create the enumerator. Therefore it can call functions like String.Split.
The AsQueryable does not hold all data to create the enumerator, it holds an Expression and a Provider. The Provider knows where the query needs to be performed on (usually a database, but it can also be a json string or a web service), and it knows how to translate the Expression into a format that the executor understands. In your case, the Provider knows how to translate the Expression into a SQL statement suitable for your database.
Of course SQL does not know your own defined functions. Although there are a lot of .NET functions that can be translated into SQL, not every function can. String.Split is one of them.
See: Supported and Unsupported LINQ Methods (LINQ to Entities)
To solve your problems you could bring the queried data to local memory. This is done using the extension function Enumerable.AsEnumerable(). After that you can use your sequence as if it is an IEnumerable.
The disadvantage of AsEnumerable is that it brings the queried data to local memory. This would be a waste if you remove a lot of this data to create your end result. Therefore make sure that you do your AsEnumerable with not too much data.
Luckily you need the Split in your final Select, so all data you need as input of your final select is thrown away, unless you will do a Skip / Take / FirstOrDefault / etc. In that case it is better to limit your Select by doing your Skip / Take etc before AsEnumerable().
I'm more familiar with MethodSyntax.
Your query divided into smaller steps (if desired make one LINQ):
// still AsQueryable:
var startLocGroups = db.UberTrips
.GroupBy(uberTrip => new {uberTip.startLoc, uberTrip.geoStartLoc)
// make asEnumerable
var localStartLocGroups = startLocGroups.AsEnumerable();
// now you can do your Select
var result = localStartLocGroups
.Select(group => new LocationGroup()
{
startLocation = startLocGroup.Key.startLoc,
latitude = group.Key.geoStartLoc.Split(',')
.Count()
.ToString(),
//longitude= sgroup.Key.
countTrips = group.Count(),
});

Since that isn't supported by the underlying Linq to Entities you'll have to do it after you get the data but before you return the results.
This way you do the query to get the data without invoking the string split function and then modify the results into the proper format.
var data = from UberTrip in db.UberTrips
group UberTrip by new { UberTrip.startLoc, UberTrip.geoStartLoc } into startLocGroup
select new {
startLocation = startLocGroup.Key.startLoc,
geoStartLoc = startLocGroup.Key.geoStartLoc,
countTrips = startLocGroup.Count()
};
return data.Select(trip => new LocationGroup() {
startLocation = trip.startLocation,
latitude = trip.geoStartLoc.Split(',')[0],
longitude= trip.geoStartLoc.Split(',')[1],
countTrips = trip.countTrips
}).ToList();

Related

IQueryable Intersect is currently not supported

When I trying to do this
//data.Photos it's IEnumerable<Photo>. Comparer worked by Id.
List<Photo> inDb = db.Photos
.Intersect(data.Photos, new PhotoComparer())
.ToList();
I get an exception:
NotSupportedException: Could not parse expression
'value(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[ReportViewer.Models.DbContexts.Photo]).Intersect(__p_0, __p_1)'
This overload of the method #x27;System.Linq.Queryable.Intersect' is currently not supported.
// This works
List<Photo> inDb = db.Photos
.ToList()
.Intersect(data.Photos, new PhotoComparer())
.ToList();
// But will it take a long time - or not ?
What did I need to use Intersect with IQueryable and IEnumerable collection?
Due to the "custom comparer", although it's functionality might be trivial, the framework is currently not able to translate your statement to SQL (which I suspect you are using).
Next, it seems that you have a in memory collection, on which you want to perform this intersect.
So if you're wondering about speed, in order to get it working you'll need to send your data to the database server, and based on the Id's retrieve your data.
So basically, you are looking for a way to perform an inner join, which would be the SQL equivalent of the intersect.
Which you could do with the flowing linq query:
//disclaimer: from the top of my head
var list= from dbPhoto in db.Photos
join dataPhoto in data.Photos on dbPhoto.Id equals dataPhoto.Id
select dbPhoto;
This will not work though, since as far as I know EF isn't able to perform an join against an in-memory dataset.
So, alternatively you could:
fetch the data as IEnumerable (but yes, you'll be retrieving the whole set first)
use a Contains, be carefull though, if you're not using primitive types this can translate to a bunch of SQL OR statements
But basically it depends on the amount of data you're querying. You might want to reconsider your setup and try to be able to query the data based on some ownership, like user or other means.

Linq: Method has no supported translation to SQL - but how to dump to memory?

I have a Linq query that reads from a SQL table and 1 of the fields it returns are from a custom function (in C#).
Something like:
var q = from my in MyTable
select new
{
ID = my.ID,
Amount = GetAmount(ID)
};
If I do a q.Dump() in LinqPad, it shows the results, which tells me that it runs the custom function without trying to send it to SQL.
Now I want to union this to another query, with:
var q1 = (from p in AnotherQuery.Union(q)...
and the I get the error that Method has no supported translation to SQL.
So, my logic tells me that I need to dump q in memory and then try to union to that. I've tried doing that with ToList() and creating a secondary query that populates itself from the List, but that leads to a long list of different errors. Am I on the right track, by trying to get q in memory and union on that, or are there better ways of doing this?
You can't use any custom functions in a LINQ query that gets translated - only the functions supported by the given LINQ provider. If you want your query to happen on the server, you need to stick with the supported functions (even if it sometimes means having to inline code that would otherwise be reused).
The difference between your two queries boils down to when (and where) the projection happens. In your first case, the data from MyTable is returned from the DB - in your sample, just the ID. Then, the projection happens on top of this - the GetAmount method is called in your application for each of ID.
On the other hand, there's no such way for this to happen in your second query, since you're not using GetAmount in the final projection.
You either need to replace the custom function with inlined query the provider understands, or refactor all your queries to use the supported functions in addition with whatever you need to do in-memory. There's no point in giving you any sample code, since it depends entirely on your actual query, and what you're really trying to query for.

Linq Check if a string contains any query from a list

I have a list of strings that are search Queries.
I want to see if a string from the database contains anyone of those terms in the Query. I'd like to do this on one line of code, that doesn't make multiple calls to the database. This should work but I want it to be more optimized.
var queries = searchQuery.Trim().Split(' ', StringSplitOptions.RemoveEmptyEntries).Distinct();
var query = context.ReadContext.Divisions.AsQueryable();
queries.ForEach(q => {
query = query.Where(d => (d.Company.CompanyCode + "-" + d.Code).Contains(q));
});
Is there a function that can do this better or a more optimal way of writing that?
There are two issues with your proposed solution:
Most LINQ to SQL providers don't understand string.Contains("xyz") so the provider will either throw an exception or fetch all the data to your machine. The right thing to do is to use SqlMethods.Like as explained in Using contains() in LINQ to SQL
Also, the code you show will check whether the division contains all of the specified strings.
To implement the 'any' behavior you need to construct a custom expression, which will not be possible using plain C#. You would need to look at the System.Linq.Expressions namespace: https://msdn.microsoft.com/en-us/library/system.linq.expressions(v=vs.110).aspx
It is possible, but quite involved.

deferred execution of where in sql

i have an editor where people can create custom sql-queries. those queries are NHibernate.IQuery objects.
in our project we are using NHibernate.Linq.LinqExtensionMethods.Query to get an IQueryable object which can be used to apply filters that are executed deferred (everything on the DB).
now i want to create another filter that is based on the custom sql-queries.
what i want is something like this:
queryable.Where(x=> <sql-query contains x>)
the only thing i can do right now is execute the sql-query beforehand and then filter the queryable with the resulting list of elements.
IList<T> elements = query.List<T>();
queryable.Where(x => elements.Contains(x)).ToList();
the problem with that approach is, that the list of elements can be huge. if its possible, i would like to perform the whole statement directly on the database, so that i dont have to transfer all objects to my application, then send all objects back to the database as the filter-parameters...
edit: the first query (the one yielding the list of elements to check for in the second query) is constructed from a plain sql-string as follows:
ISQLQuery sqlQuery = CurrentSession.CreateSQLQuery(myQueryString);//create query
sqlQuery.AddEntity(typeof (T)); //set result type
IQuery query = sqlQuery.SetReadOnly(true);
If you have id's, then you should select them in your query, and compare it to the corresponding id column in the second query.
var elementIdsToUseInContain = query
.Select(x => x.YourIdProperty);
var result = queryable
.Where(x => elementIdsToUseInContain.Contains(x.YourIdPropertyToCompareTo))
.ToList();
Note: I haven't tested it, but it is fairly standard Linq. It depends on if NHibernate supports it, which i do not know.

Expressions in Linq Query

I recently found out that i cannot call any methods from within a linq query. I am trying to write a query that, on the where clause compares two byte arrays. The value on the database is a GUID of type Raw(32) and it is returned as a byte array. This is the record ID for this table. I need to compare it to another byte array. the second byte array could be converted to a string but since i cannot call methods from within linq i was unable to compare.
I tied a custom "Compare" method, i also wrote an extension method. All received an error indicating "LINQ to Entities does not recognize the method"
Here is the code for what i am trying to do. The where clause causes this error:
LINQ to Entities does not recognize the method 'Boolean SequenceEqual[Byte] (System.Collections.Generic.IEnumerable1[System.Byte], System.Collections.Generic.IEnumerable1[System.Byte])' method, and this method cannot be translated into a store expression."
EPSGEntities dbContex = new EPSGEntities();
byte[] byteArray = ParseHex(ViewState["itemID"].ToString());
var q = (from d in dbContex.EPSG_VSOREJECTS
where d.SDSRECID.SequenceEqual(byteArray)
select d).First();
What version of EntityFramework are you using? On EF6 I am able to simply do the following against a SQL 2012 table with a varbinary column:
var q = dbContext.EPSG_VSOREJECTS.FirstOrDefault(e => e.SDSRECID == byteArray);
Is the SDSRECID property on EPSGEntities of type byte[]?
The alternative here would be to go to straight Sql to get your object. Something like:
dbContext.Database.SqlQuery<EPSG_VSOREJECT>("SELECT TOP 1 *" +
"FROM dbo.EPSGEntities" +
"WHERE SDSRECID = #byteString",
new SqlParameter
{
ParameterName = "byteString",
Value = ViewState["itemID"].ToString(),
}).FirstOrDefault();
Linq to Entities in EF is awesome for most queries, but I sometimes drop into sql when I need to do something unsupported, complex, or just fast. Hope this helps!
I'm not entirely sure this works, but I've found calling .AsEnumerable() on the IQueryable object set lets me apply pretty much any code I wish:
var q = dbContex.EPSG_VSOREJECTS.
.AsEnumerable()
.Where(d => d.SDSRECID.SequenceEqual(byteArray));
Doing so seems to prevent EF from trying to translate the Where() clause into SQL syntax, but I have no real idea what the performance hit would/will be.
This is also using method syntax, since I'm not real familiar with query syntax. HTH.
EDIT:
As some others have noted, you have to be careful with how you add any of the iterative methods (AsEnumerable(), ToList(), etc.) since past that point you are no longer building SQL against your data store. Once you start iterating, EF will execute whatever query has been built up to that point, and from then on you are filtering the result set from the LINQ query.
In this case, I don't know that this can be avoided, unless someone can build the same query as a sproc (which EF can execute on your behalf).

Categories

Resources