I recently found out that i cannot call any methods from within a linq query. I am trying to write a query that, on the where clause compares two byte arrays. The value on the database is a GUID of type Raw(32) and it is returned as a byte array. This is the record ID for this table. I need to compare it to another byte array. the second byte array could be converted to a string but since i cannot call methods from within linq i was unable to compare.
I tied a custom "Compare" method, i also wrote an extension method. All received an error indicating "LINQ to Entities does not recognize the method"
Here is the code for what i am trying to do. The where clause causes this error:
LINQ to Entities does not recognize the method 'Boolean SequenceEqual[Byte] (System.Collections.Generic.IEnumerable1[System.Byte], System.Collections.Generic.IEnumerable1[System.Byte])' method, and this method cannot be translated into a store expression."
EPSGEntities dbContex = new EPSGEntities();
byte[] byteArray = ParseHex(ViewState["itemID"].ToString());
var q = (from d in dbContex.EPSG_VSOREJECTS
where d.SDSRECID.SequenceEqual(byteArray)
select d).First();
What version of EntityFramework are you using? On EF6 I am able to simply do the following against a SQL 2012 table with a varbinary column:
var q = dbContext.EPSG_VSOREJECTS.FirstOrDefault(e => e.SDSRECID == byteArray);
Is the SDSRECID property on EPSGEntities of type byte[]?
The alternative here would be to go to straight Sql to get your object. Something like:
dbContext.Database.SqlQuery<EPSG_VSOREJECT>("SELECT TOP 1 *" +
"FROM dbo.EPSGEntities" +
"WHERE SDSRECID = #byteString",
new SqlParameter
{
ParameterName = "byteString",
Value = ViewState["itemID"].ToString(),
}).FirstOrDefault();
Linq to Entities in EF is awesome for most queries, but I sometimes drop into sql when I need to do something unsupported, complex, or just fast. Hope this helps!
I'm not entirely sure this works, but I've found calling .AsEnumerable() on the IQueryable object set lets me apply pretty much any code I wish:
var q = dbContex.EPSG_VSOREJECTS.
.AsEnumerable()
.Where(d => d.SDSRECID.SequenceEqual(byteArray));
Doing so seems to prevent EF from trying to translate the Where() clause into SQL syntax, but I have no real idea what the performance hit would/will be.
This is also using method syntax, since I'm not real familiar with query syntax. HTH.
EDIT:
As some others have noted, you have to be careful with how you add any of the iterative methods (AsEnumerable(), ToList(), etc.) since past that point you are no longer building SQL against your data store. Once you start iterating, EF will execute whatever query has been built up to that point, and from then on you are filtering the result set from the LINQ query.
In this case, I don't know that this can be avoided, unless someone can build the same query as a sproc (which EF can execute on your behalf).
Related
The geoStartLoc holds the string in this format "54.5,44.5". I am trying to split and store the results in lat and longitude. I am getting the following error
LINQ to Entities does not recognize the method 'System.String[] Split(Char[])' method, and this method cannot be translated into a store expression. (select statement)
var data = from UberTrip in db.UberTrips
group UberTrip by new { UberTrip.startLoc, UberTrip.geoStartLoc }
into startLocGroup
select new LocationGroup() {
startLocation = startLocGroup.Key.startLoc,
latitude = startLocGroup.Key.geoStartLoc.Split(',').Count().ToString(),
//longitude= startLocGroup.Key.
countTrips = startLocGroup.Count()
};
This is a schema problem in your database. It's bad to ever store delimited values in a column. You should really have two columns: one for geoLatitude, and one for geoLongitude.
But since you probably can't make that change on your own, what you'll need to do is make sure the data is pulled down to your program's code and then split it there. Right now, linq is trying to take the expression tree created by this code and convert it to an SQL query, and it can't do it for the Split() call because not all supported database targets have an analogous Split method available. You need to save that part for after the data is loaded to memory in your program.
To accomplish this, just retrieve the full geoStartLoc string (into an anonymous type if you have to), use .ToList() to force the query compile and retrieve all the data, and then use a .Select() to convert to your LocationGroup objects.
Whenever you get this error message you are performing a query AsQueryable instead of AsEnumerable.
The main difference is that AsQueryable will usually be performed in another process like a database, while AsEnumerable will be performed in-memory in your process.
An IEnumerable object knows how to create an Enumerator. An Enumerator, knows how to do two things: "Give me the first element of your sequence", and "Give me the next element of your sequence (or null if there is no next element).
If your query is AsEnumerable it knows can use all classes and data from your process to create the enumerator. Therefore it can call functions like String.Split.
The AsQueryable does not hold all data to create the enumerator, it holds an Expression and a Provider. The Provider knows where the query needs to be performed on (usually a database, but it can also be a json string or a web service), and it knows how to translate the Expression into a format that the executor understands. In your case, the Provider knows how to translate the Expression into a SQL statement suitable for your database.
Of course SQL does not know your own defined functions. Although there are a lot of .NET functions that can be translated into SQL, not every function can. String.Split is one of them.
See: Supported and Unsupported LINQ Methods (LINQ to Entities)
To solve your problems you could bring the queried data to local memory. This is done using the extension function Enumerable.AsEnumerable(). After that you can use your sequence as if it is an IEnumerable.
The disadvantage of AsEnumerable is that it brings the queried data to local memory. This would be a waste if you remove a lot of this data to create your end result. Therefore make sure that you do your AsEnumerable with not too much data.
Luckily you need the Split in your final Select, so all data you need as input of your final select is thrown away, unless you will do a Skip / Take / FirstOrDefault / etc. In that case it is better to limit your Select by doing your Skip / Take etc before AsEnumerable().
I'm more familiar with MethodSyntax.
Your query divided into smaller steps (if desired make one LINQ):
// still AsQueryable:
var startLocGroups = db.UberTrips
.GroupBy(uberTrip => new {uberTip.startLoc, uberTrip.geoStartLoc)
// make asEnumerable
var localStartLocGroups = startLocGroups.AsEnumerable();
// now you can do your Select
var result = localStartLocGroups
.Select(group => new LocationGroup()
{
startLocation = startLocGroup.Key.startLoc,
latitude = group.Key.geoStartLoc.Split(',')
.Count()
.ToString(),
//longitude= sgroup.Key.
countTrips = group.Count(),
});
Since that isn't supported by the underlying Linq to Entities you'll have to do it after you get the data but before you return the results.
This way you do the query to get the data without invoking the string split function and then modify the results into the proper format.
var data = from UberTrip in db.UberTrips
group UberTrip by new { UberTrip.startLoc, UberTrip.geoStartLoc } into startLocGroup
select new {
startLocation = startLocGroup.Key.startLoc,
geoStartLoc = startLocGroup.Key.geoStartLoc,
countTrips = startLocGroup.Count()
};
return data.Select(trip => new LocationGroup() {
startLocation = trip.startLocation,
latitude = trip.geoStartLoc.Split(',')[0],
longitude= trip.geoStartLoc.Split(',')[1],
countTrips = trip.countTrips
}).ToList();
I have a Linq query that reads from a SQL table and 1 of the fields it returns are from a custom function (in C#).
Something like:
var q = from my in MyTable
select new
{
ID = my.ID,
Amount = GetAmount(ID)
};
If I do a q.Dump() in LinqPad, it shows the results, which tells me that it runs the custom function without trying to send it to SQL.
Now I want to union this to another query, with:
var q1 = (from p in AnotherQuery.Union(q)...
and the I get the error that Method has no supported translation to SQL.
So, my logic tells me that I need to dump q in memory and then try to union to that. I've tried doing that with ToList() and creating a secondary query that populates itself from the List, but that leads to a long list of different errors. Am I on the right track, by trying to get q in memory and union on that, or are there better ways of doing this?
You can't use any custom functions in a LINQ query that gets translated - only the functions supported by the given LINQ provider. If you want your query to happen on the server, you need to stick with the supported functions (even if it sometimes means having to inline code that would otherwise be reused).
The difference between your two queries boils down to when (and where) the projection happens. In your first case, the data from MyTable is returned from the DB - in your sample, just the ID. Then, the projection happens on top of this - the GetAmount method is called in your application for each of ID.
On the other hand, there's no such way for this to happen in your second query, since you're not using GetAmount in the final projection.
You either need to replace the custom function with inlined query the provider understands, or refactor all your queries to use the supported functions in addition with whatever you need to do in-memory. There's no point in giving you any sample code, since it depends entirely on your actual query, and what you're really trying to query for.
I'm fairly new to nHibernate having come from an EF background and I'm struggling with the following query :
_patientSearchResultModel = (from patient in _patientRepository.Query(patientSearch.BuildPatientSpecification())
join admission in _admissionRepository.Query(patientSearch.BuildAdmissionSpecification())
on patient.Id equals admission.Patient.Id
orderby admission.AdmissionDate
select new PatientSearchResultModel(patient.Id,
admission.Id,
false,
_phaseTypeMapper.GetPhaseTypeModel(admission.PhaseType),
patient.Last, patient.First,
admission.InPatientLocation,
admission.AdmissionDate,
admission.DischargeDate,
admission.RRI,
null,
admission.CompletionStatus,
admission.FollowupStatus)).ToList();
The intent of this query is to allow users to filter the two queries on parameters built up using the two Build???Specification functions and return the resultset. There could be many admission records and I would only like one PatientSearchResultModel per patient object, with the admission object being the newest one by Admission Date.
These objects are coming from nHibernate and it keeps return a Not Supported exception. There is also an association between Patient and Admissions thus : Patient.Admissions but i couldn't figure out how to then add the query filters return from the function Build???Specifications.
I'd be really grateful if someone could point me in the right direction; am I up against the Linq provider implementation here in nHibernate and need to move to Criteria or is it my Linq query ?
If anyone has any links or suggestions for good books or other learning materials in this area that would also be really helpful too.
I see several potential problems:
If you're using NHibernate 2.x + Linq2NHibernate explicit joins like that are not supported; in other versions they're just considered a smell.
I dont think NHibernate supports calling parameterized constructors in select clauses
I'm very sure NHibernate does not support calling instance methods in the select lambda
I'd suggest using the lambda syntax and SelectMany to alleviate potential join issues. Points #2 & #3 can be solved by projecting into an anonymous type, calling AsEnumerable then projecting into your model type.
Overall I'd suggest restructuring your code like:
var patientSpec = patientSearch.BuildPatientSpecification();
var admissionSpec = patientSearch.BuildAdmissionSpecification();
_patientSearchResultModel = _patientRepository.Where(patientSpec)
.SelectMany(p=>p.Admissions).Where(admissionSpec)
.Select(a=> new {
PatientId = a.Patient.Id,
AdminssionId = a.Id,
a.PhaseType,
a.Patient.Last,
a.Patient.First,
a.InPatientLocation,
a.AdmissionDate,
a.DischargeDate,
a.RRI,
a.CompletionStatus,
a.FollowupStatus
}).AsEnumerable()
.Select(x=> new PatientSearchResultModel(x.PatientId, x.AdmissionId ...))
.ToList();
Divide your query into parts and check which part runs and which doesn't.
My take on this is that select new ... is not supported in Linq to nHibernate.
I would recomend using something else, because it is simply too imature and feature-less to use seriously.
As with most popular LINQ-to-Database query providers, NHibernate will try to translate the whole query into a SQL statement to run against the database. This requires that all elements of your query are possible to express in the SQL flavour you're using.
In your query, the select new statement cannot be expressed in SQL, because you're making a call to the constructor of your PatientSearchResultModel class and are making a call to a GetPhaseTypeModel method.
You should restructure your query to express what you want to execute on the SQL database, then call AsEnumerable() to force the remainder of the query to be evaluated in-memory. After that call, you can call the constructor of your class and any .NET methods, and they will be executed as native code.
This query is too complex to describe it using Linq. It would give wrong result finally (if Patient has more than one admission records, result would have duplicate entries).
I see two steps for solution:
1) At development stage, use in-memory query. So, take Patients using ToList() first (query db at this moment). Some predicates (Patient filter like MRN, First, Last) could be used at this stage.
And then do search in-memory. Not performance, but working solution. Mark it for refactor to optimize later.
2) Finally, use NHibernate IQuery (ISQLQuery) and build sql query manually to make sure it would work as expected and work fast enough on SQL Server side. This is just read-only query and do not require Nhibernate query engine (Linq to Nhibernate) at all.
I have an IQueryable that has a list of pages.
I want to do: Pages.OrderByDescending(o => CalculateSort(o.page));
the method calculate sort is similar to that here is a plain english version:
public int calculatesort(page p)
{
int rating = (from r in db.rating select r). sum();
int comments = //query database for comments;
float timedecayfactor = math.exp(-page.totalhoursago);
return sortscore = (rating +comments)* timedecayfactor;
}
when I run a code similar to the one above an error is thrown that the mothode calculatesort cannot be converted to sql.
How can I do a conver the function above to be understood by sql so that I can use it to sort the pages?
Is this not a good approach for large data? Is there another method used to sort sets of results other than dynamically at the database?
I havent slept for days trying to fix this one :(
your code is nowhere near compiling so I'm guessing a lot here but I hope this gives an idea none the less.
As several have posted you need to give Linq-2-Sql an expression tree. Using query syntax that's what happens (by compiler magic)
from p in pages
let rating = (from r in db.rating
where r.PageId == p.PageId
select r.Value).Sum()
let comments = (from c in db.Comments
where c.PageId == p.PageId
select 1).Count()
let timedecayfactor = Math.Exp(-(p.totalhoursago))
orderby (rating + comments)*timedecayfactor descending
select p;
I haven't actually tried this against a database, there's simply too many unknown based on your code, so there might still be stuff that can't be translated.
The error occurs because LINQ cannot convert custom code/methods into SQL. It can convert only Expression<Func<>> objects into SQL.
In your case, you have a complex logic to do while sorting, so it might make sense to do it using a Stored Procedure, if you want to do it in the DB Layer.
Or load all the objects into main memory, and run the calculate sort method on the objects in memory
EDIT :
I don't have the code, so Describing in english is the best I can do :
Have table with structure capable of temporarily storing all the current users data.
Have a calculated field in the Pages table that holds the value calculated from all the non-user specific fields
Write a stored procedure that uses values from these two sources (temp table and calc field) to actually do the sort.
Delete the temp table as the last part in the stored proc
You can read about stored procs here and here
var comments = db.comments.Where(...);
Pages.OrderByDescending(p=>(db.rating.Sum(r=>r.rate) + comments.Count()) * Math.Exp(-p.totalhoursago))
Linq is expecting Calculatesort to return a "queryable" expression in order to generate its own SQL.
In can embed your 'calculatesort' method in this lambda expression. (I replaced your variables with constants in order to compile in my environment)
public static void ComplexSort(IQueryable<string> Pages)
{
Pages.OrderByDescending(p =>
{
int rating = 99;//(from r in db.rating select r). sum();
int comments = 33;//query database for comments;
double timedecayfactor = Math.Exp(88);
return (rating + comments) * timedecayfactor;
});
}
Also, you can even try to run that in parallel (since .net 4.0) replacing the first line with
Pages.AsParallel().OrderByDescending(p =>
Yes, counting previous answers: the LINQ to SQL doesn't know how to translate CalculateSort method. You should convert LINQ to SQL to ordinary LINQ to Object before using custom method.
Try to use this in the way you call the CalculateSort by adding AsEnumerable:
Pages.AsEnumerable().OrderByDescending(o => CalculateSort(o.page));
Then you're fine to use the OrderByDescending extension method.
UPDATE:
LINQ to SQL will always translate the query in the code into Expression tree. It's quite almost the same concept as AST of any programming language. These expression trees are further translated into SQL expression specific to SQL Server's SQL, because currently LINQ to SQL only supports SQL Server 2005 and 2008.
I'd like to order my list by a string converted into an int:
var orderedListOfRfidTags = uow.RfidTags.OrderBy(t => Convert.ToInt32(t.Number)).ToList();
but get: The method 'ToInt32' is not supported.
What about:
var orderedListOfRfidTags = uow.RfidTags.OrderBy(t => t.Number).ToList();
remove any CLR method so ORM can transform it to a known SQL query
EDIT:
I just read want to convert it first so:
var orderedListOfRfidTags = uow.RfidTags.ToList().OrderBy(t => Convert.ToInt32(t.Number));
either to get all from DB then order it on the client (linq to object) as I mentioned before or find a method on your ORM to cast to int the order it. Before you order Select a new list with a Number converted then order by it.
Edit2:
What about the direct cast is it working with this ORM?
var orderedListOfRfidTags = uow.RfidTags.OrderBy(t => (int)t.Number).ToList();
I am one of the developers of LightSpeed.
The LINQ provider in LightSpeed 3.11 RTM doesn't support Convert.ToInt32. However we have now added support via a nightly release which is available for download now.
If you don't want to use the nightly release, you can achieve the result you want by dropping down to the Query Objects API and invoking the SQL CAST function directly. This will look something like:
Query query = new Query
{
Order = Order.By(Entity.Attribute("Number")
.Function("CAST", new LiteralExpression("INTEGER") { EmitInline = true }))
};
uow.Find<RfidTag>(query);
The reason for the rather verbose LiteralExpression for the cast type is that by default LightSpeed sends values to the database through parameters (to avoid SQL injection attacks). But for the CAST function the SQL engine needs to see CAST(Number, INTEGER) rather than CAST(Number, #p0) where p0 has the value "INTEGER". So you have to use an EmitInline expression, which bypasses parameterisation, rather than the more natural string literal.
Once again, though, the nightly release does support Convert.ToInt32 in LINQ so you only need to drop down to this level if you want to avoid taking a nightly build.
var orderedListOfRfidTags = (uow.RfidTags.ToList()).OrderBy(t => int.Parse(t.Number));
I'm not sure what kind of type "RfidTags" is, nor am I familiar with the Lightspeed ORM, but I know that when I have had similar troubles with Linq to Sql telling me that a particular method I'm trying to invoke in a Where or OrderBy clause is not supported, then I just change things around so that I'm dealing with plain old Linq instead.
For example, could you try this?
var listOfRfidTags = uow.RfidTags.ToList();
var orderedListOfRfidTags = listOfRfidTags.OrderBy(t => Convert.ToInt32(t.Number));
(yes it is possible to combine this into one line, but shown here on two lines for clarity.)
Good luck!
Try to use int.Parse instead of Convert. It's likely that Lightspeed supports one without supporting the other.
var orderedListOfRfidTags = uow.RfidTags
.OrderBy(t => int.Parse(t.Number))
.ToList();
So, here's my solution to this problem:
var query = (from q in query select q).ToList().Where(x => Convert.ToInt32(x.col_string) > 0);
I first casted the IQueryable to a list, and then converted the column of data type string to int32 for use in mathematical operations.
I hope this helps.