I use C# on WP7 (Mango). I try to use a special query because I receive an error:
Method 'Int32 orderBirthday(System.DateTime)' has no supported
translation to SQL.
Yes, I know... Linq can't use my function but I don't know the right way...
I have a database table with the columns name and birthday. In my query I will calculate how many days are to the next birthday (from all items) and then I will order with "descending".
static int orderBirthday(DateTime Birthday)
{
DateTime today = DateTime.Today;
DateTime birthday = Birthday;
DateTime next = new DateTime(today.Year, birthday.Month, birthday.Day);
if (next < today)
next = next.AddYears(1);
int numDays = (next - today).Days;
// No Conversion
return numDays;
}
public void LoadCollectionsFromDatabase()
{
DateTime today = DateTime.Today;
var toDoItemsInDB = from ToDoItem todo in toDoDB.Items
let daysToBirthday = orderBirthday(todo.ItemDate)
orderby daysToBirthday ascending
select todo;
// Query the database and load all to-do items.
AllToDoItems = new ObservableCollection<ToDoItem>(toDoItemsInDB);
.
.
.
}
You either have to pull everything from the database and sort it locally (as Enigmativity) shows, or find a way to express the sort operation in a LINQ statement itself. And since you extracted the sorting behavior into its own function, you probably want to reuse this logic. In that case your best bet is to create a function that filters an IQueryable.
Here is an example of how to do this:
public static IOrderedQueryable<Item> OrderByBirthday(
this IQueryable<Item> items)
{
return
from item in items
let today = DateTime.Today
let birthday = item.ItemDate
let next = new DateTime(today.Year, birthday.Month, birthday.Day)
let next2 = next < today ? next.AddYears(1) : next
orderby (next - today).Days
select item;
}
You can use the method as follows:
var toDoItemsInDB = OrderByBirthday(toDoDB.Items);
Or you can use it as an extension method:
var toDoItemsInDB = toDoDB.Items.OrderByBirthday();
It's easy if you do this:
var toDoItemsInDB = from ToDoItem todo in toDoDB.Items.ToArray()
let daysToBirthday = orderBirthday(todo.ItemDate)
orderby daysToBirthday ascending
select todo.;
Notice the .ToArray() added to Items. You basically bring the results into memory and them your function can work.
Two ways:
One: Pull it from Linq2SQL to Linq2Objects using ToEnumerable(), and then use orderBirthday at the C# level.
Advantage is that it's simple to code and maintain, disadvantage is that it can be less efficient (depends on just what you are doing.
Two: Write an equivalent function in SQL, let's say it was called dbo.orderBirthday. Make your orderBirthday method a non-static method of your datacontext-derived class, and then mark your method as having a SQL function equivalent:
[Function(Name="dbo.orderBirthday",IsComposable=true)] //IsComposable is true for functions that can be used within queries, false for stored procedures that must be called on their own.
public int OrderBirthday([Parameter(Name="#birthday",DbType="datetime") DateTime birthday)
{
return Helper.OrderBirthday(birthday); // just to show that we can keep the static version around if we want and call into it. Alternatively we could just move the whole body here.
}
Here the C# code is used in a non-Linq2SQL context, and the SQL code is used in composing a SQL query in a Linq2SQL context.
Advantage: Can stay within SQL longer. Disadvantage: Two versions of the same method can fall out of sync and cause bugs.
It's also possible to have the C# code call the SQL code all the time:
[Function(Name="dbo.orderBirthday",IsComposable=true)]
public int OrderBirthday([Parameter(Name="#birthday",DbType="datetime") DateTime birthday)
{
return (int)ExecuteMethodCall(this, (MethodInfo)MethodInfo.GetCurrentMethod(), birthday).ReturnValue;
}
Advantage: Keeps one version (the SQL) as the only version, so it can't fall out of synch with the C# version. Disadvantage: Calls SQL even when working on objects that have nothing to do with SQL.
If you don't want to load all the items in memory and you want the database execute the calculation, you can write a stored procedure that can execute complex calculation and call the procedure using ADO or EF.
Related
I have two datetime pickers on my form. I want a function that will return all datetimes from a specific table (which are values of a specific column) between those two dates.
My method looks like this:
public DateTime[] GetAllArchiveDates(string username = null)
{
var result = new DateTime[0];
if (username != null)
{
result = this._context.archive.OrderBy(s => s.IssuingDate).Where(s => s.insertedBy == username).Select(s => s.issuing_date).Distinct().ToArray();
}
else
{
result = this._context.archive.OrderBy(s => s.IssuingDate).Select(s => s.issuing_date).Distinct().ToArray();
}
return result;
}
But I am getting this error:
System.NotSupportedException: 'The specified type member 'IssuingDate' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.'
How to do this?
The cause of your error message
You should be aware about the differences between IEnumerable and IQueryable.
An object of a class that implements IEnumerable holds everything to enumerate over the sequence of items it represents. You can ask for the first item of the sequence, and once you've got one, you can ask for the next item, until there are no more items.
On the other hand, an object of a class that implements IQueryable holds everything to ask another process to provide data to create an IEnumerable sequence. To do this, it holds an Expression and a Provider.
The Expression is a generic representation of what kind of IEnumerable must be created once you start enumerating the IQueryable.
The Provider knows who must execute the query, and it knows how to translate the Expression into a format that the executor understands, for instance SQL.
There are two kinds of LINQ statements. Those that use deferred execution, and those that don't. The deferred functions can be recognized, because they return IQueryable<TResult> (or IEnumerable). Examples are Where, Select, GroupBy, etc.
The non-deferred functions return a TResult: ToList, ToDictionary, FirstOrDefault, Max.
As long as you concatenate deferred LINQ functions, the query is not executed, only the Expression is changed. Once you start enumerating, either explicitly using GetEnumerator and MoveNext, or implicitly using foreach, ToList, Max, etc, the Expression is sent to the Provider who will translate it to SQL and execute the query. The result is represented as an IEnumerable, on which the GetEnumerator is performed.
What has this to do with my question?
Because the Expression must be translated into SQL, it can't hold anything that you invented. After all, SQL doesn't know your functions. In fact, there are a lot of standard functions that can't be used in an IQueryable. See Supported and unsupported LINQ functions
Alas you forgot to give us the archive class definition, but I think that it is not a POCO: It contains functions and properties that do more than just get / set. I think that IssuingDate is not just get / set.
For IQueryables you should keep your classes simple: use only {get; set;} during your query, nothing more. Other functions can be called after you've materialized your IQueryable into something IEnumerable which is to be executed within your local process
Back to your question
So you have a database with a table Archive with at least columns IssuingDate and InsertedBy. It seems that InsertedBy is just a string. It could be a foreign key to a table with users. This won't influence the answer very much.
Following the entity framework code first conventions this leads to the following classes
class Archive
{
public int Id {get; set;}
public DateTime IssuingDate {get; set;}
public string InsertedBy {get; set;}
...
}
public class MyDbContext : DbContext
{
public DbSet<Archive> Archives {get; set;}
}
By the way, is there a proper reason you deviate so often from Microsoft standards about naming identifiers, especially pluralization and camel casing?
Anyway, your requirement
I have two datetime pickers on my form. I want a function that will return all datetimes from a specific table (which are values of a specific column) between those two dates.
Your code seems to do a lot more, but let's first write an extension function that meets your requirement. I'll write it as an extension method of your archive class. This will keep your archive class simple (only {get; set;}), yet it adds functionality to the class. Writing it as an extension function also enables you to use these functions as if they were any other LINQ function. See Extension methods demystified
public static IQueryable<Archive> BetweenDates(this IQueryable<Archive> archives,
DateTime startDate,
DateTime endDate)
{
return archives.Where(archive => startDate <= archive.IssuingDate
&& archive.IssuingDate <= endDate);
}
If I look at your code, you don't do anything of selecting archives between dates. You do something with a userName, ordering, select distinct... It is a bit strange that you first Order all your million archives, and then decide to keep only the ten archives that belong to userName, and if you have several same issuing dates you decide to remove the duplicates. Wouldn't it be more efficient to first limit the number of issuing dates before you start ordering them?
public static IQueryable<archive> ToIssuingDatesOfUser(this IQueryable<archive> archives,
string userName)
{
// first limit the number of archives, depdning on userName,
// then select the IssuingDate, remove duplicates, and finally Order
var archivesOfUser = (userName == null) ? archives :
archives.Where(archive => archive.InsertedBy == userName);
return archivesOfUser.Select(archive => archive.IssuingDate)
.Distinct()
.OrderBy(issuingDate => issuingDate);
}
Note: until now, I only created IQueryables. So only the Expression is changed, which is fairly efficient. The database is not communicated yet.
Example of usage:
Requirement: given a userName, a startDate and an endDate, give me the unique issuingDates of all archives that are issued by this user, in ascending order
public ICollection<string> GetIssuingDatesOfUserBetweenDates(string userName,
DateTime startDate,
DateTime endDate)
{
using (var dbContext = new MyDbContext(...))
{
return dbContext.Archives
.BetweenDates(startDate, endDate)
.ToIssuingDatesOfUser(userName)
.ToList();
}
}
I'm trying to accomplish 2 things with the below snippet of code (from ApplicationDataService.lsml.cs in the server project of my Lightswitch 2013 solution).
partial void Query1_PreprocessQuery(ref IQueryable<CandidateBasic> query)
{
query = from item in query where item.CreatedBy == this.Application.User.Name select item;
}
partial void CandidateBasics_Validate(CandidateBasic entity, EntitySetValidationResultsBuilder results)
{
var newcandidateCount = this.DataWorkspace.ApplicationData.Details.GetChanges().AddedEntities.OfType<CandidateBasic>().Count();
var databasecandidateCount = this.CandidateBasics.GetQuery().Execute().Count();
const int maxcandidateCount = 1;
if (newcandidateCount + databasecandidateCount > maxcandidateCount)
{
results.AddEntityError("Error: you are only allowed to have one candidate record");
}
}
Firstly, I want to make sure each user can only see things that he has made. This, together with a preprocess query on the table in question, works perfectly.
The next bit is designed to make sure that each user can only create one record in a certain table. Unfortunately, it seems to be looking at the whole table, and not the query I made that shows only the user's own records.
How can I get that second bit of code to limit only the user's own records, and not the global table?
You're not actually calling that query though are you? Your query is called Query1 based on the code provided yet you don't seem to be calling it. I'd do something like:
int count = DataWorkspace.ApplicationData.Query1().Count();
I am having a hard time getting this to work. I am trying to do a radius search using the following Filter helper on an IQueryable. There are a set of other filters that get applied before RadiusSearch applies. The order shouldn't really matter since the goal is to get the query to be deferred until a ToList() operation.
public static IQueryable<ApiSearchCommunity> RadiusSearch(this IQueryable<ApiSearchCommunity> communities)
{
var centerLatitude = 30.421278;
var centerLongitude = -97.426261;
var radius = 25;
return communities.Select(c => new ApiSearchCommunity()
{
CommunityId = c.CommunityId,
City = c.City,
//Distance = c.GetArcDistance(centerLatitude, centerLongitude, c.Latitude, c.Longitude, radius)
});
}
Can I somehow write a helper like GetArcDistance above which in turn calls a UDF on SQL? The query I am trying to generate is the following
SELECT
comms.community_id,
comms.city,
comms.distance
FROM (
SELECT
c.community_id,
c.city,
dbo.udf_ArcDistance(
30.421278,-97.426261,
c.community_latitude,
c.community_longitude
) AS distance
FROM communities c) AS comms
WHERE comms.distance <= 25
ORDER BY comms.distance
Ok, I think I understand the question - the gist of it is you want to be able to call a SQL UDF as part of your Linq to Entities query.
This is if you're using database or model first:
This article explains how to do it: http://msdn.microsoft.com/en-us/library/dd456847(VS.100).aspx
To sum it up, you first need to edit your edmx file in an xml editor, in the edmx:StorageModels >> Schema section you need to specify a mapping to your sql udf, eg
<Function Name="SampleFunction" ReturnType="int" Schema="dbo">
<Parameter Name="Param" Mode="In" Type="int" />
</Function>
Then you need to create a static function somewhere with the EdmFunction attribute on it, something like this:
public static class ModelDefinedFunctions
{
[EdmFunction("TestDBModel.Store", "SampleFunction")]
public static int SampleFunction(int param)
{
throw new NotSupportedException("Direct calls are not supported.");
}
}
This method will get mapped to the UDF at query time by entity framework. The first attribute argument is the store namespace - you can find this in your edmx xml file on the Schema element (look for Namespace). The second argument is the name of the udf.
You can then call it something like this:
var result = from s in context.UDFTests
select new
{
TestVal = ModelDefinedFunctions.SampleFunction(22)
};
Hope this helps.
if you use Code-First approach, then you cannot call UDFs as you want (as of EF6) - here is the proof, and another one. You are only limited to calling UDF as a part of your SQL query:
bool result = FooContext.CreateQuery<bool>(
"SELECT VALUE FooModel.Store.UserDefinedFunction(#someParameter) FROM {1}",
new ObjectParameter("someParameter", someParameter)
).First();
which is ugly IMO and error-prone.
Also - this MSDN page says:
The process for calling a custom function requires three basic steps:
Define a function in your conceptual model or declare a function in your storage model.
which essentially means you need to use Model-First approach to call UDFs.
I create a complex search query in native SQL. It's basically something like this:
SELECT ID FROM t_Product WHERE Name LIKE #criteria
SELECT publisher, count(*) as number FROM t_Product GROUP BY publisher
It has 2 SELECT statements and I want it to be sent to DB server in one round trip.
But I can't figure out how to achieve this in Nhibernate.
I considered following options but none seems to work
Use CreateMultiQuery, but this only accept HQL, not native SQL
Use CreateSQLQuery, but call to List() only return result for the first SELECT statement
Moving to a stored procedure is not an option since the whole SQL is very dynamic.
We still use Nhibernate 1.2 thus new features in later version couldn't be used either.
Advice are welcome.
Not possible using NH version 1.2
Futures was released in version 2.1 which allows you to do exactly this.
e.g.
var blogs = s.CreateCriteria<Invoice>()
.SetMaxResults(30)
.Future<Invoice>();
var countOfInvoices = s.CreateCriteria<Invoice>()
.SetProjection(Projections.Count(Projections.Id()))
.FutureValue<int>();
So you are going to either upgrade, fall back to ADO.NET and use multiple recordsets or live with what you have! Sorry!
This is really going to be scenario-specific, but if you're stuck with NH Version 1.2, and eliminating the round-trip is your goal, you could consider rewriting this as a single query using a sub-select.
Something along the lines of:
SELECT publisher, count(*) as number,
(SELECT ID FROM t_Product WHERE Name LIKE #criteria) As theId
FROM t_Product GROUP BY publisher
Would work if your subquery only returned a single value.
I don't think that it is possible, because both queries are SELECTs.
You may try a semicolon after the first query, and two line feeds between them, this is required for some databases. I successfully run query-scripts like this. If it runs, use a debugger to see what you get back ...
If this doesn't work, you need separate round trips or switch to HQL / Criteria.
You can use MultiQuery "Hack" like this:
The procudure:
CREATE PROCEDURE [dbo].[proc_Name]
AS BEGIN
SELECT * FROM t_Question where ...
SELECT * FROM t_Question where ........
END
The NHibernate Query Code:
public void ProcdureMultiTableQuery()
{
var session = Session;
var procSQLQuery = session.CreateSQLQuery("exec [proc_Name] ?,?");// prcodure returns two table
procSQLQuery.SetParameter(0, userId);
procSQLQuery.SetParameter(1, page);
procSQLQuery.AddEntity(typeof(Question));
var multiResults = session.CreateMultiQuery()
.Add(procSQLQuery)
// More table your procedure returns,more empty SQL query you should add
.Add(session.CreateSQLQuery(" ").AddEntity(typeof(Question))) // the second table returns Question Model
.List();
if (multiResults == null || multiResults.Count == 0)
{
return;
}
if (multiResults.Count != 2)
{
return;
}
var questions1 = ConvertObjectsToArray<Question>((System.Collections.IList)multiResults[0]);
var questions2 = ConvertObjectsToArray<Question>((System.Collections.IList)multiResults[1]);
}
static T[] ConvertObjectsToArray<T>(System.Collections.IList objects)
{
if (objects == null || objects.Count == 0)
{
return null;
}
var array = new T[objects.Count];
for (int i = 0; i < array.Length; i++)
{
array[i] = (T)objects[i];
}
return array;
}
Is there a "best practice" way of handling bulk inserts (via LINQ) but discard records that may already be in the table? Or I am going to have to either do a bulk insert into an import table then delete duplicates, or insert one record at a time?
08/26/2010 - EDIT #1:
I am looking at the Intersect and Except methods right now. I am gathering up data from separate sources, converting into a List, want to "compare" to the target DB then INSERT just the NEW records.
List<DTO.GatherACH> allACHes = new List<DTO.GatherACH>();
State.IState myState = null;
State.Factory factory = State.Factory.Instance;
foreach (DTO.Rule rule in Helpers.Config.Rules)
{
myState = factory.CreateState(rule.StateName);
List<DTO.GatherACH> stateACHes = myState.GatherACH();
allACHes.AddRange(stateACHes);
}
List<Model.ACH> newRecords = new List<Model.ACH>(); // Create a disconnected "record set"...
foreach (DTO.GatherACH record in allACHes)
{
var storeInfo = dbZach.StoreInfoes.Where(a => a.StoreCode == record.StoreCode && (a.TypeID == 2 || a.TypeID == 4)).FirstOrDefault();
Model.ACH insertACH = new Model.ACH
{
StoreInfoID = storeInfo.ID,
SourceDatabaseID = (byte)sourceDB.ID,
LoanID = (long)record.LoanID,
PaymentID = (long)record.PaymentID,
LastName = record.LastName,
FirstName = record.FirstName,
MICR = record.MICR,
Amount = (decimal)record.Amount,
CheckDate = record.CheckDate
};
newRecords.Add(insertACH);
}
The above code builds the newRecords list. Now, I am trying to get the records from this List that are not in the DB by comparing on the 3 field Unique Index:
AchExceptComparer myComparer = new AchExceptComparer();
var validRecords = dbZach.ACHes.Intersect(newRecords, myComparer).ToList();
The comparer looks like:
class AchExceptComparer : IEqualityComparer<Model.ACH>
{
public bool Equals(Model.ACH x, Model.ACH y)
{
return (x.LoanID == y.LoanID && x.PaymentID == y.PaymentID && x.SourceDatabaseID == y.SourceDatabaseID);
}
public int GetHashCode(Model.ACH obj)
{
return base.GetHashCode();
}
}
However, I am getting this error:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[MisterMoney.LARS.ZACH.Model.ACH] Intersect[ACH](System.Linq.IQueryable1[MisterMoney.LARS.ZACH.Model.ACH], System.Collections.Generic.IEnumerable1[MisterMoney.LARS.ZACH.Model.ACH], System.Collections.Generic.IEqualityComparer1[MisterMoney.LARS.ZACH.Model.ACH])' method, and this method cannot be translated into a store expression.
Any ideas? And yes, this is completely inline with the original question. :)
You can't do bulk inserts with LINQ to SQL (I presume you were referring to LINQ to SQL when you said "LINQ"). However, based on what you're describing, I'd recommend checking out the new MERGE operator of SQL Server 2008.
Inserting, Updating, and Deleting Data by Using MERGE
Another example here.
I recommend you just write the SQL yourself to do the inserting, I find it is a lot faster and you can get it to work exactly how you want it to. When I did something similar to this (just a one-off program) I just used a Dictionary to hold the ID's I had inserted already, to avoid duplicates.
I find LINQ to SQL is good for one record or a small set that does its entire lifespan in the LINQ to SQL.
Or you can try to use SQL Server 2008's Bulk Insert .
One thing to watch out for is if you queue more than 2000 or so records without calling SubmitChanges() - TSQL has a limit on the number of statements per execution, so you cannot simply queue up every record and then call SubmitChanges() as this will throw an SqlException, you need to periodically clear the queue to avoid this.