LINQ query optimization - c#

I retrieve data from two different repositories:
List<F> allFs = fRepository.GetFs().ToList();
List<E> allEs = eRepository.GetEs().ToList();
Now I need to join them so I do the following:
var EFs = from c in allFs.AsQueryable()
join e in allEs on c.SerialNumber equals e.FSerialNumber
where e.Year == Convert.ToInt32(billingYear) &&
e.Month == Convert.ToInt32(billingMonth)
select new EReport
{
FSerialNumber = c.SerialNumber,
FName = c.Name,
IntCustID = Convert.ToInt32(e.IntCustID),
TotalECases = 0,
TotalPrice = "$0"
};
How can I make this LINQ query better so it will run faster? I would appreciate any suggestions.
Thanks

Unless you're able to create one repository that contains both pieces of data, which would be a far preferred solution, I can see the following things which might speed up the process.
Since you'r always filtering all E's by Month and Year, you should do that before calling ToList on the IQueryable, that way you reduce the number of E's in the join (probably considerably)
Since you're only using a subset of fields from E and F, you can use an anonymous type to limit the amount of data to transfer
Depending on how many serialnumbers you're retrieving from F's, you could filter your E's by serials in the database (or vice versa). But if most of the serialnumbers are to be expected in both sets, that doesn't really help you much further
Reasons why you might not be able to combine the repositories into one are probably because the data is coming from two separate databases.
The code, updated with the above mentioned points 1 and 2 would be similar to this:
var allFs = fRepository.GetFs().Select(f => new {f.Name, f.SerialNumber}).ToList();
int year = Convert.ToInt32(billingYear);
int month = Convert.ToInt32(billingMonth);
var allEs = eRepository.GetEs().Where(e.Year == year && e.Month == month).Select(e => new {e.FSerialNumber, e.IntCustID}).ToList();
var EFs = from c in allFs
join e in allEs on c.SerialNumber equals e.FSerialNumber
select new EReport
{
FSerialNumber = c.SerialNumber,
FName = c.Name,
IntCustID = Convert.ToInt32(e.IntCustID),
TotalECases = 0,
TotalPrice = "$0"
};

Related

LINQ query with a where condition containing

I am just learning LINQ and I have come across and issue Im not sure how to do in LINQ.
string numbers = "1,3,4,5";
string[] outletsInaStringArray = outlets.Split(',');
List<string> numbersAsAList = outletsInaStringArray.ToList();
I have a field in my database which holds a number. I only want to select the lines WHERE the number in the database is IN the line list of numbers "1,3,4,5" (these numbers are just examples).
Thanks in advance
I have looked at Tim and James answers and also looked at the line that James has sent. Im still a bit confused.....Sorry. Below is my actual code. It compiles but does not work
string outlets = "1,3,4,5"
string[] outletsNeeded = outlets.Split(',');
List<string> outletsNeededList = outletsNeeded.ToList();
DashboardEntities1 db = new DashboardEntities1();
var deptSalesQuery = (
from d in db.DashboardFigures
where (d.TypeOfinformation == "DEPTSALES") && (outletsNeeded.ToString().Contains(d.OutletNo.ToString()))
select new DeptSales
{
Dn = (int)d.Number,
Dnm = "Mens",
On = d.OutletNo,
Qs = (double)d.Value_4,
Se = (double)d.Value_2,
Si = (double)d.Value_3
}
);
In the DASHBAORDFIGURES table in SQL I have 2 records where the outlets number = 1, and therefore should have come up with two records.
Sorry if this is a simple thing, its just new to me and its frustrating.
You can use Contains as tagged:
var query = db.Table
.Where(x => outletsInaStringArray.Contains(x.Number) && x.information == "SALES");
that was method syntax, if you prefer query syntax:
var query = from figure in db.Figures
where outletsInaStringArray.Contains(figure.number)
&& figure.information == "SALES"
select figure;
But the column number is int, the List<string> stores strings, maybe your LINQ provider does not support .Contains(figure.ToString()). Then convert the strings to int first:
List<int> outletsNeededList = outletsNeeded.Select(int.Parse).ToList();
The answer that Tim provided is one method. Linq and lambda are interchangeable. Have a look at the following posting as well. Link
var result = from x in db.Table.ToList()
where outletsInaStringArray.Contains(x.Number)
select x;
Also have a look the following as it offers a very similar solution to the one you are looking for:
Link
As per i understand, you want to fetch data in similar way as IN (SQL) clause does it.
SELECT <Field_List>
FROM Table
WHERE IntegerField IN (1,2,4,5)
But i'm wondering why do you want to do it that way, when you can join data and get only matches. The worse is that you're trying to mix different data type and pass comma delimited text as a set of integers (i may be wrong):
SELECT <Field_List>
FROM Table
WHERE IntegerField IN ("1,2,4,5")
Above query won't execute, because the set of integers is "packed" into comma delimited string. To be able to execute that query, a conversion between data types must be done. Numbers in a string have to be converted to a set of integers (using user define split function or Common Table Expression):
;WITH CTE AS
(
--here convertion occurs
)
SELECT t2.<Field_List>
FROM CTE As t1 INNER JOIN TableName AS t2 ON t1.MyNumber = t2.IntegerField
Linq + any programming language is more flexible. You can build a list of integers (List) to build query.
See simple example:
void Main()
{
List<MyData> data = new List<MyData>{
new MyData(1,10),
new MyData(2, 11),
new MyData(5, 12),
new MyData(8, 13),
new MyData(12, 14)
};
//you're using comma delimited string
//string searchedNumbers = "1,3,4,5";
//var qry = from n in data
// join s in searchedNumbers.Split(',').Select(x=>int.Parse(x)) on n.ID equals s
// select n;
//qry.Dump();
List<int> searchedNumbers = new List<int>{1,2,4,5};
var qry = from n in data
join s in searchedNumbers on n.ID equals s
select n;
qry.Dump();
}
// Define other methods and classes here
class MyData
{
private int id = 0;
private int weight = 0;
public MyData(int _id, int _weight)
{
id = _id;
weight = _weight;
}
public int ID
{
get{return id;}
set {id = value;}
}
public int Weight
{
get{return weight;}
set {weight = value;}
}
}
Result:
ID Weight
1 10
5 12
Cheers
Maciej
Thank you all iv now got it to work using all your suggestions
the final code that works is as follows
DeptSales myDeptSales = new DeptSales(); // Single department
List<DeptSales> myDeptSalesList = new List<DeptSales>(); // List of Departments
DashboardEntities1 db = new DashboardEntities1();
var deptSalesQuery = from d in db.DashboardFigures
join s in outlets.Split(',').Select(x => int.Parse(x)) on d.OutletNo equals s
where (d.TypeOfinformation == "DEPTSALES")
select new DeptSales
{
Dn = (int)d.Number,
Dnm = "Mens",
On = d.OutletNo,
Qs = (double)d.Value_4,
Se = (double)d.Value_2,
Si = (double)d.Value_3
};
Thanks once again.

CRM LINQ Composite join "The method 'Join' is not supported" error

Im getting a "The method 'Join' is not supported" error... Funny thing is that i simply converted the 1st LINQ into the 2nd version and it doesnt work...
What i wanted to have was LINQ version #3, but it also doesnt work...
This works
var query_join9 = from s in orgSvcContext.CreateQuery(ServiceAppointment.EntityLogicalName)
join b in orgSvcContext.CreateQuery(bh_product.EntityLogicalName)
on s["bh_contract"] equals b["bh_contract"]
where ((EntityReference)s["bh_contract"]).Id == Guid.Parse("09BDD5A9-BBAF-E111-A06E-0050568B1372")
select new
{
Events = s,
Products = b
};
This doesn't
var query_join9 = from s in orgSvcContext.CreateQuery(ServiceAppointment.EntityLogicalName)
join b in orgSvcContext.CreateQuery(bh_product.EntityLogicalName)
on new { contractid = s["bh_contract"] }
equals new { contractid = b["bh_contract"] }
where ((EntityReference)s["bh_contract"]).Id == Guid.Parse("09BDD5A9-BBAF-E111-A06E-0050568B1372")
select new
{
Events = s,
Products = b
};
Also, this doesn't, which is a composite join and what i really aim for
var query_join9 = from s in orgSvcContext.CreateQuery(ServiceAppointment.EntityLogicalName)
join b in orgSvcContext.CreateQuery(bh_product.EntityLogicalName)
on new { contractid = s["bh_contract"], serviceid = s["serviceid"] }
equals new { contractid = b["bh_contract"], serviceid = s["serviceid"] }
where ((EntityReference)s["bh_contract"]).Id == Guid.Parse("09BDD5A9-BBAF-E111-A06E-0050568B1372")
select new
{
Events = s,
Products = b
};
I tried early binding and still doesnt work...
var query_join9 = from s in orgSvcContext.CreateQuery<ServiceAppointment>()
join b in orgSvcContext.CreateQuery<bh_product>()
on new { foo = s.bh_contract.Id }
equals new { foo = b.bh_Contract.Id }
where s.bh_contract.Id == Guid.Parse("09BDD5A9-BBAF-E111-A06E-0050568B1372")
select new
{
Events = s,
Products = b
};
stil not working
var query_join9 = from s in orgSvcContext.CreateQuery<ServiceAppointment>()
join b in orgSvcContext.CreateQuery<bh_product>()
on new { s.bh_contract.Id, s.ServiceId }
equals new { b.bh_Contract.Id, ServiceId = b.bh_Service }
where s.bh_contract.Id == Guid.Parse("09BDD5A9-BBAF-E111-A06E-0050568B1372")
select new
{
Events = s,
Products = b
};
But im simply trying to do the example(s) here How to do joins in LINQ on multiple fields in single join
What am i missing?
Thanks in advance
While I'm not entirely sure which CRM you're using, I think you're misunderstanding something.
In order for a LINQ query to work, there needs to be a LINQ provider for the underlying data source -- the bit of code responsible for translating chain of e.g. Join, Where, operator usage, etc, etc, into the query API of the data source. This might be SQL, some custom query language, or some chain of methods.
Two LINQ providers (such as, one for LINQ to DataSet and some custom provider you've written yourself) don't have to support the same methods and other code. The precise subset of LINQ methods (and/or other embedded statements) a LINQ provider supports is dependent on its implementation.
Looking at it like that, it's not that surprising that the LINQ provider you're using doesn't seem to comprehend the standard syntax for joins using multiple fields, or doesn't seem to comprehend the usage of anonymous types at all.
My advice is to search the documentation of the supplied LINQ provider to see which query operations it supports (perhaps there is a note about this specific mode of query not being supported). Failing that, you'll have to resort to some sort of other query -- one not involving an equijoin. Perhaps your best option is to perform the joins separately, and then intersect the two result groups. It really depends on the specifics of the case.
Have you looked at the MSDN samples. There are some multiple-column join examples there:
using (ServiceContext svcContext = new ServiceContext(_serviceProxy))
{
var list_join = (from a in svcContext.AccountSet
join c in svcContext.ContactSet
on a.PrimaryContactId.Id equals c.ContactId
where a.Name == "Contoso Ltd" && <<--- multiple join here
a.Address1_Name == "Contoso Pharmaceuticals"
select a).ToList();
foreach (var c in list_join)
{
System.Console.WriteLine("Account " + list_join[0].Name
+ " and it's primary contact "
+ list_join[0].PrimaryContactId.Id);
}
}
This other thread might be relevant

LINQ query ideas for this table structure

I know it's a rather unusual question, but I've been spending a bit of time trying to figure out the fastest way to retrieve data from an unusually structured database. Basically, the database looks like this:
'Trip' - nice and simply structured table, no issues here
'Consignments' - each consignment links to a trip above. The interesting, and annoying, thing here is that each consignment has 2 records in the table one for the 'Sales' value, and one for the 'Costs' value - these are differentiated by a Char field ('S' or 'C').
My query basically needs to select all consignments between a date range (the date is stored on the Trip table), with the sales and costs value in the row. Here's my LINQ query now:
var cons = from cS in dc.Consignments.Where(a => a.BreakdownType == 'S')
join cC in dc.Consignments on new { NTConsignment = cS.NTConsignment, LegacyID = cS.LegacyID, TripDate = cS.TripDate, Depot = cS.Depot, TripNumber = cS.TripNumber, BreakdownType = 'C' } equals new { NTConsingment = cC.NTConsignment, LegacyID = cC.LegacyID, TripDate = cC.TripDate, Depot = cC.Depot, TripNumber = cC.TripNumber, BreakdownType = cC.BreakdownType }
join sl in dc.SageAccounts on new { LegacyID = cS.Customer, Customer = true } equals new { LegacyID = sl.LegacyID, Customer = sl.Customer }
join ss in dc.SageAccounts on sl.ParentAccount equals ss.ID
join vt in dc.VehicleTypes on cS.Trip.VehicleType.Trim() equals vt.ID.ToString() into vtg
where cS.Trip.DeliveryDate >= dates.FromDate && cS.DeliveryDate <= dates.ToDate
orderby cS.Depot, cS.TripNumber
select new GMConsignment
{
NTConsignment = cS.NTConsignment,
Trip = cS.Trip,
LegacyID = cS.LegacyID,
Costs = cC.Value ?? 0.00m,
Sales = cS.Value ?? 0.00m,
Margin = (cS.Value ?? 0.00m) - (cC.Value ?? 0.00m),
Customer = cmS.Customer,
SageID = ss.SageID,
CustomerName = ss.ShortName,
FullCustomerName = ss.Name,
Vehicle = cS.Trip.Vehicle ?? "None",
VehicleType = vtg.SingleOrDefault().VehicleTypeDescription ?? "Subcontractor",
VehicleGroup = vtg.FirstOrDefault().VehicleGroup1
};
The problem is, the above is horrifically slow. I have profiled the SQL, and there are no obvious mistakes I'm making such as missing indexes. I know there are a million possibilities why the query is slow, and it's very difficult without knowing more about it, but I'm just wondering if there are any different, and potentially faster ways of doing what I'm doing in LINQ?
I've also tried selecting the 'Cost' in the 'Select' statement (i.e Costs = dc.Costs.Where...blah blah) but it's even slower.

Adding where clause to nested Linq selects

I'm still new to Linq so if you see something I really shouldn't be doing, please feel free to suggest a change.
I am working on a new system to allow officers to sign up for overtime. Part of the data is displayed on a map with search criteria filtering unwanted positions. In order to make the data easier to work with, it is read into a hierarchy object structure using Linq. In this example, a job can contain multiple shifts and each shift can have multiple positions available. The Linq statement to read them in looks like the following.
var jobs = (from j in db.Job
join s in db.Shift on j.Id equals s.JobId into shifts
select new JobSearchResult
{
JobNumber = j.Id,
Name = j.JobName,
Latitude = j.LocationLatitude,
Longitude = j.LocationLongitude,
Address = j.AddressLine1,
Shifts = (from shift in shifts
join p in db.Position on shift.Id equals p.ShiftId into positions
select new ShiftSearchResult
{
Id = shift.Id,
Title = shift.ShiftTitle,
StartTime = shift.StartTime,
EndTime = shift.EndTime,
Positions = (from position in positions
select new PositionSearchResult
{
Id = position.Id,
Status = position.Status
}).ToList()
}).ToList()
});
That works fine and has been tested. There may be a better way to do it and if you know of a way, feel free to suggest. My problem is this. After the query is created, search criteria will be added. I know that I could add it when the query is created but for this its easier to do it after. Now, I can easy add criteria that looks like this.
jobs = jobs.Where(j => j.JobNumber == 1234);
However, I am having trouble figuring out how to do the same for Shifts or Positions. In other words, how would I could it to add the condition that a shift starts after a particular time? The following example is what I am trying to accomplish but will not (obviously) work.
jobs = jobs.Shifts.Where(s = s.StartTime > JobSearch.StartTime) //JobSearch.StartTime is a form variable.
Anyone have any suggestions?
Step 1: create associations so you can have the joins hidden behind EntitySet properties.
http://msdn.microsoft.com/en-us/library/bb629295.aspx
Step 2: construct your filters. You have 3 queryables and the possibility of filter interaction. Specify the innermost filter first so that the outer filters may make use of them.
Here are all jobs (unfiltered). Each job has only the shifts with 3 open positions. Each shift has those open positions.
Expression<Func<Position, bool>> PositionFilterExpression =
p => p.Status == "Open";
Expression<Func<Shift, bool>> ShiftFilterExpression =
s => s.Positions.Where(PositionFilterExpression).Count == 3
Expression<Func<Job, bool>> JobFilterExpression =
j => true
Step 3: put it all together:
List<JobSearchResult> jobs = db.Jobs
.Where(JobFilterExpression)
.Select(j => new JobSearchResult
{
JobNumber = j.Id,
Name = j.JobName,
Latitude = j.LocationLatitude,
Longitude = j.LocationLongitude,
Address = j.AddressLine1,
Shifts = j.Shifts
.Where(ShiftFilterExpression)
.Select(s => new ShiftSearchResult
{
Id = s.Id,
Title = s.ShiftTitle,
StartTime = s.StartTime,
EndTime = s.EndTime,
Positions = s.Positions
.Where(PositionFilterExpression)
.Select(p => new PositionSearchResult
{
Id = position.Id,
Status = position.Status
})
.ToList()
})
.ToList()
})
.ToList();

LINQ group by month question

I'm new to LINQ to SQL and I would like to know how to achieve something like this in LINQ:
Month Hires Terminations
Jan 5 7
Feb 8 8
Marc 8 5
I've got this so far, and I think there is something wrong with it but I'm not sure:
from term1 in HRSystemDB.Terminations
group term1 by new { term1.TerminationDate.Month, term1.TerminationDate.Year } into grpTerm
select new HiresVsTerminationsQuery
{
Date = Criteria.Period,
TerminationsCount = grpTerm.Count(term => term.TerminationDate.Month == Criteria.Period.Value.Month),
HiresCount = (from emp in HRSystemDB.Persons.OfType<Employee>()
group emp by new { emp.HireDate.Month, emp.HireDate.Year } into grpEmp
select grpEmp).Count(e => e.Key.Month == Criteria.Period.Value.Month)
});
Thanks in advance.
I'm not quite sure where does the Criteria.Period value come from in your sample query.
However I think you're trying to read both hires and terminations for all available months (and then you can easily filter it). Your query could go wrong if the first table (Termination) didn't include any records for some specified month (say May). Then the select clause wouldn't be called with "May" as the parameter at all and even if you had some data in the second table (representing Hires), then you wouldn't be able to find it.
This can be elegantly solved using the Concat method (see MSDN samples). You could select all termniations and all hires (into a data structure of some type) and then group all the data by month:
var terms = from t in HRSystemDB.Terminations
select new { Month = t.TerminationDate.Month,
Year = term1.TerminationDate.Year,
IsHire = false };
var hires = from emp in HRSystemDB.Persons.OfType<Employee>()
select new { Month = emp.HireDate.Month,
Year = emp.HireDate.Year
IsHire = true };
// Now we can merge the two inputs into one
var summary = terms.Concat(hires);
// And group the data using month or year
var res = from s in summary
group s by new { s.Year, s.Month } into g
select new { Period = g.Key,
Hires = g.Count(info => info.IsHire),
Terminations = g.Count(info => !info.IsHire) }
When looking at the code now, I'm pretty sure there is some shorter way to write this. On the other hand, this code should be quite readable, which is a benefit. Also note that it doesn't matter that we split the code into a couple of sub-queries. Thanks to lazy evalutation of LINQ to SQL, this should be executed as a single query.
I don't know if it shorter but you can also try this version to see if it works better with your server. I don't know exactly how these two answers turn into SQL statements. One might be better based on your indexs and such.
var terms =
from t in Terminations
group t by new {t.Month, t.Year} into g
select new {g.Key, Count = g.Count()};
var hires =
from p in Persons
group p by new {p.Month, p.Year} into g
select new {g.Key, Count = g.Count()};
var summary =
from t in terms
join h in hires on t.Key equals h.Key
select new {t.Key.Month, t.Key.Year,
Hires = h.Count, Terms = t.Count};

Categories

Resources