Is it possible to use a Linq query to search through a List? In my web app I have to potentially process over 14k records based upon a spreadsheet uploaded by the user. With each record processed, I need to compare that record against what we currently have in our database in order to make sure we either aren't adding a duplicate or I know what record I need to be updating/editing.
Instead of hitting the database 14k times or more, I wanted to pull all the records contained on this table into a List, and then perform a search based on a set of conditions.
Here is the Linq query I currently have that hits the database. The business rules are pretty... complicated so I won't bother you with the details but these are the conditions that I need to satisfy for the search. I've tested this query and it returns the expected results.
var previousZips = (from z in db.ZipCodeTerritory
where (item.ZipCode.Equals(null) ?
z.StateCode.Equals(item.StateCode) &&
z.ChannelCode.Equals(item.ChannelCode) &&
SqlFunctions.DateDiff("DAY", z.EndDate, item.EndDate) == 0 :
z.StateCode.Equals(item.StateCode) &&
z.ChannelCode.Equals(item.ChannelCode) &&
SqlFunctions.DateDiff("DAY", z.EndDate, item.EndDate) == 0 &&
(z.ZipCode.Equals(null) || z.ZipCode.Equals(item.ZipCode)))
select z).ToList();
What I would like to do, however, is create a List of all the records on the table like this:
List<ZipCodeTerritory> allRecords = (from z in db.ZipCodeTerritory
select z).ToList()
and then use a query similar to this to pull the record I'm looking for from the list:
List<ZipCodeTerritory> previousZips = allRecords.Where(
z => (item.ZipCode.Equals(null)
? z.StateCode.Equals(item.StateCode) &&
z.ChannelCode.Equals(item.ChannelCode) &&
SqlFunctions.DateDiff("DAY", z.EndDate,
item.EndDate) == 0
: z.StateCode.Equals(item.StateCode) &&
z.ChannelCode.Equals(item.ChannelCode) &&
SqlFunctions.DateDiff("DAY", z.EndDate,item.EndDate) == 0 &&
(z.ZipCode.Equals(null) || z.ZipCode.Equals(item.ZipCode))
)
).ToList();
The query above (from the List), however, throws the following error:
This function can only be invoked from LINQ to Entities.
Answered my own question. The problem here was the SqlFunction. By removing that from the query and re-writing it like this it works
List<ZipCodeTerritory> previousZips = allRecords.Where(
z => (item.ZipCode.Equals(null)
? z.StateCode.Equals(item.StateCode) &&
z.ChannelCode.Equals(item.ChannelCode) &&
z.EndDate.Date == item.EndDate.Date
: z.StateCode.Equals(item.StateCode) &&
z.ChannelCode.Equals(item.ChannelCode) &&
z.EndDate.Date == item.EndDate.Date &&
(z.ZipCode.Equals(null) || z.ZipCode.Equals(item.ZipCode))
)
).ToList();
Instead of
SqlFunctions.DateDiff("DAY", z.EndDate,item.EndDate) == 0
use
z.EndDate.Subtract(item.EndDate).TotalDays == 0
Related
In the first picture I have the result of first query, the highlighted part indicates the rows that would be excluded by applying the filter on the second query, in the second I have the result of query select * from exlusion_table
I have to make a change to the first query to have it exclude the items retrieved from the second query
the first query:
var u = from a in cx.VW_LIST
where (a.PRJ == codPrj) && (a.DATE > date_ || a.DATE == null || date_ == null)
&& (x.Contains(a.CODE) || x.Count() == 0)
select a)
the second query:
var y = from esc in cx.EXCLUSION select esc
The first query should be modified to exclude all the rows that have the value fcode = the fcode of the second query (in the case in which the fscode of the second query = null) or that (fcode = fcode of the second query && fscode = fscode of the second query )
You can use Any(). ie:
var u = from a in cx.VW_LIST
where (a.PRJ == codPrj)
&& (a.DATE > date_ || a.DATE == null || date_ == null)
&& (x.Contains(a.CODE) || x.Count() == 0)
&& (!cx.EXCLUSION.Any( y => x.fscode == y.fscode && x.fcode == y.fcode ))
select a)
There are two approaches to this, one is to use ! and ANY() to filter out records found within the other list, this will compile into WHERE NOT EXISTS(_exclusion_) filter expression
var excluded = cx.EXCLUSION.AsQueryable();
var query = from vw in cx.VW_LIST
where vw.PRJ == codPrj
where vw.DATE == null || date_ == null || vw.DATE > date_
where !x.Any() || x.Contains(vw.CODE)
where !excluded.Any(esc => vw.fcode == esc.fcode
&& (esc.fscode == null || vw.fscode == esc.fscode))
select vw;
var results = query.ToList();
The tricky element is the null for fscode in the excluded table, that needs to act as wildcard match, or negate the fscode comparison.
It is not necessary to split the excluded query out into it's own query, we could have referenced to the cx.EXCLUSION table directly and it would have exactly the same effect, this shows you an encapsulation technique for building the LINQ query in a way that you could easily increase the complexity of the exclusion lookup without creating a mess of your overall query.
You may also find need to conditionally build the query, this is where fluent syntax provides a more modular approach:
bool filterExcludedRecords = true;
...
var excluded = cx.EXCLUSION.AsQueryable();
var query = cx.VW_LIST.Where(vw => vw.PRJ == codPrj)
.Where(vw => vw.DATE == null || date_ == null || vw.DATE > date_)
.Where(vw => !x.Any() || x.Contains(vw.CODE));
if(filterExcludedRecords)
query = query.Where(vw => !excluded.Any(esc => vw.fcode == esc.fcode
&& (esc.fscode == null || vw.fscode == esc.fscode)));
var results = query.ToList();
OUTER JOIN
Another method is to use a LFET OUTER JOIN where the exclusion match is not found:
var excluded = cx.EXCLUSION.AsQueryable();
var query = from vw in cx.VW_LIST
where vw.PRJ == codPrj
where vw.DATE == null || date_ == null || vw.DATE > date_
where !x.Any() || x.Contains(vw.CODE)
from esc in excluded.Where(e => vw.fcode == e.fcode
&& (e.fscode == null || vw.fscode == e.fscode))
.DefaultIfEmpty()
where esc.fscode == null
select vw;
var results = query.ToList();
The WHERE NOT EXISTS is often superior in terms of performance, OUTER JOIN may provide better response in an un-optimised table or when the number of rows in the exclusion list is significantly small and the number of rows in the main table is very large.
I include this query option for completeness, it is not well known that you can create simple outer joins by adding a new from clause to the query.
I have the following SQL Server query and I need to have it in LINQ, Simple query but I tried several time but I can not get it working.
Here is the SQL query
select *
from td_Accountline
where
BonusPlanID = 1
and Amount > 0
and Ord_Sub_ID like '%SMPORD%'
and MONTH(Created) = 11
and YEAR(Created) = 2013
and Ord_Sub_ID not in (
select Ord_Sub_ID
from td_Accountline
where
BonusPlanID =3 and
Ord_Sub_ID like '%SMPORD%'
)
I have tried with this query but still i am confused
var account=from acc in currentDB.td_Accountline
where acc.BonusPlan.BonusPlanID == 1 && acc.Amount > 0 && acc.Ord_Sub_ID.Contains("SMPORD") && acc.Created.ToDateTime().Month == 11 && acc.Created.ToDateTime().Year == 2013
let accNot = from accN in currentDatabase.td_Accountline
where accN.BonusPlan.BonusPlanID == 3 && accN.Ord_Sub_ID.Contains("SMPORD")
select accN.Ord_Sub_ID
where !accNot.Contains("SMPORD")
select acc;
I want one query please not separate query to reduce database calling.
I think you're almost there. Instead of:
where !accNot.Contains("SMPORD")
It should be:
where !accNot.Contains(acc.Ord_Sub_ID)
Your final Linq query would be:
var account = from acc in currentDB.td_Accountline
where
acc.BonusPlan.BonusPlanID == 1
&& acc.Amount > 0
&& acc.Ord_Sub_ID.Contains("SMPORD")
&& acc.Created.Month == 11
&& acc.Created.Year == 2013
let accNot = from accN in currentDatabase.td_Accountline
where
accN.BonusPlan.BonusPlanID == 3
&& accN.Ord_Sub_ID.Contains("SMPORD")
select accN.Ord_Sub_ID
where !accNot.Contains(acc.Ord_Sub_ID)
select acc;
Regular Expressions won't work in Linq for Entity Framework but you can convert to collection in an in-memory list for you to use Regex.
Something like this:
(from x in td_Accountline where Created.Month = 11 && Created.Year = 2013 select a).ToList().Where(v => Regex.IsMatch(v.Ord_Sub_ID,#"(SMPORD)"))
That way, the match will happen in memory
Try This :
var Ord_Sub_IDs = from n in td_Accountline where n.BonusPlanID == 3 && n.Ord_Sub_ID.Contains("SMPORD") select n.Ord_Sub_ID ;
var result=from n in td_Accountline where n.BonusPlanId == 1 && n.Amount > 0 && n.Ord_Sub_ID.Contains("SMPORD") && n.Created.ToDateTime().Month == 11 && n.Created.ToDateTime().Year == 2013 && Ord_Sub_IDs.Contains(td.Ord_Sub_ID) select n;
I am currently using the following linq query
lsttask = (from d in administrationEntities.Tasks
.Include("Status")
.Include("Priority")
.Include("Batch")
.Include("Batch.ShipmentGroup")
.Include("Batch.ShipmentGroup.Shipment")
.Include("Batch.ShipmentGroup.Shipment.TOCShipmentManifests.TOCShipmentDetails.TOCShipmentProcesses")
.Include("Batch.ShipmentGroup.Shipment.Project")
.Include("TaskType")
where ((projectId == null ? true : d.Batch.ShipmentGroup.Shipment.Project.ProjectId == projectId)
&& ( statusId == null ? true : d.StatusId == statusId)
&& d.IsDeleted == false)
select d)
.ToList();
but its really consuming time , like up to 30 plus seconds
Is there an alternate way to trim on the execution time?
I added data to an ObservableCollection from a LINQ query:
foreach (var item in test4)
{
lstPareto.Add(new clsPareto(Convert.ToInt32(item.Step), Convert.ToInt32(item.LogID), test3.Where(p => p.Step.Equals(item.Step) && p.LogID.Equals(item.LogID)).Count()));
}
And this works fine. I get the items I want and convert them to an int when adding them to the list.
Then I have the following queries that pulls data from multiple databases:
int intCmbTestNr = Convert.ToInt32(m_strCmbTestNrSelectedItem);
var productIndex = (from x in m_dcSQL_ConnectionProdTest.DC3_VersionReleases
where x.TestNumber.Equals(intCmbTestNr)
select x.ProductIndex).First();
var version = ((from y in m_dcSQL_ConnectionProdTest.DC3_MainSetups
where y.ProductIndex == productIndex && y.SubVersion == 0
select y.Version).Max());
var versionIndex = (from z in m_dcSQL_ConnectionProdTest.DC3_MainSetups
where z.ProductIndex == productIndex && z.Version.Equals(version) && z.SubVersion == 0
select z.VersionIndex).First();
var subFuncName = from a in m_dcSQL_ConnectionProdTest.DC3_SubFunctions
where a.VersionIndex == versionIndex && a.FunctionNumber == lstPareto.Select(b => b.intStep) && a.SubFunctionNumber == lstPareto.Select(c => c.intStep)
select a.SubFunctionName;
Consider subFuncName. What I am trying to achieve here is to compare a.FunctionNumber to intStep and a.SubFunctionNumber to intLogID of the list lstPareto. However, it says the following: "Operator '==' cannot be applied to operands of type'int' and 'System.Collections.Generic.IEnumerable". I think I know the reason for this, seeing as I'm trying to compare a single int to a whole collection. But how do I compare to every single item intStep and intLogID of the list? I cannot seem to wrap my head around this. Do I use a foreach loop somewhere? Can somebody get me back on track?
Sorry if the title is somewhat vague, couldn't really think of a good one.
If both FunctionNumber and SubFunctionNumber is int:s. Then you can change the condition to this:
lstPareto.Select(b => b.intStep).Contains(a.FunctionNumber)
&& lstPareto.Select(c => c.intStep).Contains(a.SubFunctionNumber)
Update
The reason is most probably because you do not have .ToList(), .First(), .Single() in the query. Depends a little bit what you expect. You could change it to this:
var subFuncName = (from a in m_dcSQL_ConnectionProdTest.DC3_SubFunctions
where a.VersionIndex == versionIndex && lstPareto.Select(b => b.intStep).Contains(a.FunctionNumber)
&& lstPareto.Select(c => c.intStep).Contains(a.SubFunctionNumber)
select a.SubFunctionName).ToList();
Reference:
Enumerable.Contains Method
I want to use select query using linq to list.
in fallowing code userDtls is an list & Common.UserRightVal is an enum variable
userDtls = _userService.GetById(id);
var permission = from udtls in userDtls[0].userRightList
where udtls.object_name == "Person" &&
(udtls.user_right == (int)Common.UserRightVal.update
|| udtls.user_right == (int)Common.UserRightVal.add_update_delete_read
|| udtls.user_right == (int) Common.UserRightVal.allRight)select udtls;
Here I want to check permission details for user. My question is if 'where' condition is getting false then also permission variable is showing default records as shown by userDtls list.
Thanks to every one for response. I got answer, here it is.
var permission=(dynamic) null;
permission = (from udtls in userDtls[0].userRightList
where udtls.object_name.Contains(objectType) &&
(udtls.user_right == (int)Common.UserRightVal.readAll
|| udtls.user_right == (int)Common.UserRightVal.read_readAll
|| udtls.user_right == (int)Common.UserRightVal.add_update_delete_read
|| udtls.user_right == (int)Common.UserRightVal.allRight)
select udtls).FirstOrDefault();