Develop Reference

Why does AsEnumerable not work when using FromSqlRaw

I'm currently creating a site that showcases all my patients within a data table and I have to use FromSqlRaw in order to get the data from my database. I have a search funtion that allows me to search the patients within the table but upon entering the page I get this error when I use AsQueryable and no data is displayed in the table. It recommends me to use AsEnumerable but when I do I get an intellisense error. Any ideas on how to fix?
public async Task<IActionResult> Search(StaySearchViewModel model)
{
if (model.Cleared)
{
return Json(new
{
draw = model.Draw,
data = new object[] { },
recordsFiltered = 0,
recordsTotal = 0,
total = 0
});
}
var records = getSearchData(model);
//var records = System.Linq.Enumerable.AsEnumerable(getSearchData(model)); // Hard coding this an enumerable will break line 55, 57, and 64
//Sorting
if (!string.IsNullOrEmpty(model.SortOrder))
records = records.OrderBy(model.SortOrder);
var count = await records.CountAsync().ConfigureAwait(false);
records = records.Skip(model.Start);
if (model.Length != -1) records = records.Take(model.Length);
// Create models
var result = new List<SpStaySearchResultViewModel>();
try
{
await records.ForEachAsync(r =>
{
result.Add(new SpStaySearchResultViewModel()
{
BuildingName = r.BuildingName,
CaseManager = r.CaseManager,
CaseManagerId = r.CaseManagerId,
OccupantFileAs = r.OccupantFileAs,
StayOCFSNumber = r.StayOCFSNumber,
StayId = r.StayId,
MaxOfBillSentDate = r.MaxOfBillSentDate,
CountOfChildren = r.CountOfChildren,
StartDate = r.StartDate,
EndDate = r.EndDate,
OccupantId = r.OccupantId,
IsActive = r.IsActive,
});
}).ConfigureAwait(false);
}
catch (Exception e) { }
return Json(new
{
draw = model.Draw,
data = result,
recordsFiltered = count,
recordsTotal = count,
});
}
private IQueryable<spStaysSearch> getSearchData(StaySearchViewModel model)
{
var records = db.SpStaySearches.FromSqlRaw("dbo.spStaysSearch").AsQueryable();
if (model.OccupantId.HasValue)
records = records.Where(x => x.OccupantId == model.OccupantId);
if (!string.IsNullOrWhiteSpace(model.OccupantFileAs))
records = records.Where(x => x.OccupantFileAs == model.OccupantFileAs);
if (!string.IsNullOrWhiteSpace(model.BuildingName))
records = records.Where(x => x.BuildingName == model.BuildingName);
if (!string.IsNullOrWhiteSpace(model.CaseManager))
records = records.Where(x => x.CaseManager == model.CaseManager);
if (!string.IsNullOrWhiteSpace(model.BuildingName))
records = records.Where(x => x.BuildingName == model.BuildingName);
if (model.IntakeDateStart.HasValue && model.IntakeDateEnd.HasValue)
{
records = records.Where(x => x.StartDate >= model.IntakeDateStart && x.StartDate <= model.IntakeDateEnd);
}
else
{
if (model.IntakeDateStart.HasValue)
records = records.Where(x => x.StartDate >= model.IntakeDateStart);
if (model.IntakeDateEnd.HasValue)
records = records.Where(x => x.StartDate <= model.IntakeDateEnd);
}
if (model.ExitDateStart.HasValue && model.ExitDateEnd.HasValue)
{
records = records.Where(x => x.EndDate >= model.ExitDateStart && x.EndDate <= model.ExitDateEnd);
}
else
{
if (model.ExitDateStart.HasValue)
records = records.Where(x => x.EndDate >= model.ExitDateStart);
if (model.ExitDateEnd.HasValue)
records = records.Where(x => x.EndDate <= model.ExitDateEnd);
}
if (model.IsActive.HasValue)
records = records.Where(x => x.IsActive == model.IsActive);
return records;
}

Try this
var records = getSearchData(model).ToList();
var count = records.Count;
.....
You can't order records by model.SortOrder since it has nothing to do with records.
You can only do something like this
if (!string.IsNullOrEmpty(model.SortOrder)) records = records.OrderBy(r=> r.Id);

because your source data is a Stored Procedure, you cannot compose additional query expressions over the top of it. Instead you must load it into memory, as the error suggests, by enumerating the result set.
Including Related Data
SQL Server doesn't allow composing over stored procedure calls, so any attempt to apply additional query operators to such a call will result in invalid SQL. Use AsEnumerable or AsAsyncEnumerable method right after FromSqlRaw or FromSqlInterpolated methods to make sure that EF Core doesn't try to compose over a stored procedure.
The obvious way to interpret this in the code is to call .ToList() on the results from the SP, then to match your existing code pattern you can cast that result back to IQueryable:
var records = db.SpStaySearches.FromSqlRaw("dbo.spStaysSearch")
.ToList()
.AsQueryable()
Using AsEnumerable() is sometimes problematic as there are many different libraries that you may have implemented that might all provide an AsEnumerable() extension method.
We have to do this because even in SQL you cannot simply select from an SP and then add where clauses to it, you first have to read the results into a temporary table or a table variable, then you can re-query from the result set, that is what we are effectively doing now, we are reading the results into a C# variable (.ToList()) and then composing an in-memory query over the top of that result.
If your search logic must be encapsulated in a stored procedure, then given the technical limitations, the usual expectation is that you would add the search arguments as optional parameters to the stored procedure, rather then tring to add filter clauses on top of the results in C#.
We can help with how to move your filter logic into dbo.spStaysSearch but you'll have to post the content of that SP, ideally as a new question.
Instead of using an SP at all, where we lose practically all the goodness that EF can offer us, an alternative approach is to replace your SP entirely with a raw SQL then the rest of your logic will work as expected.
var sql = #"
SELECT
tblStays.*, tblOccupant.OccupantID,
tblOccupant.FileAs AS OccupantFileAs,
IIF(tblStays.BuildingName LIKE 'Main Shelter',
tblOccupant.OCFSMainNumber,
tblOccupant.OCFSNorthNumber) AS StayOCFSNumber,
COALESCE([CountOfOccupantStayID], 0) AS CountOfChildren,
tblCaseManager.FileAs AS CaseManager,
StaysMaxBillSentDate.MaxOfBillSentDate
FROM tblStays
LEFT JOIN tblOccupantStays ON tblStays.StayID = tblOccupantStays.StayID
LEFT JOIN tblOccupant ON tblOccupantStays.OccupantID = tblOccupant.OccupantID
LEFT JOIN (
SELECT lkpOccStays.StayID
, COUNT(tblOccupantStays.OccupantStayID) AS CountOfOccupantStayID
FROM tblOccupantStays lkpOccStays
INNER JOIN tblOccupant lkpChild ON lkpOccStays.OccupantID = lkpChild.OccupantID
WHERE lkpChild.OccupantType LIKE 'Child'
GROUP BY lkpOccStays.StayID
) OccupantStays_CountOfChildren ON tblStays.StayID = OccupantStays_CountOfChildren.StayID
LEFT JOIN tblCaseManager ON tblStays.CaseManagerID = tblCaseManager.CaseManagerID
LEFT JOIN (SELECT tblStayBillingHx.StayID
, MAX(tblStayBillingHx.BillSentDate) AS MaxOfBillSentDate
FROM tblStayBillingHx
GROUP BY tblStayBillingHx.StayID
) StaysMaxBillSentDate ON tblStays.StayID = StaysMaxBillSentDate.StayID
";
var records = db.SpStaySearches.FromSqlRaw(sql);
In this way the SP is providing the structure of the resultset, which might be necessary if you are using the Database-First approach but you are no longer executing the SP at all.
The SQL in this answer is provided as a guide to the syntax only, there is not enough information available to determine the validity of the query or that the results conform to your business requirements.

Entity Framework 4 related entity not loading

I use queries in EF4 to pull back records and process information through various other means (not EF) based on the data within, so I frequently have detached EF objects in lists.
In this case, I have a query in EntityFramework 4.0 that is not loading a related entity, even though I am using the .Include("...") method.
using (MyDBEntities ctx = new MyDBEntities())
{
ctx.ContextOptions.LazyLoadingEnabled = false;
// Get the first X records that need to be processed
var q = (from t in ctx.DBTables
.Include("Customer")
let c = t.Customer
where t.statusID == (int)Enums.Status.PostProcessing
&& c.isActive == true
select t
).Take(batchSize).ToList();
foreach (DBTable t in q)
{
// this results in c == null
Customer c = t.Customer;
// However t.CustomerID has a value, thus I know
// that t links to a real Customer record
Console.WriteLine(t.CustomerID);
}
}
Can anyone help me understand why Customer is not loading, even though I am explicitly stating to include it?

I found the root of the issue! The demon lies in the "let" command. Whenever I have a let, or a second "from" clause (like a join), then the ".Includes" get ignored!!!
// -- THIS FAILS TO RETRIEVE CUSTOMER
// Get the first X records that need to be processed
var q = (from t in ctx.DBTables
.Include("Customer")
// Using a "let" like this or
let c = t.Customer
// a "from" like this immediately causes my include to be ignored.
from ca in c.CustomerAddresses
where t.statusID == (int)Enums.Status.PostProcessing
&& c.isActive == true
&& ca.ValidAddress == true
select t
).Take(batchSize).ToList();
However, I can go get the ID's that I need to fetch in one call, then have a second "go get my includes" call, and everything works just fine.
// Get the first X record IDs that need to be processed
var q = (from t in ctx.DBTables
let c = t.Customer
from ca in c.CustomerAddresses
where t.statusID == (int)Enums.Status.PostProcessing
&& c.isActive == true
&& ca.ValidAddress == true
select t.TableID
).Take(batchSize).ToList();
// Now... go "deep-load" the records I need by ID
var ret = (from t in ctx.DBTables
.Include("Customer")
where q.Contains(t.TableID)
select t);

LINQ query of List of table object returning different results than LINQ-to-SQL query for the same data

In my ASP.NET MVC 5 project I'd like to take a LINQ-to-SQL query result and persist it in memory to do further queries on rather than hit the database each time. However when those results are put in a List and then further where clauses are applied I end up with no results as where the same where clauses applied to a query directly from the database does return results.
This returns items in my list:
DataContext db = new DataContext();
var data = db.myView.Where(x => [where clauses]).ToList();
This doesn't return items in my list:
DataContext db = new DataContext();
var data1 = db.myView.ToList();
var data2 = data1.Where(x => [where clauses]).ToList();
I've used this method to query from memory rather than directly from the database plenty of times and have never seen a difference in my results.
Here is the actual where logic:
.Where(x => ((((DateTime)x.DINVPDOF).Year == year && ((DateTime)x.DINVPDOF).Month == month && x.SOPNUMBE.Trim().ToUpper().StartsWith("SINV")) || ((x.DOCDATE >= new DateTime(DateTime.Today.Year, DateTime.Today.Month, 1).AddMonths(-1) && x.DOCDATE <= new DateTime(DateTime.Today.Year, DateTime.Today.Month, 1).AddDays(-1)) && (x.SOPNUMBE.Trim().ToUpper().StartsWith("CR") || x.SOPNUMBE.Trim().ToUpper().StartsWith("RC")))) && (x.USCATVLS_6 == "FGS") && (x.QUANTITY == 1) && !x.CUSTCLAS.Contains("SER") && (x.SLPRSNID != "HOUSE") && !(x.ITEMDESC.Contains("RETURN") && !x.ITEMDESC.Contains("CREDIT")))
Does it make sense that these two methods should ever not return the same thing? Maybe something in the where clauses that is giving me behavior different than I'm used to?
Thanks.

Looks like some of the database fields had spaces after the values. LINQ threw out records due to no string equivalency however apparently LINQ-to-SQL ignored the padding when applying the where clauses.
Trimming the fields in the LINQ query yields the same results now as the LINQ-to-SQL query.

Linq join with last instance in table

I have a workflow table that takes all the steps of a process. Lets work with 2 of those statuses:
Saved (new item saved but not submitted yet)
Submitted (item submitted for review)
Now I want to create a BatchSumbit function that will submit all the unsubmitted items. For this I need to query for all the items which has a latest workflow status of "Saved". All the historical workflow entries for the item still exist and it can go from "Submitted" back to "Saved" a few times.
Here is the table structure:
Now i want a linq query that will give me what I require:
from wasteInformation in wasteDB.WasteInformations
join workFlowHistory in wasteDB.WorkFlowHistories on wasteInformation.WasteInformationId equals workFlowHistory.WasteInformationId
// Join with last instance in workflow table (where workflowHistory.DateAdded is greatest)
where workFlowHistory.WorkFlowStep == "Saved"
&& wasteInformation.WasteProgrammeId == captureModel.WasteProgrammeId
&& wasteInformation.WasteSourceId == captureModel.WasteSourceId
select new
{
WasteInformationId = wasteInformation.WasteInformationId,
FinancialQuarter = wasteInformation.FinancialQuarter,
FinancialYear = wasteInformation.FinancialYear,
WasteProgrammeId = wasteInformation.WasteProgrammeId,
WasteMonth = wasteInformation.WasteMonth,
WasteYear = wasteInformation.WasteYear,
DateCaptured = wasteInformation.DateCaptured,
WasteSourceId = wasteInformation.WasteSourceId,
WasteDate = wasteInformation.WasteDate
}
The query as it is will give be all the saved entries for the item. I want it to give me the item if that item's last entry has a WorkFlowStep of "Saved"
Edit:
I've got something that looks like it works. Still need to test it some more:
var SavedWasteInformation = wasteDB.WasteInformations.Where(wi => wi.WorkFlowHistories.FirstOrDefault(wf => wf.DateAdded == wi.WorkFlowHistories.Max(wf_in => wf_in.DateAdded)).WorkFlowStep == "Saved"
&& wi.WasteProgrammeId == captureModel.WasteProgrammeId
&& wi.WasteSourceId == captureModel.WasteSourceId);
Edit:
My solution above and Vladimirs's below both seem to work, but after inspecting the execution plans Vladimirs's looks like the better option:

Providing that you have collection of WorkFlowHistories on your WasteInformation I believe that query will select WasteInformations with their latest WorkFlowHistory (if any):
from wasteInformation in wasteDB.WasteInformations
where wasteInformation.WasteProgrammeId == captureModel.WasteProgrammeId
&& wasteInformation.WasteSourceId == captureModel.WasteSourceId
select new
{
WasteInformation = wasteInformation,
LastSavedWorkFlowHistory = wasteInformation.WorkFlowHistories
.Where(x => x.WorkFlowStep == "Saved")
.OrderByDescending(x => x.DateAdded)
.FirstOrDefalt()
}

Building a custom|progressive query in LINQ?

I have a page with five text boxes, each one representing a field in my database table and a search button:
If I were using SQL I could build my SQL statement depending on which fields have data in them.
However, I want to use LINQ, and I'm at a loss as to how to accomplish this. For instance, take a look at the query below:
var db = new BookDBDataContext();
var q =
from a in db.Books
where a.Title.Contains(txtBookTitle) &&
a.Author.Contains(txtAuthor) &&
a.Publisher.Contains(txtPublisher)
select a.ID;
The query above will return data where all the fields match data in the table. But, what if the user didn't enter an Author in the txtAuthor field? If I were building this as a query string, I could check each field for data and add it to the query string. Since this is LINQ, I can't dynamically change the search criteria, it seems.
Any advice would be greatly appreciated!

var db = new BookDBDataContext();
var q = (from a in db.Books
where a.Title.Contains(txtBookTitle));
if(!String.IsNullOrEmpty(txtAuthor))
{
q = q.Where(a => a.Author.Contains(txtAuthor));
}
if(!String.IsNullOrEmpty(txtAuthor))
{
q = q.Where(a => a.Publisher.Contains(txtPublisher));
}
var id = q.Select(a => a.ID);

from a in db.Books
where (string.isNullorWhiteSpace(search) || a.Title.Contains(search)) &&
(string.isNullorWhiteSpace(txtAuthor) || a.Author.Contains(txtAuthor) ) &&
(string.isNullorWhiteSpace(txtPublisher) || a.Publisher.Contains(txtPublisher))
select a.ID;