Develop Reference

How can I refactor this LINQ to be more performant?

I have a console app that processes orders generated since 18:30 on the previous day. The method returns an array of order numbers.
The first operation is to retrieve a list of orders from a table called Docstore (which contains orders that have been printed, which are the only orders that are needed) into an array:
string[] orders;
using (var db = new TCRContext())
{
var query = from docs in db.Docstore
where docs.DateCreated >= fromDate && docs.DocumentType == "InvoiceCredit"
select docs.PrimaryKey.Trim();
orders = query.Distinct().ToArray();
}
Typically there may be at most 40 such orders. There are a couple of further considerations. First, there's a database VIEW called MatchingCustomerOrders which JOINs Orders (from the actual Order table, not the Docstore table) onto another table CPCustomers which contains a subset of all the company's Customers who participate in this particular operation (about 90 out of 2,370).
CREATE OR ALTER VIEW [dbo].[MatchingCustomerOrders]
AS
SELECT
Orders.order_no,
Orders.invoice_date
FROM
Orders INNER JOIN
dbo.CPCustomer ON Orders.customer = dbo.CPCustomer.Customer
WHERE
(dbo.CPCustomer.IsDeleted = 0)
GO
So we only need orders that match these customers. And we want to restrict them to those which match the invoice date parameters (i.e. from 18:30 the previous day to now) and also we don't want to process orders that have already been processed (hence the ProcessedInvoices part)
var query = from matches in db.MatchingCustomerOrders
where orders.Contains(matches.order_no.Trim()) &&
!(from processed in db.ProcessedInvoices select processed.order_no.Trim()).Contains(matches.order_no.Trim()) &&
matches.invoice_date >= fromDate.Date && matches.invoice_date <= toDate
select matches.order_no;
orders = query.Distinct().ToArray();
This is not particularly performant because the VIEW MatchingCustomerOrders has 90,000+ rows, and my suspicion is that LINQ is operating over the whole set. I've been banging my head trying to refactor this but none of my solutions seems to work.

Entity Framework Getting current date rows(while time in that day)

I have table that the value of dateime is like this
2021-12-08 10:10:54.657
2021-12-08 10:10:41.567
2021-12-08 10:09:51.960
2021-12-08 10:10:54.657
2021-12-08 10:10:41.567
2021-12-08 10:09:51.960
2021-12-08 10:10:54.657
and I want to get that day or today (now is 8 dec 2021) . So i have tried using EF in controller :
ViewBag.CByUserToday = _db.ArrayDatas.Where(a => a.CreatedBy == user && a.CreatedDate == DateTime.Today.Date).Count();
But i still did not get the rows. When i tried to debug DateTime.Today.Date , it's said DateTime.Today = {09/12/2021 00:00:00} . But when i tried to update that createddate to '2021-12-09 00:00:00.000' it can be retrieved.
So, how to retrieve that rows that i have created today(ignoring the time) ?

Check that the column is greater than or equal to today, and less than tomorrow:
DateTime today = DateTime.Today;
DateTime tomorrow = today.AddDays(1);
ViewBag.CByUserToday = _db.ArrayDatas
.Where(a => a.CreatedBy == user
&& a.CreatedDate >= today
&& a.CreatedDate < tomorrow)
.Count();
That way, your DBMS will be able to use a suitable index on the CreatedBy and CreatedDate columns to satisfy your query.
It's usually preferable to avoid calling functions on a column when you're trying to filter it, since this means the query is not SARGable. However, according to this thread on DBA, casting a datetime/datetime2 column to date is SARGable, at least in Microsoft SQL Server.

Why oracle entity framework is so slow? 700ms s vs 30 ms with raw SQL

db.Database.SqlQuery<int>(
"select sum(QUANTITY) from SomeTable
where USER_ID = :userid and Timestamp >= :someDate1and
and Timestamp < :someDate2",
userId, someDate1, someDate2
)
.First();
this takes 20 ms
var cnt = db.SomeTable.Where(x =>
x.User.Id == user.Id
&& x.Timestamp >= someDate1
&& x.Timestamp < someDate2
)
.Sum(x => x.Quantity);
This takes freaking 800 ms to execute
I set markers in the code before and after var start = DateTime.UtcNow; and then I log (DateTime.UtcNow - start).TotalMilliseconds.
If I check the generated SQL and execute it on the server directly it takes 2ms. So what is EF spending the rest 798ms for? To fetch one number?
Here is the generated SQL
SELECT "GroupBy1"."A1" AS "C1"
FROM
(SELECT SUM("Extent1"."QUANTITY") AS "A1"
FROM "FB"."SomeTable" "Extent1"
WHERE ((("Extent1"."USER_ID" = :p__linq__0)
OR (("Extent1"."USER_ID" IS NULL)
AND (:p__linq__0 IS NULL)))
AND ("Extent1"."TIMESTAMP" >= :p__linq__1)
AND ("Extent1"."TIMESTAMP" < :p__linq__2))
) "GroupBy1";

OK so as GSerg pointed the killer is OR (("Extent1"."USER_ID" IS NULL) coming from User_Id being nullable field in Oracle which itself comes from User property missing [Required] attribute. I suppose that's my fault partially, but EF is a slippery slope

SQL to linq query that uses local List<DateTime> as one of the "tables"

I want to do the sql query below using ORM..
We are calculating tickets open at 7 am in morning.
SQL query I have written
CREATE TABLE #Dates (ClosedDate datetime)
INSERT #Dates (ClosedDate) VALUES ('2014-12-21 07:00:00')
INSERT #Dates (ClosedDate) VALUES ('2014-12-22 07:00:00')
INSERT #Dates (ClosedDate) VALUES ('2014-12-23 07:00:00')
select #Dates.ClosedDate, count (t.ID) from tickets t
left join Status st on st.ID = t.StatusID
INNER JOIN #Dates ON ((st.Closed = 0) Or (st.Closed = 1 AND t.ClosedDate > #Dates.ClosedDate )) AND (t.CreatedDate < #Dates.ClosedDate )
DROP TABLE #Dates
LINQ query i have written:
List<DateTime> dates = new List<DateTime>();
for (int i = 0; i < totaldays; i++)
{
dates.Add(fromDate.AddDays(i));
}
var datesQuery = dates.AsQueryable();
var dateAndCounts = (
from t in tickets
join s in statuses
on ev.StatusID equals s.ID
join dat in dates
on (
(!s.Closed.Value || (s.Closed.Value && (ev.Closed >= dat)))
&& (t.Created < dat)
)
select new { dat, count(t.Event_ID) });
Join on date not working and I do not know how to do count on id. SQL is working fine.

Generally ORMs like linq2sql and EntityFramework don't play too nicely with mix-and-matched tables in memory and DB. (See this question for more info.)
The simplest way to get around this limitation is by just querying the database once for each date you're looking at. Since it's three, it's probably not a big deal to take a DB hit each time (unless this is in the middle of some highly performant system, in which case it can probably be solved with a little caching).
Just moving the query into the for loop and whereing on that individual DateTime, then aggregating the results manually should do the trick:
var results = new Dictionary<DateTime, int>();
for (int i = 0; i < totaldays; i++)
{
var date = fromDate.AddDays(i);
var tickets = (from t in tickets
join s in statuses on t.StatusID equals s.ID
where (!s.Closed.Value || (s.Closed.Value && t.Closed >= date)))
&& t.Created < date
).Count();
results[date] = tickets;
}
This then leaves you with a mapping of DateTimes to number of Tickets that match your filter for that particular time.

You're going to have to do this in two runs:
One get the data from the database,
Join the data from the database to the in-memory table.
You can pass the in-memory dates to the query, but if they are a series of sequential dates, it might be easier to just pass in the start and end-date.
I'd opt to grab all matching tickets in one go, not in a for loop, as that would potentially throw a lot of queries at the database:
var minDate = dates.Min();
var matchingTickets = (
from t in tickets
join s in statuses
on t.StatusId equals s.StatusId
where
(!s.Closed.HasValue || (s.Closed.HasValue && (s.Closed >= minDate)))
&& (t.Created < minDate)
select new { s.Closed, t.Created, t.EventId }).ToList();
Now that we have the tickets that match in memory, we can do the second pass and join them to the in-memory dates table. Since you want to join on an operation that is different than a simple equals we need to use the from syntax instead of a join syntax:
var datesAndCounts = from t in matchingTickets
from d in dates
where
(!t.Closed.HasValue || (t.Closed.HasValue && t.Closed >= d))
&& t.Created < d
group t by d into newGroup
orderby newGroup.Key
select newGroup;
This can be enumerated like:
foreach (var item in datesAndCounts)
{
Console.WriteLine(item.Key);
Console.WriteLine(item.Count());
}
A quick test application shows the results here on pastebin.

one query with groupby / count / select new

I have to make in c# a query with linq to sql. I can handle it in sql but in linq to
sql is the result not what I wanted to get.
So there is a table with:
a day, in datetime with date and time
and a kind of id
I have to count the ids for each date, the time isn't important. So the result
should be something like:
day: 2013-11-12 amountIDs: 4
People said to me, I can make a select new query and in this query I can set the day
and could count the ids, or I make a group by day. I read similar question, but it doesn't work in my case.
Could somebody help me?
I tried it with the statement below, but the days have to be grouped, so now the output is foreach datetime, like this
day: 12.12.2013 12:00:00 amountIDs: 1
day: 12.12.2013 12:10:10 amountIDs: 1
In sql I made this statement:
SELECT CONVERT(VARCHAR(20), data.dayandtime, 106) AS day, count(data.amountIds) as ids
FROM data
WHERE ( data.dayandtime >= DATEADD(day, -28, getdate()) AND (data.type = 100) AND (data.isSomething = 0) )
GROUP BY CONVERT(VARCHAR(20), data.dayandtime, 106), data.isSomthing and it works.
I saw similar cases, where people made a : from-select-new xyz statement, than I made a view of it and tried to group just the view. Like this
var query = data.GroupBy(g => g.day.Value).ToList();

var qry = from data in dbContext
group data by data.day into dataGrpd
select new
{
day= dataGrpd.Key,
amountIDs= dataGrpd.Select(x => x.Id).Distinct().Count()
};
Check This