EF core query group by date and subquery - c#

I have the following table:
CREATE TABLE "OrderStatusLogs" (
"Id" UNIQUEIDENTIFIER NOT NULL,
"OrderId" UNIQUEIDENTIFIER NOT NULL,
"Status" INT NOT NULL,
"StartDateTime" DATETIMEOFFSET NOT NULL,
"EndDateTime" DATETIMEOFFSET NULL DEFAULT NULL,
PRIMARY KEY ("Id"),
FOREIGN KEY INDEX "FK_OrderStatusLogs_Orders_OrderId" ("OrderId"),
CONSTRAINT "FK_OrderStatusLogs_Orders_OrderId" FOREIGN KEY ("OrderId") REFERENCES "Orders" ("Id") ON UPDATE NO_ACTION ON DELETE CASCADE
)
;
For the following entity:
[DebuggerDisplay(nameof(OrderStatusLog) + " {Status} {StartDateTime} - {EndDateTime}" )]
public class OrderStatusLog
{
public Guid Id { get; set; }
public Guid OrderId { get; set; }
public OrderStatus Status { get; set; }
public DateTimeOffset StartDateTime { get; set; }
public DateTimeOffset? EndDateTime { get; set; }
}
public enum OrderStatus
{
Unknown = 0,
Pending = 1,
Processing = 2,
Shipping = 3,
}
And i'm trying to generate a report which should show how many orders are set to a certain state for a given range.
For example, for the month oktober, we'd have the range 1 to 31 oktober.
The desired output would be something like this:
1/10/2021 Pending 21 orders
1/10/2021 Processing 23 orders
1/10/2021 Shipping 33 orders
1/10/2021 Unknown 0 orders
...
31/10/2021 Pending 1 orders
31/10/2021 Processing 3 orders
31/10/2021 Shipping 44 orders
31/10/2021 Unknown 5 orders
I'm having some difficulties writing a query in EF that would give me the right output. I can get things to work, but only client-side. I'm trying to make this work in the database instead.
So far i tried:
var logsByDayAndOrderId = orderStatusLogs.GroupBy(c => new { c.StartDateTime.Date, c.OrderId }, (key, values) => new
{
key.Date,
key.OrderId,
MaxStartDateTime = values.Max(x => x.StartDateTime)
});
var list = logsByDayAndOrderId.ToList();
var statusByDayAndOrderId = logsByDayAndOrderId.Select(c => new
{
c.Date,
c.OrderId,
orderStatusLogs.FirstOrDefault(x => x.StartDateTime == c.MaxStartDateTime && x.OrderId == c.OrderId).Status
});
//var statusByDayAndOrderId = logsByDayAndOrderId.Join(orderStatusLogs.def, inner => new { inner.OrderId, StartDateTime = inner.MaxStartDateTime }, outer => new { outer.OrderId, outer.StartDateTime }, (inner,outer) => new
//{
// inner.Date,
// inner.OrderId,
// outer.Status
//}); // TODO rem this query gives more results because of the join. we need an Outer join - but i could not get that to work. the version with select above works better, but then it does not use join so it may be slow(er).
var list1 = statusByDayAndOrderId.ToList();
var groupBy = statusByDayAndOrderId
.GroupBy(c => new { c.Date, c.Status })
.Select(c => new { c.Key.Date, c.Key.Status, Count = c.Count() });
var list2 = groupBy.ToList();
Another attempt:
var datesAndOrders = orderStatusLogs
.GroupBy(c => new { c.StartDateTime.Date, c.OrderId }, (key, values) => key);
var ordersByDateAndActiveStatusLog = orderStatusLogs
.Select(c => new
{
c.StartDateTime.Date,
c.OrderId,
ActiveStatusForDate = orderStatusLogs
.OrderByDescending(x => x.StartDateTime)
.FirstOrDefault(x => x.OrderId == c.OrderId && x.StartDateTime.Date == c.StartDateTime.Date)
.Status
});
var list = ordersByDateAndActiveStatusLog.ToList();
var orderCountByDateAndStatus = ordersByDateAndActiveStatusLog
.GroupBy(c => new { c.Date, c.ActiveStatusForDate }, (key, values) => new
{
key, count = values.Count()
});
var list1 = orderCountByDateAndStatus.ToList();
Both of these fail because of Cannot use an aggregate or a subquery in an expression used for the group by list of a GROUP BY clause..
This makes sense.
I'm hoping for someone that could help write a Linq query that generates the right data using ef core.
Notes:
I Solely use the fluent query syntax
I Have more places where i'd like to get data for each day so any other info or tips and tricks are welcome
I use net core 5 with ef core 5.0.11 with a MSSQL database

I would suggest to use EF Core extension linq2db.EntityFrameworkCore which has ability to work with local (in-memory) collections in database queries. Disclaimer: i'm one of the creators.
At first define function which generates days sequence:
public static IEnumerable<DateTime> GenerateDays(int year, int month)
{
var start = new DateTime(year, month, 1);
var endDate = start.AddMonths(1);
while (start < endDate)
{
yield return start;
start = start.AddDays(1);
}
}
Then we can use generated sequence in LINQ Query:
var days = GenerateDays(2021, 10).ToArray();
using var dc = ctx.CreateLinqToDbConnection();
var totalsQuery =
from d in days.AsQueryable(dc)
from l in orderStatusLogs.Where(l =>
(l.EndDateTime == null || l.EndDateTime >= d) && l.StartDateTime < d.AddDays(1))
.DefaultIfEmpty()
group l by new { Date = d, l.Status } into g
into g
select new
{
g.Key.Date,
g.Key.Status,
Count = g.Sum(x => x == null ? 0 : 1),
};
var result = totalsQuery.ToList();
The following SQL should be generated:
SELECT
[d].[item],
[e].[Status],
Sum(IIF([e].[OrderID] IS NULL, 0, 1))
FROM
(VALUES
('2021-05-01T00:00:00'), ('2021-05-02T00:00:00'),
('2021-05-03T00:00:00'), ('2021-05-04T00:00:00'),
('2021-05-05T00:00:00'), ('2021-05-06T00:00:00'),
('2021-05-07T00:00:00'), ('2021-05-08T00:00:00'),
('2021-05-09T00:00:00'), ('2021-05-10T00:00:00'),
('2021-05-11T00:00:00'), ('2021-05-12T00:00:00'),
('2021-05-13T00:00:00'), ('2021-05-14T00:00:00'),
('2021-05-15T00:00:00'), ('2021-05-16T00:00:00'),
('2021-05-17T00:00:00'), ('2021-05-18T00:00:00'),
('2021-05-19T00:00:00'), ('2021-05-20T00:00:00'),
('2021-05-21T00:00:00'), ('2021-05-22T00:00:00'),
('2021-05-23T00:00:00'), ('2021-05-24T00:00:00'),
('2021-05-25T00:00:00'), ('2021-05-26T00:00:00'),
('2021-05-27T00:00:00'), ('2021-05-28T00:00:00'),
('2021-05-29T00:00:00'), ('2021-05-30T00:00:00'),
('2021-05-31T00:00:00')
) [d]([item])
LEFT JOIN [OrderStatusLogs] [e] ON ([e].[EndDateTime] IS NULL OR [e].[EndDateTime] >= [d].[item]) AND [e].[StartDateTime] < DateAdd(day, 1, [d].[item])
GROUP BY
[d].[item],
[e].[Status]

Related

How do I minimize C#, LINQ code in Deleting Multiple Records

Here is a method to delete zero Inventory records from Inventory Table. I would like to reduce code/no of times that LINQ executes on Database.
Inventory Table
public class Inventory
{
public int itemCode { get; set; }
public decimal price { get; set; }
public decimal availQty { get; set; } // Can have Negative values.
}
example data
itemCode price availQty
1 10 10
1 12 -10
2 10 10
From above records, i want to delete all records of itemCode == 1, as net availQty is 0.
Here is my method
private void RemoveZeroInvs()
{
// Remove individual zero Inventorys
var rinvs = from ri in _context.Inventorys
where ri.availQty == 0
select ri;
_context.Inventorys.RemoveRange(rinvs);
_context.SaveChanges();
// Remove if group is zero in availQty, as it allows Negative Qty.
var result = from d in _context.Inventorys
group d by new
{
d.itemCode
}
into g
select new
{
g.Key.itemCode,
availQty = g.Sum(y => y.availQty)
};
var zrs = from r in result
where r.availQty == 0
select r;
foreach (var zr in zrs) // Here, zrs length may be more than 500
{
var ri = _context.Inventorys.Where(w => w.itemCode == zr.itemCode);
_context.Inventorys.RemoveRange(ri);
_context.SaveChanges();
}
}
I use Asp.Net Core 2.2. Is there any such possibility?
Also I get following error at line _context.Inventorys.RemoveRange(ri); in the loop.
A command is already in progress: SELECT t."itemCode", t."availQty"
FROM (
SELECT d."itemCode", SUM(d."availQty") AS "availQty"
FROM "Inventorys" AS d
GROUP BY d."itemCode"
) AS t
var todelete = _context.Inventorys
.GroupBy(i => i.itemCode)
.Where(g => g.Sum(i => i.availQty) == 0)
.SelectMany(g => g);
Here is a shorter versions of your code, in terms of DB excecution, one would have to compare the raw queries. But it may be lighter that your codeā€¦

Returning a nested dictionary within another dictionary with Linq

I have a little problem with creating a complex Linq query. I have the following tables:
Activities
----------
- Date : DateTime
- ProjectId : int
Projects
--------
- ProjectId
- ProjectNumber
I need to construct a query that returns a dictionary with the distinct years within the Activities table as keys. The value should be another dictionary containing all distinct months within the current distinct year as keys and then for each distinct month, I need a list of strings containing all the project numbers for that month.
So I would end up with something like this:
- 2014 //First distinct year
- 1 //January
- Contoso-2014-01 //Project number
- 3 //March
- IBM-2014-06 //Project number
- 2016 //Second distinct year
- 4 //April
- HP-2016-02 //Project number
Basically, we would have a dictionary containing two keys: 2014 and 2016
The values for the key 2014 would be a Dictionary with two KeyValuePairs. One with the key 1 and another with the key 3. Key 1 would have a list of strings as it's values containing the project number "Contoso-2014-01" and key 3 would contain "IBM-2014-06". And so on...
So now to my question: Is it even possible to query a database and get a return type structure like this? If yes, how can I achieve this?
Considering you have classes such as these
public class Activities
{
public DateTime Date { get; set; }
public Project Project { get; set; }
}
public class Project
{
public int ProjectId { get; set; }
public string ProjectNumber { get; set; }
}
Then I think this should work
public static void GetData()
{
var mainDic = new Dictionary<int, Dictionary<int, List<string>>>();
List<Activities> acts = new List<Activities>(); // Your database context.
acts.Select(x => x.Date.Year).Distinct().ToList().ForEach(
year =>
{
var yearlyDic = new Dictionary<int, List<string>>();
acts.Where(x => x.Date.Year == year).Select(x => x.Date.Month).Distinct().ToList().ForEach(
month =>
{
var projects = acts.Where(x => x.Date.Year == year && x.Date.Month == month)
.Select(x => x.Project.ProjectNumber).ToList();
yearlyDic.Add(month, projects);
});
mainDic.Add(year, yearlyDic);
});
}
I am assuming you are using SQL Server.. Here filtering Year part and Month part in the db query itself.
select YEAR(ac.[Date]) as projectYear, MONTH(ac.[Date]) as projectMonth ,pr.ProjectNumber Info from Activities ac join Projects pr on ac.ProjectId=pr.ProjectId
class ProjectActivity
{
public int Year { get; set; }
public int Month { get; set; }
public string ProjectNumber { get; set; }
public static List<ProjectActivity> GetProjectActivities()
{
//You can use above query and construct the list.
var sampleProjectActivities = new List<ProjectActivity>();
var projActivitySamp1 = new ProjectActivity()
{
Year = 2014,
Month = 1,
ProjectNumber = "Contoso-2014-01"
};
sampleProjectActivities.Add(projActivitySamp1);
var projActivitySamp2 = new ProjectActivity()
{
Year = 2014,
Month = 3,
ProjectNumber = "Contoso-2014-03"
};
sampleProjectActivities.Add(projActivitySamp2);
var projActivitySamp3 = new ProjectActivity()
{
Year = 2016,
Month = 4,
ProjectNumber = "HP-2016-02"
};
sampleProjectActivities.Add(projActivitySamp3);
var projActivitySamp4 = new ProjectActivity()
{
Year = 2016,
Month = 4,
ProjectNumber = "AnotherHP-2016-04"
};
sampleProjectActivities.Add(projActivitySamp4);
return sampleProjectActivities;
}
}
And call the code like this
var sampleProjectActivities = ProjectActivity.GetProjectActivities();
var result = sampleProjectActivities.GroupBy(projectActivity => projectActivity.Year)
.ToDictionary(k => k.Key,
v =>
{
return v.GroupBy(val => val.Month).ToDictionary(a => a.Key, b => b.Select(x => x.ProjectNumber).ToArray());
});
Assuming that your classes are named as you have mentioned (in the tables)..
This lambda should give you the dictionary that you need..
var dictionary = acts.GroupBy(activity => activity.Date.Year) // gives you year-wise groups
.ToDictionary(yearGroup => yearGroup.Key,
yearGroup => yearGroup.ToDictionary(activity => activity.Date.Month, // gives you month-wise groups
activity => yearGroup.Where(a => a.Date.Month == activity.Date.Month)
.Select(a => a.Project.ProjectNumber)
.ToList() // all the project numbers under this year and month
));
This seems to be the most straight-forward way to go:
Dictionary<int, Dictionary<int, string>> query =
(
from a in activities
join p in projects on a.ProjectId equals p.ProjectId
group new
{
a.Date.Month,
p.ProjectNumber,
} by a.Date.Year into gaps
select new
{
gaps.Key,
Value = gaps.ToDictionary(x => x.Month, x => x.ProjectNumber),
}
).ToDictionary(x => x.Key, x => x.Value);

Count and Groupby using linq from sql query

Im trying to create a table that counts all orders and groups them in a table from sql to linq to use in a bar graph with google charts.
Table`
Orders Status
8 Created
3 Delayed
4 Enroute
sql
SELECT Count (OrderID) as 'Orders', order_status FROM [ORDER]
where order_status ='Created'OR order_status= 'Delayed' OR order_status='Enroute'
group by order_status
controller
public ActionResult GetChart()
{
var Orders = db.Order.Select(a => new { a.OrderID, a.order_status })
.GroupBy(a => a.order_status);
return Json(Orders, JsonRequestBehavior.AllowGet);
}
this is not displaying the correct results as the linq seems to be wrong.
can someone please point me in the right direction? I am relatively new to this.
thanks in advance.
This should work:-
var result = db.Order.Where(x => x.order_status == "Created"
|| x.order_status == "Delayed"
|| x.order_status == "Enroute")
.GroupBy(x => x.order_status)
.Select(x => new
{
order_status = x.Key,
Orders = x.Count()
});
Or if you prefer query syntax then:-
var result = from o in db.Order
where o.order_status == "Created" || o.order_status == "Delayed"
|| o.order_status == "Enroute"
group o by o.order_status
select new
{
orderStatus = x.Key,
Counts = x.Count()
};
I think you want to group by Status and count total number of orders in each group (I build a simple console program to demonstrate). I suppose the data is:
Orders Status
8 Created
3 Delayed
4 Enroute
2 Created
1 Delayed
Order.cs
public class Order
{
public Order(int orderId, string status)
{
OrderId = orderId;
Status = status;
}
public int OrderId { get; set; }
public string Status { get; set; }
}
Program.cs
class Program
{
static void Main(string[] args)
{
// Data
var orders = new List<Order>
{
new Order(8, "Created"),
new Order(3, "Delayed"),
new Order(4, "Enroute"),
new Order(2, "Created"),
new Order(1, "Delayed"),
};
// Query
var query = orders
.GroupBy(x => x.Status)
.Select(x => new {Status = x.Key, Total = x.Count()});
// Display
foreach (var item in query)
{
Console.WriteLine(item.Status + ": " + item.Total);
}
Console.ReadLine();
}
}
The one you need to focus in is query. After using GroupBy, you will have a list of groups. For each group, the Key is the criteria to group (here is the Status). Then, we call Count() to get the total number of element in that group.
So, from the program above, the output should be:
Created: 2
Delayed: 2
Enroute: 1

returning specified type for a linq query with anonymous type

i have this query:
DbQuery<Logs> logs = context.GetQuery<Logs>();
var MessageLogs =
logs.Where(
s =>
s.DATE == date.Date
.GroupBy(s => new {s.DATE, s.ID})
.Select(
g => new {Date = g.Key.DATE, SID = g.Key.ID, Count = g.Count()})
.GroupBy(x => x.SID, x => new {x.Date, x.Count});
and I have these two classess:
public class Data
{
public Values[] Val { get; set; }
public string Key { get; set; }
}
and this:
public class Values
{
public string type1 { get; set; }
public string type2 { get; set; }
}
all i want to do is using that query to return type of Data.
key in class Data is SID and list of values should be counts and date as type1 and type2.
i know i can do this with anonymous type but i dont know how, i tried many ways but all of them was wrong.
EDIT:
i have this query
logs.Where(
s =>
s.DATE == date.Date
.GroupBy(s => new {s.DATE, s.ID})
.Select(
g => new {Date = g.Key.DATE, SID = g.Key.ID, Count = g.Count()})
this query returns something like this:
key date count
----------------------------
1021 2012 1
1021 2013 5
1022 2001 10
1023 2002 14
what i want is base on each id a list of values
in fact return type should be type of Data which this ids are key fore example
key=1021 and Values[] should be type1=2012, type2=1 and type1=2013, type2=5
Given that your current query returns elements with key/date/count, it sounds like you probably just want:
var result = query.GroupBy(
x => x.Key,
(key, rows) => new Data {
Key = key,
Val = rows.Select(r => new Values { type1 = r.Date, type2 = r.Count })
.ToArray();
});
Basically this overload takes:
A source
A key selector
A transformation from a key and matching rows to a result element (an instance of Data in your case)

how to get an ordered list with default values using linq

I have an ICollection of records (userID,itemID,rating) and an IEnumerable items
for a specific userID and each itemID from a set of itemIDs, i need to produce a list of the users rating for the items or 0 if no such record exists. the list should be ordered by the items.
example:
records = [(1,1,2),(1,2,3),(2,3,1)]
items = [3,1]
userID = 1
result = [0,2]
my attempt:
dataset.Where((x) => (x.userID == uID) & items.Contains(x.iID)).Select((x) => x.rating);
it does the job but it doesn't return 0 as default value and it isnt ordered...
i'm new to C# and LINQ, a pointer in the correct direction will be very appreciated.
Thank you.
This does the job:
var records = new int[][] { new int[] { 1, 1, 2 }, new int[] { 1, 2, 3 }, new int[] { 2, 3, 1 } };
var items = new int[] { 3, 1 };
var userId = 1;
var result = items.Select(i =>
{
// When there's a match
if (records.Any(r => r[0] == userId && r[1] == i))
{
// Return all numbers
return records.Where(r => r[0] == userId && r[1] == i).Select(r => r[2]);
}
else
{
// Just return 0
return new int[] { 0 };
}
}).SelectMany(r => r); // flatten the int[][] to int[]
// output
result.ToList().ForEach(i => Console.Write("{0} ", i));
Console.ReadKey(true);
How about:
dataset.Where((x) => (x.userID == uID)).Select((x) => items.Contains(x.iID) ? x.rating : 0)
This does the job. But whether it's maintainable/readable solution is topic for another discussion:
// using your example as pseudo-code input
var records = [(1,1,2),(1,2,3),(2,3,1)];
var items = [3,1];
var userID = 1;
var output = items
.OrderByDescending(i => i)
.GroupJoin(records,
i => i,
r => r.ItemId,
(i, r) => new { ItemId = i, Records = r})
.Select(g => g.Records.FirstOrDefault(r => r.UserId == userId))
.Select(r => r == null ? 0 : r.Rating);
How this query works...
ordering is obvious
the ugly GroupJoin - it joins every element from items with all records that share same ItemId into annonymous type {ItemId, Records}
now we select first record for each entry that matches userId - if none is found, null will be returned (thanks to FirstOrDefault)
last thing we do is check whether we have value (we select Rating) or not - 0
How about this. your question sounds bit like an outer join from SQL, and you can do this with a GroupJoin, SelectMany:
var record1 = new Record() { userID = 1, itemID = 1, rating = 2 };
var record2 = new Record() { userID = 1, itemID = 2, rating = 3 };
var record3 = new Record() { userID = 2, itemID = 3, rating = 1 };
var records = new List<Record> { record1, record2, record3 };
int userID = 1;
var items = new List<int> { 3, 1 };
var results = items
.GroupJoin( records.Where(r => r.userID == userID), item => item, record => record.itemID, (item, record) => new { item, ratings = record.Select(r => r.rating) } )
.OrderBy( itemRating => itemRating.item)
.SelectMany( itemRating => itemRating.ratings.DefaultIfEmpty(), (itemRating, rating) => rating);
To explain what is going on
For each item GroupJoin gets the list of rating (or empty list if no rating) for the specified user
OrderBy is obvious
SelectMany flattens the ratings lists, providing a zero if the ratings list is empty (by DefaultIfEmpty)
Hope this makes sense.
Be aware, if there is more than one rating for an item by a user, they will all appear in the final list.

Categories

Resources