I have two tables (one-to-many). MeterReadings(0..1) and MeterReadingDetails(*)
I want to join these tables and group by date. Date field is in MeterReadings and Others are in MeterReadingDetails.
I used this code:
Linq
public static IEnumerable<MeterReadingsForChart> GetCustomerTotal(int CustomerId, int MeterTypeId, DateTime StartDate, DateTime EndDate, MeterReadingsTimeIntervals DateRangeType)
{
var customerReadings = from m in entity.MeterReadings
join n in entity.MeterReadingDetails on m.sno equals n.ReadingId
where m.Meters.CustomerId == CustomerId && m.ReadDate >= StartDate && m.ReadDate <= EndDate && m.Meters.TypeId == MeterTypeId
group n by new { date = new DateTime(m.ReadDate.Value.Year, m.ReadDate.Value.Month, 1) } into g
select new MeterReadingsForChart
{
ReadDate = g.Key.date,
Value = g.Sum(x => x.Value),
Name = g.FirstOrDefault().MeterReadingTypes.TypeName
};
return customerReadings;
}
MeterReadinsForChart.cs
public class MeterReadingsForChart
{
public DateTime ReadDate { get; set; }
public string Name { get; set; }
public double Value { get; set; }
}
But I got this error:
Only parameterless constructors and initializers are supported in LINQ to Entities
How can I join, group, and sum?
Try the following:
var customerReadings = (from m in entity.MeterReadings
join n in entity.MeterReadingDetails on m.sno equals n.ReadingId
where m.Meters.CustomerId == CustomerId && m.ReadDate >= StartDate && m.ReadDate <= EndDate && m.Meters.TypeId == MeterTypeId
group n by new { Year = m.ReadDate.Value.Year, Month = m.ReadDate.Value.Month} into g
select new
{
Key = g.Key,
Value = g.Sum(x => x.Value),
Name = g.FirstOrDefault().MeterReadingTypes.TypeName
}).AsEnumerable()
.Select(anon => new MeterReadingsForChart
{
ReadDate = new DateTime(anon.Key.Year, anon.Key.Month, 1),
Value = anon.Value,
Name = anon.Name
});
Unf. its ugly, but entity framework won't let you create a DateTime (being a struct it has no parameterless constructors). So in this case we want most of the result from the db and then as this streams we construct the date in memory.
Related
My Entity structure is as follows:
public class DisbursementItem
{
public int DisbursementNumber;
public int IDDisbursementItem;
public int IDReceiptItem;
public decimal Amount;
public decimal MeasureUnit;
public decimal PricePerMU;
public decimal PriceTotal;
public Disbursement Disbursement_IDDisbursement;
public int IDDisbursementNumber;
}
public class Disbursement
{
public int DisbursementNumber;
DateTime date;
DisbursementType DType;
string Note;
string Subscriber;
Subscriber SubscriberModel;
string ItemType;
int ProcessNumber;
}
public class Subscriber
{
public string Name
public string Address;
public string City;
}
public class DisbursementDescription
{
public int IDDisbursementItem;
public string Description;
}
public class Receipt
{
public int IDReceiptItem;
public int ItemNumber;
}
public class StorageCard
{
public int ItemNumber;
public string StorageCardGroup;
public string StorageCardName;
}
And my EF6 LINQ query is:
DateTime from;
DateTime to;
var result = context.DisbursementItem
.Where(x => x.Disbursement_IDDisbursement.Date <= to && x.Disbursement_IDDisbursement.Date >= from)
.Join(context.DisbursementDescription, di => di.IDDisbursementItem, dd => dd.IDDisbursementItem, (di, dd) => new {di = di, desc = dd.Description})
.Join(context.Receipt, x => x.di.IDReceiptItem, r => r.IDReceiptItem, (x, r) => new { di = x.di, desc = x.desc, r = r })
.Join(context.StorageCard, x => x.r.ItemNumber, sc => sc.ItemNumber, (x, sc) => new { di = x.di, desc = x.desc, r = x.r, sc = sc})
.GroupBy(g => new {g.di.DisbursementNumber, g.sc.ItemNumber, g.di.MeasureUnit})
.Select(x => new
{
Date = x.FirstOrDefault().di.Disbursement_IDDisbursement.Date,
DisbursementNumber = x.Key.DisbursementNumber,
DType = x.FirstOrDefault().di.Disbursement_IDDisbursement.DType,
Note = x.FirstOrDefault().di.Disbursement_IDDisbursement.Note,
Subscriber = x.FirstOrDefault().di.Disbursement_IDDisbursement.Subscriber,
SubscriberName = x.FirstOrDefault().di.Disbursement_IDDisbursement.SubscriberModel.Name,
SubscriberAddress = x.FirstOrDefault().di.Disbursement_IDDisbursement.SubscriberModel.Address,
SubscriberCity = x.FirstOrDefault().di.Disbursement_IDDisbursement.SubscriberModel.City,
ItemNumber = x.FirstOrDefault().sc.ItemNumber,
StorageCardGroup = x.FirstOrDefault().sc.StorageCardGroup,
StorageCardName = x.FirstOrDefault().sc.StorageCardName,
Amount = x.Sum(y => y.di.Amount),
PricePerMU = x.FirstOrDefault().di.PricePerMU,
PriceTotal = x.Sum(y => y.di.PriceTotal),
MeasureUnit = x.Key.MeasureUnit
Desc = x.FirstOrDefault().desc,
})
SELECT
di.Date,
di.DisbursementNumber,
d.DType,
d.Note,
d.Subscriber,
subs.Name,
subs.Address,
subs.City,
sc.ItemNumber,
sc.StorageCardGroup,
sc.StorageCardName,
Sum(di.Amount) as Amount,
di.PricePerMU,
Sum(di.PriceTotal) as PriceTotal,
di.MeasureUnit,
dd.Description
FROM
DisbursementItem as di
INNER JOIN Disbursement as d
ON di.IDDisbursementNumber = d.DisbursementNumber
INNER JOIN Receipt as r
ON di.IDReceiptItem = r.IDReceiptItem
INNER JOIN StorageCard as sc
ON r.ItemNumber = sc.ItemNumber
INNER JOIN DisbursementDescription dd
ON di.IDDisbuzrsementItem = dd.IDDisbursementItem
WHERE
di.Date <= ... and di.Date >= ...
GROUP BY
di.DisbursementNumber, sc.ItemNumber, di.MeasureUnit
That is the query in SQL that I want to achieve in EF
This query can take over a minute for a few hundred rows. How can I optimize it? I suspect the multiple joins is a problem and maybe also the Sum of some fields.
Also the database schema cannot be modified.
The query it generate is enormous. It's like a SELECT in SELECT in SELECT for like 40 times.
Easiest way is to add all fields which are needed for result to grouping key. Rewritten query to Query syntax for readability and maintainability:
DateTime from;
DateTime to;
var query =
from di in context.DisbursementItem
where di.Disbursement_IDDisbursement.Date <= to && di.Disbursement_IDDisbursement.Date >= from
join dd in context.DisbursementDescription on di.IDDisbursementItem equals dd.IDDisbursementItem
join r in context.Receipt on di.IDReceiptItem equals r.IDReceiptItem
join sc in context.StorageCard on r.ItemNumber equals sc.ItemNumber
group di by new
{
di.DisbursementNumber,
sc.ItemNumber,
di.MeasureUnit,
di.Disbursement_IDDisbursement.Date,
di.Disbursement_IDDisbursement.DType,
di.Disbursement_IDDisbursement.Note,
Subscriber = di.Disbursement_IDDisbursement.Subscriber,
SubscriberName = di.Disbursement_IDDisbursement.SubscriberModel.Name,
SubscriberAddress = di.Disbursement_IDDisbursement.SubscriberModel.Address,
SubscriberCity = di.Disbursement_IDDisbursement.SubscriberModel.City,
sc.ItemNumber,
sc.StorageCardGroup,
sc.StorageCardName,
di.PricePerMU,
Desc = dd.Description
} into g
select new
{
g.Key.Date,
g.Key.DisbursementNumber,
g.Key.DType,
g.Key.Note,
g.Key.Subscriber,
g.Key.SubscriberName,
g.Key.SubscriberAddress,
g.Key.SubscriberCity,
g.Key.ItemNumber,
g.Key.StorageCardGroup,
g.Key.StorageCardName,
g.Key.PricePerMU,
g.Key.MeasureUnit,
g.Key.Desc,
Amount = g.Sum(x => x.Amount),
PriceTotal = g.Sum(x => x.PriceTotal)
}
you could try some kind of multithreading
the query could be splitted in parts and each part assigned to a task. in here you should find something useful (parallel section):
https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks?view=net-6.0
I have one table in database named Balance and a list of dates as follows:
List<string> allDates = { "2021-01-02", "2021-01-03", "2021-01-04" }
Balance table:
Id, Amount, BalanceDate
1, 233, "2021-01-02"
2, 442, "2021-01-03
I need to fetch the records in Balance table with amount 0 for the missing dates. For example:
233, "2021-01-02"
442, "2021-01-03"
0, "2021-01-04"
I have tried the following:
balnces.GroupJoin(allDates,
balance => balance.Date,
d => d,
(balance, d) => balance);
But the records are still the same (only the ones in the balance table)
Given a data structure from database:
private class balance
{
public int id { get; set; }
public double amount { get; set; }
public string date { get; set; }
}
You get your data as you want (this is only a mock-up)
List<string> allDates = new List<string> { "2021-01-02", "2021-01-03", "2021-01-04" };
List<balance> balances = new List<balance>();
balances.Add(new balance { id = 1, amount = 233 , date = "2021-01-02" });
balances.Add(new balance { id = 2, amount = 442, date = "2021-01-03" });
you can get your desired result this way:
List<balance> result = allDates.Select(d=>
new balance {
amount =
balances.Any(s=> s.date == d)?
balances.FirstOrDefault(s => s.date == d).amount:0,
date = d
}).ToList();
If your default contains a 0 in amount instead a null, you can skip the .Any check
Assumption
Balance query had been materialized and data are returned from the database.
Solution 1: With .DefaultIfEmpty()
using System.Linq;
var result = (from a in allDates
join b in balances on a equals b.Date.ToString("yyyy-MM-dd") into ab
from b in ab.DefaultIfEmpty()
select new { Date = a, Amount = b != null ? b.Amount : 0 }
).ToList();
Sample Program for Solution 1
Solution 2: With .ToLookup()
var lookup = balances.ToLookup(x => x.Date.ToString("yyyy-MM-dd"));
var result = (from a in allDates
select new
{
Date = a,
Amount = lookup[a] != null && lookup[a].Count() > 0 ? lookup[a].First().Amount : 0
}
).ToList();
Sample Program for Solution 2
I have the following table:
CREATE TABLE "OrderStatusLogs" (
"Id" UNIQUEIDENTIFIER NOT NULL,
"OrderId" UNIQUEIDENTIFIER NOT NULL,
"Status" INT NOT NULL,
"StartDateTime" DATETIMEOFFSET NOT NULL,
"EndDateTime" DATETIMEOFFSET NULL DEFAULT NULL,
PRIMARY KEY ("Id"),
FOREIGN KEY INDEX "FK_OrderStatusLogs_Orders_OrderId" ("OrderId"),
CONSTRAINT "FK_OrderStatusLogs_Orders_OrderId" FOREIGN KEY ("OrderId") REFERENCES "Orders" ("Id") ON UPDATE NO_ACTION ON DELETE CASCADE
)
;
For the following entity:
[DebuggerDisplay(nameof(OrderStatusLog) + " {Status} {StartDateTime} - {EndDateTime}" )]
public class OrderStatusLog
{
public Guid Id { get; set; }
public Guid OrderId { get; set; }
public OrderStatus Status { get; set; }
public DateTimeOffset StartDateTime { get; set; }
public DateTimeOffset? EndDateTime { get; set; }
}
public enum OrderStatus
{
Unknown = 0,
Pending = 1,
Processing = 2,
Shipping = 3,
}
And i'm trying to generate a report which should show how many orders are set to a certain state for a given range.
For example, for the month oktober, we'd have the range 1 to 31 oktober.
The desired output would be something like this:
1/10/2021 Pending 21 orders
1/10/2021 Processing 23 orders
1/10/2021 Shipping 33 orders
1/10/2021 Unknown 0 orders
...
31/10/2021 Pending 1 orders
31/10/2021 Processing 3 orders
31/10/2021 Shipping 44 orders
31/10/2021 Unknown 5 orders
I'm having some difficulties writing a query in EF that would give me the right output. I can get things to work, but only client-side. I'm trying to make this work in the database instead.
So far i tried:
var logsByDayAndOrderId = orderStatusLogs.GroupBy(c => new { c.StartDateTime.Date, c.OrderId }, (key, values) => new
{
key.Date,
key.OrderId,
MaxStartDateTime = values.Max(x => x.StartDateTime)
});
var list = logsByDayAndOrderId.ToList();
var statusByDayAndOrderId = logsByDayAndOrderId.Select(c => new
{
c.Date,
c.OrderId,
orderStatusLogs.FirstOrDefault(x => x.StartDateTime == c.MaxStartDateTime && x.OrderId == c.OrderId).Status
});
//var statusByDayAndOrderId = logsByDayAndOrderId.Join(orderStatusLogs.def, inner => new { inner.OrderId, StartDateTime = inner.MaxStartDateTime }, outer => new { outer.OrderId, outer.StartDateTime }, (inner,outer) => new
//{
// inner.Date,
// inner.OrderId,
// outer.Status
//}); // TODO rem this query gives more results because of the join. we need an Outer join - but i could not get that to work. the version with select above works better, but then it does not use join so it may be slow(er).
var list1 = statusByDayAndOrderId.ToList();
var groupBy = statusByDayAndOrderId
.GroupBy(c => new { c.Date, c.Status })
.Select(c => new { c.Key.Date, c.Key.Status, Count = c.Count() });
var list2 = groupBy.ToList();
Another attempt:
var datesAndOrders = orderStatusLogs
.GroupBy(c => new { c.StartDateTime.Date, c.OrderId }, (key, values) => key);
var ordersByDateAndActiveStatusLog = orderStatusLogs
.Select(c => new
{
c.StartDateTime.Date,
c.OrderId,
ActiveStatusForDate = orderStatusLogs
.OrderByDescending(x => x.StartDateTime)
.FirstOrDefault(x => x.OrderId == c.OrderId && x.StartDateTime.Date == c.StartDateTime.Date)
.Status
});
var list = ordersByDateAndActiveStatusLog.ToList();
var orderCountByDateAndStatus = ordersByDateAndActiveStatusLog
.GroupBy(c => new { c.Date, c.ActiveStatusForDate }, (key, values) => new
{
key, count = values.Count()
});
var list1 = orderCountByDateAndStatus.ToList();
Both of these fail because of Cannot use an aggregate or a subquery in an expression used for the group by list of a GROUP BY clause..
This makes sense.
I'm hoping for someone that could help write a Linq query that generates the right data using ef core.
Notes:
I Solely use the fluent query syntax
I Have more places where i'd like to get data for each day so any other info or tips and tricks are welcome
I use net core 5 with ef core 5.0.11 with a MSSQL database
I would suggest to use EF Core extension linq2db.EntityFrameworkCore which has ability to work with local (in-memory) collections in database queries. Disclaimer: i'm one of the creators.
At first define function which generates days sequence:
public static IEnumerable<DateTime> GenerateDays(int year, int month)
{
var start = new DateTime(year, month, 1);
var endDate = start.AddMonths(1);
while (start < endDate)
{
yield return start;
start = start.AddDays(1);
}
}
Then we can use generated sequence in LINQ Query:
var days = GenerateDays(2021, 10).ToArray();
using var dc = ctx.CreateLinqToDbConnection();
var totalsQuery =
from d in days.AsQueryable(dc)
from l in orderStatusLogs.Where(l =>
(l.EndDateTime == null || l.EndDateTime >= d) && l.StartDateTime < d.AddDays(1))
.DefaultIfEmpty()
group l by new { Date = d, l.Status } into g
into g
select new
{
g.Key.Date,
g.Key.Status,
Count = g.Sum(x => x == null ? 0 : 1),
};
var result = totalsQuery.ToList();
The following SQL should be generated:
SELECT
[d].[item],
[e].[Status],
Sum(IIF([e].[OrderID] IS NULL, 0, 1))
FROM
(VALUES
('2021-05-01T00:00:00'), ('2021-05-02T00:00:00'),
('2021-05-03T00:00:00'), ('2021-05-04T00:00:00'),
('2021-05-05T00:00:00'), ('2021-05-06T00:00:00'),
('2021-05-07T00:00:00'), ('2021-05-08T00:00:00'),
('2021-05-09T00:00:00'), ('2021-05-10T00:00:00'),
('2021-05-11T00:00:00'), ('2021-05-12T00:00:00'),
('2021-05-13T00:00:00'), ('2021-05-14T00:00:00'),
('2021-05-15T00:00:00'), ('2021-05-16T00:00:00'),
('2021-05-17T00:00:00'), ('2021-05-18T00:00:00'),
('2021-05-19T00:00:00'), ('2021-05-20T00:00:00'),
('2021-05-21T00:00:00'), ('2021-05-22T00:00:00'),
('2021-05-23T00:00:00'), ('2021-05-24T00:00:00'),
('2021-05-25T00:00:00'), ('2021-05-26T00:00:00'),
('2021-05-27T00:00:00'), ('2021-05-28T00:00:00'),
('2021-05-29T00:00:00'), ('2021-05-30T00:00:00'),
('2021-05-31T00:00:00')
) [d]([item])
LEFT JOIN [OrderStatusLogs] [e] ON ([e].[EndDateTime] IS NULL OR [e].[EndDateTime] >= [d].[item]) AND [e].[StartDateTime] < DateAdd(day, 1, [d].[item])
GROUP BY
[d].[item],
[e].[Status]
I need some help, i have a one working join, need the other one for third table? How can i create it? My orderby does not work either with year and need some help also. This is my logic as below and using Linq in sql;
// controller
public IList<ExtractionViewModel> GetExtractionViewModels()
{
ProductionManagementEntities db = new ProductionManagementEntities();
var scheduleList = (from p in db.ProductionDays
join w in db.Weeks on p.WeekId equals w.WeekId
// need other join here for the second table
orderby w.Year ascending // this is not working, year starts in 2017 instead of 2021 downwards
where(w.WeekNum == 9)
select new ExtractionViewModel
{
Year = w.Year,
Week = w.WeekNum,
Day = p.ProductionDate,
}).ToList();
return scheduleList;
}
// Model
public class ExtractionViewModel
{
public string Year { get; set; }
public int Week { get; set; }
public DateTime Day { get; set; }
public string VW250 { get; set; }
public string VW270 { get; set; }
public string VW250_2PA { get; set; }
public string VW_270_PA { get; set; }
}
//Controller
public IList<ExtractionViewModel> GetExtractionViewModels()
{
ProductionManagementEntities db = new ProductionManagementEntities();
var scheduleList = (from p in db.ProductionDays
from m in db.Models
join w in db.Weeks on p.WeekId equals w.WeekId
orderby w.Year descending
orderby m.Name ascending
where(m.Name== "VW250")
where(w.WeekNum == 9)
select new ExtractionViewModel
{
Year = w.Year,
Week = w.WeekNum,
Day = p.ProductionDate,
VW250 = m.Name
}).ToList();
return scheduleList;
}
Using a Linq query, this should work:
var scheduleList = (from p in db.ProductionDays
join w in db.Weeks on p.WeekId equals w.WeekId
join n in db.NewTable on p.WeekId equals n.WeekId
where w.WeekNum equals 9 and m.Name equals "VW250"
orderby w.Year ascending
select new ExtractionViewModel
{
Year = w.Year,
Week = w.WeekNum,
Day = p.ProductionDate,
Property = n.Property
}).ToList();
A simpler way, and also seems to run a bit quicker as well, is:
var scheduleList = db.ProductionDays
.Include(x => x.Weeks)
.Include(x => x.NewTable)
.Where(x => x.Week.WeekNum == 9)
.OrderBy(x => x.Week.Year)
.Select(x => new ExtractionViewModel {
x.Week.Year,
x.Week.WeekNum,
x.ProductionDate,
x.NewTable.Property
})
.ToList();
The second one is linq method and when debugging and stepping through I notice that they seem to be quicker than linq queries.
The problem with your query seemed to be the syntax on the where clause. You had where(w.WeekNum == 9) which may work but I have never seen that syntax. With linq, I have only worked with lambda expressions or where property equals value type syntax. I haven't tested this so if there is an error you will probably need to move the .OrderBy() statement to the bottom, but it should be fine.
In the question you don't mention what third table you would like to join but indicate that there is a third table. I added NewTable and NewTable.Property to indicate the third table and one of its columns/Properties.
How can I write the following LEFT OUTER JOIN SQL query against my Calendar and Sales tables for the purpose of grouping summed sales by day, week or month in LINQ so that it can be materialised by LINQ-to-SQL?
SELECT c.CalendarDate, c.FirstDayOfWeek, c.FirstDayOfMonth,
ISNULL(s.Total, 0) as Total
FROM Calendar as c
LEFT OUTER JOIN Sales as s
on s.SaleDate >= c.CalendarDateTime and
s.SaleDate < c.NextDayDateTime
WHERE s.SaleDate BETWEEN #since and #until
I managed to get an inner join working in LINQ, but I need an outer join to retrieve days with zero sales. Here is the code I use for an inner join:
var sales = from s in db.Sales
from c in db.Calendars
where
s.SaleDate >= c.CalendarDate && s.SaleDate < c.NextDayDateTime
&& s.SaleDate >= sinceDate && s.SaleDate < dateEnd
select new
{
c.CalendarDate,
c.FirstDateOfWeek,
c.FirstDateOfMonth,
s.Total
};
I can then switch on a date interval and group sales as follows:
Daily:
var groupedSales = sales.GroupBy(x => x.CalendarDate);
Weekly:
var groupedSales = sales.GroupBy(x => x.FirstDateOfWeek);
Monthly:
var groupedSales = sales.GroupBy(x => x.FirstDateOfMonth);
Finally:
var salesReport = from g in groupedSales
orderby g.Key
select new {
Date = g.Key,
Total = g.Sum(x => x.Total)
};
Alternatively, it could also work to inject zero sale records into my report after retrieving sales for non-zero days only.
What about this?
var sales = from s in db.Sales
from c in db.Calendars.DefaultIfEmpty()
where
s.SaleDate >= c.CalendarDate && s.SaleDate < c.NextDayDateTime
&& s.SaleDate >= sinceDate && s.SaleDate < dateEnd
EDIT:
You can create helper classes like this:
class CalendarSealesHelper
{
public DateTime CalendarDate {get; set;}
public DateTime NextDayDateTime {get; set;}
}
class CalendarSealesHelperComparer : IEqualityComparer<CalendarSealesHelper>
{
public bool Equals(CalendarSealesHelper c1, CalendarSealesHelper c2)
{
if (c2.CalendarDate >= c1.CalendarDate
&& c2.NextDayDateTime < c1.NextDayDateTime)
{
return true;
}
else
{
return false;
}
}
public int GetHashCode(CalendarSealesHelper c)
{
int hCode = (int)c.CalendarDate.Ticks ^ (int)c.NextDayDateTime.Ticks;
return hCode.GetHashCode();
}
}
Then try this:
var query = db.Calendars.GroupJoin(
db.Sales,
c => new CalendarSealesHelper{c.CalendarDate, c.NextDayDateTime},
s => new CalendarSealesHelper{s.SaleDate, s.SaleDate},
(c, s) => new {Calendars = c, Sales = s},
new CalendarSealesHelperComparer())
.SelectMany(s => s.Sales.DefaultIfEmpty(),
(c, s) => new {
CalendarDate = c.CalendarDate,
FirstDayOfWeek = c.FirstDayOfWeek,
FirstDayOfMonth = c.FirstDayOfMonth,
Total = s.Total,
SaleDate = s.SaleDate
})
.Where(r => r.SaleDate >= sinceDate && r.SaleDate <= dateEnd);