I have one table in database named Balance and a list of dates as follows:
List<string> allDates = { "2021-01-02", "2021-01-03", "2021-01-04" }
Balance table:
Id, Amount, BalanceDate
1, 233, "2021-01-02"
2, 442, "2021-01-03
I need to fetch the records in Balance table with amount 0 for the missing dates. For example:
233, "2021-01-02"
442, "2021-01-03"
0, "2021-01-04"
I have tried the following:
balnces.GroupJoin(allDates,
balance => balance.Date,
d => d,
(balance, d) => balance);
But the records are still the same (only the ones in the balance table)
Given a data structure from database:
private class balance
{
public int id { get; set; }
public double amount { get; set; }
public string date { get; set; }
}
You get your data as you want (this is only a mock-up)
List<string> allDates = new List<string> { "2021-01-02", "2021-01-03", "2021-01-04" };
List<balance> balances = new List<balance>();
balances.Add(new balance { id = 1, amount = 233 , date = "2021-01-02" });
balances.Add(new balance { id = 2, amount = 442, date = "2021-01-03" });
you can get your desired result this way:
List<balance> result = allDates.Select(d=>
new balance {
amount =
balances.Any(s=> s.date == d)?
balances.FirstOrDefault(s => s.date == d).amount:0,
date = d
}).ToList();
If your default contains a 0 in amount instead a null, you can skip the .Any check
Assumption
Balance query had been materialized and data are returned from the database.
Solution 1: With .DefaultIfEmpty()
using System.Linq;
var result = (from a in allDates
join b in balances on a equals b.Date.ToString("yyyy-MM-dd") into ab
from b in ab.DefaultIfEmpty()
select new { Date = a, Amount = b != null ? b.Amount : 0 }
).ToList();
Sample Program for Solution 1
Solution 2: With .ToLookup()
var lookup = balances.ToLookup(x => x.Date.ToString("yyyy-MM-dd"));
var result = (from a in allDates
select new
{
Date = a,
Amount = lookup[a] != null && lookup[a].Count() > 0 ? lookup[a].First().Amount : 0
}
).ToList();
Sample Program for Solution 2
Related
I have the following table:
CREATE TABLE "OrderStatusLogs" (
"Id" UNIQUEIDENTIFIER NOT NULL,
"OrderId" UNIQUEIDENTIFIER NOT NULL,
"Status" INT NOT NULL,
"StartDateTime" DATETIMEOFFSET NOT NULL,
"EndDateTime" DATETIMEOFFSET NULL DEFAULT NULL,
PRIMARY KEY ("Id"),
FOREIGN KEY INDEX "FK_OrderStatusLogs_Orders_OrderId" ("OrderId"),
CONSTRAINT "FK_OrderStatusLogs_Orders_OrderId" FOREIGN KEY ("OrderId") REFERENCES "Orders" ("Id") ON UPDATE NO_ACTION ON DELETE CASCADE
)
;
For the following entity:
[DebuggerDisplay(nameof(OrderStatusLog) + " {Status} {StartDateTime} - {EndDateTime}" )]
public class OrderStatusLog
{
public Guid Id { get; set; }
public Guid OrderId { get; set; }
public OrderStatus Status { get; set; }
public DateTimeOffset StartDateTime { get; set; }
public DateTimeOffset? EndDateTime { get; set; }
}
public enum OrderStatus
{
Unknown = 0,
Pending = 1,
Processing = 2,
Shipping = 3,
}
And i'm trying to generate a report which should show how many orders are set to a certain state for a given range.
For example, for the month oktober, we'd have the range 1 to 31 oktober.
The desired output would be something like this:
1/10/2021 Pending 21 orders
1/10/2021 Processing 23 orders
1/10/2021 Shipping 33 orders
1/10/2021 Unknown 0 orders
...
31/10/2021 Pending 1 orders
31/10/2021 Processing 3 orders
31/10/2021 Shipping 44 orders
31/10/2021 Unknown 5 orders
I'm having some difficulties writing a query in EF that would give me the right output. I can get things to work, but only client-side. I'm trying to make this work in the database instead.
So far i tried:
var logsByDayAndOrderId = orderStatusLogs.GroupBy(c => new { c.StartDateTime.Date, c.OrderId }, (key, values) => new
{
key.Date,
key.OrderId,
MaxStartDateTime = values.Max(x => x.StartDateTime)
});
var list = logsByDayAndOrderId.ToList();
var statusByDayAndOrderId = logsByDayAndOrderId.Select(c => new
{
c.Date,
c.OrderId,
orderStatusLogs.FirstOrDefault(x => x.StartDateTime == c.MaxStartDateTime && x.OrderId == c.OrderId).Status
});
//var statusByDayAndOrderId = logsByDayAndOrderId.Join(orderStatusLogs.def, inner => new { inner.OrderId, StartDateTime = inner.MaxStartDateTime }, outer => new { outer.OrderId, outer.StartDateTime }, (inner,outer) => new
//{
// inner.Date,
// inner.OrderId,
// outer.Status
//}); // TODO rem this query gives more results because of the join. we need an Outer join - but i could not get that to work. the version with select above works better, but then it does not use join so it may be slow(er).
var list1 = statusByDayAndOrderId.ToList();
var groupBy = statusByDayAndOrderId
.GroupBy(c => new { c.Date, c.Status })
.Select(c => new { c.Key.Date, c.Key.Status, Count = c.Count() });
var list2 = groupBy.ToList();
Another attempt:
var datesAndOrders = orderStatusLogs
.GroupBy(c => new { c.StartDateTime.Date, c.OrderId }, (key, values) => key);
var ordersByDateAndActiveStatusLog = orderStatusLogs
.Select(c => new
{
c.StartDateTime.Date,
c.OrderId,
ActiveStatusForDate = orderStatusLogs
.OrderByDescending(x => x.StartDateTime)
.FirstOrDefault(x => x.OrderId == c.OrderId && x.StartDateTime.Date == c.StartDateTime.Date)
.Status
});
var list = ordersByDateAndActiveStatusLog.ToList();
var orderCountByDateAndStatus = ordersByDateAndActiveStatusLog
.GroupBy(c => new { c.Date, c.ActiveStatusForDate }, (key, values) => new
{
key, count = values.Count()
});
var list1 = orderCountByDateAndStatus.ToList();
Both of these fail because of Cannot use an aggregate or a subquery in an expression used for the group by list of a GROUP BY clause..
This makes sense.
I'm hoping for someone that could help write a Linq query that generates the right data using ef core.
Notes:
I Solely use the fluent query syntax
I Have more places where i'd like to get data for each day so any other info or tips and tricks are welcome
I use net core 5 with ef core 5.0.11 with a MSSQL database
I would suggest to use EF Core extension linq2db.EntityFrameworkCore which has ability to work with local (in-memory) collections in database queries. Disclaimer: i'm one of the creators.
At first define function which generates days sequence:
public static IEnumerable<DateTime> GenerateDays(int year, int month)
{
var start = new DateTime(year, month, 1);
var endDate = start.AddMonths(1);
while (start < endDate)
{
yield return start;
start = start.AddDays(1);
}
}
Then we can use generated sequence in LINQ Query:
var days = GenerateDays(2021, 10).ToArray();
using var dc = ctx.CreateLinqToDbConnection();
var totalsQuery =
from d in days.AsQueryable(dc)
from l in orderStatusLogs.Where(l =>
(l.EndDateTime == null || l.EndDateTime >= d) && l.StartDateTime < d.AddDays(1))
.DefaultIfEmpty()
group l by new { Date = d, l.Status } into g
into g
select new
{
g.Key.Date,
g.Key.Status,
Count = g.Sum(x => x == null ? 0 : 1),
};
var result = totalsQuery.ToList();
The following SQL should be generated:
SELECT
[d].[item],
[e].[Status],
Sum(IIF([e].[OrderID] IS NULL, 0, 1))
FROM
(VALUES
('2021-05-01T00:00:00'), ('2021-05-02T00:00:00'),
('2021-05-03T00:00:00'), ('2021-05-04T00:00:00'),
('2021-05-05T00:00:00'), ('2021-05-06T00:00:00'),
('2021-05-07T00:00:00'), ('2021-05-08T00:00:00'),
('2021-05-09T00:00:00'), ('2021-05-10T00:00:00'),
('2021-05-11T00:00:00'), ('2021-05-12T00:00:00'),
('2021-05-13T00:00:00'), ('2021-05-14T00:00:00'),
('2021-05-15T00:00:00'), ('2021-05-16T00:00:00'),
('2021-05-17T00:00:00'), ('2021-05-18T00:00:00'),
('2021-05-19T00:00:00'), ('2021-05-20T00:00:00'),
('2021-05-21T00:00:00'), ('2021-05-22T00:00:00'),
('2021-05-23T00:00:00'), ('2021-05-24T00:00:00'),
('2021-05-25T00:00:00'), ('2021-05-26T00:00:00'),
('2021-05-27T00:00:00'), ('2021-05-28T00:00:00'),
('2021-05-29T00:00:00'), ('2021-05-30T00:00:00'),
('2021-05-31T00:00:00')
) [d]([item])
LEFT JOIN [OrderStatusLogs] [e] ON ([e].[EndDateTime] IS NULL OR [e].[EndDateTime] >= [d].[item]) AND [e].[StartDateTime] < DateAdd(day, 1, [d].[item])
GROUP BY
[d].[item],
[e].[Status]
Here is a method to delete zero Inventory records from Inventory Table. I would like to reduce code/no of times that LINQ executes on Database.
Inventory Table
public class Inventory
{
public int itemCode { get; set; }
public decimal price { get; set; }
public decimal availQty { get; set; } // Can have Negative values.
}
example data
itemCode price availQty
1 10 10
1 12 -10
2 10 10
From above records, i want to delete all records of itemCode == 1, as net availQty is 0.
Here is my method
private void RemoveZeroInvs()
{
// Remove individual zero Inventorys
var rinvs = from ri in _context.Inventorys
where ri.availQty == 0
select ri;
_context.Inventorys.RemoveRange(rinvs);
_context.SaveChanges();
// Remove if group is zero in availQty, as it allows Negative Qty.
var result = from d in _context.Inventorys
group d by new
{
d.itemCode
}
into g
select new
{
g.Key.itemCode,
availQty = g.Sum(y => y.availQty)
};
var zrs = from r in result
where r.availQty == 0
select r;
foreach (var zr in zrs) // Here, zrs length may be more than 500
{
var ri = _context.Inventorys.Where(w => w.itemCode == zr.itemCode);
_context.Inventorys.RemoveRange(ri);
_context.SaveChanges();
}
}
I use Asp.Net Core 2.2. Is there any such possibility?
Also I get following error at line _context.Inventorys.RemoveRange(ri); in the loop.
A command is already in progress: SELECT t."itemCode", t."availQty"
FROM (
SELECT d."itemCode", SUM(d."availQty") AS "availQty"
FROM "Inventorys" AS d
GROUP BY d."itemCode"
) AS t
var todelete = _context.Inventorys
.GroupBy(i => i.itemCode)
.Where(g => g.Sum(i => i.availQty) == 0)
.SelectMany(g => g);
Here is a shorter versions of your code, in terms of DB excecution, one would have to compare the raw queries. But it may be lighter that your code…
This is my table structure
ID A B C D
1 null 10 5 null
2 3 5 null D2
3 8 null 2 D2
4 null 4 3 D1
5 4 6 1 D2
This is c# class and its property to store query result.
public class GrillTotals
{
public int? SumOfA {get; set;}
public int? SumOfB{get; set;}
public int? SumOfC{get; set;}
public int? CountOfD1{get; set;}
public int? CountOfD2{get; set;}
}
What I expect is:
SumOfA = 15
SumOfB = 20
SumOfC = 11
CountOfD1 = 1
CountOfD2 = 3
What I am getting is :
SumOfA = null,
SumOfB = null,
SumOfC = null,
CountOfD1 = 0,
CountOfD2 = 0
Here is a code what I have tried.
var _FinalResult = from s in dbContext.tblSchedules
group s by new
{
s.A,
s.B,
s.C,
s.D,
} into gt
select new GrillTotals
{
SumOfA = gt.Sum(g => g.A),
SumOfB = gt.Sum(g => g.B),
SumOfC = gt.Sum(g => g.C),
CountOfD1 = gt.Count(g => g.D == "D1"),
CountOfD2 = gt.Count(g => g.D == "D2"),
};
Try to correct me if I am doing something wrong or incorrectly.Any help will be appreciated.
You should not be grouping by the fields you want to calculate aggregates. When you group by them, every aggregate (Sum, Min, Max etc) will return the value itself (and Count 1 or 0 depending of the condition).
From what I see you are trying to return several aggregates with single SQL query. If that's correct, it's possible by using group by constant technique.
Just replace
group s by new
{
s.A,
s.B,
s.C,
s.D,
} into gt
with
group s by 1 // any constant
into gt
Try this:
var _FinalResult = from s in dbContext.tblSchedules
group s by new
{
s.A,
s.B,
s.C,
s.D,
} into gt
select new GrillTotals
{
SumOfA = gt.Sum(g => g.A ?? 0),
SumOfB = gt.Sum(g => g.B ?? 0),
SumOfC = gt.Sum(g => g.C ?? 0),
CountOfD1 = gt.Count(g => g.D == "D1"),
CountOfD2 = gt.Count(g => g.D == "D2"),
};
I am fetching data from database from separater tables and storing in separate variables.
this is first list
var res = (from v in context.MasterValues
join c in context.MasterCategories on v.Category_Id equals c.Id
where c.Model_Key == (int)model && c.Name == tableName && v.Version_Id == version && v.Active == "Y" && c.Name.Equals("FXRates")
select new
{
Id = v.Id,
VersionId = v.Version_Id,
Text1 = v.Text1,
}).ToList();
This is second list
var fxview = context.MasterFXRates.Select
(x => new
{
Currency_code = x.Currency_code,
Rate = x.Rate,
Effective_dt = x.Effective_dt
}
).ToList();
So now How to filter Data from my second list fxview based on data from my first list ?
i.e.
i need to filter data where Currency_code's data of list2 matches with Text1 of List1 where effective_dt(datetime column) is maximum/Latest date
For Example if second lists data has
ABC , 100 , 2010-10-10
ABC , 120 , 2014-12-12
DEF ,700 , 2013-08-02
DEF ,500 ,2015-06-06
And List 1(res) has following data
1 , 1 , ABC
2 , 1 , DEF
So after filtering my final list must have following output
ABC ,120 (Since 2014-12-12 is latest date , the corresponding value is fetched and duplicate value (ABC,100) should be filtered.)
2.DEF ,500 (Since 2015-06-06 is latest date , the corresponding value is fetched and duplicate value (DEF,&00) should be filtered.)
var result = from masterFxRate in masterFxRates
join masterValue in masterValues on masterFxRate.Currency_code equals masterValue.Text1
group masterFxRate by
new
{
masterFxRate.Currency_code
} into groupedRates
select new
{
groupedRates.Key.Currency_code,
Rate = groupedRates.FirstOrDefault(g => g.Effective_dt != null
&& g.Effective_dt == groupedRates.Max(c => c.Effective_dt)).Rate
};
foreach (var item in result)
{
Console.WriteLine("{0} : {1} ", item.Currency_code, item.Rate);
}
fxView = fxView.OrderByDescending(x => x.effectiveDate).ToList();
var result = new List();
res.ForEach((x) => result.Add(fxView.First(y => x.text1 == y.currency_code)));
If effectiveDate is already a DateTime this should work otherwise convert it to a DateTime
var fxview = context.MasterFXRates.Select
(x => new
{
Currency_code = x.Currency_code,
Rate = x.Rate,
Effective_dt = Convert.ToDateTime(x.Effective_dt)
}
).ToList();
I'm still learning LINQ and have a task where I need to group Booking objects by four properties and then by weekly intervals depending on the input timerange.
public class Booking
{
public string Group { get; set; }
public BookingType Type { get; set; }
public BookingStatus Status { get; set; }
public decimal Price { get; set; }
public bool Notification { get; set; }
public DateTime Date { get; set; }
}
Let's say we have the following Bookings:
IList<Booking> Bookings = new List<Booking>
{
new Booking{Group = "Group1", Type = BookingType.Online, Status = BookingStatus.New, Price = 150, Date = new DateTime(2012,06,01)},
new Booking{Group = "Group1", Type = BookingType.Online, Status = BookingStatus.New, Price = 100, Date = new DateTime(2012,06,02)},
new Booking{Group = "Group1", Type = BookingType.Online, Status = BookingStatus.New, Price = 200, Date = new DateTime(2012,06,03)},
new Booking{Group = "Group2", Type = BookingType.Phone, Status = BookingStatus.Accepted, Price = 80, Date = new DateTime(2012,06,10)},
new Booking{Group = "Group2", Type = BookingType.Phone, Status = BookingStatus.Accepted, Price = 110, Date = new DateTime(2012,06,12)},
new Booking{Group = "Group3", Type = BookingType.Store, Status = BookingStatus.Accepted, Price = 225, Date = new DateTime(2012,06,20)},
new Booking{Group = "Group3", Type = BookingType.Store, Status = BookingStatus.Invoiced, Price = 300, Date = new DateTime(2012,06,21)},
new Booking{Group = "Group3", Type = BookingType.Store, Status = BookingStatus.Invoiced, Price = 140, Date = new DateTime(2012,06,22)},
};
That would result in the following lines on the final printout:
Week 22 Week 23 Week 24 Week 25 Week 26
May28-Jun3 Jun4-10 Jun11-17 Jun18-24 Jun25-Jul1
Group Type Status Cnt. Price Cnt. Price Cnt. Price Cnt. Price Cnt. Price
----------------------------------------------------------------------------------------------
Group1 Online New 3 450 0 0 0 0 0 0 0 0
Group2 Phone Accepted 0 0 1 80 1 110 0 0 0 0
Group3 Store Accepted 0 0 0 0 0 0 1 225 0 0
Group3 Store Invoiced 0 0 0 0 0 0 2 440 0 0
I have created 2 additional classes to represent a line with the possibility to include any number of weeks:
public class GroupedLine
{
public string Group { get; set; }
public BookingType Type { get; set; }
public BookingStatus Status { get; set; }
public List<WeeklyStat> WeeklyStats { get; set; }
}
public class WeeklyStat
{
public DateTime WeekStart { get; set; }
public decimal Sum { get; set; }
public int Count { get; set; }
}
If I have the following time period:
var DateFrom = new DateTime(2012, 05, 28);
var DateTo = new DateTime(2012, 7, 01);
Firstly, I need to identify what weeks are necessary in the statistics: in this case week 22-26.
For that I have the following code:
var DateFrom = new DateTime(2012, 05, 28);
var DateTo = new DateTime(2012, 7, 01);
var firstWeek = GetFirstDateOfWeek(DateFrom, DayOfWeek.Monday);
IList<DateTime> weeks = new List<DateTime> { firstWeek };
while(weeks.OrderByDescending(w => w).FirstOrDefault().AddDays(7) <= DateTo)
{
weeks.Add(weeks.OrderByDescending(w => w).FirstOrDefault().AddDays(7));
}
And now, I'd need some LINQ magic to do the grouping both by the 4 properties and the aggregation (count of bookings and sum of prices) for the weeks.
I can attach code sample of the LINQ I got so far tomorrow, as I don't have access to it now.
Sorry for the long post, hope it's clear what I mean.
Edit: 2012-11-07
I have to modify the question a bit, so that the grouped weeks only include those weeks, that actually have data.
Updated example output:
May28-Jun3 Jun4-10 Jun18-24
Group Type Status Cnt. Price Cnt. Price Cnt. Price
---------------------------------------------------------------
Group1 Online New 3 450 0 0 0 0
Group2 Phone Accepted 0 0 1 80 0 0
Group3 Store Accepted 0 0 0 0 1 225
Group3 Store Invoiced 0 0 0 0 2 440
In this case there were no Bookings in period Jun 11-17 and Jun25-Jul1 so they are omitted from the results.
This query will get all data
var query = from b in Bookings
where b.Date >= dateFrom && b.Date <= dateTo
group b by new { b.Group, b.Type, b.Status } into g
select new GroupedLine()
{
Group = g.Key.Group,
Type = g.Key.Type,
Status = g.Key.Status,
WeeklyStats = (from b in g
let startOfWeek = GetFirstDateOfWeek(b.Date)
group b by startOfWeek into weekGroup
orderby weekGroup.Key
select new WeeklyStat()
{
WeekStart = weekGroup.Key,
Count = weekGroup.Count(),
Sum = weekGroup.Sum(x => x.Price)
}).ToList()
};
I leave UI output to you :)
This will also return WeekStats for all weeks (with 0 values, if we do not have booking groups on some week):
// sequence contains start dates of all weeks
var weeks = Bookings.Select(b => GetFirstDateOfWeek(b.Date))
.Distinct().OrderBy(date => date);
var query = from b in Bookings
group b by new { b.Group, b.Type, b.Status } into bookingGroup
select new GroupedLine()
{
Group = bookingGroup.Key.Group,
Type = bookingGroup.Key.Type,
Status = bookingGroup.Key.Status,
WeeklyStats = (from w in weeks
join bg in bookingGroup
on w equals GetFirstDateOfWeek(bg.Date) into weekGroup
orderby w
select new WeeklyStat()
{
WeekStart = w,
Count = weekGroup.Count(),
Sum = weekGroup.Sum(b => b.Price)
}).ToList()
};
Keep in mind, that if you need date filter (from, to), then you need to apply it both to weeks query and bookings query.
var query = Bookings.GroupBy(book => new GroupedLine()
{
Group = book.Group,
Type = book.Type,
Status = book.Status
})
.Select(group => new
{
Line = group.Key,
Dates = group.GroupBy(book => GetWeekOfYear(book.Date))
.Select(innerGroup => new
{
Week = innerGroup.Key,
Count = innerGroup.Count(),
TotalPrice = innerGroup.Sum(book => book.Price)
})
});
public static int GetWeekOfYear(DateTime date)
{
return date.DayOfYear % 7;
}
This variant generates "empty" spots for weeks with no data:
// I group by week number, it seemed clearer to me , but you can change it
var firstWeek = cal.GetWeekOfYear(DateFrom, dfi.CalendarWeekRule, dfi.FirstDayOfWeek);
var lastWeek = cal.GetWeekOfYear(DateTo, dfi.CalendarWeekRule, dfi.FirstDayOfWeek);
var q = from b in Bookings
group b by new { b.Group, b.Type, b.Status }
into g
select new
{
Group = g.Key,
Weeks =
from x in g
select new
{
Week = cal.GetWeekOfYear(x.Date,
dfi.CalendarWeekRule,
dfi.FirstDayOfWeek),
Item = x
}
} into gw
from w in Enumerable.Range(firstWeek, lastWeek-firstWeek+1)
select new
{
gw.Group,
Week = w,
Item = from we in gw.Weeks
where we.Week == w
group we by we.Item.Group into p
select new
{
p.Key,
Sum = p.Sum (x => x.Item.Price),
Count = p.Count()
}
};