linq-to-sql Concat() throwing a System.IndexOutofRangeException - c#

I'm struggling with an exception using linq-to-sql Concat()
I've got 2 tables.
The first table, ParsedMessages, has the following fields
* ParsedMessageID (int)
* MessageTypeID (int)
* TextMessage (varchar(max))
The second table, ParsedMessageLinks, has the following fields
* ParsedMessageID (int)
* AnotherID (int)
* NumberOfOccurences (int)
This is what I need to achieve using a single linq query but I'm not sure if it's possible or not.
Through a join, retrieves ParsedMessage records that links to a certain AnotherID. In example SQL and linq code, the AnotherID will have the value 0 just for the purpose of having an example.
For each ParsedMessage record, I also need the NumberOfOccurences (field of table #2)
Retrieve only the top(100) ParsedMessage records for each MessageTypeID. So for example, if there is 275 records in ParsedMessages that links to AnotherID==0 where the first 150 records have MessageTypeID == 0 and the remaining 125 records having MessageTypeID == 1, I want my query to end up returning 200 records, the top(100) with MessageTypeID == 0 and the top(100) with MessageTypeID == 1
After a lot of search, I've found that the plain SQL equivalent of I what I want to do is this. I knew that this exists first end, but I tried to find something else without Union all at first and fail to do so (my SQL knowledge is not that good) :
SELECT TOP(100) PM.*,
PML.NumberOfOccurences FROM
ParsedMessages PM INNER JOIN
ParsedMessageLinks PML ON
PM.ParsedMessageID =
PML.ParsedMessageID WHERE
PML.AnotherID = 0 AND PM.MessageTypeID
= 0 ORDER BY PM.ParsedMessageID DESC UNION ALL
SELECT TOP(100) PM.*,
PML.NumberOfOccurences FROM
ParsedMessages PM INNER JOIN
ParsedMessageLinks PML ON
PM.ParsedMessageID =
PML.ParsedMessageID WHERE
PML.AnotherID = 0 AND PM.MessageTypeID
= 1 ORDER BY PM.ParsedMessageID DESC UNION ALL
SELECT TOP(100) PM.*,
PML.NumberOfOccurences FROM
ParsedMessages PM INNER JOIN
ParsedMessageLinks PML ON
PM.ParsedMessageID =
PML.ParsedMessageID WHERE
PML.AnotherID = 0 AND PM.MessageTypeID
= 2 ORDER BY PM.ParsedMessageID DESC
So basically, the only way to retrieve the data I need is to do 3 sql queries in a single pass where only the PM.MessageTypeID is different for each query.
Now I wanted to achieve this using linq-to-sql. After googling, I've found that I could use the Linq Concat() method to reproduce a SQL Union All.
Here are some links pointing to what I thought would work :
http://blog.benhall.me.uk/2007/08/linq-to-sql-difference-between-concat.html
EF. How to union tables, sort rows, and get top entities?
I end up having this exception :
System.IndexOutOfRangeException : "Index was outside the bounds of the array."
Here's the faulty code :
IQueryable<MyObject> concatquery;
int[] allMessageTypeIDs = new int[] { 0, 1, 2 };
for (int mt = 0; mt < allMessageTypeIDs.Length; mt++)
{
if (mt == 0)
{
concatquery = (from pm in db.ParsedMessages
join pml in db.ParsedMessageLinks on pm.ParsedMessageID equals pml.ParsedMessageID
where pml.AnotherID == 0 && pm.MessageTypeID == allMessageTypeIDs[mt]
orderby pm.ParsedMessageID descending
select new MyObject
{
NumberOfOccurences = pml.Occurrences,
ParsedMessage = pm
}).Take(100);
}
else
{
concatquery = concatquery.Concat(from pm in db.ParsedMessages
join pml in db.ParsedMessageLinks on pm.ParsedMessageID equals pml.ParsedMessageID
where pml.AnotherID == 0 && pm.MessageTypeID == allMessageTypeIDs[mt]
orderby pm.ParsedMessageID descending
select new MyObject
{
NumberOfOccurences = pml.Occurrences,
ParsedMessage = pm
}).Take(100);
}
}
var results = concatquery.ToArray();
I've declared the int array allMessageTypeIDs, for simplicity. But remember that the values it holds may differ, so that's why I've added the for loop. Maybe it's "illegal" to use a Concat() in a loop that way, but I could not find any relevant information on this exception.
The class MyObject basically hold a int (NumberOfOccurences) and a ParsedMessage database object, nothing else.
Any suggestions on what could be wrong with my code that causes the exception?
Thanks
Francis

Never use the variable you're looping with in your Linq queries. It just doesn't work. You want to assign a new temporary variable to use instead.
IQueryable<MyObject> concatquery;
int[] allMessageTypeIDs = new int[] { 0, 1, 2 };
for (int mt = 0; mt < allMessageTypeIDs.Length; mt++)
{
var myItem = allMessageTypeIDs[mt]; // <-- HERE!
if (mt == 0)
{
concatquery = (from pm in db.ParsedMessages
join pml in db.ParsedMessageLinks on pm.ParsedMessageID equals pml.ParsedMessageID
where pml.AnotherID == 0 && pm.MessageTypeID == myItem
orderby pm.ParsedMessageID descending
select new MyObject
{
NumberOfOccurences = pml.Occurrences,
ParsedMessage = pm
}).Take(100);
}
else
{
concatquery = concatquery.Concat(from pm in db.ParsedMessages
join pml in db.ParsedMessageLinks on pm.ParsedMessageID equals pml.ParsedMessageID
where pml.AnotherID == 0 && pm.MessageTypeID == myItem
orderby pm.ParsedMessageID descending
select new MyObject
{
NumberOfOccurences = pml.Occurrences,
ParsedMessage = pm
}).Take(100);
}
}
var results = concatquery.ToArray();

Related

Multiple AND conditions on the same column [Servicestack.OrmLite]

I was wondering if it's possible to have multiple AND conditions on the same column using Servicestack.OrmLite. This is the SELECT statement I printed out, but It always returns 0. I should get the product count from the products having both specifications with id 1016 and 17.
SELECT COUNT(DISTINCT "Product"."Id")
FROM "Product"
INNER JOIN "ProductManufacturer"
ON ("Product"."Id" = "ProductManufacturer"."ProductId")
INNER JOIN "ProductSpecificationAttribute"
ON ("Product"."Id" = "ProductSpecificationAttribute"."ProductId")
WHERE ("ProductManufacturer"."ManufacturerId" = 6)
AND ("ProductSpecificationAttribute"."SpecificationAttributeOptionId" = 1016)
AND ("ProductSpecificationAttribute"."SpecificationAttributeOptionId" = 17)
A single column value can't possibly have two values at the same time.
What you want is either:
AND
(
ProductSpecificationAttribute.SpecificationAttributeOptionId = 1016
OR
ProductSpecificationAttribute.SpecificationAttributeOptionId = 17
)
Or, more succinctly:
AND
(
ProductSpecificationAttribute.SpecificationAttributeOptionId
IN (1016, 17)
)
And turn off whatever option is forcing your tool to "inject" "double" "quotes" "around" "every" "entity" "name" because it makes the query text unmanageable. You might also consider using aliases and schema prefixes, like INNER JOIN dbo.ProductSpecificationAttribute AS psa...
After further clarification... the goal is to find products where they have both of those attributes on different rows, which isn't clear from the description or the code ORMLite barfed out. Here's what you want in that case (there are several ways to do this, but converting everything to EXISTS also allows you to remove the DISTINCT from the COUNT, which is never free):
SELECT COUNT(Product.Id) FROM dbo.Product AS p
WHERE EXISTS
(
SELECT 1 FROM dbo.ProductManufacturer AS pm
WHERE pm.ProductId = p.Id AND pm.ManufacturerId = 6
)
AND EXISTS
(
SELECT 1 FROM dbo.ProductSpecificationAttribute AS psa
WHERE psa.ProductId = p.Id
AND psa.SpecificationAttributeOptionId = 1016
)
AND EXISTS
(
SELECT 1 FROM dbo.ProductSpecificationAttribute AS psa
WHERE psa.ProductId = p.Id
AND psa.SpecificationAttributeOptionId = 17
);
If ProductSpecificationAttribute is poorly index and this leads to two scans, you could change that by saying something like this (untested, but I'm happy to test it out if you can produce a db<>fiddle:
SELECT COUNT(Product.Id) FROM dbo.Product AS p
WHERE EXISTS
(
SELECT 1 FROM dbo.ProductManufacturer AS pm
WHERE pm.ProductId = p.Id
AND pm.ManufacturerId = 6
)
AND EXISTS
(
SELECT 1 FROM dbo.ProductSpecificationAttribute AS psa
WHERE psa.ProductId = p.Id
AND psa.SpecificationAttributeOptionId IN (17, 1016)
GROUP BY psa.ProductId, psa.SpecificationAttributeOptionId
HAVING COUNT(DISTINCT psa.SpecificationAttributeOptionId) > 1
);
It's also really weird that the table ProductManufacturer has a list of ProductIDs in it that point back to Product - usually Product would have a ManufacturerID that points in the other direction.
Anyway, you might consider using stored procedures that your ORM can call if it has problems creating queries beyond basic CRUD (which is unfortunately a limitation of all ORMs to some degree - they're great at the basics, covering 80% of the use case, but they're terrible at the other 20% - unfortunately most of us end up needing that other 20% before too long).
You can get all the product ids that you want if you group by product and set the conditions in the HAVING clause:
SELECT p.Id
FROM Product p
INNER JOIN ProductManufacturer pm ON p.Id = pm.ProductId
INNER JOIN ProductSpecificationAttribute psa ON p.Id = psa.ProductId
WHERE pm.ManufacturerId = 6 AND psa.SpecificationAttributeOptionId IN (17, 1016)
GROUP BY p.Id
HAVING COUNT(DISTINCT psa.SpecificationAttributeOptionId) = 2; -- both specifications must exist
If you want to count these products you could either use the above query as a subquery or a cte and count the rows:
WITH cte AS (
SELECT p.Id
FROM Product p
INNER JOIN ProductManufacturer pm ON p.Id = pm.ProductId
INNER JOIN ProductSpecificationAttribute psa ON p.Id = psa.ProductId
WHERE pm.ManufacturerId = 6 AND psa.SpecificationAttributeOptionId IN (17, 1016)
GROUP BY p.Id
HAVING COUNT(DISTINCT psa.SpecificationAttributeOptionId) = 2;
)
SELECT COUNT(*) FROM cte;
or, use COUNT() window function:
SELECT DISTINCT COUNT(*) OVER ()
FROM Product p
INNER JOIN ProductManufacturer pm ON p.Id = pm.ProductId
INNER JOIN ProductSpecificationAttribute psa ON p.Id = psa.ProductId
WHERE pm.ManufacturerId = 6 AND psa.SpecificationAttributeOptionId IN (17, 1016)
GROUP BY p.Id
HAVING COUNT(DISTINCT psa.SpecificationAttributeOptionId) = 2;

How to get closest smallest number using linq C#

I have a sql table like this,
SomeDouble SomeInt
1.00 121
1.50 124
2.00 200
2.50 321
and so on... up to 10,000 for SomeDouble
Now I can have a decimal number anywhere between 0.0 to 10,000.00 and I need to find the correct row for it. For example if number is 1.12 then I want it to return 121.
1.49 should return 121, 1.50 should return 124, 1.51 should return 124.
Trimmed version of what I am trying is,
var myValue = 1.12
var SomeInt = (from mainTable in table1
join table3 in table2 on mainTable.someId equals table3.someId
where table3.Column1 == Column1 && mainTable.SomeDouble >= myValue
select mainTable.SomeInt).FirstOrDefault();
but my output is 124. How can I change above to get me the closest smallest number then myValue ?
Because the SomeDouble values are integers and half-integers, you can round up myValue to the next multiple of 0.5:
var myValueToLookUp = Math.Ceiling(myValue * 2) / 2;
and then look up the value of SomeInt directly with mainTable.SomeDouble == myValueToLookUp to avoid any confusion or inefficiency with <= or >=.
In SQL, you can express the closest as:
select t.*
from t
order by abs(SomeDouble - 1.12)
fetch first 1 row only;
A more efficient method would narrow it down to two rows first:
select t.*
from ((select t.*
from t
where t <= 1.12
order by SomeDouble desc
fetch first 1 row only
) union all
((select t.*
from t
where t > 1.12
order by SomeDouble asc
fetch first 1 row only
)
) t
order by (SomeDouble - 1.12)
fetch first 1 row only;
Using Linq queries:
var above = (from mainTable in table1
join table3 in table2 on mainTable.someId equals table3.someId
where table3.Column1 == Column1 && mainTable.SomeDouble >= myValue
orderby mainTable.SomeDouble
select new {SomeInt = mainTable.SomeInt, SomeDouble = mainTable.SomeDouble}).FirstOrDefault();
var below = (from mainTable in table1
join table3 in table2 on mainTable.someId equals table3.someId
where table3.Column1 == Column1 && mainTable.SomeDouble < myValue
orderby mainTable.SomeDouble descending
select new {SomeInt = mainTable.SomeInt, SomeDouble = mainTable.SomeDouble}).FirstOrDefault();
int SomeInt;
if (above == null)
SomeInt = below.SomeInt;
else if (below == null)
SomeInt = above.SomeInt;
else if (Math.Abs(below.SomeDouble - myValue) <= Math.Abs(above.SomeDouble - myValue))
SomeInt = below.SomeInt;
else
SomeInt = above.SomeInt;
Here's the linq extension method to order the records by absolute difference of SomeDouble, then by SomeInt to get the smallest first for 2 or more matches, and then we get the first one. It looks like both columns exist on main table, so I'm guessing we can limit that first then join whatever you want to it.
mainTable.OrderBy(x => Math.Abs(x.SomeDouble - myValue)).ThenBy(x => x.SomeInt).First()
If you can do it in SQL, then
SELECT COALESCE(MAX(SomeInt), 0)
FROM DoubleToInt
WHERE SomeDouble <= 1.12

How do I rewrite this SQL query in LINQ format?

I have a sql query that returns the surrounding rows for a given ID. So lets say I'm looking to find 3 rows for a given MediaID of 8000. (Previous Row, Current Row, Next Row).
I'm not even sure if this is the best way to achieve those results but here's my query anyway:
SELECT * FROM (
SELECT TOP 1 * FROM Media WHERE MediaTypeID = 1 and MediaID < 8000 order by MediaID DESC
UNION
SELECT * FROM Media WHERE MediaID = 8000
UNION
SELECT TOP 1 * FROM Media WHERE MediaTypeID = 1 and MediaID > 8000
) AS TBL
ORDER BY TBL.MediaID
I'm importing this query into a C# web application and would like to convert the query over to LINQ format. I'm struggling a bit with this. I think im close.
Here's my LINQ code:
//Get Prev record, current record, next record
var Results = (from m in DB.Media where m.MediaTypeID == 1 && m.MediaID < 8000 orderby m.MediaID descending select m).Take(1).Union(
from m in DB.Media where m.MediaID == 8000 select m).Union(
from m in DB.Media where m.MediaTypeID == 1 && m.MediaID > 8000 select m).Take(1);
Thanks for your help.
It does look really close. I think you need an additional set of parentheses around the final statement your "union"ing (so that the Take(1) only applies to that last statement and not the entire unioned LINQ statement up to that point) and a final OrderBy:
var Results = (from m in DB.Media
where m.MediaTypeID == 1
&& m.MediaID < 8000
orderby m.MediaID descending select m).Take(1)
.Union(
from m in DB.Media
where m.MediaID == 8000
select m)
.Union(
(from m in DB.Media
where m.MediaTypeID == 1
&& m.MediaID > 8000
select m).Take(1))
.OrderBy(m => m.MediaID);
Consider breaking this up into separate lines for clarity:
var lessThan8000 = (from m in DB.Media
where m.MediaTypeID == 1
&& m.MediaID < 8000
orderby m.MediaID descending select m).Take(1);
var equalTo8000 = (from m in DB.Media
where m.MediaID == 8000
select m);
var greaterThan8000 = (from m in DB.Media
where m.MediaTypeID == 1
&& m.MediaID > 8000
select m).Take(1));
var Results = lessThan8000.Union(equalTo8000)
.Union(greaterThan8000)
.OrderBy(m => m.MediaId);

LINQ to EF, Left Join and group by clause

I have this SQL:
select o.prod_id, SUM(o.[count]) as [count]
into #otgr
from otgr o
where o.[date]<= #date
group by o.prod_id
select f.prod_id, SUM(f.[count]) as [count]
into #factory
from factory f
where f.[date]<= #date
group by f.prod_id
select p.name, p.id, f.[count] - ISNULL(o.[count],0) as av_count
from products p
join #factory f on f.prod_id = p.id
left join #otgr o on o.prod_id = p.id
where f.[count] - ISNULL(o.[count],0) > 0
How can I translate this into Linq? I'm stuck with this code:
from otgrr in db.otgr
where otgrr.date <= date
group otgrr by otgrr.prod_id into otgrs
from fac in db.factory
where fac.date <= date
group fac by fac.prod_id into facs
from prod in db.products
join fac2 in facs on prod.id equals fac2.Key
join otg2 in otgrs.DefaultIfEmpty(new {id = 0, av_count = 0 }) on prod.id equals otg2.Key
where (fac2.SUM(a=>a.av_count) - otg2.SUM(a=>a.av_count)) > 0
select new products { id = prod.id, name = prod.name, av_count = (fac2.SUM(a=>a.av_count) - otg2.SUM(a=>a.av_count))
Thank to everyone, and sorry for my bad english
You can also check LINQPad.
Of course, you can split this into multiple LINQ queries (after all, the execution is deferred, so it will be executed all as one single query, without using temporary tables. It should be faster in 99% of the cases).
But in your case it can be written more simply, by using navigation properties you probably have already set up:
var result= from p in products
select new {Name=p.Name,
Id = p.Id,
Count = p.Factories.Where(f=> f.date <= date).Sum(f=>f.Count)
- p.otgrs.Where(o=> o.date <= date).Sum(o=>o.Count)
};

Problem with decimal precision in SQL using Linq to SQL

I have a simple query:
var results = from k in db.tree_nodes
join s in db.stocks
on k.tree_nodes_id equals s.tree_nodes_id
into tmpTable
from rowtmp in tmpTable.DefaultIfEmpty()
select new
{
stock = (rowtmp.amount == null) ?
((k.code == null) ? (decimal?)null : (decimal?)0)
:
rowtmp.amount - rowtmp.amount_in_use,
};
This is the generated SQL code:
SELECT
(CASE
WHEN ([t1].[amount]) IS NULL THEN
(CASE
WHEN [t0].[code] IS NULL THEN CONVERT(Decimal(33,4),NULL)
ELSE CONVERT(Decimal(33,4),0)
END)
ELSE CONVERT(Decimal(33,4),[t1].[amount] - [t1].[amount_in_use])
END) AS [stock]
FROM [dbo].[tree_nodes] AS [t0]
LEFT OUTER JOIN [dbo].[stocks] AS [t1] ON [t0].[tree_nodes_id] = [t1].[tree_nodes_id]
The problem is, the generator created Decimal(33,4) when converting the results. So I'm getting "123.4560" in results instead of "123.456" All of my fields in this query are decimal(14,3). I don't mind the 33 part but I need to change the ,4 to ,3. How can I do this?
You could round the decimal values to 3 decimals?
var results = from k in db.tree_nodes
join s in db.stocks
on k.tree_nodes_id equals s.tree_nodes_id
into tmpTable
from rowtmp in tmpTable.DefaultIfEmpty()
select new
{
stock = (rowtmp.amount == null) ?
((k.code == null) ? (decimal?)null : (decimal?)0)
:
decimal.Round(rowtmp.amount,3) - decimal.Round(rowtmp.amount_in_use == null ? 0 : rowtmp.amount_in_use,3),
};
Dunno of any way to prevent linq-to-sql from type conversion.

Categories

Resources