Can you please suggest how to write an optmized LINQ query for the following operation?
foreach (DataRow entry1 in table1.Rows)
{
var columnA = entry1["ColumnA"] as string;
if (!string.IsNullOrEmpty(columnA))
{
foreach (string entry2 in table2)
{
var dataExists = table3.Any(rows3 =>
!string.IsNullOrEmpty(rows3[entry2] as string)
&& columnA.IsEqual(rows3["ColumnB"] as string));
if (dataExists)
{
entry1[entry2] = Compute(columnA, entry2);
}
}
}
}
I tried with this, but the results don't match in terms of the unique iteration counts.
var t2t3Pair = from entry2 in table2
let entry3 = table3.FirstOrDefault(x =>
!string.IsNullOrEmpty(x[entry2] as string))
where entry3 != null
select new { entry2, entry3 };
var t1t3Pair = from pair in t2t3Pair
from entry1 in table1.AsEnumerable()
let columnA = entry1["ColumnA"] as string
where !string.IsNullOrEmpty(columnA)
&& columnA.IsEqual(pair.entry3["ColumnB"] as string)
select new { Entry1Alias = entry1, Entry2Alias = pair.entry2 };
foreach (var pair in t1t3Pair)
{
var columnA = (string)pair.Entry1Alias["ColumnA"];
pair.Entry1Alias[pair.Entry2Alias] = Compute(columnA, pair.Entry2Alias);
}
Note: IsEqual is my extension method to compare string without case sensitivity.
Apparently the bottleneck is the line
var dataExists = table3.Any(rows3 =>
!string.IsNullOrEmpty(rows3[entry2] as string)
&& columnA.IsEqual(rows3["ColumnB"] as string));
which is executed inside the innermost loop.
As usual, it can be optimized by preparing in advance a fast lookup data structure and use it inside the critical loop.
For your case, I would suggest something like this:
var dataExistsMap = table3.AsEnumerable()
.GroupBy(r => r["ColumnB"] as string)
.Where(g => !string.IsNullOrEmpty(g.Key))
.ToDictionary(g => g.Key, g => new HashSet<string>(
table2.Where(e => g.Any(r => !string.IsNullOrEmpty(r[e] as string)))
// Include the proper comparer if your IsEqual method is using non default string comparison
//, StringComparer.OrdinalIgnoreCase
)
);
foreach (DataRow entry1 in table1.Rows)
{
var columnA = entry1["ColumnA"] as string;
if (string.IsNullOrEmpty(columnA)) continue;
HashSet<string> dataExistsSet;
if (!dataExistsMap.TryGetValue(columnA, out dataExistsSet)) continue;
foreach (string entry2 in table2.Where(dataExistsSet.Contains))
entry1[entry2] = Compute(columnA, entry2);
}
Related
I'm new to using Linq so I don't understand some things or its syntax. I want to group a list and then loop through it with foreach, like my logic below. Obviously my logic doesn't work.
My code:
var final = finalv.Union(finalc);
final = final.GroupBy(x => x.Clave);
foreach (var articulo in final)
{
Articulo articulo2 = new Articulo();
articulo2.ArtID = articulo.ArtID;
articulo2.Clave = articulo.Clave;
articulo2.ClaveAlterna = articulo.ClaveAlterna;
lista.Add(articulo2);
}
First, such usage is syntactically consistent with this overloaded method of GroupBy: GroupBy<TSource,TKey>(IEnumerable<TSource>, Func<TSource,TKey>), and it will return a IEnumerable<IGrouping<TKey,TSource>> variable.
That means, if you run final.GroupBy(x => x.Clave), let's assume he returns finalWithGrouped, then finalWithGrouped.Key is the key and finalWithGrouped.ToList() is a collection of all variables with the same key(at here, it is with the same Clave).
And for your code, try this:
var final = finalv.Union(finalc);
var finalWithGrouped = final.GroupBy(x => x.Clave);
foreach (var articulosWithSameClavePair in finalWithGrouped)
{
var clave = articulosWithSameClavePair.Key;
var articulos = articulosWithSameClavePair.ToList();
foreach(var articulo in articulos)
{
Articulo articulo2 = new Articulo();
articulo2.ArtID = articulo.ArtID;
articulo2.Clave = articulo.Clave;
articulo2.ClaveAlterna = articulo.ClaveAlterna;
lista.Add(articulo2);
}
}
I suggest you read some examples of using GroupBy.
When you group a list, it will return a key and groued list and you are trying reach a single property of a list.
When you group an data, you can convert it to dictionary, It is not nessesary but better way for me. You can try this code:
var final = finalv.Union(finalc);
final = final.GroupBy(x => x.Clave).ToDictionary(s=> s.Key, s=> s.ToList();
foreach (var articulo in final)
{
foreach (var articuloItem in articulo.value)
{
Articulo articulo2 = new Articulo();
articulo2.ArtID = articuloItem.ArtID;
articulo2.Clave = articuloItem.Clave;
articulo2.ClaveAlterna = articuloItem.ClaveAlterna;
lista.Add(articulo2);
}
}
Below is my code
var dbClaimLink = this.Context.Set<ClaimLink>();
var claims = await DbSet
.Include(claim => claim.Parent)
.Include(link => link.ParentLinks)
.ToListAsync();
var newClaimLink = await dbClaimLink.ToListAsync();
var processedClaims = claims.Select(x =>
{
var claimLinks = x.ParentLinks;
if (!claimLinks.Any())
{
return x;
}
var hiddenParents = claimLinks.Select(p => claims.Find(t => t.Id == p.ClaimLinkId));
x.HiddenParents = hiddenParents;
return x;
});
foreach (var objClaim in processedClaims)
{
if (objClaim.Children == null)
objClaim.Children = new List<Claim>();
var lst = newClaimLink.Where(k=> k.ClaimLinkId == objClaim.Id).ToList();
if (lst.Any())
{
foreach (var item in lst)
{
IEnumerable<Claim> newChildren = claims.Where(p => p.Id == item.ClaimId);
objClaim.Children.Concat(newChildren);
}
}
}
it always return old children set without concatenate with new children. I want to those old and new children set concatenate in side of foreach loop
the Concat method returns a new collection with both values and does not alter the original.
Concat will return new object - result of concatination, so you need to save it somewhere: var result = objClaim.Children.Concat(newChildren);
Where is lazy operation, it does not execute in place, only after materialization (ToArray, or foreach call): claims.Where(p => p.Id == item.ClaimId).ToArray()
I've table with following data.
I want to group the data by DocumentID and then want to marge the DocPropIdentifyName and meta value together using comma separator. The output will be like following:
I'm doing it by using a foreach like below:
var test = (from r in lstDocSearch
group r by r.DocumentID
into g
select new
{
DocumentID = g.Key,
MetaValues = g.ToList()
}).ToList();
List<DocSearch> list = new List<DocSearch>();
foreach (var item in test)
{
foreach (var item2 in item.MetaValues)
{
var check = list.Exists(x => x.DocumentID == item2.DocumentID);
if (check)
{
var find = list.FirstOrDefault(x => x.DocumentID == item2.DocumentID);
find.MetaValue = find.MetaValue + ", " + item2.MetaValue;
find.DocPropIdentifyName = find.DocPropIdentifyName + ", " + item2.DocPropIdentifyName;
}
else
{
DocSearch objDocSearch = new DocSearch();
objDocSearch.DocumentID = item2.DocumentID;
objDocSearch.DocPropIdentifyID = item2.DocPropIdentifyID;
objDocSearch.DocPropIdentifyName = item2.DocPropIdentifyName;
objDocSearch.MetaValue = item2.MetaValue;
list.Add(objDocSearch);
}
}
}
But I would like to do this with linq rather than looping through the collection. Is it possible?
Do you mean something like this ? :
var test = (from r in lstDocSearch
group r by r.DocumentID
into g
select new
{
DocumentID = g.Key,
MetaValues = String.Join(",", g.Select(o => o.MetaValue)),
DocPropIdentifyNames = String.Join(",", g.Select(o => o.DocPropIdentifyName)),
}).ToList();
I have a function returning a report object, but currently i am going through a foreach look and then using the asQueryable method.
I would like to do it in one query and not have to use the AsQueryable function.
var query = from item in context.Dealers
where item.ManufacturerId == manufacturerId
select item;
IList<DealerReport> list = new List<DealerReport>();
foreach (var deal in query)
{
foreach (var bodyshop in deal.Bodyshops1.Where(x => x.Manufacturer2Bodyshop.Select(s => s.ManufacturerId).Contains(manufacturerId)))
{
DealerReport report = new DealerReport();
report.Dealer = deal.Name;
report.Bodyshop = bodyshop.Name;
short stat = bodyshop.Manufacturer2Bodyshop.FirstOrDefault(x => x.ManufacturerId == manufacturerId).ComplianceStatus;
report.StatusShort = stat;
list.Add(report);
}
}
return list.OrderBy(x => x.Dealer).AsQueryable();
I think you want something like this:
var query = from deal in context.Dealers
where deal.ManufacturerId == manufacturerId
from bodyshop in deal.Bodyshops1
where bodyshop.Manufacturer2Bodyshop.Select(s => s.ManufacturerId).Contains(manufacturerId)
let stat = bodyshop.Manufacturer2Bodyshop.FirstOrDefault(x => x.ManufacturerId == manufacturerId)
orderby deal.Name
select new DealerReport
{
Dealer = deal.Name,
Bodyshop = bodyshop.Name,
StatusShort = stat != null ? stat.ComplianceStatus : 0, // or some other default
};
return query;
I am able to filter the data with the following two parameters id1 and id2, and get accurate result of 10 records, from which have 9 with a price_type=cs and other with price-type=ms.
However, if I add price_type to the parameters id1 and id2 (id1=23456,567890&id2=6782345&price_type=ms), I get 3000 records instead of getting one record.
Am I missing something in the code. Any help would be very much appreciated.
var data = db.database_BWICs.AsQueryable();
var filteredData = new List<IQueryable<database_Data>>();
if (!string.IsNullOrEmpty(query.name))
{
var ids = query.name.Split(',');
foreach (string i in ids)
{
filteredData.Add(data.Where(c => c.Name != null && c.Name.Contains(i)));
}
}
if (!string.IsNullOrEmpty(query.id2))
{
var ids = query.id2.Split(',');
foreach (string i in ids)
{
filteredData.Add(data.Where(c => c.ID2!= null && c.ID2.Contains(i)));
}
}
if (!string.IsNullOrEmpty(query.id1))
{
var ids = query.id1.Split(',');
foreach (string i in ids)
{
filteredData.Add(data.Where(c => c.ID1!= null && c.ID1.Contains(i)));
}
}
if (query.price_type != null)
{
var ids = query.price_type.Split(',');
foreach (string i in ids)
{
filteredData.Add(data.Where(c => c.Type.Contains(i)));
}
}
if (filteredData.Count != 0)
{
data = filteredData.Aggregate(Queryable.Union);
}
Updated Code:
var data = db.database_BWICs.AsQueryable();
if (!string.IsNullOrEmpty(query.name))
{
var ids = query.name.Split(',');
data = data.Where(c => c.Name != null && ids.Contains(c.Name));
}
if (query.price_type != null)
{
var ids = query.price_type.Split(',');
data = data.Where(c => ids.Contains(c.Cover));
}
if (!String.IsNullOrEmpty(query.id1))
{
var ids = query.id1.Split(',');
data = data.Where(c => c.ID1!= null && ids.Contains(c.ID1));
}
Because you don't add filter to restrict, every filter adds datas to result.
It means you make OR between your filters, not AND.
And your usage of contains looks rather strange too : you're using String.Contains, while I would guess (maybe wrong) that you want to see if a value is in a list => Enumerable.Contains
You should rather go for something like this (withoud filteredData)
var data = db.database_BWICs.AsQueryable();
if (!string.IsNullOrEmpty(query.name))
{
var ids = query.name.Split(',');
data = data.Where(c => c.Name != null && ids.Contains(c.Name)));
}
//etc.
if (query.price_type != null)
{
var ids = query.price_type.Split(',');
data = data.Where(c => ids.Contains(c.Type));
}
EDIT
Well, if you wanna mix and or conditions, you could go for PredicateBuilder
Then your code should look like that (to be tested).
//manage the queries with OR clause first
var innerOr = Predicate.True<database_BWICs>();//or the real type of your entity
if (!String.IsNullOrEmpty(query.id1))
{
var ids = query.id1.Split(',');
innerOr = innerOr.Or(c => c.ID1!= null && ids.Contains(c.ID1));
}
if (!String.IsNullOrEmpty(query.id2))
{
var ids = query.id2.Split(',');
innerOr = innerOr.Or(c => c.ID2!= null && ids.Contains(c.ID2));
}
//now manage the queries with AND clauses
var innerAnd = Predicate.True<database_BWICs>();//or the real type of your entity
if (query.price_type != null)
{
var ids = query.price_type.Split(',');
innerAnd = innerAnd.And(c => ids.Contains(c.Type));
}
//etc.
innerAnd = innerAnd.And(innerOr);
var data = db.database_BWICs.AsQueryable().Where(innerAnd);