Hello I have just started to try and understand Parallel Linq and with my first attempt I am having no success. I am using EF 4.0 and a repository pattern class that I created to query the data. I don't believe that the repository pattern is the problem but I could be mistaken.
The database that I have isn't setup the way that I would like it but hey I inherited the system. The code that I am having the problem with is below:
var gId = Sql.ToGuid(Request["ID"]);
var lOrdersGridList = new OrdersGridList(); //Class that only contains properties
var lOrdersForContact = new BaseRepository<ORDER>()
.Find(i => i.ORDERS_CONTACTS.Where(b => b.CONTACT_ID == gId).Count() > 0).AsParallel()
.Select(i =>
new OrdersGridList
{
ORDER_ID = i.ID,
ORDER_NUM = i.ORDER_NUM,
SHIPPING_ACCOUNT_ID = i.ORDERS_ACCOUNTS.Where(b => b.ORDER_ID == i.ID && b.ACCOUNT_ROLE == "Ship To").First().ACCOUNT_ID,
SHIPPING_ACCOUNT_NAME = i.ORDERS_ACCOUNTS.Where(b => b.ORDER_ID == i.ID && b.ACCOUNT_ROLE == "Ship To").First().ACCOUNT.NAME,
SHIPPING_CONTACT_ID = i.ORDERS_CONTACTS.Where(b => b.ORDER_ID == i.ID && b.CONTACT_ROLE == "Ship To").First().CONTACT_ID,
SHIPPING_CONTACT_NAME = i.ORDERS_CONTACTS.Where(b => b.ORDER_ID == i.ID && b.CONTACT_ROLE == "Ship To")
.Select(b => new { SHIPPING_CONTACT_NAME = (b.CONTACT.FIRST_NAME + ' ' + b.CONTACT.LAST_NAME) }).First().SHIPPING_CONTACT_NAME,
NAME = i.NAME
}).DefaultIfEmpty(lOrdersGridList).ToList<OrdersGridList>();
grdMain.DataSource = lOrdersForContact.ToDataTable().DefaultView; //ToDataTable extension function converts the List Object to a datatable.
If I run the Code without AsParallel the code executes with no problem however, once I add AsParallel I receive the following error:
Also just in case you wanted to see this is the class that I am declaring as new for the Select Above:
public class OrdersGridList : EntityObject
{
public string ORDER_NUM { get; set; }
public Guid ORDER_ID { get; set; }
public Guid SHIPPING_ACCOUNT_ID { get; set; }
public string SHIPPING_ACCOUNT_NAME { get; set; }
public Guid SHIPPING_CONTACT_ID { get; set; }
public string SHIPPING_CONTACT_NAME { get; set; }
public string NAME { get; set; }
}
If I remove all of the relationships that are used to retrieve data in the select I don't receive any errors:
var lOrdersForContact = new BaseRepository<ORDER>()
.Find(i => i.ORDERS_CONTACTS.Where(b => b.CONTACT_ID == gId).Count() > 0).AsParallel()
.Select(i =>
new OrdersGridList
{
ORDER_ID = i.ID,
ORDER_NUM = i.ORDER_NUM,
//SHIPPING_ACCOUNT_ID = i.ORDERS_ACCOUNTS.Where(b => b.ORDER_ID == i.ID && b.ACCOUNT_ROLE == "Ship To").First().ACCOUNT_ID,
//SHIPPING_ACCOUNT_NAME = i.ORDERS_ACCOUNTS.Where(b => b.ORDER_ID == i.ID && b.ACCOUNT_ROLE == "Ship To").First().ACCOUNT.NAME,
//SHIPPING_CONTACT_ID = i.ORDERS_CONTACTS.Where(b => b.ORDER_ID == i.ID && b.CONTACT_ROLE == "Ship To").First().CONTACT_ID,
//SHIPPING_CONTACT_NAME = i.ORDERS_CONTACTS.Where(b => b.ORDER_ID == i.ID && b.CONTACT_ROLE == "Ship To")
// .Select(b => new { SHIPPING_CONTACT_NAME = (b.CONTACT.FIRST_NAME + ' ' + b.CONTACT.LAST_NAME) }).First().SHIPPING_CONTACT_NAME,
NAME = i.NAME
}).DefaultIfEmpty(lOrdersGridList).ToList<OrdersGridList>();
I would be more than happy to give more information if required. Any help you can provide on using PLinq I would appreciate it.
To me it appears like the BaseRepository class creates some kind of LINQ to Entities query using the Find and Select parameters.
Using AsParellel is made for LINQ to Objects, where your code actually evaluates the expressions you pass. Other LINQ dialects, including LINQ to Entities, translate it into a different query language like SQL. The SQL server may do reasonable parallelisation itself.
Related
I have a query to search in single table where records are at Parent, Child Relationship like below
Table
Id
ParentId
Name
Status
so my query is like
var projects = from p in _projectSetupRepository.Table
from all in _projectSetupRepository.Table
where p.Status == status && p.ParentId == null &&
all.Status == status && ((all.ParentId == null && all.Id == p.Id) || all.ParentId == p.Id)
select p;
if (!string.IsNullOrEmpty(search))
projects = projects.Where(c => c.Name.Contains(search)).OrderBy(c => c.Name);
but I don't get actual results of parents if search with the child's name. What was the issue in the query?
PS
table contains thousands of data and performance is very important
PS
public class ProjectSetup
{
public int Id { get; set; }
public int? ParentId { get; set; }
public string Name { get; set; }
public bool Status { get; set; }
public ProjectSetup Project{ get; set; }
public virtual ICollection<ProjectSetup> SubProjects { get; set; }
}
Id
ParentId
Name
Status
1
null
P1
true
2
1
T1
true
3
1
T2
true
4
3
Q1
true
You can find all parents with specific child name(this query searches Level1 childs only):
var status = true;
var search = "T";
var projects = (from parent in context.Projects
join child in context.Projects on parent.Id equals child.ParentId into joinedT
from pd in joinedT.DefaultIfEmpty()
where parent.Status == status
&& parent.ParentId == null //filter top level parents only?
&& pd.Name.Contains(search)
select parent).Distinct().OrderBy(c => c.Name);
foreach(var p in projects)
{
Console.WriteLine(p.Id+":"+p.Name);
}
Console.WriteLine("Found results:"+ projects.Count());
here's fiddle: https://dotnetfiddle.net/PaCidA
If you are looking for multi-level solution I suggest you to take a look at hierarchyid data type and its usage in EF LINQ
https://softwarehut.com/blog/tech/hierarchyid-entity-framework
https://kariera.future-processing.pl/blog/hierarchy-in-the-entity-framework-6-with-the-hierarchyid-type/
Check for performance the folloging algorithm. Idea that we can Include several Project and check on the client what to load again. Algorithm uses EF Core navigation properties fixup.
var query = in_projectSetupRepository.Table
.Include(p => p.Project.Project.Project) // you can increase count of loaded parents
.Where(p => p.Status == status)
.AsQueryable();
var loadedDictionary = new Dictionary<int, ProjectSetup>>();
var projects = query;
if (!string.IsNullOrEmpty(search))
projects = projects.Where(c => c.Name.Contains(search));
while (true)
{
var loaded = projects.ToList();
// collect loaded with parents
foreach(var p in loaded)
{
var current = p;
do
{
if (!loadedDictionary.ContainsKey(current.Id))
loadedDictionary.Add(current.Id, current);
current = current.Project;
}
while (current != null)
}
// collect Ids of incomplete
var idsToLoad = loadedDictionary.Values
.Where(x => x.ParentId != null && x.Project == null)
.Select(x => x.ParentId.Value)
.ToList();
// everything is ok
if (idsToLoad.Count == 0)
break;
// make query to load another portion of objects
projects = query.Where(p => idsToLoad.Contains(p.Id));
}
var result = loadedDictionary.Values
.OrderBy(c => c.Name)
.ToList();
So I've found a way to do this, but I think its rather ugly and a bit hacky.
I have a Contact model which contains a list of messages, my models can be seen below:
public class Contact
{
[Key]
public int ContactId { get; set; }
public string Name { get; set; }
public string Owner { get; set; }
public ICollection<Message> Messages { get; set; } = new Collection<Message>();
}
public class Message
{
[Key]
public int MessageId { get; set; }
public string MessageContent { get; set; }
public string Sender { get; set; }
public string Receiver { get; set; }
public DateTime DateCreated { get; set; }
}
what I wanted to do, and which I have done (badly) is combine the list of messages from both contacts models like by creating a viewModel containing a list of messages, and looping through the results of the contact messages and adding into the view model as seen below:
public class ContactMessageViewModel
{
public ICollection<Message> Messages { get; set; } = new Collection<Message>();
}
public async Task<ContactMessageViewModel> GetContactMessages(string username, string contactName)
{
ContactMessageViewModel MessagesList = new ContactMessageViewModel();
var messagesListUser = await _dbContext.Contact
.Include(m => m.Messages)
.Where(x =>
(x.Owner == username) && x.Name == contactName).ToListAsync();
var messagesListContact = await _dbContext.Contact
.Include(m => m.Messages)
.Where(x =>
(x.Owner == contactName) && x.Name == username).ToListAsync();
foreach(var message in messagesListContact)
{
foreach(var m in message.Messages)
{
MessagesList.Messages.Add(m);
}
}
foreach (var message in messagesListUser)
{
foreach (var m in message.Messages)
{
MessagesList.Messages.Add(m);
}
}
return MessagesList;
}
This just feels super duper ugly to me, and I feel there is a better way to do this. I'm wondering if there is a better looking (and better performing) way to do this? Possibly just using linq?
Thanks!
You can combine the queries into a single query, and then return all the messages:
return new ContactMessageViewModel
{
Messages = await _dbContext.Contact
.Include(m => m.Messages)
.Where(x =>
(x.Owner == username && x.Name == contactName) ||
(x.Owner == contactName && x.Name == username))
.SelectMany(x => x.Messages)
.ToListAsync()
};
Are you able to use something similar to the below which checks for either.
var messagesList = await (from c in_dbContext.Contact
from m in c.Messages
where (c.Owner == username && c.Name == contactName) || (c.Owner == contactName && c.Name == username)
select m).ToListAsync();
The code above assume owner and name will never be the same, otherwise you could store the username and contact name in an array and check the owner and name are both in the array.
The alternative is to store the IQueryable<Messages> in a variable, without doing a ToList. This won't hit the database until you need to enumerate. The benefit of this is you can manipulate the IQueryable and not hit the database until you need the items.
It seems you can do it in one query:
var messagesList = await _dbContext.Contact
.Include(m => m.Messages)
.Where(x => (x.Owner == username && x.Name == contactName)
|| (x.Owner == contactName && x.Name == username))
.ToListAsync();
and use it to build MessagesList.Messages.
Or, if it is ok to get in results entries with x.Owner == x.Name (or such situations are impossible) you can do:
var names = new []{contactName, username};
var messagesList = await _dbContext.Contact
.Include(m => m.Messages)
.Where(x => names.Contains(x.Owner) && names.Contains(x.Name))
.ToListAsync();
UPD
It seems that you need only messages, so you can try something like this:
var messagesList = await _dbContext.Contact
.Where(x => (x.Owner == username && x.Name == contactName)
|| (x.Owner == contactName && x.Name == username))
.SelectMany(m => m.Messages)
.ToListAsync();
// foreach m in messagesList MessagesList.Messages.Add(m);
I have a problem with type convert var to List<SubToSubMenu>.
Firstly, I select data from database that is ok !!!!. I received data with var variable, but i can't convert var type to List<SubToSubMenu> data type.
This is my LINQ statement:
var ss =
db
.SubToSubMenus
.Join(
db.MenuPermissions,
s => s.ID,
p => p.SubToSubMenuId,
(s, p) => new { s, p })
.Where(w => w.s.Active == true && w.p.RoleId == roleId && w.p.hasPermission == true)
.Select(s => new
{
ID = s.s.ID,
SubToSubMenuName = s.s.SubToSubMenuName,
Description = s.s.Description,
})
.ToList();
This is SubToSubMenu class:
[Table("SubToSubMenus")]
public class SubToSubMenu : AceEntityBase
{
public SubToSubMenu()
{ }
[Key]
public string ID { get; set; }
public string SubToSubMenuName { get; set; }
public string Description { get; set; }
public string SubMenuID { get; set; }
}
var is not a type, it's syntactic sugar. You have an anonymous type that has no relation whatsoever to your SubToSubMenu type.
Instead of projecting into an anonymous type:
.Select(s => new { ... })
Project into the type you want:
.Select(s => new SubToSubMenu { ... })
You shouldn't need to project here. You're already selecting the SubToSubMenu from the database, so there should be no need to 'recreate' the class later in the expression chain.
var ss =
db
.SubToSubMenus
.Join(
db.MenuPermissions,
s => s.ID,
p => p.SubToSubMenuId,
(s, p) => new { s, p })
.Where(w => w.s.Active == true && w.p.RoleId == roleId && w.p.hasPermission == true)
This is good so far. You've joined two tables and applied the correct filters.
.Select(s => new
{
ID = s.s.ID,
SubToSubMenuName = s.s.SubToSubMenuName,
Description = s.s.Description,
})
.ToList();
OK stop here. If the ultimate goal of this query is to select only SubToSubMenu entities, you can replace this part with just
.Select(s => s.s);
...and ignore the rest of the subsequent statements.
However, you could also go one step further and make the association between the SubToSubMenu and MenuPermissions entities implicit in your EF configuration, so you'll have no need to .Join in LINQ. Given this, the eventual query should be similar to:
var ss = db.SubToSubMenus
.Where(stsm => stsm.Active
&& stsm.MenuPermissions.RoleId == roleId
&& stsm.MenuPermissions.HasPermission);
Try this:
var ss =
db
.SubToSubMenus
.Join(
db.MenuPermissions,
s => s.ID,
p => p.SubToSubMenuId,
(s, p) => new { s, p })
.Where(w => w.s.Active == true && w.p.RoleId == roleId && w.p.hasPermission == true)
.Select(s => new
{
ID = s.s.ID,
SubToSubMenuName = s.s.SubToSubMenuName,
Description = s.s.Description,
})
.ToList()
.Select(s => new SubToSubMenu()
{
ID = s.ID,
SubToSubMenuName = s.SubToSubMenuName,
Description = s.Description,
})
.ToList();
I have added a simple projection to the end of your query. This is to keep the code as close to how you had it to begin with and to aid with any future refactoring.
In this case, it can certainly be coded as a single projection.
I have written a code like this in my .NET project:
var v = ctx.Items
.Where(x => x.userid== user.userid)
.Select(e => new MyViewModel
{
Title = e.Title,
CurrentPrice = e.CurrenctPrice.Value,
ItemID = e.ItemID.ToString(),
Sales = e.Transactions.Where(p => p.TransactionDate >= intoPast && p.TransactionDate <= endDate).Sum(x => x.QuantityPurchased)
})
.Where(x => x.Sales > 0 && ((filterWord == "") || (filterWord != "" && x.Title.ToLower().Contains(filterWord.ToLower()))));
where "ctx" is my object context variable...
And this is the ViewModelClass that I use:
public class MyViewModel
{
public string Title { get; set; }
public int Sales { get; set; }
public string ItemID { get; set; }
public double CurrentPrice { get; set; }
}
The thing that most bugs me here is the sales property... As you can see i set its value in select statement. This way all my data gets enumerated every time...
What I was thinking here is to create a method called "getsales()"... And then to just simply call the GetSales method in my where statement like this:
.Where(x=>X.GetSales(/*neccesary parameters here*/)...)
In order to avoid having multiple enumerations...
But I'm not really sure how to do it...
Can someone help me out here?
I think this is what you're looking for:
var v = ctx.Items
.Where(x =>
x.userid == user.userid &&
(filterWord == "" || x.Title.ToLower().Contains(filterWord.ToLower())))
.Select(e => new MyViewModel
{
Title = e.Title,
CurrentPrice = e.CurrentPrice.Value,
ItemID = e.ItemID.ToString(),
Sales = e.Transactions
.Where(p =>
p.TransactionDate >= intoPast &&
p.TransactionDate <= endDate)
.Sum(x => x.QuantityPurchased)
})
.Where(x => x.Sales > 0);
I have a linq query like
var tmp = (from t in db.sometable
where (bunch of conditions)
group t by new { t.somefield} into g
select new
{
Name = g.Key.someobject.Name,
xColor = g.Sum(a => (int?)a.LineItems.Where(m => m.Product == "x" && m.Color == true)
.Sum(m => m.Quantity)),
xBW = g.Sum(a => (int?)a.LineItems.Where(m => m.Product == "x" && m.Color == false)
.Sum(m => m.Quantity))
})
Now I want to optimize this query. If I had to write a stored procedure I would have written some thing like:
if(a.LineItems.Product == 'x')
{
if(m.Color == true)
xColor++;
else
xBW++;
}
so that I get aggregated value in one scan. Basically I wanted to optimize this query to avoid multiple scans.
check out this simple code snippet in Linqpad, which does the way you expect, for running in visual studio, remove the Dump method call. I have simplified the class structure a bit for a simple demo, it is just a POCO, not a complex type containing another list inside
void Main()
{
var testList = Test.CreateList();
var tmp = (from t in testList
group t by new { t.Name } into g
select new
{
Name = g.Key.Name,
xColor = g.Sum(a =>
{
if (a.Product == "x" && a.Color == true)
return a.Quantity;
return 0;
}),
xBW = g.Sum(a =>
{
if (a.Product == "x" && a.Color == false)
return a.Quantity;
return 0;
})
});
tmp.Dump();
}
public class Test
{
public string Name { get; set; }
public string Product { get; set;}
public bool Color { get; set;}
public int? Quantity { get; set;}
public static List<Test> CreateList()
{
return new List<Test>()
{
new Test {Name="A",Color = true,Product="x",Quantity=5},
new Test {Name="A",Color = true,Product="x",Quantity=5},
new Test {Name="A",Color = true,Product="x",Quantity=5},
new Test {Name="B",Color = true,Product="x",Quantity=5},
new Test {Name="B",Color = true,Product="x",Quantity=5},
new Test {Name="B",Color = true,Product="x",Quantity=5}
};
}
}
However whether this is efficient or not can still be debated
You can do the same thing that would work in SQL. If there's a subset of data (lines matching particular criteria) then select those first into an array. Then you can execute your larger query against that subset.