I have a piece of code that's performing badly, and need to rewite it to introduce a proper where clause before starting the .ToList however, that's where I'm getting stuck.
Currently the code looks lke this (roughly, I've taken some of the search criteria out to make it easier to display)
var Widgets = from b in _caspEntities.Widgets.Include("WidgetRegionLogs")
.Include("WidgetStatusLogs").Include("WidgetVoltageTests")
select b;
IEnumerable<Widget> results = Widgets.ToList();
if (comboBoxRegion.SelectedValue.ToString() != "0")
{
results = from b in results
where b.CurrentRegionLog != null && b.CurrentRegionLog.RegionId == int.Parse(comboBoxRegion.SelectedValue.ToString())
select b;
}
if (comboBoxStatus.SelectedValue != null)
{
results = from b in results
where b.CurrentStatusLog != null && b.CurrentStatusLog.StatusId == comboBoxStatus.SelectedValue.ToString()
select b;
}
if (txtCode.Text.Trim().Length > 0)
{
results = from b in results
where b.CodeNumber == txtCode.Text.Trim()
select b;
}
dataGridViewWidget.DataSource = results.ToList();
I can write the SQL easily enough, essentially the model is simple, I have a Widget it has a RegionLog and a StatusLog, both of which store a history. The current region and status are retrieved from this by grouping by WidgetID and selecting the most recent Date Updated (and then going off to Region and Status tables to get the actual value).
So, I need to translate this into LINQ, but to be honest I don't have a clue but am ken and willing to learn. In my head, I think I need to add some better where clauses, and then do the Widget.toList after I have applied the where clauses. I'm struggling with the CurrentRegionLog and CurrentStatusLog concepts as they are not populated until I run the IEnumerable.
If anyone can give some pointers, I'd be grateful,
Thanks
Edit - Added
public BatteryRegionLog CurrentRegionLog
{
get { return _currentRegionLog; }
}
private BatteryRegionLog _currentRegionLog
{
get
{
if (this.BatteryRegionLogs.Count > 0)
{
BatteryRegionLog log = this.BatteryRegionLogs.OrderByDescending(item => item.LastModifiedDate).First();
return log;
}
else
{
return null;
}
}
}
You can compose the query like this:
if (comboBoxRegion.SelectedValue.ToString() != "0")
{
var id = int.Parse(comboBoxRegion.SelectedValue.ToString()
Widgets = from b in Widgets
let currentRegionLog =
b.BatteryRegionLogs
.OrderByDescending(item => item.LastModifiedDate)
.FirstOrDefault()
where currentRegionLog.RegionId == id)
select b;
}
... // Same for the other criteria.
dataGridViewWidget.DataSource = Widgets.ToList();
The whole query is not executed before you do ToList(). As everything is translated to SQL you don't need the null check b.CurrentRegionLog != null. SQL will evaluate b.CurrentRegionLog.RegionId == id just fine when there is no CurrentRegionLog.
Edit
Since CurrentRegionLog is a calculated property of your Widget class it cannot be translated into SQL. I made an effort to incorporate the code of calculated property into the query in a way that only the basic navigation property is used, so EF can translate it to SQL again.
try remove this line:
IEnumerable<Widget> results = Widgets.ToList();
and just use the Widgets variable you get in at the top
The .ToList() goes to the database and materialiaze all the data into entities.
if you don't call the .ToList() the query is still "open" for a where clause
Related
First I'll explain a tad exactly what is happening, most of my values from my database are fine when I execute a linq to grab them, but specific ones such as ItemNumber and JobReference don't seem to carry across to the code, so I can't compare them.
Here is my linq statement for getting all the items.
public List<PurchaseOrderItem> GetPurchaseOrderItems()
{
return _context.PurchaseOrderItems.Include(p => p.PurchaseOrderHeader).ToList();
}
Here is the statement for getting a specific item:
public PurchaseOrderItem GetPurchaseOrderItem(PurchaseOrderItem selectedPoLine)
{
foreach (var item in GetPurchaseOrderItems())
if (item.JobReference == selectedPoLine.JobReference &&
item.ItemNumber == selectedPoLine.ItemNumber)
return item;
return null;
//return _context.PurchaseOrderItems.FirstOrDefault(p => p.JobReference == selectedPoLine.JobReference && p.ItemNumber == selectedPoLine.ItemNumber);
}
Here is my problem
See how the item.JobReference is empty?
Well this doesn't make sense because if I check in my database, the value for the JobReference is there and is correct ...
Basically I want to know if it is possible using as an example, Entity Framework, to retrieve all records from the DB table when the provided id doesn't match any of the id's in the table, but if there are id's that match then only retrieve those records.
It is possible to do obviously if you use an if statement or a ?: expression, as an example below.
var dbDocuments = new List<tblSalesQuoteDocument>();
if (id < 0)
dbDocuments = dbContext.tblSalesQuoteDocuments.ToList();
else
dbDocuments = dbContext.tblSalesQuoteDocuments.Where(x => x.HeaderId == id).ToList();
But I find this pretty ugly because if you want all records your URL is basically Documents/Index/-1 or any value less than 0.
Is there a better way?
Why I want one ActionResult is because I do a lot of filtering and code specific stuff in it. I could use two methods, 1 for all records, and another for specific records.
So should I do it as my question above or just use two methods and abstract all my filtering and other code away in Helper Methods to reduce code duplication?
You could add your filter expression on demand. Example:
ActionResult MyAction(int? id = null)
{
// ...
IQueryable<QuoteDocuments> docs = dbContext.tblSalesQuoteDocuments;
if (id != null)
{
docs = docs.Where(x => x.HeaderId == id.Value);
}
var list = docs.ToList();
// ...
}
docs = dbContext.tblSalesQuoteDocuments.Any(x => x.HeaderId == id)? dbContext.tblSalesQuoteDocuments.Where(x => x.HeaderId == id) : dbContext.tblSalesQuoteDocuments.ToList();
You should rather use
if(dbContext.tblSalesQuoteDocuments.Any(x => x.HeaderId == id)){
...
}
I am using the following parameterized linq query expression to query from approximately 100,000 records in a SQL server. Is there a better way?
public IList<Article> GetArticles(string language, string category, string subcategory, bool exclusives, int pageIndex, int pageSize = 200)
{
IQueryable<Article> query;
query = db.Articles.Where(t => t.IsActive && t.ArticleStatus);
if (exclusives) { query = query.Where(t => t.IsExclusive); }
if (language.ToUpper() != "ALL")
{
query = query.Where(t => t.Language.ToUpper() == language);
}
if (category.ToUpper() != "ALL")
{
query = query.Where(t => t.Category.ToUpper() == category);
}
if (subcategory.ToUpper() != "ALL")
{
query = query.Where(t => t.SubCategory.ToUpper() == subcategory);
}
query = query.Where(t => t.ArticleDate <= DateTime.Now);
query = query.OrderByDescending(t => t.ArticleNo).Skip((pageIndex - 1) * pageSize).Take(pageSize);
if (query.Any() == false)
{
return null;
}
else
{
return query.ToList();
}
}
Entity Framework isn't known for being the fastest option at runtime--it's supposed to be maintainable, easy, and fast to write. If you want something fast, go with a different ORM that lets you write T-SQL manually (or use EF's Database.SqlQuery<T>(string) methods).
That said, you're actually running your query twice.
if (query.Any() == false)
{
return null;
}
else
{
return query.ToList();
}
Each of these methods will execute a varied form of your query against the database.
Instead, test out performance with only calling with once.
var returnList = query.ToList();
if (returnList.Count == 0)
{
return null;
}
else
{
return returnList;
}
The improvement here should be evident, but it's possible that statistics will be messed up on your database, so definitely test.
Aside from that, I would just check indexes. Entity Framework builds its queries in sometimes-unusual ways, so I like to run the query plan explorer (or SQL Profiler) against them.
Simplest way to do that is to put a breakpoint at your ToList() call for a few inputs, hover or use the Watch windows to get query.ToString(), paste that in something that can get query plans (like VS's database tooling or SSMS), and see if it has anything obvious.
It's sometimes the case that you an rearrange calls to EF to improve the query, but generally the case that you'll want to reassess indexes server-side to meet what EF is giving you.
But, again, remember to do that for varied inputs--you're changing your query, so you want to make sure you've got all your bases covered.
I need to get the list of names that starts with special characters or numbers in the linq to sql query for my asp.net mvc(C#) application.
I have tried like this (Which may not be efficient):
public List<User> GetUsersStartingWithNonCharacter()
{
List<User> _dbUsers = this.GetAllUsers();
return _dbUsers.Where(p => ((p.FirstName != null && p.FirstName != string.Empty && !char.IsLetter(p.FirstName.ToLower()[0])) || (p.FirstName == null || p.FirstName == string.Empty))).ToList();
}
public List<Users> GetAllUsers()
{
return (from u in _context.pu_Users
where !u.Is_Deleted
select new User
{
UserId = u.User_Id,
Email = u.Email_Address,
FirstName = u.First_Name,
LastName = u.Last_Name
}).ToList();
}
Can anyone suggest the most efficient way to do this in linq to sql?
How do you know if it isn't already efficient? Use somekind of profiler tool, like SQL Server Profiler if you're using MSSQL, that way you can trace your call against the database and see the actual SQL. Of course you can only debug the code to see the generated SQL but it's easier with a profiler tool and you'll see how long time the query takes.
EDIT: I see one part in how you can make it more efficient:
public List<User> GetUsersStartingWithNonCharacter()
{
List<User> _dbUsers = this.GetAllUsers();
return _dbUsers.Where(p => ((p.FirstName != null && p.FirstName != string.Empty && !char.IsLetter(p.FirstName.ToLower()[0])) || (p.FirstName == null || p.FirstName == string.Empty))).ToList();
}
public IQueryable<Users> GetAllUsers()
{
return from u in _context.pu_Users
where !u.Is_Deleted
select new User
{
UserId = u.User_Id,
Email = u.Email_Address,
FirstName = u.First_Name,
LastName = u.Last_Name
};
}
Changing your GetAllUsersto return IQueryable will delay the query to execute until you've applied your filters. This might affect some other aspects of your design but you should consider it since that change might make your where clause run in the database instead of in the code which will result in less data traffic between your application and database. Again, use a profiler to see the difference :).
I'll use Regular Expression in this scenerio
Here is my sample code
return _dbUsers.Where(p=>p.FirstName!=String.Empty)
. Where(p => Regex.Match(p.Firstname[0].ToString(), "[a-zA-Z]").Success).ToList();
I suspect all rows will be retrieved and filted in your application due to the condition:
char.IsLetter(p.FirstName.ToLower()[0])
(Using a regular expression like suggested in another answer will also pull in all rows, and filter them on the client.)
It is possible to check characters in a string with the PATINDEX function, but it's seems only to be available for LINQ via the Entity framework.
You could write a stored procedure using PATINDEX directly to check for the first character to optimize your query. Sample queries can be found at http://www.databasejournal.com/features/mssql/article.php/3071531/Using-SQL-Servers-CHARINDEX-and-PATINDEX.htm.
Sometimes LINQ to whatever will not yield the most optimized solution, but that's just life. In most cases it will give clearer code, but special cases might require work arounds in order to use special operators of the underlying system.
What is the best way to assemble a dynamic WHERE clause to a LINQ statement?
I have several dozen checkboxes on a form and am passing them back as: Dictionary<string, List<string>> (Dictionary<fieldName,List<values>>) to my LINQ query.
public IOrderedQueryable<ProductDetail> GetProductList(string productGroupName, string productTypeName, Dictionary<string,List<string>> filterDictionary)
{
var q = from c in db.ProductDetail
where c.ProductGroupName == productGroupName && c.ProductTypeName == productTypeName
// insert dynamic filter here
orderby c.ProductTypeName
select c;
return q;
}
(source: scottgu.com)
You need something like this? Use the Linq Dynamic Query Library (download includes examples).
Check out ScottGu's blog for more examples.
I have similar scenario where I need to add filters based on the user input and I chain the where clause.
Here is the sample code.
var votes = db.Votes.Where(r => r.SurveyID == surveyId);
if (fromDate != null)
{
votes = votes.Where(r => r.VoteDate.Value >= fromDate);
}
if (toDate != null)
{
votes = votes.Where(r => r.VoteDate.Value <= toDate);
}
votes = votes.Take(LimitRows).OrderByDescending(r => r.VoteDate);
You can also use the PredicateBuilder from LinqKit to chain multiple typesafe lambda expressions using Or or And.
http://www.albahari.com/nutshell/predicatebuilder.aspx
A simple Approach can be if your Columns are of Simple Type like String
public static IEnumerable<MyObject> WhereQuery(IEnumerable<MyObject> source, string columnName, string propertyValue)
{
return source.Where(m => { return m.GetType().GetProperty(columnName).GetValue(m, null).ToString().StartsWith(propertyValue); });
}
It seems much simpler and simpler to use the ternary operator to decide dynamically if a condition is included
List productList = new List();
productList =
db.ProductDetail.Where(p => p.ProductDetailID > 0 //Example prop
&& (String.IsNullOrEmpty(iproductGroupName) ? (true):(p.iproductGroupName.Equals(iproductGroupName)) ) //use ternary operator to make the condition dynamic
&& (ID == 0 ? (true) : (p.ID == IDParam))
).ToList();
I came up with a solution that even I can understand... by using the 'Contains' method you can chain as many WHERE's as you like. If the WHERE is an empty string, it's ignored (or evaluated as a select all). Here is my example of joining 2 tables in LINQ, applying multiple where clauses and populating a model class to be returned to the view. (this is a select all).
public ActionResult Index()
{
string AssetGroupCode = "";
string StatusCode = "";
string SearchString = "";
var mdl = from a in _db.Assets
join t in _db.Tags on a.ASSETID equals t.ASSETID
where a.ASSETGROUPCODE.Contains(AssetGroupCode)
&& a.STATUSCODE.Contains(StatusCode)
&& (
a.PO.Contains(SearchString)
|| a.MODEL.Contains(SearchString)
|| a.USERNAME.Contains(SearchString)
|| a.LOCATION.Contains(SearchString)
|| t.TAGNUMBER.Contains(SearchString)
|| t.SERIALNUMBER.Contains(SearchString)
)
select new AssetListView
{
AssetId = a.ASSETID,
TagId = t.TAGID,
PO = a.PO,
Model = a.MODEL,
UserName = a.USERNAME,
Location = a.LOCATION,
Tag = t.TAGNUMBER,
SerialNum = t.SERIALNUMBER
};
return View(mdl);
}
Just to share my idea for this case.
Another approach by solution is:
public IOrderedQueryable GetProductList(string productGroupName, string productTypeName, Dictionary> filterDictionary)
{
return db.ProductDetail
.where
(
p =>
(
(String.IsNullOrEmpty(productGroupName) || c.ProductGroupName.Contains(productGroupName))
&& (String.IsNullOrEmpty(productTypeName) || c.ProductTypeName.Contains(productTypeName))
// Apply similar logic to filterDictionary parameter here !!!
)
);
}
This approach is very flexible and allow with any parameter to be nullable.
You could use the Any() extension method. The following seems to work for me.
XStreamingElement root = new XStreamingElement("Results",
from el in StreamProductItem(file)
where fieldsToSearch.Any(s => el.Element(s) != null && el.Element(s).Value.Contains(searchTerm))
select fieldsToReturn.Select(r => (r == "product") ? el : el.Element(r))
);
Console.WriteLine(root.ToString());
Where 'fieldsToSearch' and 'fieldsToReturn' are both List objects.
This is the solution I came up with if anyone is interested.
https://kellyschronicles.wordpress.com/2017/12/16/dynamic-predicate-for-a-linq-query/
First we identify the single element type we need to use ( Of TRow As DataRow) and then identify the “source” we are using and tie the identifier to that source ((source As TypedTableBase(Of TRow)). Then we must specify the predicate, or the WHERE clause that is going to be passed (predicate As Func(Of TRow, Boolean)) which will either be returned as true or false. Then we identify how we want the returned information ordered (OrderByField As String). Our function will then return a EnumerableRowCollection(Of TRow), our collection of datarows that have met the conditions of our predicate(EnumerableRowCollection(Of TRow)). This is a basic example. Of course you must make sure your order field doesn’t contain nulls, or have handled that situation properly and make sure your column names (if you are using a strongly typed datasource never mind this, it will rename the columns for you) are standard.
System.Linq.Dynamic might help you build LINQ expressions at runtime.
The dynamic query library relies on a simple expression language for formulating expressions and queries in strings.
It provides you with string-based extension methods that you can pass any string expression into instead of using language operators or type-safe lambda extension methods.
It is simple and easy to use and is particularly useful in scenarios where queries are entirely dynamic, and you want to provide an end-user UI to help build them.
Source: Overview in Dynamic LINQ
The library lets you create LINQ expressions from plain strings, therefore, giving you the possibility to dynamically build a LINQ expression concatenating strings as you require.
Here's an example of what can be achieved:
var resultDynamic = context.Customers
.Where("City == #0 and Age > #1", "Paris", 50)
.ToList();