I'm writing a UI that allows a someone to lookup users by their first and/or last name. For example, if you typed in "Mike" for the first name and "Jo" for the last name, it would return "Mike Jones", "Mike Johnson" and "Mike Jobs". I use the following LINQ statement for this search:
var users = (from u in context.TPM_USER
where u.LASTNAME.ToLower().Contains(LastName.ToLower())
&& u.FIRSTNAME.ToLower().Contains(FirstName.ToLower())
select u);
(There may or may not be a better way to do a case-insensitive like clause, but this seems to work)
The problem is if the user types in a first or last name, but then leaves the other field empty. If I type in "Mike" for the first name and leave the Last Name field blank, I want to return all Mikes regardless of their last name. The above query returns no results unless both fields are filled in with at least something.
I tried:
var users = (from u in context.TPM_USER
where (LastName == "" || u.LASTNAME.ToLower().Contains(LastName.ToLower()))
&& (FirstName == "" || u.FIRSTNAME.ToLower().Contains(FirstName.ToLower()))
select u);
However, I still get no results unless both fields are filled out. I've verified under the debugger that LastName == "" is indeed true.
UPDATE:
I did some more debugging and this is actually an Oracle issue. The query being generated is:
--Replaced the field list with * for brevity
SELECT * FROM TPMDBO.TPM_USER "Extent1"
WHERE (('jones' = '') OR ((INSTR(LOWER("Extent1".LASTNAME), LOWER('jones'))) > 0)) AND (('' = '') OR ((INSTR(LOWER("Extent1".FIRSTNAME), LOWER(''))) > 0))
Which at first glance appears to be correct. However, Oracle does not seem to correctly short-circuit the phrase ('' = ''). In fact, if I do:
select * from TPM_USER where '' = ''
I get zero rows. I'm not enough of an Oracle expert to know how this query should be written, but either way it's an Entity Framework dialect bug.
Just add the predicates conditionally:
var users = from u in context.TPM_USER select u;
if (!string.IsNullOrWhiteSpace(FirstName))
users = users.Where(u => u.FIRSTNAME.ToLower().Contains(FirstName.ToLower()));
if (!string.IsNullOrWhiteSpace(LastName))
users = users.Where(u => u.LASTNAME.ToLower().Contains(LastName.ToLower()));
Or only the LASTNAME predicate as conditional one.
Later addition:
An expression like Where(u => u.FIRSTNAME.ToLower()... is better to be avoided. They cause any indexes on FIRSTNAME to be ignored, because the field value is converted first and then compared (see here for more details).
There's a big chance you don't need these lower-case conversions. Check the database collation of the field. If it's case-insensitive (CI), which it probably is, you don't need these conversions.
Are you sure that FirstName and LastName aren't null?
You might try writing it like this instead...
string LowerFirstName = (FirstName + "").ToLower();
string LowerLastName = (LastName + "").ToLower();
var users = (from u in context.TPM_USER
where (LowerLastName == "" || u.LASTNAME.ToLower().Contains(LowerLastName))
&& (LowerFirstName == "" || u.FIRSTNAME.ToLower().Contains(LowerFirstName))
select u);
FYI, if anyone runs into this issue with Oracle, here's a workaround:
var users = (from u in context.TPM_USER
where (LastName == null|| u.LASTNAME.ToLower().Contains(LastName.ToLower()))
&& (FirstName == null || u.FIRSTNAME.ToLower().Contains(FirstName.ToLower()))
select u);
This will get converted to:
'' is null
In SQL, which Oracle interprets as true.
You could simply create a conditional statement around your query:
if (String.IsNullOrWhiteSpace(LastName) && !String.IsNullOrWhiteSpace(FirstName))
{
var users = (from u in context.TPM_USER
where (u.FIRSTNAME.ToLower().Contains(FirstName.ToLower()))
select u);
}
else if (String.IsNullOrWhiteSpace(FirstName) && !String.IsNullOrWhiteSpace(LastName))
{
var users = (from u in context.TPM_USER
where (u.LASTNAME.ToLower().Contains(LastName.ToLower()))
select u);
}
May be you can try checking the length of the search terms to see if it is working in Oracle PL/SQL.
var users = (from u in context.TPM_USER
where ((LastName ?? "").Trim().Length == 0 || u.LASTNAME.ToLower().Contains(LastName.ToLower()))
&& ((FirstName ?? "").Trim().Length == 0 || u.FIRSTNAME.ToLower().Contains(FirstName.ToLower()))
select u);
Related
I have to Perform Global Search on table means if user enters any keyword or multiple keywords and clicks on search button then based on the entered keywords it should bring all the combination records.
We have to search those 2 keywords in every column of a table (Like clause in SQL with OR operator for multiple keywords) and query should fetch the data.
I have around 200k of records in the database.
First calling function to load the data
if ((Role)user.Role == Role.InternalAdministrator || (Role)user.Role ==
Role.InternalStaff)
{
listJobs = (
from d in db.Jobs
where d.TimeCreated.Value.Year >= 2020
select new JobModel()
{
AlternatePickupDelivery = d.AlternatePickupDelivery,
Branch = (
from b in db.Branches
where b.BranchId == d.ProcessingCity
select b.Branch1
).FirstOrDefault(),
ClientName = d.ClientName,
ClientId = d.ClientId,
ContactName = d.ContactName,
MatterReference = d.MatterReference,
JMSNumber = d.JmsNumber,
JobDescription = d.JobDescription,
JobId = d.JobId,
JobShortDescription = d.JobShortDescription,
OrderType = d.OrderType,
OrderTypeDisplay = (
from jt in db.JobTypes
where jt.Id == d.OrderType
select jt.JobTypeName
).FirstOrDefault(),
ProcessingCity = d.ProcessingCity ?? 0,
DisplayProcessingCity = (
from jt in db.ProcessingCities
where jt.ProcessingCityId == d.ProcessingCity
select jt.ProcessingCity1
).FirstOrDefault(),
Status = d.Status,
DisplayStatus = (
from jt in db.JobStatuses
where jt.Id == d.Status
select jt.JobStatusName
).FirstOrDefault(),
StatusDisplayOrder = (from js in db.JobStatuses
where js.Id == d.Status
select js.DisplayOrder).FirstOrDefault(),//d.JobStatus.DisplayOrder,
StatusLastModifiedBy = (
from u in db.Users
where (u.UserId == d.StatusLastModifiedById)
select u.FirstName + " " + u.LastName
).FirstOrDefault(),
StatusLastModifiedById = d.StatusLastModifiedById,
StatusLastModified = d.StatusLastModified ?? DateTime.UtcNow,
CreatedByDisplay = (
from u in db.Users
where (u.UserId == d.CreatedById)
select u.FirstName + " " + u.LastName
).FirstOrDefault(),
CreatedById = d.CreatedById,
ModifiedByDisplay = (
from u in db.Users
where (u.UserId == d.LastModifiedById)
select u.FirstName + " " + u.LastName
).FirstOrDefault(),
LastModifiedById = d.LastModifiedById,
TimeCreated = d.TimeCreated ?? DateTime.UtcNow,
TimeDelivered = (d.Status == (int)JMS4.Utilities.JobStatus.Delivered) ? d.StatusLastModified :
null,
TimeDue = d.TimeDue ?? DateTime.UtcNow,
TimeReady = d.TimeReady ?? DateTime.UtcNow,
TimeZoneId = timeZoneId.ToString(),
ExtClientId = d.ExtClientId,
Address = d.Address,
ReceivedBy = d.ReceivedBy,
ContactPhone = d.ContactPhone,
AfterHourContactNumber = d.AfterHoursContactNumber,
Email = d.Email,
CostEstimateNumber = d.CostEstimateNumber,
LastModifiedBy = d.LastModifiedBy,
MatterType = d.MatterType,
QaData = d.QaData,
InternalInstructions = d.InternalInstructions,
GlobalSearch = d.GlobalSearch
}
);
Then calling second function if search textbox have any keyword/keywords to search
jobs = jobs.Where(x => x.JMSNumber.ToLower().Contains(keyword.ToLower())
|| (x.ClientName != null && x.ClientName.ToLower().Contains(keyword.ToLower()))
|| (x.MatterReference != null && x.MatterReference.ToLower().Contains(keyword.ToLower()))
|| (x.ContactName != null && x.ContactName.ToLower().Contains(keyword.ToLower()))
|| (x.JobShortDescription != null &&
x.JobShortDescription.ToLower().Contains(keyword.ToLower()))
|| (x.StatusLastModifiedBy != null &&
x.StatusLastModifiedBy.ToLower().Contains(keyword.ToLower()))
|| (x.Address != null && x.Address.ToLower().Contains(keyword.ToLower()))
|| (x.Email != null && x.Email.ToLower().Contains(keyword.ToLower()))
|| (x.LastModifiedBy != null &&
x.LastModifiedBy.ToString().ToLower().Contains(keyword.ToLower()))
|| (x.CostEstimateNumber != null &&
x.CostEstimateNumber.ToLower().Contains(keyword.ToLower()))
|| (x.ClientId != null && x.ClientId.ToString().ToLower().Contains(keyword.ToLower()))
|| (x.JobDescription != null && !String.IsNullOrEmpty(x.JobDescription.ToString()) &&
x.JobDescription.ToString().ToLower().Contains(keyword.ToLower()))
|| (x.CreatedByDisplay != null && !String.IsNullOrEmpty(x.CreatedByDisplay.ToString()) &&
x.CreatedByDisplay.ToString().ToLower().Contains(keyword.ToLower()))
|| (x.ModifiedByDisplay != null && !String.IsNullOrEmpty(x.ModifiedByDisplay.ToString()) &&
x.ModifiedByDisplay.ToString().ToLower().Contains(keyword.ToLower()))
|| (x.InternalInstructions != null &&
x.InternalInstructions.ToString().ToLower().Contains(keyword.ToLower()))
);
AFter using these queries, it is taking more than 3+ minutes to fetch the records.
Please suggest how can i improve the search performance and optimize the query.
To optimise a query like this against a database there are a few rules to try and follow
Make sure the query is passed through to the database, do not operate in memory
Remove or reduce the use of functions
Don't bother comparing nulls
Split the query into multiple parallel queries
Improve the structure to optimise the query
The general idea is that you want to make your comparison directly in the index entries, function or conversions on records in the database will not use the indexes. Databases are specifically optimised to query against Indexes, so it will be important to also create the necessary indexes on your search columns.
You have tagged this as linq-to-sql so we assume that your query is being passed through to the DB, it is important that you make sure it does. The following code and advice will only work if the LINQ expression is a genuine IQueryable<T> that will be resolved into SQL.
If your database uses a CASE INSENSITIVE collation, then you can drop all the .ToLower() function calls, you want to avoid function calls so any indexes can be accessed directly.
Event though C# is sensitive to case by default, if the LINQ query is translated to SQL then it will obey to collation settings for standard LIKE '%' + #param + '%' comparison.
Skip the null comparison, just like the .ToLower() it is not necessary in SQL to check the nullability of a field first before executing a comparison on that field.
This is already a far better filter:
jobs = jobs.Where(x => x.JMSNumber.Contains(keyword)
|| x.ClientName.Contains(keyword)
|| x.MatterReference.Contains(keyword)
|| x.ContactName.Contains(keyword)
|| x.JobShortDescription.Contains(keyword)
|| x.StatusLastModifiedBy.Contains(keyword)
|| x.Address.Contains(keyword)
|| x.Email.Contains(keyword))
|| x.LastModifiedBy.Contains(keyword))
|| x.CostEstimateNumber.Contains(keyword))
|| x.ClientId.ToString().Contains(keyword))
|| x.JobDescription.Contains(keyword))
|| x.CreatedByDisplay.Contains(keyword))
|| x.ModifiedByDisplay.Contains(keyword))
|| x.InternalInstructions.Contains(keyword))
);
Store values in a searchable format.
We can do better that the above if you really need to search on numeric fields, like in this case ClientId then it helps to store the numeric values in a string based column because our search argument is a string.
The easiest way to implement a variant of a field in the database is to use a computed column, however it needs to be a write computed or peristed column to realise the benefit for a search index. Make the computed column out of the expression:
CAST(ClientId as char(10))
The same rule applies for any other column that might need a function applied to it, you will see greater performance if you move the function evaluation to the time that the record is INSERT or UPDATE, which happens with a much lower frequency to reads via SELECT.
Normalize the structure, most of your comparisons are on a user - displayname if the query joins on to a user table, then you only have one seek for any user that matches instead of 1 separate seek for each user
There is a clear related table here for the User who is applying the data modifications. This can add a great deal of redundant information in the search query. Ideally we do not search across the user fields, as any match there would bring up all records that are associated with them, it is not usually a good search candidate, unless users do not edit many records. So if you can, exclude them from the general search, and allow the user to pick from a list of users to scope the results, or to search from the users in parallel with the main search
Now the search is much quicker: (this assumes a new column called ClientIdString)
jobs = jobs.Where(x => x.JMSNumber.Contains(keyword)
|| x.ClientName.Contains(keyword)
|| x.MatterReference.Contains(keyword)
|| x.ContactName.Contains(keyword)
|| x.JobShortDescription.Contains(keyword)
|| x.Address.Contains(keyword)
|| x.Email.Contains(keyword))
|| x.CostEstimateNumber.Contains(keyword))
|| x.ClientIdString.Contains(keyword))
|| x.JobDescription.Contains(keyword))
|| x.InternalInstructions.Contains(keyword))
);
Hypothetical User Search for searching and then filtering by the users:
var userIds = db.Users.Where(u => u.UserName.Contains(keyword))
.Select(u => u.Id)
.ToList();
//Filter to only rows that match the user lookup
if (jobs.Any())
{
jobs = jobs.Where(x => userIds.Contains(x.StatusLastModifiedByUserId)
|| userIds.Contains(x.LastModifiedByUserId)
|| userIds.Contains(x.CreatedByUserId)
|| userIds.Contains(x.ModifiedByUserId)
);
}
If the schema is NOT already normalised, then I strongly suggest you do at least normalise out the users into their own table, indexing the possible users is much more efficient than searching across 200K+ records.
It is also possible that we can write the query directly in SQL. Sometimes we can write much more efficient SQL by hand than we might be able to achieve through LINQ don't feel bad about that, just recognise that it is one of many tools at your disposal.
Most Linq providers will give you an explicit mechanism for executing raw SQL that will return into a linq expression. The detail for this is out of scope, but searching is a specific scenario where this is accepted.
There are other external options too like SQL Server Full Text Search or MySQL FULLTEXT Indexes or even Microsoft Azure Search or Elastic Search. These external mechanisms can be used to return the search content directly or they might return references that you can use to access the records in your local DB.
Many NoSQL providers can be used to construct an efficient search index, the products I listed above as simply designed for searching and are likely to implement a lot of industry
Indexes
All of the above assumes that you have implemented adequate indexes on the underlying data store. Searching can have such a large impact on the user experience, it is worth putting in the effort to get it right.
Assessing and Implementing Indexes is out of scope for this question
I want to get the records with "Restricted" at top.
here Is my query:
var records = (from entry in db.Table1
orderby entry.Always_Prohibited
select new
{
RowID = entry.RowID,
VehicleMake = entry.Make,
VehicleModel = entry.Model,
Restricted = (entry.Always_Prohibited == null || entry.Always_Prohibited == "False") ? "Restricted" : "Not Restricted"
}).ToList();
I tried by Orderby but it is not working because entry.Always_Probibited is a string field.
Please suggest me.
If you have only two values, simply order descending:
from entry in db.Table1
orderby entry.Always_Prohibited descending
If you have more, assign integer values to your strings:
from entry in db.Table1
orderby entry.Always_Prohibited=="A" ? 0 : entry.Always_Prohibited=="B" ? 1 : 2 // and so on
As a side note, strings are a pretty terrible way of storing state in databases. You should redesign it to store it as well defined integers (preferably as foreign keys in master lookup tables, ie strongly typed enums).
User then below give code. It will help to you.
var records = (from entry in db.Table1.AsQueryable();
orderby entry.Always_Prohibited
select new
{
RowID = entry.RowID,
VehicleMake = entry.Make,
VehicleModel = entry.Model,
Restricted = (entry.Always_Prohibited == null || entry.Always_Prohibited == "False") ? "Restricted" : "Not Restricted"
}).ToList();
i have the following code:
int selectedcourseId = Convert.ToInt32(c1.Text);
var cid = (from g in re.Sections where g.CourseID == selectedcourseId select g.CourseID);
int selectedinstructorid = Convert.ToInt32(c2.Text);
var iid = (from u in re.Sections where u.InstructorID == selectedinstructorid select u.InstructorID);
i want to compare the two (selectedcourseId) with (cid) and (selectedinstructorid) with (iid) in if-statement such as:
if (selectedcourseId = cid && selectedinstructorid = iid)
{
MessageBox.Show("it already exists");
}
i have tried many things that didnt work our because i have limited knowledge.
thank you very much in advance for any comment or answer
You can change your code as: (but it is meaningless for your situation to check this)
if (selectedcourseId == cid.First() && selectedinstructorid == iid.First())
First of all for checking equality in if statement you must use ==, not =. And the second is the IQueryable<T> allows you to execute a query against a specific data source, but it uses deferred execution. For executing it in your case, you can use First().
But, I suggest that you are just learning how to use LINQ and therefore you have written this code.
I don't know what you are trying to achive. But, if you want to search if there is any result with that ID's, the you must use Any():
var result1 = from g in re.Sections where g.CourseID == selectedcourseId select g.CourseID;
var result2 = from u in re.Sections where u.InstructorID == selectedinstructorid select u.InstructorID;
if(result1.Any() && result2.Any()) { ... }
Or, if you want to find if there is any row which has specified CourseID and InstructorID, then you can call one Any():
if(re.Sections.Any(x => x.CourseID == selectedcourseId && x.InstructorID == selectedinstructorid))
{ ... }
Let me try to find the X from this XY-problem. I guess you want to check if there is already a combination of courseid + instructorid. Then use a single query:
var data = from section in re.Sections
where section.InstructorID == selectedinstructorid
&& section.CourseID == selectedcourseId
select section;
if(data.Any())
{
MessageBox.Show("it already exists");
}
You should not do it in two queries, because the two results might be related to two different rows. This would lead to "false positives" when an instructor handles some section, and a course has some instructors, but the two matches do not belong to the same row:
course instructor
------ ----------
100 10
101 15
102 20
If you are looking for a combination (101, 10) it is not enough to see that 100 is present and 10 is present; you need to check that the two belong to the same row in order to consider it a duplicate.
You can fix this by making a "check presence" query, like this:
var existing = re.Sections
.Any(s => s.InstructorID == selectedinstructorid && s.CourseID == selectedcourseId);
if (existing) {
MessageBox.Show("it already exists");
}
if (selectedcourseId = cid && selectedinstructorid = iid)
this will not work, since single '=' is an assignment, not a comparation (which is '==')
also, you can try to do something like this
var cid = (int)((from g in re.Sections where g.CourseID == selectedcourseId select g.CourseID).FirstOrDefault());
so you select the first or default record from your list and cast it to int
I am trying in a linq query to check if one of the fields is null but i get this error whatever i do.
"Non-static method requires a target."
This is my code :
var users = from s in db.Users
where s.DepartmentId == booking.Item.DepartmentId && s.UserEmail != null
select s;
is any way to go through this error and be able to actually check if UserEmail is null?
p.s : i am using asp.net mvc entity framework.
Be sure that the first 2 letters are uppercase (DBNull.Value).
EDIT:
Try to copy your booking item into a local variable.
var departmentId = booking.Item.DepartmentId;
var users = from s in db.Users
where s.DepartmentId == departmentId && s.UserEmail != null
select s;
var users = from s in db.Users
where s.DepartmentId == booking.Item.DepartmentId && s.UserEmail != DbNull.Value
select s;
You need to compare it to DbNull.Value
I am thoroughly frustrated right now. I am having an issue with LINQ-To-SQL. About 80% of the time, it works great and I love it. The other 20% of the time, the query that L2S creates returns the correct data, but when actually running it from code, it doesn't return anything. I am about to pull my hair out. I am hoping somebody can see a problem or has heard of this before. Google searching isn't returning much of anything.
Here is the linq query...
var query = from e in DataLayerGlobals.GetInstance().db.MILLERTIMECARDs
where e.deleted_by == -1
&& e.LNAME == lastName
&& e.FNAME == firstName
&& e.TIMECARDDATE == startDate.ToString("MM/dd/yyyy")
group e by e.LNAME into g
select new EmployeeHours
{
ContractHours = g.Sum(e => e.HRSCONTRACT),
MillerHours = g.Sum(e => e.HRSSHOWRAIN + e.HRSOTHER),
TravelHours = g.Sum(e => e.HRSTRAVEL)
};
This is the generated query....
SELECT SUM([t0].[HRSCONTRACT]) AS [ContractHours],
SUM([t0].[HRSSHOWRAIN] + [t0].[HRSOTHER]) AS [MillerHours],
SUM([t0].[HRSTRAVEL]) AS [TravelHours]
FROM [dbo].[MILLERTIMECARD] AS [t0]
WHERE ([t0].[deleted_by] = #p0)
AND ([t0].[LNAME] = #p1)
AND ([t0].[FNAME] = #p2)
AND ([t0].[TIMECARDDATE] = #p3)
GROUP BY [t0].[LNAME]
Now when I plug in the EXACT same values that the linq query is using into the generated query, I get the correct data. When I let the code run, I get nothing.
Any ideas?
What type is TIMECARDDATE? Date, datetime, datetime2, smalldatetime, datetimeoffset or character?
Any chance local date/time settings are messing up the date comparison of startDate.ToString(...)? Since you're sending #p3 as a string, 01/02/2009 may mean Feb 1st or January 2nd, depending on the date/time setting on the server.
My instinct is telling me that you need to be pulling out DataLayerGlobals.GetInstance().db.MILLERTIMECARDs into an IQueryable variable and executing your Linq query against that, although there really should be no difference at all (other than maybe better readability).
You can check the results of the IQueryable variable first, before running the Linq query against it.
To extend this concept a bit further, you can create a series of IQueryable variables that each store the results of a Linq query using each individual condition in the original query. In this way, you should be able to isolate the condition that is failing.
I'd also have a look at the LNAME & FNAME data types. If they're NCHAR/NVARCHAR you may need to Trim the records, e.g.
var query = from e in DataLayerGlobals.GetInstance().db.MILLERTIMECARDs
where e.deleted_by == -1
&& e.LNAME.Trim() == lastName
&& e.FNAME.Trim() == firstName
&& e.TIMECARDDATE == startDate.ToString("MM/dd/yyyy")
group e by e.LNAME into g
select new EmployeeHours
{
ContractHours = g.Sum(e => e.HRSCONTRACT),
MillerHours = g.Sum(e => e.HRSSHOWRAIN + e.HRSOTHER),
TravelHours = g.Sum(e => e.HRSTRAVEL)
};