Develop Reference

C# for comparing two lists that may contain null values

I am comparing two excel sheets using C#.
I have assigned the sheets to datatables and I hope to compare the values in each cell. I have converted the datatables to lists and I'm iterating through them comparing them. How do I avoid a NullReferenceException if one of the values contains a null?
I am hitting a NullReferenceException at this point in the code:
if (lst1[i].ToString() != lst2[j].ToString())
Is there a way to guard against nulls here or do I need to handle null values earlier in the code?
Thanks

One good way to do this is to ask if the value is null before call .ToString().
So, would be like:
if (lst1[i] != null && lst2[j] != null && lst1[i].ToString() != lst2[j].ToString())

if (lst1[i]?.ToString() != lst2[j]?.ToString())
More info on the ?. operator

Just test for null before dereferencing lst1[i] and lst2[j]
if(lst1[i] is not null and lst2[j] is not null)
{
if (lst1[i].ToString() != lst2[j].ToString())
{
...
}
}

Assuming
if both items are null then they are equal;
if both items are not null and their string representation are equal then they are equal;
in any other case they are not equal;
you can make comparison of your lists more readable With following LINQ query.
var query = list1
.Zip(list2, (a, b) => new { a, b })
.Any(m => !((m.a == null && m.b == null) || (m.a != null && m.b != null && m.a.ToString() == m.b.ToString())));
if (query)
{
Console.WriteLine("Lists are not equal.");
}
else
{
Console.WriteLine("Lists are equal.");
}

Since you're doing ToString() it looks like both lists are typed as objects. Since they're coming from Excel they're presumably all either strings or simple value types.
In that case you can do
if(lst1.SequenceEquals(lst2))
That compares each item in both lists and ensure that they're equal. It accounts for nulls. You wouldn't need to compare each item one at a time. This compares the entire list. I'd write unit tests to make sure it doesn't throw any curves.
If for some reason you wanted to do a text-based comparison ("2" == 2) then you could create a comparer like this:
public class ObjectTextComparer : IEqualityComparer<object>
{
public bool Equals(object x, object y)
{
return x?.ToString() == y?.ToString();
}
public int GetHashCode(object obj)
{
return obj?.GetHashCode() ?? 0;
}
}
and then this would return true:
var x = new object[] { "x", 2, null, "y" };
var y = new object[] { "x", "2", null, "y" };
var comparer = new ObjectTextComparer();
var listsAreEqual = x.SequenceEqual(y, comparer);
That second option is a can of worms, though, if the values in one list are stored in Excel as text and the other ones as numbers or dates.
For example if one list contains a date and the other list contains the same date stored as text, the comparer is going to convert the first one to a string, but it might not be formatted the same way as the other one stored as text, and the two strings won't match.

unable to cast of type 'System.Data.EnumerableRowCollection`1[System.String]' to type 'System.IConvertible'

I need the column value(FundSpreadDurationContribution) of a data table(residing in dataset) through LINQ which fetches the above error (heading)
Elaborated: when a row cell has value Spread Duration--IR Swap, need the corresponding column cell FundSpreadDurationContribution value.
double testvalue = Convert.ToDouble(raptorDS.Tables[RaptorTable.DurationContribBySector].AsEnumerable().Where(r =>
r.Field<string>(RaptorColumns.FundCode) == fundDescriptionColumn &&
r.Field<string>(RaptorColumns.Component) == Component.B8_DURATION_CONTRIBUTION_BY_SECTOR &&
r.Field<string>(RaptorColumns.Sector) == "Spread Duration--IR Swap").Select(s => s.Field<string>(RaptorColumns.FundSpreadDurationContribution)))
I am learner of LINQ.

// First check if this query will return any results
var records =
raptorDS.Tables[RaptorTable.DurationContribBySector].AsEnumerable().Where(r =>
r.Field<string>(RaptorColumns.FundCode) == fundDescriptionColumn &&
r.Field<string>(RaptorColumns.Component) == Component.B8_DURATION_CONTRIBUTION_BY_SECTOR &&
r.Field<string>(RaptorColumns.Sector) == "Spread Duration--IR Swap");
// CHeck if any result
if (records.Any())
{
// We have results so let's go through each record and try to get that value
// converted to a double
List<double> values = new List<double>();
List<string> badValues = new List<string>();
foreach (var thisRecord in records)
{
var fsdc = thisRecord.Field<string>(RaptorColumns.FundSpreadDurationContribution);
if (!string.IsNullOrWhiteSpace(fsdc))
{
double val = 0;
if (double.TryParse(fsdc, val))
{
values.Add(val);
}
else
{
badValues.Add(fsdc);
}
}
}
// Do whatever you need to do with values here
// and bad values here
}

Where returns a collection, which cannot be converted to a double. Use one of these Linq methods to get one answer to convert:
Single
SingleOrDefault
First
FirstOrDefault

This is just playing around with Datatypes which fixed the type casting issue
var sumFundSpreadDuration = raptorDS.Tables[RaptorTable.DurationContribBySector].AsEnumerable().Where(r =>
r.Field<string>(RaptorColumns.FundCode) == fundDescriptionColumn &&
r.Field<string>(RaptorColumns.Component) == Component.B8_DURATION_CONTRIBUTION_BY_SECTOR &&
r.Field<string>(RaptorColumns.Sector) == "Spread Duration--IR Swap")
.Select(s => s.Field<double?>(RaptorColumns.FundSpreadDurationContribution)).FirstOrDefault();
If you see the datatype of variable is changed to Variant from double. Also, the datatype of column FundSpreadDurationContribution is changed to double. Thanks to CodingYoshi for providing insights of datatypes in linq

check whether LINQ query returns rows

I need to check whether a query returns rows and if it does, change it to a string, but if it doesn't, return "In Progress". I thought the below code would work but this is always true:
if (System.Linq.Enumerable.Count(columns) == 0)<--- always true but it shouldn't be
And when there isn't a row returned to columns I get the following error in my jQuery Ajax:
"The cast to value type \u0027Int32\u0027 failed because the materialized value is null. Either the result type\u0027s generic parameter or the query must use a nullable type."
Here's my WebMethod:
using (dbPSREntities5 myEntities = new dbPSREntities5())
{
var thisId = myEntities.tbBreadCrumbs.Where(x => x.ProjectID == projectID && x.StatusID == statusID).Max(x => x.BreadCrumbID);
var columns = myEntities.tbBreadCrumbs
.Where(x => x.BreadCrumbID == thisId)
.Select(x => x.CreateDateTime)
.ToList();
if (System.Linq.Enumerable.Count(columns) == 0)
{
var formattedList = columns
.Select(d => null != d
? d.Value.ToString("MMM dd, yyyy")
: string.Empty) // this is just one example to handle null
.ToList();
return formattedList;<-- return this if there is a BreadCrumbID (columns would have a row)
}
else
{
return "In Progress";<--- there was no BreadCrumbID (columns would have 0 rows)
}
}

You could use Any()
MSDN Enumerable.Any Method

You first check for Count == 0, doesn't sound right, I guess you need the opposite check. You should use Any or Count() > 0 check, Like:
if (columns.Any()) //Or columns.Count() > 0
{
var formattedList = columns
.Select(d => null != d
? d.Value.ToString("MMM dd, yyyy")
: string.Empty) // this is just one example to handle null
.ToList();
return formattedList;<-- return this if there is a BreadCrumbID (columns would have a row)
}
else
{
return "In Progress";<--- there was no BreadCrumbID (columns would have 0 rows)
}
You have to convert your List to string, in order for your method to return a string.

Your condition is wrong. Count must be greater than zero

Use the .Any() method provided by List<T>:
If (columns.Any())
{
// do your bidding
} else {
// in progress code
}
Also your method has two different return signatures. That won't compile. You can't return a List or a string, unless the return type is object (not recommended).
I suggest to return a null list and check if it's null in your UI layer and then display the appropriate string or date values, since they end up as strings in the UI anyway.

You only needs to ckeck .Count() > 0
if (columns.Count() > 0)
{
var formattedList = columns
.Select(d => null != d
? d.Value.ToString("MMM dd, yyyy")
: string.Empty) // this is just one example to handle null
.ToList();
return formattedList;<-- return this if there is a BreadCrumbID (columns would have a row)
}
else
{
return "In Progress";<--- there was no BreadCrumbID (columns would have 0 rows)
}

Linq to SQL Strange SQL Translation

I have a simple query that is generating some odd SQL translations, which is blowing up my code when the object is saturated.
from x in DataContext.MyEntities
select new
{
IsTypeCDA = x.EntityType == "CDA"
}
I would expect this query should translate to:
SELECT (CASE WHEN [t0].[EntityType] = #p1 THEN 1 ELSE 0 END) as [IsTypeCDA]
...
Instead I get this :
SELECT
(CASE
WHEN #p1 = [t0].[EntityType] THEN 1
WHEN NOT (#p1 = [t0].[EntityType]) THEN 0
ELSE NULL
END) AS [IsTypeCDA]
...
Since I'm saturating a POCO where IsTypeCDA is a bool, it blows up stating I can't assign null to bool.
Any thoughts?
Edit: fixed the property names so they make sense...

from x in DataContext.MyEntities
select new
{
IsTypeCDA = x.EntityType == null
}
c# interpretation (false) or sql interpretation (null)?
This runs in sql so sql interpretation. That's why the funky translation - the operation does return a nullable bool.
Use this query to punt that nullable bool into a plain old bool.
from x in DataContext.MyEntities
select new
{
IsTypeCDA = ((bool?)(x.EntityType == "CDA")) ?? false
}

Linq to SQL does the "strange" comparison because database values can be NULL. Unfortunately, it doesn't seem to do well translating the tri-valued comparison result. At first blush, I'd wonder why any of your entities have NULL for their EntityType. Modifying your schema to disallow NULL values would be the most straightforward solution.
However, assuming the table is defined this way for legitimate business reasons you can work around it in a couple ways.
First off, you could coalesce the NULLs into a placeholder value.
from x in DataContext.MyEntities
select new
{
IsTypeCDA = (x.EntityType ?? "") == "CDA"
}
Alternately, using String.Equals generates a more thorough comparison that handles NULLs.
from x in DataContext.MyEntities
select new
{
IsTypeCDA = string.Equals(x.EntityType, "CDA")
}
Neither of these will generate the simple SQL you were expecting, but they'll both get the job done.

I would have IsTypeCDA as a get only property and select into that class:
public class SomeName
{
public string EntityType { get; set; }
public bool IsTypeCDA { get { return EntityType == EntityType.CDA; } }
}
...
from x in DataContext.MyEntities
select new SomeName
{
EntityType = x.EntityType
}

Compare nullable types in Linq to Sql

I have a Category entity which has a Nullable ParentId field. When the method below is executing and the categoryId is null, the result seems null however there are categories which has null ParentId value.
What is the problem in here, what am I missing?
public IEnumerable<ICategory> GetSubCategories(long? categoryId)
{
var subCategories = this.Repository.Categories.Where(c => c.ParentId == categoryId)
.ToList().Cast<ICategory>();
return subCategories;
}
By the way, when I change the condition to (c.ParentId == null), result seems normal.

Other way:
Where object.Equals(c.ParentId, categoryId)
or
Where (categoryId == null ? c.ParentId == null : c.ParentId == categoryId)

The first thing to do is to put on logging, to see what TSQL was generated; for example:
ctx.Log = Console.Out;
LINQ-to-SQL seems to treat nulls a little inconsistently (depending on literal vs value):
using(var ctx = new DataClasses2DataContext())
{
ctx.Log = Console.Out;
int? mgr = (int?)null; // redundant int? for comparison...
// 23 rows:
var bosses1 = ctx.Employees.Where(x => x.ReportsTo == (int?)null).ToList();
// 0 rows:
var bosses2 = ctx.Employees.Where(x => x.ReportsTo == mgr).ToList();
}
So all I can suggest is use the top form with nulls!
i.e.
Expression<Func<Category,bool>> predicate;
if(categoryId == null) {
predicate = c=>c.ParentId == null;
} else {
predicate = c=>c.ParentId == categoryId;
}
var subCategories = this.Repository.Categories
.Where(predicate).ToList().Cast<ICategory>();
Update - I got it working "properly" using a custom Expression:
static void Main()
{
ShowEmps(29); // 4 rows
ShowEmps(null); // 23 rows
}
static void ShowEmps(int? manager)
{
using (var ctx = new DataClasses2DataContext())
{
ctx.Log = Console.Out;
var emps = ctx.Employees.Where(x => x.ReportsTo, manager).ToList();
Console.WriteLine(emps.Count);
}
}
static IQueryable<T> Where<T, TValue>(
this IQueryable<T> source,
Expression<Func<T, TValue?>> selector,
TValue? value) where TValue : struct
{
var param = Expression.Parameter(typeof (T), "x");
var member = Expression.Invoke(selector, param);
var body = Expression.Equal(
member, Expression.Constant(value, typeof (TValue?)));
var lambda = Expression.Lambda<Func<T,bool>>(body, param);
return source.Where(lambda);
}

My guess is that it's due to a rather common attribute of DBMS's - Just because two things are both null does not mean they are equal.
To elaborate a bit, try executing these two queries:
SELECT * FROM TABLE WHERE field = NULL
SELECT * FROM TABLE WHERE field IS NULL
The reason for the "IS NULL" construct is that in the DBMS world, NULL != NULL since the meaning of NULL is that the value is undefined. Since NULL means undefined, you can't say that two null values are equal, since by definition you don't know what they are.
When you explicitly check for "field == NULL", LINQ probably converts that to "field IS NULL". But when you use a variable, I'm guessing that LINQ doesn't automatically do that conversion.
Here's an MSDN forum post with more info about this issue.
Looks like a good "cheat" is to change your lambda to look like this:
c => c.ParentId.Equals(categoryId)

You need to use operator Equals:
var subCategories = this.Repository.Categories.Where(c => c.ParentId.Equals(categoryId))
.ToList().Cast<ICategory>();
Equals fot nullable types returns true if:
The HasValue property is false, and the other parameter is null. That is, two null values are equal by definition.
The HasValue property is true, and the value returned by the Value property is equal to the other parameter.
and returns false if:
The HasValue property for the current Nullable structure is true, and the other parameter is null.
The HasValue property for the current Nullable structure is false, and the other parameter is not null.
The HasValue property for the current Nullable structure is true, and the value returned by the Value property is not equal to the other parameter.
More info here Nullable<.T>.Equals Method

Or you can simply use this. It will also translate to a nicer sql query
Where((!categoryId.hasValue && !c.ParentId.HasValue) || c.ParentId == categoryId)

What about something simpler like this?
public IEnumerable<ICategory> GetSubCategories(long? categoryId)
{
var subCategories = this.Repository.Categories.Where(c => (!categoryId.HasValue && c.ParentId == null) || c.ParentId == categoryId)
.ToList().Cast<ICategory>();
return subCategories;
}

Linq to Entities supports Null Coelescing (??) so just convert the null on the fly to a default value.
Where(c => c.ParentId == categoryId ?? 0)