Develop Reference

How to ignore Symbols in string comparison and dictionary lookup

It looks like I have a problem with the symbols encodings between string built by my program and string retrieved from another datasource.
Here is a .NET Fiddle and here is the explanation:
var context = new List<Foo>
{
new Foo { Name = "SoW.Probing.4GEPCCore.CaptSite[1].S1U" },
new Foo { Name = "SoW.Probing.4GEPCCore.CaptSite[2].S1U" },
new Foo { Name = "SoW.Probing.2G3GPSCore.CaptSite[1].GnGpU" },
new Foo { Name = "SoW.Probing.2G3GPSCore.CaptSite[2].GnGpU" }
};
var nameToCheckPresence = GetStringFromAnotherDataSource(); // the value of the string is for example: "SoW.Probing.4GEPCCore.CaptSite.S1U"
nameToCheckPresence = nameToCheckPresence.Replace("CaptSite", "CaptSite[1]");
var foo = context.FirstOrDefault(f => f.Name == nameToCheckPresence); // Should return an object since one object does have that name
My problem is that foo is null. It works if I use this code line:
var foo = context .FirstOrDefault(f => CultureInfo.CurrentCulture.CompareInfo.Compare(f.Name, nameToCheckPresence , CompareOptions.IgnoreSymbols) == 0);
So clearly, I have a problem with symbols encoding (the .? the [ ]?). My true problem is that later, I am doing the same thing with a dictionary. The hashcode of the strings are different and the dictionary lookup also failed:
var dictionary = context.ToDictionary(f => f.Name);
var foo = dictionary[nameToCheckPresence]; // Should return the object but failed and throw a KeyNotFoundException
Is there a way to change the string symbols encoding in a global manner in the application? (WPF application in my case)
As the context can be very large, it is planed to use a Dictionary also in the first place. So if you provide me a solution that only works with Dictionary, it is not a problem.
Just for the record, the datasource is a SQLite database in which is a copy of the data of a MySQL database filled by another WPF application (running on the same computer, no specific culture setup). Finally, the nameToCheckPresence is extracted from a larger string by ANTLR4CS.

This is not a satisfactory answer, but that's all I find to solve the problem. Instead of looking into the dictionary through the indexor, I am doing a linq query:
dictionary.FirstOrDefault(pair => CultureInfo.CurrentCulture.CompareInfo.Compare(pair.Key, localFactName, CompareOptions.IgnoreSymbols) == 0).Value;
But doing this, I lost all the benefit of the dictionary access complexity. If anyone has a better solution, I will take it!

Why doesn't IEnumerable.Where() find my objects in DynamoDBContext.Scan() results?

While using AWS DynamoDB "Object Persistence Model" in C#, ran into an interesting issue; parsing results of a Scan operation. In the code below, my entries in Datastore.userIndicators (which is a Dictionary with Lists of objects, indexed by usernames) are always empty lists.
var allIndicators = context.Scan<Indicator>();
Datastore.globalIndicators = allIndicators.Where(i => i.UserName == "*").ToList();
var userDefinedIndicators = allIndicators.Where(i => i.UserName != "*");
foreach (var username in userDefinedIndicators.Select(i => i.UserName).Distinct())
{
Datastore.userIndicators[username] = userDefinedIndicators.Where(i => i.DynamoDbRangeKey.StartsWith(username)).ToList();
}
So, for example, if I have entries in my table that include an attribute "UserName" with value "userA", when running this code the Dictionary "Datastore.userIndicators" will end up with an entry for key "userA" but the corresponding value will be an empty list.

After fiddling with this and following a hunch, I modified the assignment of
var allIndicators = context.Scan<Indicator>();
to
var allIndicators = context.Scan<Indicator>().ToList();
Voila!
It turns out (as confirmed by AWS SDK documentation), the return of the DynamoDBContext.Scan() method is lazy-loaded. Calling .ToList() forces enumeration and loads all the results.

Returning Id's from a list in Entity Framework

I'm having an issue with Entity Framework, when I execute SaveChanges, the context has quite a few objects associated with it, some of the objects are updated and some are added, afterwards I want to use the Id's of all these items (the Id's for the added items are only assigned on insert in the database). After the save changes a list of all the objects is empty.
I've seen samples on the site where the object is updated after the save so I suspect it might be how I'm getting the list of objects in the first place
Here's my code:
// Lots of processing to create or update objects
using (var localContext = this.context)
{
var updatedObjects = localContext.ChangeTracker.Entries().Where(e => e.Entity is GenerationEvent && (e.State == EntityState.Modified || e.State == EntityState.Added));
var updatedEvents = updatedObjects.Select(e => (GenerationEvent)e.Entity);
// The list has 5 items in at this point
localContext.SaveChanges();
// This list is now empty
DoSomethingWithList(updatedEvents);
}
Thanks in advance for any help.

The variable updatedEvents is a Linq query. Linq queries aren't executed immediately. By the time it is executed in your code it won't find any updated object anymore. Putting .ToList() after the Linq query will execute it immediately.
var updatedEvents = updatedObjects.Select(e => (GenerationEvent)e.Entity).ToList();

first your "using" statement is odd.
it should be
using (var context = new MyContext()) //passing in optional connection string.
{
}
Then the way you access your entities seem odd or i have no clue what you are doing there...
var updatedObjects = localContext.ChangeTracker.Entries().Where(e => e.Entity is GenerationEvent && (e.State == EntityState.Modified || e.State == EntityState.Added));
var updatedEvents = updatedObjects.Select(e => (GenerationEvent)e.Entity);
Seems like you are asking the context for all items which are considered "Add" or "Updated"
Then you are accepting the changes to the context. eg SaveChanges().
I fully expect "updatedEvents" to be empty, after save-changes is called.
Change you stuff... to something like
using (var context = new MyContext()) //passing in optional connection string.
{
LIst<EntityType> listOfChangedEntities = //ToDo:either passed in or generated
context.EntityType.AddRandge(listOfChangedEntities);
context.SaveChanges();
//after SaveChanges has been done all the entities in the
//listOfChangedEntities will now have there id's
//for update, fetch the entities... change them and Update them
}
I suspect that you are trying to create some sort of generic code to handle any type of Entity without specifying its type. Your code is not suited for this as it is, if this is what you are trying to do, I would modify the question to ask what you are trying to achieve. But the above is the Normal way of getting the Id's of the entities which have been inserted.
The other examples you are passably talking about is where they use foreign keys and navigation properties to automatically associate related entities, but your code looks way off from that.
UPDATE
routine
public static DoWork()
{
var context = new MyContext();
List<GenerationEvent > internalEntityType = new List<GenerationEvent ();
foreach(var item in SomeList)
{
var newItemEntity = new GenerationEvent();
newItemEntity.Name = "Test";
context.GenerationEvent.Add(newItemEntity);
//add to internal list
internalEntityType.Add(newItemEntity )
}
context.SaveChanges();
var first_id = internalEntityType.FirstOrDefault().Id;
//first_id will not be 0 it will be the Id the database gave it.
}

New object, but error: an object with the same key already exists in the objectstatemanager. the objectstatemanager cannot track multiple (...)

Student and Teacher are i relationship many-to-many.
When I have Student existed in database, and want to add new Teacher, I try code below.
But in line:
addedTeachers.ForEach(a => dbStudent.Teacher.Add(a)); I get error
"an object with the same key already exists in the objectstatemanager. the objectstatemanager cannot track multiple objects with the same key".
What's wrong?
void Update(Student s)
{
using(var context = new MyEntities(connectionString))
{
context.ContextOptions.LazyLoadingEnabled = false;
var dbStudent = context.Student.Include("Teacher").Where(a => a.Id == s.Id).SingleOrDefault();
var dbTeachers = dbStudent.Teacher.ToList();
var newTeachers = s.Teacher.ToList();
var addedTeachers = newTeachers.Except(dbTeachers).ToList();
var deletedTeachers = dbTeachers.Except(newTeachers).ToList();
addedTeachers.ForEach(a => dbStudent.Teacher.Add(a));
deletedTeachers.ForEach(a => dbStudent.Teacher.Remove(a));
context.SaveChanges();
}
}
EDIT
What's strange else:
just before this line with exception dbStudent.Teacher.Count is 0. But after exception is 1. Of course addedTeachers.Count is also 1, but debugger doesn't reach the next line.

The problem is s and dbStudent has the same primary key. Your Teacher instances in s.Teacher collection may refer to the s instance. Then when you call addedTeachers.ForEach(a => dbStudent.Teacher.Add(a));, EF will identity all objects linked to teacher instance and try to add them too.
Try
addedTeachers.ForEach(a => { a.Students.Remove(s);
a.Students.Add(dbStudent);
dbStudent.Teacher.Add(a);});

I think your problem is your Except statement. It is comparing your collection items using the Default comparer which is comparing to see if the items reference the same object in memory when you actually intend it to compare the values of your Teacher objects.
The solution is to provide your own IEqualityComparer and specify exactly how the Teacher objects should be compared to each other. MSDN provides a good example of how to do this.

C# EF / LINQ hack fix hitting perfomance? Other way of fixing?

I've been learning C# / LINQ / ASP.NET / MVC 3 / EF for a few months now comming from Java / Icefaces / Ibatis base (Real world uses .NET D;). I really enjoy LINQ / Entity Framework from the .NET Framework but I'm having a few issues understand what's really happening behind the scenes.
Here's my problem:
I'm using a AJAX / JSON fed jQuery datatable (that I highly recommend to anyone in need of a free web datatable system by the way). I have a method in my MVC3 application that returns a JSON result of the data needed by the table, doing the sorting and all. Everything is working nicely and smoothly. However, I'm having a concern with the "dirty" hack I had to do to make this work.
Here's the complete code:
//inEntities is the Entity Framework Database Context
//It includes the following entities:
// Poincon
// Horaire
// HoraireDetail
//Poincon, Horaire and HoraireDetail are "decorated" using the Metadata technic which
//adds properties methods and such to the Entity (Like getEmploye which you will see in
//the following snippet)
//
//The Entity Employe is not a database data and therefor not handled by the EF.
//Instead, it is a simple object with properties that applies Lazy Loading to get an
//Employe Name based off of his Employe ID in the Active Directory. An employe object
//can be constructed with his Employe ID which will expose the possibility of getting
//the Employe Name from the AD if needed.
[HttpPost]
public JsonResult List(FormCollection form)
{
String sEcho;
int iDisplayStart;
int iDisplayLength;
String sSearch;
int iSortingCols;
Dictionary<String, String> sorting;
try
{
sEcho = form["sEcho"];
iDisplayStart = int.Parse(form["iDisplayStart"]);
iDisplayLength = int.Parse(form["iDisplayLength"]);
sSearch = form["sSearch"];
iSortingCols = int.Parse(form["iSortingCols"]);
sorting = new Dictionary<string,string>();
for (int i = 0; i < iSortingCols; i++)
sorting.Add(form["mDataProp_" + form["iSortCol_" + i]].ToUpper(), form["sSortDir_" + i].ToUpper());
}
catch
{
HttpContext.Response.StatusCode = 500;
return null;
}
var qPoincon = inEntities.Poincons.AsEnumerable();
var lPoincon = qPoincon.Select(o => new
{
o.id,
emp = o.getEmploye(),
o.poinconStart,
o.poinconEnd,
o.commentaire,
o.codeExceptions
}).AsEnumerable();
//Search
lPoincon = lPoincon.Where(p => (p.emp.empNoStr.Contains(sSearch) || p.emp.empNom.Contains(sSearch) || (p.commentaire != null && p.commentaire.Contains(sSearch))));
//Keep count
int iTotalDisplayRecords = lPoincon.Count();
//Sorting
foreach(KeyValuePair<String,String> col in sorting)
{
switch (col.Key)
{
case "EMPNO":
if (col.Value == "ASC")
lPoincon = lPoincon.OrderBy(h => h.emp.empNo);
else
lPoincon = lPoincon.OrderByDescending(h => h.emp.empNo);
break;
case "POINCONSTART":
if (col.Value == "ASC")
lPoincon = lPoincon.OrderBy(h => h.poinconStart);
else
lPoincon = lPoincon.OrderByDescending(h => h.poinconStart);
break;
case "POINCONEND":
if (col.Value == "ASC")
lPoincon = lPoincon.OrderBy(h => h.poinconEnd);
else
lPoincon = lPoincon.OrderByDescending(h => h.poinconEnd);
break;
case "COMMENTAIRE":
if (col.Value == "ASC")
lPoincon = lPoincon.OrderBy(h => h.commentaire);
else
lPoincon = lPoincon.OrderByDescending(h => h.commentaire);
break;
}
}
//Paging
lPoincon = lPoincon.Skip(iDisplayStart).Take(iDisplayLength);
//Building Response
var jdt = new
{
iTotalDisplayRecords = iTotalDisplayRecords,
iTotalRecords = inEntities.Poincons.Count(),
sEcho = sEcho,
aaData = lPoincon
};
return Json(jdt);
}
As you can see, when I'm grabbing the entire list of "Poincons" from the EF and turning it into a Enumerable. From my current understanding, turning the LINQ query into a Enumerable "kills" the link to the EF, or in other words, will generate the SQL required to get that list at that point instead of keeping the LINQ data until the end and execute a percise query that will return only the data you require. After turning this LINQ Query into a Enumerable, I'm heavily filtering the LINQ (since there is paging, sorting, searching in the datatable). This leads me to thinkg that what my code is currently doing is "Grab all the "Poincons" from the database and put it into the web server's memory as a Enumerable, do your work with the Enumerable then serialize the result as a JSON string and send it to the client.
If I'm correct, the performance hit is quite heavy when you hit the couple thousand of entries (which will happen quite fast once in production... everytime an employe comes to work, it will add 1 entry. 100 employes, ~300 work days a year, you get the idea).
The reason for this hack is that the EF does not know what "getEmploye" method of "Poincon" is, therefor throwing an exception at runtime similar to this:
LINQ to Entities ne reconnaît pas la méthode « PortailNorclair.Models.Employe getEmploye() », et cette dernière ne peut pas être traduite en expression de magasin.
Approximated traduction (If anyone can let me know in a comment how to configure IIS / ASP.NET to display errors in english while keeping the globalization in a foreign language, I would be really grateful. French information about error messages is sometimes lacking):
LINQ to Entity does not recognize the method " PortailNorclair.Models.Employe getEmploye()" and the following could not be translated to a SQL expression.
The "getEmploye" method instances and returns a Employe object with the employe id found in the Poincon object. That Employe object has properties that "lazy loads" information like the employe name from the Active Directory.
So the question is: How can I avoid the performance hit from using .AsEnumerable() on the non-filtered list of objects?
Thanks a lot!

The "getEmploye" method instances and returns a Employe object with
the employe id found in the Poincon object. That Employe object has
properties that "lazy loads" information like the employe name from
the Active Directory.
You should be storing the Employee Name in the database, so you can then order, sort, skip and take in your Linq Query without having to load every employee object.
If empNoStr, empNom, and empNo were all in the database, you could retrieve just the records you want, and call getEmploye() (loading whatever else you need from active directory, or wherever) for each of those.

There are some classes on which your program performs its main work.
There are other classes which represent to database rows.
If you keep them separated, you can also separate actions you intend to occur in the database from actions you intend to perform locally. This makes it trivial to avoid loading the full table, when specific rows are required.
I see you're also doing Paging locally, while the database can do that and save your webserver some memory.