Develop Reference

Weird behaviour of c# compiler due caching delegate

Suppose I have following program:
static void SomeMethod(Func<int, int> otherMethod)
{
otherMethod(1);
}
static int OtherMethod(int x)
{
return x;
}
static void Main(string[] args)
{
SomeMethod(OtherMethod);
SomeMethod(x => OtherMethod(x));
SomeMethod(x => OtherMethod(x));
}
I cannot understand compiled il code (it uses too extra code). Here is simplified version:
class C
{
public static C c;
public static Func<int, int> foo;
public static Func<int, int> foo1;
static C()
{
c = new C();
}
C(){}
public int b(int x)
{
return OtherMethod(x);
}
public int b1(int x)
{
return OtherMethod(x);
}
}
static void Main()
{
SomeMethod(new Func<int, int>(OtherMethod));
if (C.foo != null)
SomeMethod(C.foo)
else
{
C.foo = new Func<int, int>(c, C.b)
SomeMethod(C.foo);
}
if (C.foo1 != null)
SomeMethod(C.foo1)
else
{
C.foo1 = new Func<int, int>(c, C.b1)
SomeMethod(C.foo1);
}
}
Why does compiler create not static equal methods b/b1? Equal means that they have the same code

Your question is: why did the compiler not realize that the two lines
SomeMethod(x => OtherMethod(x));
SomeMethod(x => OtherMethod(x));
Are the same and write this as
if ( delegate is not created )
create the delegate and stash it away
SomeMethod( the delegate );
SomeMethod( the delegate );
? Well let me answer that question in several ways.
First off, is the compiler permitted to make that optimization? Yes. The specification calls out that a C# compiler is permitted to make two lambdas that do exactly the same thing into a single delegate. And in fact you can see that it already does this optimization in part: it creates each delegate once and saves it away so that it doesn't have to create it again later when the code is called again. Notice that this is a waste of memory in the case where code is only called once.
Second, is the compiler required to make the caching optimization? No. The specification calls out that the compiler is only permitted to make the optimization, but not required to.
Is the compiler required to make the optimization you want? Obviously not, because it doesn't. It is permitted to, and maybe a future version of the compiler will. The compiler is open-source; if you care about this optimization, go write it and submit a pull request.
Third, is it possible to make the optimization you want? Yes. The compiler could take all pairs of lambdas that appear in the same method, compile them to the internal tree format, and do a tree comparison to see if they have the same content, and then generate the same static backing field for both.
So now we have a situation: the compiler is permitted to make a particular optimization, and it doesn't. And you've asked "why not"? That's an easy question to answer: all optimizations are not implemented until someone spends the considerable time and effort to:
Carefully design the optimization: under precisely what conditions is the optimization triggered and not triggered? How general should the optimization be? You've suggested that they detect similar lambda bodies but why stop there? You have two identical statements of code, so why not generate the code for those statements once instead of twice? What if you had a repeated group of statements? There is a huge amount of design work to do here.
In particular, an important aspect of the design is: could the user reasonably do the optimization "by hand" while still keeping the code readable. In this case, yes they could, easily. Just assign the duplicated lambda to a variable and then use the variable. An optimization which does automatically something that a user who cared could have done themselves easily is not really a very interesting or compelling optimization.
Your examples are trivial; real-world code is not. What does your proposed design do with identical nested lambdas? And so on.
Does your optimization cause the behaviour of the code in the debugger to "look weird"? You have probably noticed that when debugging code that was compiled with optimizations turned on, the debugger seems to behave weirdly; that's because there's no longer a clear mapping between the generated code and the original code. Does your optimization make that worse? Is it acceptable to users? Does the debugger need to be aware of the optimization? If so, you'll have to change the debugger. In this case, probably not, but these are questions you have to ask and answer.
Get the design reviewed by experts; this takes up their time, and will likely result in changes to the design
Make estimates of the pros and cons of the optimization -- optimizations often have hidden costs, like the memory leak I mentioned before. In particular, optimizations often preclude other optimizations which might be better.
Make estimates as to the total savings world-wide of this optimization. Does the optimization actually affect real-world code? Does it change the correctness of that code? Is there any production code, anywhere in the world, that would break with this optimization and cause the CTO of company X to call the CTO of Microsoft demanding a fix? If the answer is yes then maybe you might want to not do this optimization. C# is not a toy. Millions and millions of people depend on its correct operation every day.
What's the estimated burden of doing the optimization on compile time? Compilation doesn't have to happen between keystrokes but it does have to be pretty fast. Anything which introduces a superlinear algorithm in a common code path in the compiler is going to be unacceptable. Can you implement your optimization so that it is linear in code size? Note that the algorithm I sketched before -- compare all pairs -- is superlinear in code size. (Exercise: what's the worst case asymptotic performance of doing a tree comparison on all pairs of lambdas?)
Actually implement the optimization. I encourage you to do so.
Test the optimization; does it actually produce better code? On what metric? An optimization which causes no change to any metric is not an optimization.
Sign up to fix bugs in the optimization forever.
The optimization you want simply doesn't meet the bar. No one writes code like that. If they did, and they cared that it duplicated an object, they could easily fix it themselves. So the optimization optimizes code that doesn't exist, in order to get a "win" that is the construction of a single object amongst the millions and millions of objects the program will allocate. Not worth it.
But again, if you think it is, go ahead and implement it and submit a pull request. Make sure to submit the results of the investigations I noted above, because those are where the real work is. The implementation is usually the smallest part of the total effort spent on a feature; that's why C# is a successful language.

Is it bad practice to purposely rely on Linq Side Effects?

A programming pattern like this comes up every so often:
int staleCount = 0;
fileUpdatesGridView.DataSource = MultiMerger.TargetIds
.Select(id =>
{
FileDatabaseMerger merger = MultiMerger.GetMerger(id);
if (merger.TargetIsStale)
staleCount++;
return new
{
Id = id,
IsStale = merger.TargetIsStale,
// ...
};
})
.ToList();
fileUpdatesGridView.DataBind();
fileUpdatesMergeButton.Enabled = staleCount > 0;
I'm not sure there is a more succinct way to code this?
Even if so, is it bad practice to do this?

No, it is not strictly "bad practice" (like constructing SQL queries with string concatenation of user input or using goto).
Sometimes such code is more readable than several queries/foreach or no-side-effect Aggregate call. Also it is good idea to at least try to write foreach and no-side-effect versions to see which one is more readable/easier to prove correctness.
Please note that:
it is frequently very hard to reason what/when will happen with such code. I.e. you sample hacks around the fact of LINQ queries executed lazily with .ToList() call, otherwise that value will not be computed.
pure functions can be run in parallel, once with side effects need a lot of care to do so
if you ever need to convert LINQ-to-Object to LINQ-to-SQL you have to rewrite such queries
generally LINQ queries favor functional programming style without side-effects (and hence by convention readers would not expect side-effects in the code).

Why not just code it like this:
var result=MultiMerger.TargetIds
.Select(id =>
{
FileDatabaseMerger merger = MultiMerger.GetMerger(id);
return new
{
Id = id,
IsStale = merger.TargetIsStale,
// ...
};
})
.ToList();
fileUpdatesGridView.DataSource = result;
fileUpdatesGridView.DataBind();
fileUpdatesMergeButton.Enabled = result.Any(r=>r.IsStale);
I would consider this a bad practice. You are making the assumption that the lambda expression is being forced to execute because you called ToList. That's an implementation detail of the current version of ToList. What if ToList in .NET 7.x is changed to return an object that semi-lazily converts the IQueryable? What if it's changed to run the lambda in parallel? All of a sudden you have concurrency issues on your staleCount. As far as I know, both of those are possibilities which would break your code because of bad assumptions your code is making.
Now as far as repeatedly calling MultiMerger.GetMerger with a single id, that really should be reworked to be a join as the logic for doing a join (w|c)ould be much more efficient than what you have coded there and would scale a lot better, especially if the implementation of MultiMerger is actually pulling data from a database (or might be changed to do so).
As far as calling ToList() before passing it to the Datasource, if the Datasource doesn't use all the fields in your new object, you would be (much) faster and take less memory to skip the ToList and let the datasource only pull the fields it needs. What you've done is highly couple the data to the exact requirements of the view, which should be avoided where possible. An example would be what if you all of a sudden need to display a field that exists in FileDatabaseMerger, but isn't in your current anonymous object? Now you have to make changes to both the controller and view to add it, where if you just passed in an IQueryable, you would only have to change the view. Again, faster, less memory, more flexible, and more maintainable.
Hope this helps.. And this question really should be posted of code review, not stackoverflow.
Update on further review, the following code would be much better:
var result=MultiMerger.GetMergersByIds(MultiMerger.TargetIds);
fileUpdatesGridView.DataSource = result;
fileUpdatesGridView.DataBind();
fileUpdatesMergeButton.Enabled = result.Any(r=>r.TargetIsStale);
or
var result=MultiMerger.GetMergers().Where(m=>MultiMerger.TargetIds.Contains(m.Id));
fileUpdatesGridView.DataSource = result;
fileUpdatesGridView.DataBind();
fileUpdatesMergeButton.Enabled = result.Any(r=>r.TargetIsStale);

Cost of mapping POCO's in a high load system C#

I have a poco that needs to be mapped to another poco in a high traffic system. I intend to map these objects together in a simple mapper similar to this:
public class a
{
public int MyValue { get;set; }
public string YAV { get; set; }
}
public class B
{
public int aTestValue { get;set; }
public string YetAnotherValue { get; set; }
}
public class Mapper
{
public static B MapIt(A a)
{
return new B { aTestValue = a.MyValue, YetAnotherValue = a.YAV };
}
}
How much does a mapping like this really affect performance? Ignore the fact that we'll have to write a mapping for all our types and just focus on the performance lost doing the actual mapping.

How much does a mapping like this really affect performance?
I would say that such mapping wouldn't affect performance even in a high traffic system. The cost of calling getters and setters will probably be negligible compared to other operations you might be doing.
Obviously that's just some 2 cents, if you want real stuff do performance benchmarks and measure the difference with and without the mapping.
At least that's what I would do: make something that corresponds to the requirements, then benchmark it, then two possibilities: you are satisfied with the result => ship in production and enjoy life, or you are not satisfied with the results and those benchmarks have allowed you to identify that this part is the bottleneck for your application => refactor the code and start thinking about optimizing it. But never do premature optimization or you will hardly respect the project deadlines.

From our experience, the overhead won't be much. I tested this recently by retrieving 75,000 rows of data using Linq to SQL and then mapping the L2S entities to POCO entities using mapping code we wrote. The cost of doing this was amazing small. If I recall correctly, it was something like 75 to 100 Ms to map 75K rows.

It's almost impossible to know how this will affect performance without knowing something about the scale of the system, the looping structure in which this mapping occurs, etc.
In general, these types of simple mappings are quick, but you can always run into issues that are associated with the scaling issues I mentioned when things such as serialization are involved.

Best thing to do is hook it up to the profiler and make some measurements. Doing a manual mapping like that is a fairly light way to do it so shouldn't be significant. The AutoMapper tool is also available and will reduce the coding time but has a little more overhead as it does other services besides just mapping:
Analyzing AutoMapper Performance

How about using conversion operators. Only worry about its performance if a profiler shows it to be a bottleneck.

How slow is Reflection

I recently created an interface layer to distinguish the DataAccessProvider from our Business logic layer.
With this approach we can change our choice of DataAccessProvider whenever we want by changing the values in the Web/App.Config.
(more details can be given if needed).
Anyway, to do this we use reflection to accomplish our DataProvider class on which we can work.
/// <summary>
/// The constructor will create a new provider with the use of reflection.
/// If the assembly could not be loaded an AssemblyNotFoundException will be thrown.
/// </summary>
public DataAccessProviderFactory()
{
string providerName = ConfigurationManager.AppSettings["DataProvider"];
string providerFactoryName = ConfigurationManager.AppSettings["DataProviderFactory"];
try
{
activeProvider = Assembly.Load(providerName);
activeDataProviderFactory = (IDataProviderFactory)activeProvider.CreateInstance(providerFactoryName);
}
catch
{
throw new AssemblyNotFoundException();
}
}
But now I'm wondering how slow reflection is?

In most cases: more than fast enough. For example, if you are using this to create a DAL wrapper object, the time taken to create the object via reflection will be minuscule compared to the time it needs to connect to a network. So optimising this would be a waste of time.
If you are using reflection in a tight loop, there are tricks to improve it:
generics (using a wrapper where T : new() and MakeGenericType)
Delegate.CreateDelegate (to a typed delegate; doesn't work for constructors)
Reflection.Emit - hardcore
Expression (like Delegate.CreateDelegate, but more flexible, and works for constructors)
But for your purposes, CreateInstance is perfectly fine. Stick with that, and keep things simple.
Edit: while the point about relative performance remains, and while the most important thing, "measure it", remains, I should clarify some of the above. Sometimes... it does matter. Measure first. However, if you find it is too slow, you might want to look at something like FastMember, which does all the Reflection.Emit code quietly in the background, to give you a nice easy API; for example:
var accessor = TypeAccessor.Create(type);
List<object> results = new List<object>();
foreach(var row in rows) {
object obj = accessor.CreateNew();
foreach(var col in cols) {
accessor[obj, col.Name] = col.Value;
}
results.Add(obj);
}
which is simple, but will be very fast. In the specific example I mention about a DAL wrapper—if you are doing this lots, consider something like dapper, which again does all the Reflection.Emit code in the background to give you the fastest possible but easy to use API:
int id = 12345;
var orders = connection.Query<Order>(
"select top 10 * from Orders where CustomerId = #id order by Id desc",
new { id }).ToList();

Its slower compared to non-reflective code. The important thing is not if its slow, but if its slow where it counts. For instance, if you instantiate objects using reflection in web environment where expected concurency can rise up to 10K, it will be slow.
Anyway, its good not to be concerned about performance in advance. If things turns out to be slow, you can always speed them up if you designed things correctly so that parts that you expected might be in need of optimisation in future are localised.
You can check this famous article if you need speed up:
Dynamic... But Fast: The Tale of Three Monkeys, A Wolf and the DynamicMethod and ILGenerator Classes

Here are some links that might help:
This guy did some tests and provides a few metrics. This article is from 2006, so I made a Gist of the code to test Reflection Performance. The results are similar (although it's much faster now obviously).
Constructor
Time elapsed (ms): 15 which is 151588 ticks for 1000000 calls.
Constructor using reflection
Time elapsed (ms): 38 which is 381821 ticks for 1000000 calls.
Method call
Time elapsed (ms): 5 which is 57002 ticks for 1000000 calls.
Method call using reflection
Time elapsed (ms): 252 which is 2529507 ticks for 1000000 calls.
Setting properties
Time elapsed (ms): 294 which is 2949458 ticks for 1000000 calls.
Setting properties using reflection
Time elapsed (ms): 1490 which is 14908530 ticks for 1000000 calls.
MSDN article "Dodge Common Performance Pitfalls to Craft Speedy Applications"

I thought I'd do a quick test to demonstrate how slow reflection is compared to without.
With Reflection
Instantiating 58 objects by iterating through each of their Attributes and matching
Total Time: 52254 nanoseconds
while (reader.Read()) {
string[] columns = reader.CurrentRecord;
CdsRawPayfileEntry toAdd = new CdsRawPayfileEntry();
IEnumerable<PropertyInfo> rawPayFileAttributes = typeof(CdsRawPayfileEntry).GetProperties().Where(prop => Attribute.IsDefined(prop, typeof(CustomIndexAttribute)));
foreach (var property in rawPayFileAttributes) {
int propertyIndex = ((CustomIndexAttribute)property.GetCustomAttribute(typeof(CustomIndexAttribute))).Index;
if (propertyIndex < columns.Length)
property.SetValue(toReturn, columns[propertyIndex]);
else
break;
}
}
Without Reflection
Instantiating 58 Objects by creating a new object
Total Time: 868 nanoseconds
while (reader2.Read()) {
string[] columns = reader2.CurrentRecord;
CdsRawPayfileEntry toAdd = new CdsRawPayfileEntry() {
ColumnZero = columns[0],
ColumnOne = columns[1],
ColumnTwo = columns[2],
ColumnThree = columns[3],
ColumnFour = columns[4],
ColumnFive = columns[5],
ColumnSix = columns[6],
ColumnSeven = columns[7],
ColumnEight = columns[8],
ColumnNine = columns[9],
ColumnTen = columns[10],
ColumnEleven = columns[11],
ColumnTwelve = columns[12],
ColumnThirteen = columns[13],
ColumnFourteen = columns[14],
ColumnFifteen = columns[15],
ColumnSixteen = columns[16],
ColumnSeventeen = columns[17]
};
}
Albeit, not completely fair since the reflection also has to retrieve a specific attribute of every property 58*18 times on top of creating a new object via reflection, but it at least provides some perspective.

Reflection is not THAT slow. Invoking a method by reflection is about 3 times slower than the normal way. That is no problem if you do this just once or in non-critical situations. If you use it 10'000 times in a time-critical method, I would consider to change the implementation.

Other than following the links given in other answers and ensuring you're not writing "pathalogically bad" code then for me the best answer to this is to test it yourself.
Only you know where you bottle necks are, how many times your reflection code will be user, whether the reflection code will be in tight loops etc. You know your business case, how many users will access your site, what the perf requirements are.
However, given the snippet of code you've shown here then my guess would be that the overhead of reflection isn't going to be a massive problem.
VS.NET web testing and performance testing features should make measuring the performance of this code pretty simple.
If you don't use reflection, what will your code look like? What limitations will it have? It may be that you can't live with the limitations that you find yourself with if you remove the reflection code. It might be worth trying to design this code without the reflection to see if it's possible or it the alternative is desirable.

I was doing somethign similar until I started playing with IoC. I would use a Spring object definition to specify the data provider - SQL, XML, or Mocks!

IDataReader and "HasColumn", Best approach?

I've seen two common approaches for checking if a column exists in an IDataReader:
public bool HasColumn(IDataReader reader, string columnName)
{
try
{
reader.getOrdinal(columnName)
return true;
}
catch
{
return false;
}
}
Or:
public bool HasColumn(IDataReader reader, string columnName)
{
reader.GetSchemaTable()
.DefaultView.RowFilter = "ColumnName='" + columnName + "'";
return (reader.GetSchemaTable().DefaultView.Count > 0);
}
Personally, I've used the second one, as I hate using exceptions for this reason.
However, on a large dataset, I believe RowFilter might have to do a table scan per column, and this may be incredibly slow.
Thoughts?

I think I have a reasonable answer for this old gem.
I would go with the first approach cause its much simpler. If you want to avoid the exception you can cache the field names and do a TryGet on the cache.
public Dictionary<string,int> CacheFields(IDataReader reader)
{
var cache = new Dictionary<string,int>();
for (int i = 0; i < reader.FieldCount; i++)
{
cache[reader.GetName(i)] = i;
}
return cache;
}
The upside of this approach is that it is simpler and gives you better control. Also, note, you may want to look into case insensitive or kana insensitive compares, which would make stuff a little trickier.

A lot depends on how you're using HasColumn. Are you calling it just once or twice, or repeatedly in a loop? Is the column likely to be there or is that completely unknown in advance?
Setting a row filter probably would do a table scan each time. (Also, in theory, GetSchemaTable() could generate an entirely new table with every call, which would be even more expensive -- I don't believe SqlDataReader does this, but at the IDataReader level, who knows?) But if you only call it once or twice I can't imagine this being that much of an issue (unless you have thousands of columns or something).
(I would, however, at least store the result of GetSchemaTable() in a local var within the method to avoid calling it twice in quick succession, if not cache it somewhere on the off chance that your particular IDataReader DOES regenerate it.)
If you know in advance that under normal circumstances the column you ask for will be present, the exception method is a bit more palatable (because the column not being there is, in fact, an exceptional case). Even if not, it might perform slightly better, but again unless you're calling it repeatedly you should ask yourself if performance is really that much of a concern.
And if you ARE calling it repeatedly, you probably should consider a different approach anyway, such as: call GetSchemaTable() once up front, loop through the table, and load the field names into a Dictionary or some other structure that is designed for fast lookups.

I wouldn't worry about the performance impact. Even if you had a table with 1000 columns (which would be an enormous table), you are still only doing a "table scan" of 1000 rows. That is likely to be trivial.
Premature optimization will just lead you toward an unnecessarily complex implementation. Implement the version that seems best to you, and then measure the performance impact. If it is unacceptable compared to your performance requirements, then consider alternatives.