Develop Reference

How does the compiler/runtime determine a lambda expression's type?

I was wondering how the the compiler/runtime determines a lambda expression's type?
For example, the following System.Linq Select extension method (not select query)
//var recordIds = new List<int>(records.Select(r => r?.RecordId ?? 0));
//var recordIds = new List<int>(records.Where(r => r != null).Select(r => r.RecordId));
var recordIds = new List<int>(records.Select(r => r.RecordId));
is defined as
Enumerable.Select<TSource, TResult> Method (IEnumerable<TSource>, Func<TSource, TResult>)
and so takes the lambda r => r.RecordId as a Func<TSource, TResult>.
How is the lambda's type determined, and once it is, is it simply cast to that type?

I was wondering how the the compiler/runtime determines a lambda expression's type?
It is rather complicated. Implementing this feature was the "long pole" for the version of Visual Studio that shipped C# 3 -- so every day that I was over schedule on this was a day that VS would slip! -- and the feature has only gotten more complicated with the introduction of covariance and contravariance in C# 4.
As others have noted, consult the specification for exact details. You might also read the various articles I've written about it over the years.
I can give you a quick overview though. Suppose we have
records.Select(r => r.RecordId)
where records is of type IEnumerable<Record> and RecordId is of type int.
The first question is "does IEnumerable<Record> have any applicable method called Select? No, it does not. We therefore go to the extension method round.
The second question then is: "is there any static type with an accessible extension method that is applicable to this call?"
SomeType.Select(records, r => r.RecordId)
Let's suppose that Enumerable is the only such type. It has two versions of Select, and one of them takes a lambda that takes two parameters. We can automatically discard that one. That leaves us with, as you note:
static IEnumerable<R> Select<A, R>(IEnumerable<A>, Func<A, R>)
Third question: Can we deduce the type arguments corresponding to type parameters A and R?
In the first round of type inference we consider all the non-lambda arguments. We only have one. We deduce that A could be Record. However, IEnumerable<T> is covariant, so we make a note that it could be a type more general than Record as well. It cannot be a type that is more specific than Record though.
Now we ask "are we done?" No. We still don't know R.
"Are there any inferences left to make?" Yes. We haven't checked the lambda yet.
"Are there any contradictions or additional facts to know about A?" Nope.
We therefore "fix" A to Record and move on. What do we know so far? We have:
static IEnumerable<R> Select<Record, R>(IEnumerable<Record>, Func<Record, R>)
We then say OK, the argument must be (Record r) => r.RecordId. Can we infer the return type of this lambda knowing that? Plainly yes, it is int. So we put a note on R saying that it could be int.
Are we done? Yes. Is there anything else we can make a deduction about? No. Did we infer all the type parameters? Yes. So we "fix" R to int, and we're done.
Now we do a final round that checks to make sure that Select<Record, int> does not produce any errors; for example, if Select had a generic constraint that was violated by <Record, int> we would reject it at this point.
We've deduced that records.Select(r=>r.RecordId) means the same thing as Enumerable.Select<Record, int>(records, (Record r) => { return (int)r.RecordId; } ) and that's a legal call to that method, so we're done.
Things get rather more complicated when you introduce multiple bounds. See if you can figure out how something like a Join works where there are four type parameters to infer.

I think the best place to look for such information is the language spec:
Section 7.15
An anonymous function is an expression that represents an “in-line”
method definition. An anonymous function does not have a value or type
in and of itself, but is convertible to a compatible delegate or
expression tree type. The evaluation of an anonymous function
conversion depends on the target type of the conversion: If it is a
delegate type, the conversion evaluates to a delegate value
referencing the method which the anonymous function defines. If it is
an expression tree type, the conversion evaluates to an expression
tree which represents the structure of the method as an object
structure.
The above explains the nature of lambda expressions, basically that they have no types by themselves, unlike say, an int literal 5. Another point to note here is that the lambda expression is not casted to the delegate type like you said. It is evaluated, just like how 3 + 5 is evaluated to be of int type.
Exactly how these types are figured out (type inference) is described in section 7.5.2.
...assume that the generic method has the following signature:
Tr M<X1…Xn>(T1
x1 … Tm xm)
With a method call of the form M(E1 …Em) the
task of type inference is to find unique type arguments
S1…Sn for each of the type parameters
X1…Xn so that the call
M1…Sn>(E1…Em)becomes
valid.
The whole algorithm is quite well documented but it's kind of long to post it here. So here's a link to download the language spec.
https://msdn.microsoft.com/en-us/library/ms228593(v=vs.120).aspx

inference from methods are not allowed? [duplicate]

Given this code:
class C
{
C()
{
Test<string>(A); // fine
Test((string a) => {}); // fine
Test((Action<string>)A); // fine
Test(A); // type arguments cannot be inferred from usage!
}
static void Test<T>(Action<T> a) { }
void A(string _) { }
}
The compiler complains that Test(A) can't figure out T to be string.
This seems like a pretty easy case to me, and I swear I've relied far more complicated inference in other generic utility and extension functions I've written. What am I missing here?
Update 1: This is in the C# 4.0 compiler. I discovered the issue in VS2010 and the above sample is from a simplest-case repro I made in LINQPad 4.
Update 2: Added some more examples to the list of what works.

Test(A);
This fails because the only applicable method (Test<T>(Action<T>)) requires type inference, and the type inference algorithm requires that each each argument be of some type or be an anonymous function. (This fact is inferred from the specification of the type inference algorithm (§7.5.2)) The method group A is not of any type (even though it is convertable to an appropriate delegate type), and it is not an anonymous function.
Test<string>(A);
This succeeds, the difference being that type inference is not necessary to bind Test, and method group A is convertable to the required delegate parameter type void Action<string>(string).
Test((string a) => {});
This succeeds, the difference being that the type inference algorithm makes provision for anonymous functions in the first phase (§7.5.2.1). The parameter and return types of the anonymous function are known, so an explicit parameter type inference can be made, and a correspondense is thereby made between the types in the anonymous function (void ?(string)) and the type parameter in the delegate type of the Test method’s parameter (void Action<T>(T)). No algorithm is specified for method groups that would correspond to this algorithm for anonymous functions.
Test((Action<string>)A);
This succeeds, the difference being that the untyped method group parameter A is cast to a type, thereby allowing the type inference of Test to proceed normally with an expression of a particular type as the only argument to the method.
I can think of no reason in theory why overload resolution could not be attempted on the method group A. Then—if a single best binding is found—the method group could be given the same treatment as an anonymous function. This is especially true in cases like this where the method group contains exactly one candidate and it has no type parameters. But the reason it does not work in C#4 appears to be the fact that this feature was not designed and implemented. Given the complexity of this feature, the narowness of its application, and the existance of three easy work-arounds, I am not going to be holding my breath for it!

I think it's because it's a two-step inference:
It has to infer that you want to convert A to a generic delegate
It has to infer what the type of the delegate parameter should be
I'm not sure if this is the reason, but my hunch is that a two-step inference isn't necessarily easy for the compiler.
Edit:
Just a hunch, but something is telling me the first step is the problem. The compiler has to figure out to convert to a delegate with a different number of generic parameters, and so it can't infer the types of the parameters.

This looks like a vicious circle to me.
Test method expects a parameter of delegate type constructed from generic type Action<T>. You pass in a method group instead: Test(A). This means compiler has to convert your parameter to a delegate type (method group conversion).
But which delegate type? To know the delegate type we need to know T. We didn't specify it explicitly, so compiler has to infer it to figure out the delegate type.
To infer the type parameters of the method we need to know the types of the method arguments, in this case the delegate type. Compiler doesn't know the argument type and thus fails.
In all other cases either type of argument is apparent:
// delegate is created out of anonymous method,
// no method group conversion needed - compiler knows it's Action<string>
Test((string a) => {});
// type of argument is set explicitly
Test((Action<string>)A);
or type parameter is specified explicitly:
Test<string>(A); // compiler knows what type of delegate to convert A to
P.S. more on type inference

You're passing the name of the Method A. The .Net framework CAN convert it to an Action, but it's implicit and it will not take responsibility for it.
But still, a method name is NOT an explicit Action<> Object. And therefor it won't infer the type as an Action type.

I could be wrong, but I imagine the real reason C# cannot infer the type is due to method overloading and the ambiguity that arises. For example, suppose I have the following methods: void foo (int) and void foo (float). Now if I write var f = foo. Which foo should the compiler pick? Likewise, the same problem happens with your example using Test(foo).

Why is Foo(Action bar) and Foo(Func<Task> bar) ambiguous when used with method groups? [duplicate]

The following call to the overloaded Enumerable.Select method:
var itemOnlyOneTuples = "test".Select<char, Tuple<char>>(Tuple.Create);
fails with an ambiguity error (namespaces removed for clarity):
The call is ambiguous between the following methods or properties:
'Enumerable.Select<char,Tuple<char>>
(IEnumerable<char>,Func<char,Tuple<char>>)'
and
'Enumerable.Select<char,Tuple<char>>
(IEnumerable<char>, Func<char,int,Tuple<char>>)'
I can certainly understand why not specifying the type-arguments explicitly would result in an ambiguity (both the overloads would apply), but I don't see one after doing so.
It appears clear enough to me that the intention is to call the first overload, with the method-group argument resolving to Tuple.Create<char>(char). The second overload should not apply because none of the Tuple.Create overloads can be converted to the expected Func<char,int,Tuple<char>> type. I'm guessing the compiler is confused by Tuple.Create<char, int>(char, int), but its return-type is wrong: it returns a two-tuple, and is hence not convertible to the relevant Func type.
By the way, any of the following makes the compiler happy:
Specifying a type-argument for the method-group argument: Tuple.Create<char> (Perhaps this is actually a type-inference issue?).
Making the argument a lambda-expression instead of a method-group: x => Tuple.Create(x). (Plays well with type-inference on the Select call).
Unsurprisingly, trying to call the other overload of Select in this manner also fails:
var itemIndexTwoTuples = "test".Select<char, Tuple<char, int>>(Tuple.Create);
What's the exact problem here?

First off, I note that this is a duplicate of:
Why is Func<T> ambiguous with Func<IEnumerable<T>>?
What's the exact problem here?
Thomas's guess is essentially correct. Here are the exact details.
Let's go through it a step at a time. We have an invocation:
"test".Select<char, Tuple<char>>(Tuple.Create);
Overload resolution must determine the meaning of the call to Select. There is no method "Select" on string or any base class of string, so this must be an extension method.
There are a number of possible extension methods for the candidate set because string is convertible to IEnumerable<char> and presumably there is a using System.Linq; in there somewhere. There are many extension methods that match the pattern "Select, generic arity two, takes an IEnumerable<char> as the first argument when constructed with the given method type arguments".
In particular, two of the candidates are:
Enumerable.Select<char,Tuple<char>>(IEnumerable<char>,Func<char,Tuple<char>>)
Enumerable.Select<char,Tuple<char>>(IEnumerable<char>,Func<char,int,Tuple<char>>)
Now, the first question we face is are the candidates applicable? That is, is there an implicit conversion from each supplied argument to the corresponding formal parameter type?
An excellent question. Clearly the first argument will be the "receiver", a string, and it will be implicitly convertible to IEnumerable<char>. The question now is whether the second argument, the method group "Tuple.Create", is implicitly convertible to formal parameter types Func<char,Tuple<char>>, and Func<char,int, Tuple<char>>.
When is a method group convertible to a given delegate type? A method group is convertible to a delegate type when overload resolution would have succeeded given arguments of the same types as the delegate's formal parameter types.
That is, M is convertible to Func<A, R> if overload resolution on a call of the form M(someA) would have succeeded, given an expression 'someA' of type 'A'.
Would overload resolution have succeeded on a call to Tuple.Create(someChar)? Yes; overload resolution would have chosen Tuple.Create<char>(char).
Would overload resolution have succeeded on a call to Tuple.Create(someChar, someInt)? Yes, overload resolution would have chosen Tuple.Create<char,int>(char, int).
Since in both cases overload resolution would have succeeded, the method group is convertible to both delegate types. The fact that the return type of one of the methods would not have matched the return type of the delegate is irrelevant; overload resolution does not succeed or fail based on return type analysis.
One might reasonably say that convertibility from method groups to delegate types ought to succeed or fail based on return type analysis, but that's not how the language is specified; the language is specified to use overload resolution as the test for method group conversion, and I think that's a reasonable choice.
Therefore we have two applicable candidates. Is there any way that we can decide which is better than the other? The spec states that the conversion to the more specific type is better; if you have
void M(string s) {}
void M(object o) {}
...
M(null);
then overload resolution chooses the string version because string is more specific than object. Is one of those delegate types more specific than the other? No. Neither is more specific than the other. (This is a simplification of the better-conversion rules; there are actually lots of tiebreakers, but none of them apply here.)
Therefore there is no basis to prefer one over the other.
Again, one could reasonably say that sure, there is a basis, namely, that one of those conversions would produce a delegate return type mismatch error and one of them would not. Again, though, the language is specified to reason about betterness by considering the relationships between the formal parameter types, and not about whether the conversion you've chosen will eventually result in an error.
Since there is no basis upon which to prefer one over the other, this is an ambiguity error.
It is easy to construct similar ambiguity errors. For example:
void M(Func<int, int> f){}
void M(Expression<Func<int, int>> ex) {}
...
M(x=>Q(++x));
That's ambiguous. Even though it is illegal to have a ++ inside an expression tree, the convertibility logic does not consider whether the body of a lambda has something inside it that would be illegal in an expression tree. The conversion logic just makes sure that the types check out, and they do. Given that, there's no reason to prefer one of the M's over the other, so this is an ambiguity.
You note that
"test".Select<char, Tuple<char>>(Tuple.Create<char>);
succeeds. You now know why. Overload resolution must determine if
Tuple.Create<char>(someChar)
or
Tuple.Create<char>(someChar, someInt)
would succeed. Since the first one does and the second one does not, the second candidate is inapplicable and eliminated, and is therefore not around to become ambiguous.
You also note that
"test".Select<char, Tuple<char>>(x=>Tuple.Create(x));
is unambiguous. Lambda conversions do take into account the compatibility of the returned expression's type with the target delegate's return type. It is unfortunate that method groups and lambda expressions use two subtly different algorithms for determining convertibility, but we're stuck with it now. Remember, method group conversions have been in the language a lot longer than lambda conversions; had they been added at the same time, I imagine that their rules would have been made consistent.

I'm guessing the compiler is confused by Tuple.Create<char, int>(char, int), but its return-type is wrong: it returns a two-tuple.
The return type isn't part of the method signature, so it isn't considered during overload resolution; it's only verified after an overload has been picked. So as far as the compiler knows, Tuple.Create<char, int>(char, int) is a valid candidate, and it is neither better nor worse than Tuple.Create<char>(char), so the compiler can't decide.

Method Inference does not work with method group

Consider
void Main()
{
var list = new[] {"1", "2", "3"};
list.Sum(GetValue); //error CS0121
list.Sum(s => GetValue(s)); //works !
}
double GetValue(string s)
{
double val;
double.TryParse(s, out val);
return val;
}
The description for the CS0121 error is
The call is ambiguous between the following methods or properties:
'System.Linq.Enumerable.Sum<string>(System.Collections.Generic.IEnumerable<string>,
System.Func<string,decimal>)' and
'System.Linq.Enumerable.Sum<string>(System.Collections.Generic.IEnumerable<string>,
System.Func<string,decimal?>)'
What I don't understand is, what information does s => GetValue(s) give the compiler that simply GetValue doesn't - isn't the latter syntactic sugar for the former ?

Mark's answer is correct but could use a bit more explanation.
The problem is indeed due to a subtle difference between how method groups are handled and how lambdas are handled.
Specifically, the subtle difference is that a method group is considered to be convertible to a delegate type solely on the basis of whether the arguments match, not also on the basis of whether the return type matches. Lambdas check both the arguments and the return type.
The reason for this odd rule is that method group conversions to delegates are essentially a solution of the overload resolution problem. Suppose D is the delegate type double D(string s) and M is a method group containing a method that takes a string and returns a string. When resolving the meaning of a conversion from M to D, we do overload resolution as if you had said M(string). Overload resolution would pick the M that takes a string and returns a string, and so M is convertible to that delegate type even though the conversion will result in an error later. Just as "regular" overload resolution would succeed if you said "string s = M(null);" -- overload resolution succeeds, even though that causes a conversion failure later.
This rule is subtle and a bit weird. The upshot of it here is that your method group is convertible to all the different delegate types that are the second arguments of every version of Sum that takes a delegate. Since no best conversion can be found, the overload resolution on method group Sum is ambiguous.
Method group conversions rules are plausible but a bit odd in C#. I am somewhat vexed that they are not consistent with the more "intuitively correct" lambda conversions.

s => GetValue(s) is a lambda expression and GetValue is a method group, which is a completely different thing. They can both be considered syntactic sugar for new Func<string,double>(...) but the only way they are related to each other is that the lambda expression includes a call to GetValue(). When it comes to converting to a delegate, method groups have different conversion rules than lambda expressions with respect to return types. See Why is Func<T> ambiguous with Func<IEnumerable<T>>? and Overloaded method-group argument confuses overload resolution?.

Is the C# compiler unable to infer method type parameters by expected return type?

This seems odd to me, but I remember a thread where Eric Lippert commented on the inability (by design, or at least convention, I think) of C# to overload methods based on return type, so perhaps it's in some convoluted way related to that.
Is there any reason this does not work:
public static T Test<T>() where T : new()
{
return new T();
}
// Elsewhere
SomeObject myObj = Test();
But this does:
var myObj = Test<SomeObject>();
From a certain perspective, they're both fine, in that you're not Repeating Yourself (in a very small way), but is this just a different pass of the compiler?

First off, this is "overloading based on return type":
void M(int x){...}
int M(int x){...}
string M(int x){...}
The declarations are not legal; you can't overload a method based on return type because the return type is not part of the signature, and the signature has to be unique.
What you are talking about is method type inference based on the method's return type. We don't support that either.
The reason is because the return type might be what you are trying to figure out.
M(Test());
What's the return type of Test? That depends on which overload of M we choose. What overload of M do we choose? That depends on the return type of Test.
In general, C# is designed so that every subexpression has a type, and the types are worked out from the "inside" towards the "outside", not from the outside to the inside.
The notable exceptions are anonymous functions, method groups, and null:
M(x=>x+1)
What's the type of x=>x+1? It depends on which overload of M is called.
M(N); // N is a method group
what's the type of N? Again, it depends on which overload of M is called.
And so on. In these cases we do reason from "outside" to "inside".
Type inference involving lambdas is extremely complicated and was difficult to implement. We don't want to have that same complication and difficulty throughout the compiler.

Except for typeless expressions (null, method groups, and lambda expressions), the type of an expression must be statically determinable by the expression itself, regardless of context.
In other words, the type of an expression Test() cannot depend on what you're assigning it to.

Check C# Language Specification §7.5.2, the declaring type of a variable is not an attestation for type inference, and obviously it shouldn't be. Consider the following code:
Base b = Test<Derived>();
Derived d = Test<Derived>();
The return type of the method probably differs from the declaring type of the variable, since we have implicit convert in C#.

Type inference by the compiler doesn't use the "expected type" of an assignment as part of the logic.
So, the "scope of consideration" for type inference is not this:
SomeObject myObj = Test();
but this:
Test();
And, there are no clues here as to the expected type.

If you want an example of why the type of an expression needs to be able to be determined by the expression itself, consider the following two cases:
We don't use the return value at all - we're just calling the method for its side-effects.
We pass the return value directly into an overloaded method
Using the "expected type" of the return value when it comes to generic type resolution would introduce a whole whack of additional complexity into the compiler, and all you've gained is that sometimes you need to explicitly specify the type and sometimes you don't, and whether you need to or not can change based on unrelated changes elsewhere in the code.