Develop Reference

Non-nullable reference types' default values VS non-nullable value types' default values

This isn't my first question about nullable reference types as it's been few months I'm experiencing with it. But the more I'm experiencing it, the more I'm confused and the less I see the value added by that feature.
Take this code for example
string? nullableString = default(string?);
string nonNullableString = default(string);
int? nullableInt = default(int?);
int nonNullableInt = default(int);
Executing that gives:
nullableString => null
nonNullableString => null
nullableInt => null
nonNullableInt => 0
The default value of an (non-nullable) integer has always been 0 but
to me it doesn't make sense a non-nullable string's default value is null.
Why this choice? This is opposed to the non-nullable principles we've always been used to.
I think the default non-nullable string's default value should have been String.Empty.
I mean somewhere deep down in the implementation of C# it must be specified that 0 is the default value of an int. We also could have chosen 1 or 2 but no, the consensus is 0. So can't we just specify the default value of a string is String.Empty when the Nullable reference type feature is activated? Moreover it seems Microsoft would like to activate it by default with .NET 5 projects in a near future so this feature would become the normal behavior.
Now same example with an object:
Foo? nullableFoo = default(Foo?);
Foo nonNullableFoo = default(Foo);
This gives:
nullableFoo => null
nonNullableFoo => null
Again this doesn't make sense to me, in my opinion the default value of a Foo should be new Foo() (or gives a compile error if no parameterless constructor is available).
Why by default setting to null an object that isn't supposed to be null?
Now extending this question even more
string nonNullableString = null;
int nonNullableInt = null;
The compiler gives a warning for the 1st line which could be transformed into an error with a simple configuration in our .csproj file: <WarningsAsErrors>CS8600</WarningsAsErrors>.
And it gives a compilation error for the 2nd line as expected.
So the behavior between non-nullable value types and non-nullable reference types isn't the same but this is acceptable since I can override it.
However when doing that:
string nonNullableString = null!;
int nonNullableInt = null!;
The compiler is completely fine with the 1st line, no warning at all.
I discovered null! recently when experiencing with the nullable reference type feature and I was expecting the compiler to be fine for the 2nd line too but this isn't the case. Now I'm just really confused as for why Microsoft decided to implement different behaviors.
Considering it doesn't protect at all against having null in a non-nullable reference type variable, it seems this new feature doesn't change anything and doesn't improve developers' lives at all (as opposed to non-nullable value types which could NOT be null and therefore don't need to be null-checked)
So at the end it seems the only value added is just in terms of signatures. Developers can now be explicit whether or not a method's return value could be null or not or if a property could be null or not (for example in a C# representation of a database table where NULL is an allowed value in a column).
Beside that I don't see how can I efficiently use this new feature, could you please give me other useful examples on how you use nullable reference types?
I really would like to make good use of this feature to improve my developer's life but I really don't see how...
Thank you

You are very confused by how programming language design works.
Default values
The default value of an (non-nullable) integer has always been 0 but to me it doesn't make sense a non-nullable string's default value is null. Why this choice? This is completely against non-nullable principles we've always been used to. I think the default non-nullable string's default value should have been String.Empty.
Default values for variables are a basic feature of the language that is in C# since the very beginning. The specification defines the default values:
For a variable of a value_type, the default value is the same as the value computed by the value_type's default constructor ([see] Default constructors).
For a variable of a reference_type, the default value is null.
This makes sense from a practical standpoint, as one of the basic usages of defaults is when declaring a new array of values of a given type. Thanks to this definition, the runtime can just zero all the bits in the allocated array - default constructors for value types are always all-zero values in all fields and null is represented as an all-zero reference. That's literally the next line in the spec:
Initialization to default values is typically done by having the memory manager or garbage collector initialize memory to all-bits-zero before it is allocated for use. For this reason, it is convenient to use all-bits-zero to represent the null reference.
Now Nullable Reference Types feature (NRT) was released last year with C#8. The choice here is not "let's implement default values to be null in spite of NRT" but rather "let's not waste time and resources trying to completely rework how the default keyword works because we're introducing NRTs". NRTs are annotations for programmers, by design they have zero impact on the runtime.
I would argue that not being able to specify default values for reference types is a similar case as for not being able to define a parameterless constructor on a value type - runtime needs a fast all-zero default and null values are a reasonable default for reference types. Not all types will have a reasonable default value - what is a reasonable default for a TcpClient?
If you want your own custom default, implement a static Default method or property and document it so that the developers can use that as a default for that type. No need to change the fundamentals of the language.
I mean somewhere deep down in the implementation of C# it must be specified that 0 is the default value of an int. We also could have chosen 1 or 2 but no, the consensus is 0. So can't we just specify the default value of a string is String.Empty when the Nullable reference type feature is activated?
As I said, the deep down is that zeroing a range of memory is blazingly fast and convenient. There is no runtime component responsible for checking what the default of a given type is and repeating that value in an array when you create a new one, since that would be horribly inefficient.
Your proposal would basically mean that the runtime would have to somehow inspect the nullability metadata of strings at runtime and treat an all-zero non-nullable string value as an empty string. This would be a very involved change digging deep into the runtime just for this one special case of an empty string. It's much more cost-efficient to just use a static analyzer to warn you when you're assigning null instead of a sensible default to a non-nullable string. Fortunately we have such analyzer, namely the NRT feature, which consistently refuses to compile my classes that contain definitions like this:
string Foo { get; set; }
by issuing a warning and forcing me to change that to:
string Foo { get; set; } = "";
(I recommend turning on Treat Warnings As Errors by the way, but it's a matter of taste.)
Again this doesn't make sense to me, in my opinion the default value of a Foo should be new Foo() (or gives a compile error if no parameterless constructor is available). Why by default setting to null an object that isn't supposed to be null?
This would, among other things, render you unable to declare an array of a reference type without a default constructor. Most basic collections use an array as the underlying storage, including List<T>. And it would require you to allocate N default instances of a type whenever you make an array of size N, which is again, horribly inefficient. Also the constructor can have side effects. I'm not going to further ponder how many things this would break, suffice to say it's hardly an easy change to make. Considering how complicated NRT was to implement anyway (the NullableReferenceTypesTests.cs file in the Roslyn repo has ~130,000 lines of code alone), the cost-efficiency of introducing such a change is... not great.
The bang operator (!) and Nullable Value Types
The compiler is completely fine with the 1st line, no warning at all. I discovered null! recently when experiencing with the nullable reference type feature and I was expecting the compiler to be fine for the 2nd line too but this isn't the case. Now I'm just really confused as for why Microsoft decided to implement different behaviors.
The null value is valid only for reference types and nullable value types. Nullable types are, again, defined in the spec:
A nullable type can represent all values of its underlying type plus an additional null value. A nullable type is written T?, where T is the underlying type. This syntax is shorthand for System.Nullable<T>, and the two forms can be used interchangeably. (...) An instance of a nullable type T? has two public read-only properties:
A HasValue property of type bool
A Value property of type T
An instance for which HasValue is true is said to be non-null. A non-null instance contains a known value and Value returns that value.
The reason for which you can't assign a null to int is rather obvious - int is a value type that takes 32-bits and represents an integer. The null value is a special reference value that is machine-word sized and represents a location in memory. Assigning null to int has no sensible semantics. Nullable<T> exists specifically for the purpose of allowing null assignments to value types to represent "no value" scenarios. But note that doing
int? x = null;
is purely syntactic sugar. The all-zero value of Nullable<T> is the "no value" scenario, since it means that HasValue is false. There is no magic null value being assigned anywhere, it's the same as saying = default -- it just creates a new all-zero struct of the given type T and assigns it.
So again, the answer is -- no one deliberately tried to design this to work incompatibly with NRTs. Nullable value types are a much more fundamental feature of the language that works like this since its introduction in C#2. And the way you propose it to work doesn't translate to a sensible implementation - would you want all value types to be nullable? Then all of them would have to have the HasValue field that takes an additional byte and possible screws up padding (I think a language that represents ints as a 40-bit type and not 32 would be considered heretical :) ).
The bang operator is used specifically to tell the compiler "I know that I'm dereferencing a nullable/assigning null to a non-nullable, but I'm smarter than you and I know for a fact this is not going to break anything". It disables static analysis warnings. But it does not magically expand the underlying type to accommodate for a null value.
Summary
Considering it doesn't protect at all against having null in a non-nullable reference type variable, it seems this new feature doesn't change anything and doesn't improve developers' life at all (as opposed to non-nullable value types which could NOT be null and therefore don't need to be null-checked)
So at the end it seems the only value added is just in terms of signatures. Developers can now be explicit whether or not a method's return value could be null or not or if a property could be null or not (for example in a C# representation of a database table where NULL is an allowed value in a column).
From the official docs on NRTs:
This new feature provides significant benefits over the handling of reference variables in earlier versions of C# where the design intent can't be determined from the variable declaration. The compiler didn't provide safety against null reference exceptions for reference types (...) These warnings are emitted at compile time. The compiler doesn't add any null checks or other runtime constructs in a nullable context. At runtime, a nullable reference and a non-nullable reference are equivalent.
So you're right in that "the only value added is just in terms of signatures" and static analysis, which is the reason we have signatures in the first place. And that is not an improvement on developers' lives? Note that your line
string nonNullableString = default(string);
gives off a warning. If you did not ignore it (or even better, had Treat Warnings As Errors on) you'd get value - the compiler found a bug in your code for you.
Does it protect you from assigning null to a non-null reference type at runtime? No. Does it improve developers' lives? Thousand times yes. The power of the feature comes from warnings and nullable analysis done at compile time. If you ignore warnings issued by NRT, you're doing it at your own peril. The fact that you can ignore the compiler's helpful hand does not make it useless. After all you can just as well put your entire code in an unsafe context and program in C, doesn't mean that C# is useless since you can circumvent its safety guarantees.

Again this doesn't make sense to me, in my opinion the default value of a Foo should be new Foo() (or gives a compile error if no parameterless constructor is available)
That's an opinion, but: that isn't how it is implemented. default means null for reference-types, even if it is invalid according to nullability rules. The compiler spots this and warns you about it on the line Foo nonNullableFoo = default(Foo);:
Warning CS8600 Converting null literal or possible null value to non-nullable type.
As for string nonNullableString = null!; and
The compiler is completely fine with the 1st line, no warning at all.
You told it to ignore it; that's what the ! means. If you tell the compiler to not complain about something, it isn't valid to complain that it didn't complain.
So at the end it seems the only value added is just in terms of signatures.
No, it has lots more validity, but if you ignore the warnings that it does raise (CS8600 above), and if you suppress the other things that it does for you (!): yes, it will be less useful. So... don't do that?

How to detect whether a type can be nullable at runtime?

I'm trying to detect whether a type can be nullable or not at runtime to convert that type to the corresponding GraphQL type so that, for example:
With nullable reference types enabled:
string is converted to String!
string? is converted to String
With nullable reference types disabled:
string is converted to String
NonNull<string> is converted to String! (NonNull is a custom library type)
I'm having trouble adapting the code that detected the nullability of a type:
bool isNullable = !typeInfo.IsValueType;
How can I change it so that it works with nullable reference types both enabled and disabled?

Note that there are good ways to check for "the old" nullable types that applies to value types that is documented thoroughly here on Stack Overflow.
I will then instead only focus on nullable reference types, and provide the means to check if one of those are in effect.
Let me sum up my comments on the question first as they are rather important.
Contrary to the name of the new feature, nullable reference types is not about the types, but rather about the things these types are used for. These things are:
Fields
Properties
Method return values
Method parameters
Now, of course this also applies to local variables but you need a whole 'nother kind of introspection to deal with decoding instructions. I do not know how, or even if, this kind of information is encoded in the actual instructions for local variables.
OK, so with that out of the way, let's look at some code (btw, I'm using LINQPad with Roslyn experimental mode to test all this):
public string? Nullable;
public string NonNullable;
These are two public fields. Ignore whether this is a good idea or not. How would you check the type of these fields and detect the presence or lack of this question mark?
Well, let's try the simple route:
Type nullable = GetType().GetField("Nullable").FieldType;
Type nonNullable = GetType().GetField("NonNullable").FieldType;
Console.WriteLine(ReferenceEquals(nullable, nonNullable));
Running this gives me:
True
So clearly this doesn't work. The Type objects are the exact same instance. They don't just compare equal, I got the same thing back, no difference. Basically, FieldType is oblivious to the presence or lack of this question mark.
My comments up above has some of those details but the main reason for this is at all existing nuget packages and compiled code will still work with this new support because of this. There is no need for any code to be rewritten to handle something like NullableReferenceType<T> suddenly. This is a good thing, but also means that you will still be passing null-references around and getting them back from existing nuget packages.
OK, so then, how would we detect this? The answer is that the information about the nullability is not attached to the type, as I mentioned above, but rather to the thing that has the type, in this case the fields.
Let's show attributes on these fields (again I'm using LINQPad):
GetType().GetField("Nullable").GetCustomAttributes().Dump();
GetType().GetField("NonNullable").GetCustomAttributes().Dump();
This gives this output:
As you can see here, the Nullable field has an additional attribute, NullableAttribute. I must confess that I do not know what that other attribute is about, I will have to investigate more.
This NullableAttribute attribute is much more complex than this simple example shows, as it has a collection property with bool values. Let's look at a slightly more complex example:
public List<string>? Nullable1;
public List<string?>? Nullable2;
Here, both fields are nullable references to a list, the difference is that I've said that one of the lists contains nullable references to strings, the other doesn't.
Here's some reflection to look at these collections:
GetType().GetField("Nullable1").GetCustomAttributesData().Dump();
GetType().GetField("Nullable2").GetCustomAttributesData().Dump();
and their output:
Here you can see that there is a difference on the second element in this collection (I've "circled" them with red ... rectangles ...), I expect the first element applies to the list, the second one to the first generic type parameter. If you have generic lists containing generic types, the number of parameters will increase accordingly.
You can also find more information on this excellent blog post by Rico Suter.

What is this cast code doing?

What is this code doing? Specifically the default(XX) part. I've never seen it before.
Entities.BizTalkRequestResult result = default(Entities.BizTalkRequestResult);

It's not a cast; it compiles to the default value of Entities.BizTalkRequestResult. For a reference type, e.g., that's probably null. See MSDN: http://msdn.microsoft.com/en-us/library/xwth0h0d(v=vs.80).aspx

There is a misconception; this is not casting at all. The default operator or function returns the default value. ex: 0 for int and null for reference types.
default is often used with generics (default(T)) because we don't know the actual type at compile time.

It gives you the default value for the particular type inside the parentheses. E.g. 0 for primitives numeric types like int or float, or null for reference types. It's useful particularly when the type could vary, and you want to write general code that is applicable for all possible types.

How to represent data that may not be set

I have a class containing a number of properties, something like:
public class Update
{
public int Quantity { get; set; }
public decimal Price { get; set; }
public string Name { get; set; }
}
Each instance of Update does not necessarily have each property set, and another part of the system needs to know which have been set, and which haven't.
One option I've got is to make all the value types Nullable and so a null value would represent the concept of not being set. Whilst this would work, I don't really like the idea of having some properties explicitly Nullable (the value types) and some nullable by virtue of being a reference type. The class definition would look ugly and I'm not convinced a null check is semantically the best approach.
I could create a class very similar to Nullable<T> that has no constraint on the T with an IsSet property. I prefer this option to using Nullable, but I'd still like to see if anyone has an alternative representation that's better than the options I've suggested.

You really should stick to the preexisting idiom here. Use the built-in nullability.
I see your concern with nullability being different for value and ref types. Your workaround would work. But it is just a cosmetic change which gains you litte. I recommend that you change yourself instead of changing the code in this case. Try to fit yourself to the existing conventions.
Edit: Sometimes you need to be able to make a value optional in generic code. In this case you need to use some custom option type. By from experience I can tell that this is pretty nasty to use. It is not my solution of choice.

The reference types are already nullable, effectively - if you use int?, decimal? and string then every one of your properties can be null.
The problem comes if you ever want to set the string value to a null reference - if null is effectively a valid value which is set.
You certainly could write a Maybe<T> type, but I'm not sure I would - I would probably just use null... aside from anything else, it'll be more familiar for others reading the code who are used to C# idioms. For all the "anti-null" sentiment around (which I do share in many situations) there are cases where it's the simplest approach.

I don't really like the idea of having some properties Nullable (the
value types) and some not (the reference types)
Reference types are obviously nullable.
string t = null; //is totally valid

I'd say that Nullable is exactly what you want to use for this purpose.
You could wrap the members with properties (as you already do) to have the class show normal values to the outside alongside with "is it set"-methods to check if needed. But on the inside I'd use Nullable.

To suggest you something new... If you just talking about one class like Update with a limited number of members I would use just IsSet.
But if you have i.e. a number of similar classes with this behavior or a lot of properties I could suggest you using t4 templates. You can, in example, get class properties (of needed type or attribute) as it described in this article and auto generate code based on the list of properties (implementing any design you want automatically)
I could describe it more if one is interested...

The solution here to
use nullable wherever null is not a valid option
use a default value wherever null is a valid option but you are perfectly sure that a given value won't occur
use a boolean flag for each property where null is a valid value and you can't name a default which won't be used ever.
Examples:
Quantity should be nullable, because if it is set its value won't be null ever
Name should be defaulted to "" if Name might be null (the lack of a name) and you are sure Name will never be ""
A flag, let's say nameSet should be used if Name might have a null value and you can't think of a default value. This flag would be false by default and when you first set the value of Name, the flag should be set to true too.
If you want to handle all your properties in the same way a solution would be to create a class which would contain an Object and a boolean flag. The Object would store the value of the property and the flag would store whether the property was initialized, but I don't like this, because it creates a boolean flag even if it's not needed.

Why shouldn't I always use nullable types in C#

I've been searching for some good guidance on this since the concept was introduced in .net 2.0.
Why would I ever want to use non-nullable data types in c#? (A better question is why wouldn't I choose nullable types by default, and only use non-nullable types when that explicitly makes sense.)
Is there a 'significant' performance hit to choosing a nullable data type over its non-nullable peer?
I much prefer to check my values against null instead of Guid.empty, string.empty, DateTime.MinValue,<= 0, etc, and to work with nullable types in general. And the only reason I don't choose nullable types more often is the itchy feeling in the back of my head that makes me feel like it's more than backwards compatibility that forces that extra '?' character to explicitly allow a null value.
Is there anybody out there that always (most always) chooses nullable types rather than non-nullable types?
Thanks for your time,

The reason why you shouldn't always use nullable types is that sometimes you're able to guarantee that a value will be initialized. And you should try to design your code so that this is the case as often as possible. If there is no way a value can possibly be uninitialized, then there is no reason why null should be a legal value for it. As a very simple example, consider this:
List<int> list = new List<int>()
int c = list.Count;
This is always valid. There is no possible way in which c could be uninitialized. If it was turned into an int?, you would effectively be telling readers of the code "this value might be null. Make sure to check before you use it". But we know that this can never happen, so why not expose this guarantee in the code?
You are absolutely right in cases where a value is optional. If we have a function that may or may not return a string, then return null. Don't return string.Empty(). Don't return "magic values".
But not all values are optional. And making everything optional makes the rest of your code far more complicated (it adds another code path that has to be handled).
If you can specifically guarantee that this value will always be valid, then why throw away this information? That's what you do by making it a nullable type. Now the value may or may not exist, and anyone using the value will have to handle both cases. But you know that only one of these cases is possible in the first place. So do users of your code a favor, and reflect this fact in your code. Any users of your code can then rely on the value being valid, and they only have to handle a single case rather than two.

Because it's inconvenient to always have to check whether the nullable type is null.
Obviously there are situations where a value is genuinely optional, and in those cases it makes sense to use a nullable type rather than magic numbers etc, but where possible I would try to avoid them.
// nice and simple, this will always work
int a = myInt;
// compiler won't let you do this
int b = myNullableInt;
// compiler allows these, but causes runtime error if myNullableInt is null
int c = (int)myNullableInt;
int d = myNullableInt.Value;
// instead you need to do something like these, cumbersome and less readable
int e = myNullableInt ?? defaultValue;
int f = myNullableInt.HasValue ? myNullableInt : GetValueFromSomewhere();

I think the language designers feel that 'reference types being nullable by default' was a mistake, and that non-nullable is the only sensible default, and you should have to opt into nullness. (This is how it is in many modern functional languages.) "null" is usually a heap of trouble.

You seem to have 2 different questions...
Why would I ever want to use non-nullable data types in C#?
Simple, because the value-type data you're relying on is guaranteed by the compiler to actually have a value!
Why wouldn't I choose nullable types by default, and only use non-nullable types when that explicitly makes sense?
As Joel has already mentioned, a type can only be null if it is a reference type. Value types are guaranteed by the compiler to have a value. If your program depends on a variable to have a value, then this is the behavior you will want by not choosing a nullable type.
Of course, when your data is coming from anywhere that is not your program, then all bets are off. The best example is from a database. Database fields can be null, so you would want your program variable to mimic this value - not just create a "magic" value (i.e. -1, 0, or whatever) that "represents" null. You do this with nullable types.

Although null values can be convenient for using as "not-initialized-yet" or "not-specified" values, they make the code more complex, mainly because you're overloading the meaning of null as well as the variable (number-or-null vs. just-a-number).
NULL values are favoured by many database designers and SQL database programmers but with a small change in thinking about the problem you can do away with null values and actually have simpler and more reliable code (e.g., no worrying about NullReferenceExceptions).
There's actually a large demand for a "T!" operator that makes any reference type non-nullable, similar to how "T?" makes value types nullable, and Anders Hejlsberg, the inventor of C#, wished he had included the ability.
See also the question, Why is “null” present in C# and java?

I tend to use Nullable types wherever they make sense -- I won't care about performance until it's a problem, then I'll fix the few areas where it is and be done with it.
However, I also find that in general, most of my values end up being non-nullable. In fact, there are many times I'd actually like a NotNullable I can use with reference types to find out about a null problem when I get the null, not later on when I try to use it.

The only time that a Nullable Type should ever be used, is in the case that a certain field in a table of the database absolutely requires that a null be sent or received by the application at some point. Even in such a case, one should always try to find a way around using the Nullable Type. Bool isn't always your best friend.