Develop Reference

How to move two enumerators simultaneously?

I have two big arrays of two readonly structs: A and B. I need to enumerate over them but I need to do it in the most optimized way.
readonly struct A {
long a;
long b;
long c;
long d;
long e;
long g;
}
readonly struct B {
byte a;
ushort b;
}
As you can see Type A is very large struct to copy it over and over again so I made my custom enumerator that returns this struct by reference (ref return). Type B doesn't need it because its' size is only 4 bytes. Now I need to make two enumerators move simultenously using only single foreach cycle because it doesn't look much native calling Enumerator's methods directly (i.e. MoveNext, Current) and so on.
My problem is that when I try to save two structs in one single aggregate struct A is copied and I lose my perfomance gain.
public readonly struct Aggregate
{
public readonly A ItemA;
public readonly B ItemB;
public Aggregate(A itemA, in B itemB)
{
ItemA = itemA;
ItemB = itemB;
}
}
The total size of this aggregate will have size of 56 bytes. It sounds logical that C# copies the struct when I assign it to the property of another struct. But what can I do with it? I need only the reference to the element of an array. Using unsafe code to get a pointer is not a right way, I think, because GC can move my array (if it's small enough and not located in LOH area).
So what solutions could you propose to me?

I don't know whether I understand your question correct or not, but your talking about arrays.
Assumption: Both are arrays of same size
You can do that
for (int i = 0; i < length; i++)
{
ref var aItem = ref arrayA[i];
ref var bItem = ref arrayB[i];
//do your stuff
}

You can use the enumerators explicitly, without a foreach-loop:
var aEnumerator = listA.GetEnumerator();
var bEnumerator = listB.GetEnumerator();
while (aEnumerator.MoveNext() && bEnumerator.MoveNext()) {
var aItem = aEnumerator.Current;
var bItem = bEnumerator.Current;
//TODO: do some work
}

Accessing fields of structs before completely initializing the struct

Consider the following code:
public struct Color {
public int R;
public int G;
public int B;
}
public class App
{
static void Main()
{
Color c;
c.B = 0xFF;
int b = c.B;
}
}
csc compiles the code happily. I always thought all fields of a struct have to be assigned to before one can access data members of the struct. Is this a specialty of csc.exe?
I think NullReferenceExceptions are not the right solution, since we are talking about structs here.

From MSDN:
When you create a struct object using the new operator, it gets
created and the appropriate constructor is called. Unlike classes,
structs can be instantiated without using the new operator. In such a
case, there is no constructor call, which makes the allocation more
efficient. However, the fields will remain unassigned and the object
cannot be used until all of the fields are initialized.
From MSDN:
Compiler Error CS0170: Use of possibly unassigned field 'field'. A
field in a structure was used without first being initialized. To
solve this problem, first determine which field was uninitialized and
then initialize it before you try to access it.
From MSDN:
Compiler Error CS0165: Use of unassigned local variable 'name'. The C# compiler does not allow the use of uninitialized variables. If
the compiler detects the use of a variable that might not have been
initialized, it generates compiler error CS0165.
This is wrong:
I always thought all fields of a struct have to be assigned to before one can access data members of the struct
The correct one is:
All fields of a struct have to be assigned to before one can access the struct.
Color c;
c.B = 0xFF;
int b = c.B; // Okay. You have assigned B
int r = c.R; // Error CS0170! Use of possibly unassigned field
Color cc = c; // Error CS0165! Use of unassigned local variable. The object cannot be used until all of the fields are initialized

Refer this link
msdn ink
If you use Color c; fields are not initialized, but if you do Color c = new Color(); all fields will be initialized.
if you run the below code. it will failed to compile.
Color c;
int b = c.B;
But this will be compiled.
Color c = new Color();
// c.B = 0xFF;
int b = c.B;

Strange unmarshalling behavior with union in C#

I want to export a C-like union into a byte array, like this :
[StructLayout(LayoutKind.Explicit)]
struct my_struct
{
[FieldOffset(0)]
public UInt32 my_uint;
[FieldOffset(0)]
public bool other_field;
}
public static void Main()
{
var test = new my_struct { my_uint = 0xDEADBEEF };
byte[] data = new byte[Marshal.SizeOf(test)];
IntPtr buffer = Marshal.AllocHGlobal(data.Length);
Marshal.StructureToPtr(test, buffer, false);
Marshal.Copy(buffer, data, 0, data.Length);
Marshal.FreeHGlobal(buffer);
foreach (byte b in data)
{
Console.Write("{0:X2} ", b);
}
Console.WriteLine();
}
The output we get (https://dotnetfiddle.net/gb1wRf) is 01 00 00 00 instead of the expected EF BE AD DE.
Now, what do we get if we change the other_field type to byte (for instance) ?
Oddly, we get the output we wanted in the first place, EF BE AD DE (https://dotnetfiddle.net/DnXyMP)
Moreover, if we swap the original two fields, we again get the same output we wanted (https://dotnetfiddle.net/ziSQ5W)
Why is this happening? Why would the order of the fields matter ? Is there a better (reliable) solution for doing the same thing ?

This is an inevitable side-effect of the way a structure is marshaled. Starting point is that the structure value is not blittable, a side-effect of it containing a bool. Which takes 1 byte of storage in the managed struct but 4 bytes in the marshaled struct (UnmanagedType.Bool).
So the struct value cannot just be copied in one fell swoop, the marshaller needs to convert each individual member. So the my_uint is first, producing 4 bytes. The other_field is next, also producing 4 bytes at the exact same address. Which overwrites everything that my_uint produced.
The bool type is an oddity in general, it never produces a blittable struct. Not even when you apply [MarshalAs(UnmanagedType.U1)]. Which in itself has an interesting effect on your test, you'll now see that the 3 upper bytes produced by my_int are preserved. But the result is still junk since the members are still converted one-by-one, now producing a single byte of value 0x01 at offset 0.
You can easily get what you want by declaring it as a byte instead, now the struct is blittable:
[StructLayout(LayoutKind.Explicit)]
struct my_struct {
[FieldOffset(0)]
public UInt32 my_uint;
[FieldOffset(0)]
private byte _other_field;
public bool other_field {
get { return _other_field != 0; }
set { _other_field = (byte)(value ? 1 : 0); }
}
}

I admit that I don't have an authoritative answer for why Marshal.StructureToPtr() behaves this way, other than that clear it is doing more than just copying bytes. Rather, it must be interpreting the struct itself, marshaling each field individually to the destination via the normal rules for interpreting that field. Since bool is defined to only ever be one of two values, the non-zero value gets mapped to true, which marshals to raw bytes as 0x00000001.
Note that if you really just want the raw bytes from the struct value, you can do the copying yourself instead of going through the Marshal class. For example:
var test = new my_struct { my_uint = 0xDEADBEEF };
byte[] data = new byte[Marshal.SizeOf(test)];
unsafe
{
byte* pb = (byte*)&test;
for (int i = 0; i < data.Length; i++)
{
data[i] = pb[i];
}
}
Console.WriteLine(string.Join(" ", data.Select(b => b.ToString("X2"))));
Of course, for that to work you will need to enable unsafe code for your project. You can either do that for the project in question, or build the above into a separate helper assembly where unsafe is less risky (i.e. where you don't mind enabling it for other code, and/or don't care about the assembly being verifiable, etc.).

Cycle in the struct layout that doesn't exist

This is a simplified version of some of my code:
public struct info
{
public float a, b;
public info? c;
public info(float a, float b, info? c = null)
{
this.a = a;
this.b = b;
this.c = c;
}
}
The problem is the error Struct member 'info' causes a cycle in the struct layout. I'm after struct like value type behaviour. I could simulate this using a class and a clone member function, but I don't see why I should need to.
How is this error true? Recursion could perhaps cause construction forever in some similar situations, but I can't think of any way that it could in this case. Below are examples that ought to be fine if the program would compile.
new info(1, 2);
new info(1, 2, null);
new info(1, 2, new info(3, 4));
edit:
The solution I used was to make "info" a class instead of a struct and giving it a member function to returned a copy that I used when passing it. In effect simulating the same behaviour as a struct but with a class.
I also created the following question while looking for an answer.
Value type class definition in C#?

It's not legal to have a struct that contains itself as a member. This is because a struct has fixed size, and it must be at least as large as the sum of the sizes of each of its members. Your type would have to have 8 bytes for the two floats, at least one byte to show whether or not info is null, plus the size of another info. This gives the following inequality:
size of info >= 4 + 4 + 1 + size of info
This is obviously impossible as it would require your type to be infinitely large.
You have to use a reference type (i.e. class). You can make your class immutable and override Equals and GetHashCode to give value-like behaviour, similar to the String class.

The reason why this creates a cycle is that Nullable<T> is itself a struct. Because it refers back to info you have a cycle in the layout (info has a field of Nullable<info> and it has a field of info) . It's essentially equivalent to the following
public struct MyNullable<T> {
public T value;
public bool hasValue;
}
struct info {
public float a, b;
public MyNullable<info> next;
}

The real problem is on this line:
public info? c;
Since this is a struct, C# needs to know the inner info/s layout before it could produce outer info's layout. And the inner info includes an inner inner info, which in turn includes an inner inner inner info, and so on. The compiler cannot produce a layout because of this circular reference issue.
Note: info? c is a shorthand for Nullable<info> which is itself a struct.

There isn't any way to achieve mutable value semantics of variable-sized items (semantically, I think what you're after is to have MyInfo1 = MyInfo2 generate a new linked list which is detached from the one started by MyInfo2). One could replace the info? with an info[] (which would always either be null or else populated with a single-element array), or with a holder class that wraps an instance of info, but the semantics would probably not be what you're after. Following MyInfo1 = MyInfo2, changes to MyInfo1.a would not affect MyInfo2.a, nor would changes to MyInfo1.c affect MyInfo2.c, but changes to MyInfo1.c[0].a would affect MyInfo2.c[0].a.
It would be nice if a future version of .net could have some concept of "value references", so that copying a struct wouldn't simply copy all of its fields. There is some value to the fact that .net does not support all the intricacies of C++ copy constructors, but there would also be value in allowing storage locations of type 'struct' to have an identity which would be associated with the storage location rather than its content.
Given that .net does not presently support any such concept, however, if you want info to be mutable, you're going to have to either put up with mutable reference semantics (including protective cloning) or with weird and wacky struct-class-hybrid semantics. One suggestion I would have if performance is a concern would be to have an abstract InfoBase class with descendants MutableInfo and ImmutableInfo, and with the following members:
AsNewFullyMutable -- Public instance -- Returns a new MutableInfo object, with data copied from the original, calling AsNewFullyMutable on any nested references.
AsNewMutable -- Public instance -- Returns a new MutableInfo object, with data copied from the original, calling AsImmutable on any nested references.
AsNewImmutable -- Protected instance -- Returns a new ImmutableInfo object, with data copied from the orignal, calling AsImmutable (not AsNewImmutable) on any nested references.
AsImmutable -- Public virtual -- For an ImmutableInfo, return itself; for a MutableInfo, call AsNewImmutable on itself.
AsMutable -- Public virtual -- For a MutableInfo, return itself; for an ImmutableInfo, call AsNewMutable on itself.
When cloning an object, depending upon whether one expected that the object or its descendants would be cloned again before it had to be mutated, one would call either AsImmutable, AsNewFullyMutable, or AsNewMutable. In scenarios where one would expect an object to be repeatedly defensively cloned, the object would be replaced by an immutable instance which would then no longer have to be cloned until there was a desire to mutate it.

Disclaimer: This may not achieve the goal of "struct like value type behaviour."
One solution is to use an array of one item to essentially get a reference the recursively referenced structure. Adapting my approach to your code looks something like this.
public struct info
{
public float a, b;
public info? c
{
get
{
return cArray[nextIndex];
}
set
{
steps[nextIndex] = value;
}
}
private info?[] cArray;
public info(float a, float b, info? c = null)
{
this.a = a;
this.b = b;
this.cArray = new info?[] { c }
this.c = c;
}
}

Why can I not use Marshal.Copy() to update a struct?

I have some code intended to get a struct from a byte array:
public static T GetValue<T>(byte[] data, int start) where T : struct
{
T d = default(T);
int elementsize = Marshal.SizeOf(typeof(T));
GCHandle sh = GCHandle.Alloc(d, GCHandleType.Pinned);
Marshal.Copy(data, start, sh.AddrOfPinnedObject(), elementsize);
sh.Free();
return d;
}
However, the structure d is never modified, and always returns its default value.
I have looked up the 'correct' way to do this and am using that instead, but am still curious, as I cannot see why the above should not work.
Its as simple as can be: allocate some memory, d, get a pointer to it, copy some bytes into the memory pointed at by this, return.
Not only that, but when I use similar code but with d being an array of T, it works fine.
Unless sh.AddrOfPinnedObject() isn't really pointing to d, but then what is the point of it?
Can anyone tell me why the above does not work?

GCHandle sh = GCHandle.Alloc(d, GCHandleType.Pinned);
That's where your problem started. A struct is a value type, GCHandle.Alloc() can only allocate handles for reference types. The kind whose objects are allocated on the garbage collected heap. And the kind that make pinning sensible. The C# compiler is being a bit too helpful here, it automatically emits a boxing conversion to box the value and make the statement work. Which is normally very nice and creates the illusion that value types are derived from System.Object. Quacks-like-a-duck typing.
Problem is, Marshal.Copy() will update the boxed copy of the value. Not your variable. So you don't see it change.
Directly updating the structure value is only possible with Marshal.PtrToStructure(). It contains the required smarts to convert the published layout of the struct (StructLayout attribute) to the internal layout. Which is not the same and otherwise undiscoverable.

Warning implementation detail alert, this may not be true in future versions of .Net.
structs are value types and are (generally) stored on the stack (*) , not on the heap. An address of a struct is meaningless since they are passed by value, not by reference. The array of struct is a reference type, that is a pointer to memory on the heap, so the address in memory is perfectly valid.
The point of AddrOfPinnedObject is to get the address of an object thats memory is pinned, not a struct.
Additionally, Eric Lippert has written a series of very good blog posts on the subject of reference types and value types.
(*) Unless:
1 They are fields on a class
2 They are boxed
3 They are "captured variables"
4 They are in an iterator block
(nb points 3 and 4 are corollaries of point 1)

Here's a working example:
public static T GetValue<T>(byte[] data, int start) where T : struct
{
int elementsize = Marshal.SizeOf(typeof(T));
IntPtr ptr = IntPtr.Zero;
try
{
ptr = Marshal.AllocHGlobal(elementsize);
Marshal.Copy(data, start, ptr, elementsize);
return (T)Marshal.PtrToStructure(ptr, typeof(T));
}
finally
{
if (ptr != IntPtr.Zero)
{
Marshal.FreeHGlobal(ptr);
}
}
}
But I would use explicit layout here because of struct alignment.
[StructLayout(LayoutKind.Explicit, Size = 3)]
public struct TestStruct
{
[FieldOffset(0)]
public byte z;
[FieldOffset(1)]
public short y;
}