Develop Reference

Best way to write bytes to a byte array in a struct?

I'm trying to create a packet struct that is basically a byte builder of fixed length. I have a WriteByte function written in 3 different ways. Just wondering which is best (or if there's a better way altogether) and which will keep the GC happy. BTW, I have a Position field that has to be updated when a byte(s) is written. The functions will expand to include WriteUInt16, WriteFloat etc... Not sure what the best approach is. Any advice is appreciated.
Should this be a struct? I'd like to do as little allocation as possible because these Packs will be created very frequently.
Should I put the WriteByte in the struct itself as a method (option 1), as an extension of Pack (passed by ref) (option 2), or just use a static helper class (passed by ref) (option 3)?
Here's the code:
public struct Pack
{
public byte[] Data { get; internal set; }
public int Pos { get; internal set; }
public Pack(byte opcode, ushort size)
{
++size; // make room for byte opcode
Data = new byte[2 + size]; // make room to prepend the size (ushort)
BitConverter.GetBytes(size).CopyTo(Data, 0);
Data[2] = opcode;
Pos = 3; // start writing at position 3
}
// option 1
// use: pack.WriteByte(0x01)
public void WriteByte(byte value) => Data[Pos++] = value;
}
public static class SPackExtensions
{
// option 2
// use: pack.WriteByte(0x01)
public static void WriteByte(ref this Pack pack, byte value)
{
pack.Data[pack.Pos++] = value;
}
}
public static class PackWriter
{
// option 3
// use: PackWriter.WriteByte(ref pack, 0x01)
public static void WriteByte(ref Pack pack, byte value)
{
pack.Data[pack.Pos++] = value;
}
}

Should this be a struct? I'd like to do as little allocation as possible because these Packs will be created very frequently.
Maybe. Even with a struct you will still be allocating two objects, one when converting the size to an array, and one for the array itself. So the overhead for allocating the object will probably be negligible, especially since it will probably be much smaller than the array. However, structs are recommended to be immutable, since it can be quite confusing when passing around a object if the data-array is shared, but the Pos is not.
But you should probably use BitConverter.TryWriteBytes or BinaryPrimitives.TryWriteInt16LittleEndian to avoid the allocation when writing the length.
Should I put the WriteByte in the struct itself as a method (option 1), as an extension of Pack (passed by ref) (option 2), or just use a static helper class (passed by ref) (option 3)?
The generated code should be identical, so go for whatever is easiest to use and most readable. I would probably argue for option 1. In my opinion, using extension methods or helper classes would bring little advantages in this case.
these Packs will be created very frequently
When writing performance optimized code, a good principle is avoiding allocations altogether. And as far as I can see this object would be impossible to put into a object pool and reuse, since the position parameter is not possible to reset. So I would consider basing my object around a span that is fetched from a pool of fixed size buffers.
Also, I would highly recommend that you profile your code. "Very frequently" might range from a 1hz to 10^9hz depending on context. So you should check if this is an actual problem in your particular context.

What happens when I assign a `ref` value to an `out` argument in C#?

I am writing an class that retrieves binary data and supports generically converting it into primitive types. It's supposed to be as efficient as possible. Here is what it looks like right now:
public abstract class MemorySource {
public abstract Span<byte> ReadBytes(ulong address, int count);
public unsafe bool TryRead<T>(ulong address, out T result) where T : unmanaged {
Span<byte> buffer = ReadBytes(address, sizeof(T));
result = default;
// If the above line is commented, `result = ref <...>` won't compile, showing CS0177.
if (!buffer.IsEmpty) {
result = ref Unsafe.As<byte, T>(ref buffer.GetPinnableReference());
return true;
} else
return false;
}
}
Since I'm working with large amounts of memory, and my code is going to be performing a lot of small reading operations. I want to minimize the amount of times the memory is copied around.
The ReadBytes implementation will either a) create a span across a part an already existing array on the heap, or b) stackalloc a buffer and fill it with data from a remote source (depending on the data I'll be working with). The point is, it will not be allocating anything on the heap by itself.
I want my TryRead<T> method to return a typed reference to the span's memory, rather than copy that memory into a new value, and I want to know if that's possible. I've noticed that I can't assign a ref value to an out argument without initializing it, but I can after, which makes little sense if we assume that I'm assigning a reference.
I guess what I'm asking is, what's really going on in this code? Am I returning a reference to an existing value, or is that value being copied into a new stack-allocated one? And how would the behavior be different with stack-allocated and heap-allocated spans? Would GC know to update the reference of type T when moving the data, in case of a heap-allocated span being used?

Primitive types are value types anyway, so you shouldn't worry about allocating on the heap when reading them.
You cannot use stackalloc for this code, because you can't (or shouldn't try to) return a pointer to it, as it will be destroyed at the end of the function.
The code you have so far is dangerous, because you are returning a pinnable reference which is not actually pinned.
The reason you are having problems with the ref parameter is because in the else you are not assigning it at all. You should move the result = default; line into the else branch.
Either way, you are far better off using MemoryMarshal for all of this, note that this does not require unsafe code
public bool TryRead<T>(ulong address, out T result) where T : unmanaged
{
ReadOnlySpan<byte> buffer = ReadBytes(address, sizeof(T));
if (!buffer.IsEmpty)
{
result = MemoryMarshal.Read<T>(buffer);
return true;
}
result = default;
return false;
}

Allocation of value types

When you assign an instance of a value type to another instance, the object is copied bit-by-bit to the target location:
private struct Word
{
public Word(char c) { ... }
}
public void Method(Word a)
{
Word b = a; //a is copied and stored in b
}
But given the following code:
private Word _word;
public void Method() {
_word = new Word('x');
}
I suspect that the right-hand side (RHS) expression is evaluated first - which instantiates a value type on the stack - and then the value is copied and stored on the location of the _word field, which is on the heap.
The alternative would be to take the left-hand side into consideration, and instantiate the value type directly on _word, avoiding having to copy the object.
Is my suspicion correct? If it is, I suppose it's safe to assume that the first block of code would perform better than the second.
//1 instantiation + 10k copies
Word[] words = new Word[10000];
Word word = new Word('x');
for (int i = 0; i < 10000; i++)
words[i] = word;
//10k instantiations + 10k copies
Word[] words = new Word[10000];
for (int i = 0; i < 10000; i++)
words[i] = new Word('x');
Note: I'm not trying to micro-optimize anything.
Edit: The core of my question is, as Lee puts it: Are structs allocated in place directly, or do they need to be allocated then copied?

When you assign an instance of a value type to another instance, the object is copied bit-by-bit to the target location
When you assign an instance of a value type to a variable of the same type, the value is copied to the target location, yes. But that is true of reference types as well: the reference is copied bit by bit to the target location. The referent of course stays right where it is.
I suspect that the right-hand side (RHS) expression is evaluated first
The specification states that the left hand side is evaluted to produce a variable, then the right hand side is evaluated to produce a value, and then the assignment happens.
In the examples you give the evaluation of the left hand side does not produce an observable side effect and therefore its evaluation can be re-ordered by the optimizers in the C# compiler, the jitter or the CPU if any of them so choose. But if you had something like
x[i++] = v();
then the side effect on the left hand side has to happen before the call on the right hand side.
The core of my question is: Are structs allocated in place directly, or do they need to be allocated then copied?
The specification states that structures are allocated in a temporary location -- which would typically be the stack or a register in practice -- and then copied to their final destination. However, there are some situations in which the optimizer can determine that it is impossible for the user to notice if the mutation happens "in place" at the final destination. This is a copy elision optimization, and the C# compiler will perform this optimization if it feels it can get away with it.
For more details see my article on the subject:
http://ericlippert.com/2010/10/11/debunking-another-myth-about-value-types/

Are either preferred?
What is the business case for multiple identical structs?
If you need multiple identical objects is a struct the best choice?
A struct (re)initialized in that manner is probably not a good solution for the example use case
In the new array the WordStruct is allocated and initialized using the default ctor (no ctor)
You don't have the option to initialize a struct array with another ctor
If you do need identical structs then this would be preferred
WordStruct[] WordStructS = new WordStruct[1000];
for (int i = 0; i < WordStructS.Length; i++) { WordStructS[i].C = 'x'; }
If multiple identical objects what you really need to do then consider a class
A class new array is allocated but not yet initialized
You don't waste resources initializing with the default constructor
WordClass[] WordClassS = new WordClass[1000];
for (int i = 0; i < WordClassS.Length; i++) { WordClassS[i] = new WordClass('x'); }
If you want to generalize deep copy of an object (struct or class) then consider IConable
In the case of a struct I suspect it is more efficient than a bit wise copy (but I am not positive)
In the case of class it will make a clone (deep copy) not a reference
public struct WordStruct : ICloneable
{
public char C;
public WordStruct(char C)
{
this.C = C;
}
public object Clone()
{
WordStruct newWordStruct = (WordStruct)this.MemberwiseClone();
return newWordStruct;
}
}
I know in a comment you said curiosity but that is not clear in the question
In the question you state first block of code is preferred over the second
I get it is an interesting question of curiosity
But if it is just curiosity then the question should have stopped at
Is my suspicion correct?

Cycle in the struct layout that doesn't exist

This is a simplified version of some of my code:
public struct info
{
public float a, b;
public info? c;
public info(float a, float b, info? c = null)
{
this.a = a;
this.b = b;
this.c = c;
}
}
The problem is the error Struct member 'info' causes a cycle in the struct layout. I'm after struct like value type behaviour. I could simulate this using a class and a clone member function, but I don't see why I should need to.
How is this error true? Recursion could perhaps cause construction forever in some similar situations, but I can't think of any way that it could in this case. Below are examples that ought to be fine if the program would compile.
new info(1, 2);
new info(1, 2, null);
new info(1, 2, new info(3, 4));
edit:
The solution I used was to make "info" a class instead of a struct and giving it a member function to returned a copy that I used when passing it. In effect simulating the same behaviour as a struct but with a class.
I also created the following question while looking for an answer.
Value type class definition in C#?

It's not legal to have a struct that contains itself as a member. This is because a struct has fixed size, and it must be at least as large as the sum of the sizes of each of its members. Your type would have to have 8 bytes for the two floats, at least one byte to show whether or not info is null, plus the size of another info. This gives the following inequality:
size of info >= 4 + 4 + 1 + size of info
This is obviously impossible as it would require your type to be infinitely large.
You have to use a reference type (i.e. class). You can make your class immutable and override Equals and GetHashCode to give value-like behaviour, similar to the String class.

The reason why this creates a cycle is that Nullable<T> is itself a struct. Because it refers back to info you have a cycle in the layout (info has a field of Nullable<info> and it has a field of info) . It's essentially equivalent to the following
public struct MyNullable<T> {
public T value;
public bool hasValue;
}
struct info {
public float a, b;
public MyNullable<info> next;
}

The real problem is on this line:
public info? c;
Since this is a struct, C# needs to know the inner info/s layout before it could produce outer info's layout. And the inner info includes an inner inner info, which in turn includes an inner inner inner info, and so on. The compiler cannot produce a layout because of this circular reference issue.
Note: info? c is a shorthand for Nullable<info> which is itself a struct.

There isn't any way to achieve mutable value semantics of variable-sized items (semantically, I think what you're after is to have MyInfo1 = MyInfo2 generate a new linked list which is detached from the one started by MyInfo2). One could replace the info? with an info[] (which would always either be null or else populated with a single-element array), or with a holder class that wraps an instance of info, but the semantics would probably not be what you're after. Following MyInfo1 = MyInfo2, changes to MyInfo1.a would not affect MyInfo2.a, nor would changes to MyInfo1.c affect MyInfo2.c, but changes to MyInfo1.c[0].a would affect MyInfo2.c[0].a.
It would be nice if a future version of .net could have some concept of "value references", so that copying a struct wouldn't simply copy all of its fields. There is some value to the fact that .net does not support all the intricacies of C++ copy constructors, but there would also be value in allowing storage locations of type 'struct' to have an identity which would be associated with the storage location rather than its content.
Given that .net does not presently support any such concept, however, if you want info to be mutable, you're going to have to either put up with mutable reference semantics (including protective cloning) or with weird and wacky struct-class-hybrid semantics. One suggestion I would have if performance is a concern would be to have an abstract InfoBase class with descendants MutableInfo and ImmutableInfo, and with the following members:
AsNewFullyMutable -- Public instance -- Returns a new MutableInfo object, with data copied from the original, calling AsNewFullyMutable on any nested references.
AsNewMutable -- Public instance -- Returns a new MutableInfo object, with data copied from the original, calling AsImmutable on any nested references.
AsNewImmutable -- Protected instance -- Returns a new ImmutableInfo object, with data copied from the orignal, calling AsImmutable (not AsNewImmutable) on any nested references.
AsImmutable -- Public virtual -- For an ImmutableInfo, return itself; for a MutableInfo, call AsNewImmutable on itself.
AsMutable -- Public virtual -- For a MutableInfo, return itself; for an ImmutableInfo, call AsNewMutable on itself.
When cloning an object, depending upon whether one expected that the object or its descendants would be cloned again before it had to be mutated, one would call either AsImmutable, AsNewFullyMutable, or AsNewMutable. In scenarios where one would expect an object to be repeatedly defensively cloned, the object would be replaced by an immutable instance which would then no longer have to be cloned until there was a desire to mutate it.

Disclaimer: This may not achieve the goal of "struct like value type behaviour."
One solution is to use an array of one item to essentially get a reference the recursively referenced structure. Adapting my approach to your code looks something like this.
public struct info
{
public float a, b;
public info? c
{
get
{
return cArray[nextIndex];
}
set
{
steps[nextIndex] = value;
}
}
private info?[] cArray;
public info(float a, float b, info? c = null)
{
this.a = a;
this.b = b;
this.cArray = new info?[] { c }
this.c = c;
}
}

Not able to modify object of struct in loop

I have a List of structure.In the loop i am trying to modify the object's property,which is happening,but when i (Quick look in Visual studio)look into the list object ,the new value is not reflecting.Is it by virtue that the structure's object cannot be modified when in a collection?
I am using generics list with the struct as the type in the list

You mention "modify the object's property" in the context of a struct, but importantly a struct is not an object. Other people have answered as to the issue with structs being copied (and changes discarded), but to take that further the real problem here is that you have a mutable (changeable) struct at all. Unless you are on XNA (or similar) there is simply no need.
If you want to be able to change properties, make it a class:
public class Foo {
public string Bar {get;set;}
}
This is now a reference-type, and your changes (obj.Bar = "abc";) will be preserved through the foreach. If you really want/need a struct, make it immutable:
public struct Foo {
private readonly string bar;
public string Bar { get {return bar; }}
public Foo(string bar) {this.bar = bar;}
}
Now you can't make the mistake of changing the value of a copy; you would instead have to use the indexer to swap the value (list[i] = new Foo("abc");). More verbose (and you can't use foreach), but correct.
But IMO, use a class. Structs are pretty rare, to be honest. If you aren't sure: class.

If you are using a foreach loop you probably got
Compiler Error CS1654
Error Message Cannot modify members of
'variable' because it is a 'read-only
variable type'
This error occurs when you try to
modify members of a variable which is
read-only because it is in a special
construct.
One common area that this occurs is
within foreach loops. It is a
compile-time error to modify the value
of the collection elements. Therefore,
you cannot make any modifications to
elements that are value types,
including structs.
You could however try
struct MyStruct
{
public int i;
}
List<MyStruct> list = new List<MyStruct>
{ new MyStruct { i = 1 }, new MyStruct { i = 2 } };
for(int i = 0; i < list.Count; i++)
{
MyStruct val = list[i];
val.i++;
list[i] = val;
}
EDIT
See also Structs Tutorial
Structs vs. Classes
Structs may seem similar to classes,
but there are important differences
that you should be aware of. First of
all, classes are reference types and
structs are value types.

I THINK i know what the problem might be.
struct Astruct
{
int amember;
}
List < Astruct > listofStructs;
foreach(Astruct A in listofStructs)
{
A.amember = 1337;
}
if this is what you are doing...
when you use structs in c# they are not referenced but copied! so that means the contents of your list is being COPIED to A, so when you change A it doesn't change the value in the list!
to solve this problem (if this is your problem...) either use CLASSES rather than STRUCTS, that way A would be a reference, OR use a manual iterating for loop instead, ie:
for(int i=0;i < listofStructs.Count;i++)
{
listofStructs[i].amember = 1337;
}
alternatively, if you’re using a list, you maybe should use an iterator or something... but the above should definitely fix that problem.

Given the information in your post (although I'd have liked to see the code itself), let me put forth the most probable issue and its fix.
foreach(var s in listOfStructs)
{
s.Property = x;
}
s is assigned to a copy of the actual struct in the collection. s.set_Property is now modifying the copy which is thrown away at the end of the current iteration.
This is because 2 value type variables cannot point to the same instance.
struct1 = new MyStruct(100, 200);
struct2 = struct1; // struct2 is now a copy of struct1
Now to the problem of how do you modify the instances in a collection:
Get the object to modify in a local variable (copy created). Modify it. Now remove the original object and insert the copy. use listOfStructs[i] = modifiedInstance.