Develop Reference

Read binary data from UDP to model

In my app I am receiving binary data through UDP, I wrote a binary reader that is going field by field and reading appropriate type and assigning it to model manually. Is there a way to define model in a way that will tell some kind of (de)serializer where to look for certain field instead of assigning everything?
Some fields may be arrays - their sizes are known and fixed, but model is unaware of this.
Example (current code):
byte Field1 = binaryReader.ReadByte();
sbyte Field2 = binaryReader.ReadSByte();
float[] Field3 = ...
//etc.
Desired code:
public class Model
{
[BinaryPosition(1)]
public byte Field1 { get; set;}
[BinaryPosition(2)]
public sbyte Field2 { get; set; }
[BinaryPosition(3)]
public float[] Field3 { get; set; }
}
//...
var model = BinaryDeserializer.Deserialize<Model>(byteBuffer);

There are many ways of doing this, increasing in cost and complexity the more features you require.
Starting from the simplest, a POCO struct that works with, as POCO implies, plain C types (byte, int, basically unmanaged value types, no string or anything). In your example, that would be:
struct Model
{
public byte Field1;
public sbyte Field2;
}
There's many ways of converting your byte[] array to that structure, in order of efficiency (and version of .Net): MemoryMarshal.Cast, fixed+pointers and Marshal.PtrToStructure.
If you require more complex objects like strings, you need to use the MarshalAs attribute to declare how you want to pack the string, typically as a value array or less typically as a pointer, and then you need to use Marshal.PtrToStructure to have the marshaller silently handle all the extra copies for you. Obviously, much less efficient.

Is there a way to have an interface method do an operation over variable size structs?

Having various C# structs of varying sizes, most of them do have one member in common: a CRC field of type ushort. This field is always the last in each struct. (Yes, we are doing some hardware programming here)
So, I have an IHasCRC interface which implements the required calculations in the default methods. yet, I didn't figure out a way to have the interface method access the structs bytes without "external help". The CRC routines all access the data through a ReadOnlySpan<byte> .
See what I mean:
public interface IHasCrc
{
public ushort CalculateCrc(ReadOnlySpan<byte> dataBuffer)
{
return CRC.ComputeChecksum(dataBuffer[0..^2]);
}
public bool IsCrcValid(ReadOnlySpan<byte> dataBuffer)
{
ushort crc = MemoryMarshal.Cast<byte, ushort>(dataBuffer[^2..])[0];
ushort crc_computed = CRC.ComputeChecksum(dataBuffer[0..^2]);
return crc == crc_computed;
}
}
For this work, the caller must provide the data buffer (the struct cast to a read only span of type byte) instead of the method implementation casting from ´this´. Like this:
HWSTRUCT_xxx data; // ... somewhere initialized
data.CRC = (data as IHasCrc).CalculateCrc(StructExtensions.AsSpan<HWSTRUCT_xxx>(ref data));
The AsSpan method does the magic through MemoryMarshal's Cast and CreateReadOnlySpan, which requires a ref reference, thus can't pass this.
How could I implement a CalculateCrc (and IsCrcValid) method in the interface that does all that without the need to pass the Span as parameter?? Remember that all structs have different sizes and thus the CRC field is on different byte positions in the struct, but it's always the last 2 bytes.
Update:
After some discussions online and a bit of researching, I came to a little improvement:
public interface IHasCrc2<T> where T : struct
{
private unsafe ReadOnlySpan<byte> GetSpan(ref T myStruct)
{
return new ReadOnlySpan<byte>(Unsafe.AsPointer(ref myStruct), Unsafe.SizeOf<T>());
}
public ushort CalculateCrc();
public ushort CalculateCrcImplementation(ref T me)
{
return CRC.ComputeChecksum(this.GetSpan(ref me));
}
public bool IsCrcValid();
}
With that interface, details are better hidden and easier to implement, although not yet totally transparent.
Implementing structs are like this, it all comes down to implementing the two methods of the interface, with basically a one liner each, only replacing the struct name. Unfortunately boxing is needed to access the default implementation in the interface. Although this might go into an helper method I guess and thus avoid boxing.
public struct HWSTRUCT_xxx : IHasCrc2<HWSTRUCT_xxx>
{
public HWSTRUCT_base Base;
// other data
public ushort CRC;
public ushort CalculateCrc()
{
return this.CRC = (this as IHasCrc2<HWSTRUCT_xxx>).CalculateCrcImplementation(ref this);
}
public bool IsCrcValid()
{
return (this as IHasCrc2<HWSTRUCT_xxx>).CalculateCrcImplementation(ref this) == this.CRC;
}
}

C# fixed size array

I started to learn about C# and I usually use C++.
There is a bunch of things that I'm trying to adapt, but std::array seem like impossible...
I just want to run this kind of code:
public struct Foo {};
public struct Test
{
public Foo value[20];
};
I don't want to allocate each time I use this struct and I don't want to use a class ever...
I saw fixed keyword but it works only for basic types...
There is not equivalent to something as simple as std::array?
I can even do that in C.
How would you sove this problem? (Even if it's still dynamically alocated..)

Using a fixed size buffer (fixed) is only possible for primitive types since its use is intended for interop. Array types are reference types, and so they can have dynamic size:
public struct Test
{
public Foo[] value;
}
Note however that copying the struct will only copy the reference, so the arrays will be identical. I suggest you either make the type immutable (by disabling writing to the array), or change struct to class and control cloning explicitly.
There is no such thing as a fixed size by-value array type in C# (although I have proposed it once). The closest thing you can get to it is a value tuple.

So it seems like there is no way to not do something as stupid as dynamically allocate something know at compile time. But that's C# so I just need to... try to close my eyes.
Anyway I did something to solve array alias and fixed array at the same time (I didn't ask about array alias on this question thought).
public abstract
class Array<T>
{
private T[] data;
protected Array(int size) { data = new T[size]; }
public T this[int i]
{
get { return data[i]; }
set { data[i] = value; }
}
};
public Alias : Array<int>
{
static public int Length = 10;
public Area() : base(Length) {}
};
And some people say it's quicker to write code with C#...
If someone have better I'll glady take it!

Data structure in .Net keeping heterogeneous structs contiguous in memory

I'm looking for a data structure in .Net which keep heterogeneous structs contiguous in memory in order to be cpu-cache-friendly.
This type of data structure is explained in this blog : T-machine.org at the Iteration 4.
In .Net an array of value types (structs) keeps data contiguous in memory, but this is only working for a no-generic array.
I tried to create a ValueType[], but structs are boxed. So the references are contiguous in memory but not the real data.
After many tries I don't think this is possible natively in .Net. The only possible solution I see, it's to manually managed the seralization and deserialization of structs in a byte array, but I don't think it will be performant.
Did you find a native solution? or a better solution that mine?
Edit 1:
I'm trying to implement an entity-component-system as described in the T-Machine.org blog.

No. There is no way to do Iteration 4 in C#. You can't decide where in memory a .NET struct or class will be put. There is nothing similar to Placement New of C++.
But note that even Iteration 4 seems to have more problems than solutions:
At this point, our iterations are quite good, but we’re seeing some recurring problems:
Re-allocation of arrays when Components are added/removed (I’ve not covered this above – if you’re not familiar with the problem, google “C dynamic array”)
Fragmentation (affects every iteration after Iteration 1, which doesn’t get any worse simple because it’s already as bad as it could be)
Cross-referencing (which I skipped)
but
if you have struct of around the same size, the union trick could be enough...
public enum StructType
{
Velocity = 0,
Position = 1,
Foo = 2,
Bar = 3,
}
public struct Velocity
{
public int Vx;
public int Vy;
}
public struct Position
{
public int X;
public int Y;
public int Z;
}
public struct Foo
{
public double Weight;
public double Height;
public int Age;
}
public struct Bar
{
public int ColorR;
public int ColorG;
public int ColorB;
public int Transparency;
}
[StructLayout(LayoutKind.Explicit)]
public struct SuperStruct
{
[FieldOffset(0)]
public StructType StructType;
[FieldOffset(4)]
public Velocity Velocity;
[FieldOffset(4)]
public Position Position;
[FieldOffset(4)]
public Foo Foo;
[FieldOffset(4)]
public Bar Bar;
}
"officially" in C# there are no C unions. But thorugh the use of FixedLayout and FieldOffset you can create them. Note that they are totally incompatible with reference types, and clearly the size of the SuperStruct will be the size of the biggest possible element. In this case, 32 bytes, because Foo is 20 bytes, but there is some padding needed before and after it to align to the 8 bytes boundary.
Clearly your array would be of SuperStruct types. Note that by following the Iterion 4 example, the StructType isn't strictly necessary, because the types of the elements is written in other places.

Restricting use of a structure in C#

Ok so lets say I have a structure A like that:
Struct A{
private String _SomeText;
private int _SomeValue;
public A(String someText, int SomeValue) { /*.. set the initial values..*/ }
public String SomeText{ get { return _SomeText; } }
public int SomeValue{ get { return _SomeValue; } }
}
Now what I want to be able to do is to return that Structure A as a result of a method in a Class ABC, like that:
Class ABC{
public A getStructA(){
//creation of Struct A
return a;
}
}
I don't want any programmer using my library (which will have Struct A and Class ABC and some more stuff) to ever be able to create an instance of Struct A.
I want the only way for it to be created is as a return from the getStructA() method. Then the values can be accessed through the appropriate getters.
So is there any way to set a restrictions like that? So a Structure can't be instantiated outside of a certain class? Using C#, .Net4.0.
Thanks for your help.
---EDIT:----
To add some details on why am I trying to achieve this:
My class ABC has some "status" a person can query. This status has 2 string values and then a long list of integers.
There never will be a need to create an object/instance of "Status" by the programmer, the status can only be returned by "getStatus()" function of the class.
I do not want to split these 3 fields to different methods, as to obtain them I am calling Windows API (p/invoke) which returns similar struct with all 3 fields.
If I was indeed going to split it to 3 methods and not use the struct, I would have to either cache results or call the method from Windows API every time one of these 3 methods is called...
So I can either make a public struct and programmers can instantiate it if they want, which will be useless for them as there will be no methods which can accept it as a parameter. Or I can construct the library in such a way that this struct (or change it to a class if it makes things easier) can be obtained only as a return from the method.

If the "restricted" type is a struct, then no, there is no way to do that. The struct must be at least as public as the factory method, and if the struct is public then it can be constructed with its default constructor. However, you can do this:
public struct A
{
private string s;
private int i;
internal bool valid;
internal A(string s, int i)
{
this.s = s;
this.i = i;
this.valid = true;
}
...
and now you can have your library code check the "valid" flag. Instances of A can only be made either (1) by a method internal to your library that can call the internal constructor, or (2) by the default constructor. You can tell them apart with the valid flag.
A number of people have suggested using an interface, but that's a bit pointless; the whole point of using a struct is to get value type semantics and then you go boxing it into an interface. You might as well make it a class in the first place. If it is going to be a class then it is certainly possible to make a factory method; just make all the ctors of the class internal.
And of course I hope it goes without saying that none of this gear should be used to implement code that is resistant to attack by a fully-trusted user. Remember, this system is in place to protect good users from bad code, not good code from bad users. There is nothing whatsoever that stops fully trusted user code from calling whatever private methods they want in your library via reflection, or for that matter, altering the bits inside a struct with unsafe code.

Create a public interface and make the class private to the class invoking it.
public ISpecialReturnType
{
String SomeText{ get; }
int SomeValue{ get; }
}
class ABC{
public ISpecialReturnType getStructA(){
A a = //Get a value for a;
return a;
}
private struct A : ISpecialReturnType
{
private String _SomeText;
private int _SomeValue;
public A(String someText, int SomeValue) { /*.. set the initial values..*/ }
public String SomeText{ get { return _SomeText; } }
public int SomeValue{ get { return _SomeValue; } }
}
}

What exactly are you concerned about? A structure is fundamentally a collection of fields stuck together with duct tape. Since struct assignment copies all of the fields from one struct instance to another, outside the control of the struct type in question, structs have a very limited ability to enforce any sort of invariants, especially in multi-threaded code (unless a struct is exactly 1, 2, or 4 bytes, code that wants to create an instance which contains a mix of data copied from two different instances may do so pretty easily, and there's no way the struct can prevent it).
If you want to ensure that your methods will not accept any instances of a type other than those which your type has produced internally, you should use a class that either has only internal or private constructors. If you do that, you can be certain that you're getting the instances that you yourself produced.
EDIT
Based upon the revisions, I don't think the requested type of restriction is necessary or particularly helpful. It sounds like what's fundamentally desired to stick a bunch of values together and store them into a stuck-together group of variables held by the caller. If you declare a struct as simply:
public struct QueryResult {
public ExecutionDuration as Timespan;
public CompletionTime as DateTime;
public ReturnedMessage as String;
}
then a declaration:
QueryResult foo;
will effectively create three variables, named foo.ExecutionDuration, foo.CompletionTime, and foo.ReturnedMessage. The statement:
foo = queryPerformer.performQuery(...);
will set the values of those three variables according to the results of the function--essentially equivalent to:
{
var temp = queryPerformer.performQuery(...);
foo.ExecutionDuration = temp.ExecutionDuration
foo.CompletionTime = temp.CompletionTime;
foo.ReturnedMessage = temp.ReturnedMessage;
}
Nothing will prevent user code from doing whatever it wants with those three variables, but so what? If user code decides for whatever reason to say foo.ReturnedMessage = "George"; then foo.ReturnedMessage will equal George. The situation is really no different from if code had said:
int functionResult = doSomething();
and then later said functionResult = 43;. The behavior of functionResult, like any other variable, is to hold the last thing written to it. If the last thing written to it is the result of the last call to doSomething(), that's what it will hold. If the last thing written was something else, it will hold something else.
Note that a struct field, unlike a class field or a struct property, can only be changed either by writing to it, or by using a struct assignment statement to write all of the fields in one struct instance with the values in corresponding fields of another. From the consumer's perspective, a read-only struct property carries no such guarantee. A struct may happen to implement a property to behave that way, but without inspecting the code of the property there's no way to know whether the value it returns might be affected by some mutable object.