I am calling a C++ function from C#. As arguments it receives a pointer to an array of structs.
struct A
{
int data;
}
int CFunction (A* pointerToFirstElementOfArray, int NumberOfArrayElements)
In C# I have created the same struct (as a class) and I marshall it correctly (the first element in the array is received correctly). Here is my C# definition of the C++ struct:
[StructLayout(LayoutKind.Sequential), Serializable]
class A
{
int data;
}
The first element is read correctly so I know all the fields in the struct are marshalled correctly. The problem occurs when I try to send over an array of elements. How do I create an array of classes that will be in a single memory block (chunk), so the C++ function can increment the pointer to the array?
I guess I would need something similar to stackalloc, however I belive that only works for primitive types?
Thank you for your help.
This is not intended to be a definitive answer, but merely an example of you could accomplish this using stackalloc and unsafe code.
public unsafe class Example
{
[DllImport("library.dll")]
private static extern int CFunction(A* pointerToFirstElementOfArray, int NumberOfArrayElements);
public void DoSomething()
{
A* a = stackalloc A[LENGTH];
CFunction(a, LENGTH);
}
}
Also, pay attention to the packing of the struct that the API accepts. You may have to play around with the Pack property of the StructLayout attribute. I believe the default is 4, but some APIs expect 1.
Edit:
For this to work you will have to change the declaration of A from a class to a struct.
public struct A
{
public int data;
}
Related
I came across something very much like the below at work. I have never worked with a C# codebase that makes such heavy use of structs before.
I have used fixed before to prevent the garbage collector from moving things while I work on something unsafe using pointers. But I've never seen it used while taking a pointer and passing it to a Span like this and then using the Span outside of the fixed statement.
Is this okay? I guess since Span is managed, once we pass the location to it, then if the GC moves the location of MyStruct, it should get the new location okay right?
[StructLayout(LayoutKind.Sequential)]
public unsafe struct MyInnerStruct
{
public uint InnerValueA;
public uint InnerValueB;
public float InnerValueC;
public long InnerValueD;
}
[StructLayout(LayoutKind.Sequential)]
public unsafe struct MyStruct
{
public MyInnerStruct Value0;
public MyInnerStruct Value1;
public MyInnerStruct Value2;
public MyInnerStruct Value3;
public MyInnerStruct Value4;
public MyInnerStruct Value5;
public MyInnerStruct Value6;
public MyInnerStruct Value7;
public MyInnerStruct Value8;
public MyInnerStruct Value9;
public int ValidValueCount;
public Span<MyInnerStruct> Values
{
get
{
fixed (MyInnerStruct* ptr = &Value0)
{
return new Span<MyInnerStruct>(ptr, ValidValueCount);
}
}
}
}
This is unsafe
Returning a Span<T> from a method is, in the general case, unsafe. It works, as long as one guarantees that the object it references to does not get out of scope and is not moved. These conditions apply only to stack variables that live at most as long as the Span itself. With the above code, consider this example:
public Span<MyInnerStruct> Foo()
{
MyStruct s = default;
s.Value0.InnerValueA = 3;
s.Values[1].InnerValueB = 4; // this is fine
return s.Values; // Peng! We're returning a reference to s, which is illegal
}
With the use of a pointer to initialize the Span, the runtime will also lose all means of tracking the reference it points to. So if MyStruct is part of an object, you'll easily get a dangling reference after the next GC run. .NET 5.0 and above has a method MemoryMarshal.CreateSpan<T> that works around some of the issues, but it's use is still dangerous.
A detailed analysis of the problems with such a struct can be found here. The discussed struct ValueArray<T> looks very similiar to the example given in the OP. In the end, the decision was to remove it from the code base, because it's use is just too dangerous.
This is the source code in .NET 6
public unsafe Span(void* pointer, int length)
{
_pointer = new ByReference<T>(ref Unsafe.As<byte, T>(ref *(byte*)pointer));
_length = length;
}
Your code:
fixed (MyInnerStruct* ptr = &Value0)
{
return new Span<MyInnerStruct>(ptr, ValidValueCount);
}
I did some tests, and using the span like you did is always safe to do even when the PTR gets moved around by the garbarge collector.
I also tested to write to the Span after the ptr had been moved and the original object was still correctly updated.
The Span will always return the correct values because of the ByReference
I started to learn about C# and I usually use C++.
There is a bunch of things that I'm trying to adapt, but std::array seem like impossible...
I just want to run this kind of code:
public struct Foo {};
public struct Test
{
public Foo value[20];
};
I don't want to allocate each time I use this struct and I don't want to use a class ever...
I saw fixed keyword but it works only for basic types...
There is not equivalent to something as simple as std::array?
I can even do that in C.
How would you sove this problem? (Even if it's still dynamically alocated..)
Using a fixed size buffer (fixed) is only possible for primitive types since its use is intended for interop. Array types are reference types, and so they can have dynamic size:
public struct Test
{
public Foo[] value;
}
Note however that copying the struct will only copy the reference, so the arrays will be identical. I suggest you either make the type immutable (by disabling writing to the array), or change struct to class and control cloning explicitly.
There is no such thing as a fixed size by-value array type in C# (although I have proposed it once). The closest thing you can get to it is a value tuple.
So it seems like there is no way to not do something as stupid as dynamically allocate something know at compile time. But that's C# so I just need to... try to close my eyes.
Anyway I did something to solve array alias and fixed array at the same time (I didn't ask about array alias on this question thought).
public abstract
class Array<T>
{
private T[] data;
protected Array(int size) { data = new T[size]; }
public T this[int i]
{
get { return data[i]; }
set { data[i] = value; }
}
};
public Alias : Array<int>
{
static public int Length = 10;
public Area() : base(Length) {}
};
And some people say it's quicker to write code with C#...
If someone have better I'll glady take it!
I'm looking for a data structure in .Net which keep heterogeneous structs contiguous in memory in order to be cpu-cache-friendly.
This type of data structure is explained in this blog : T-machine.org at the Iteration 4.
In .Net an array of value types (structs) keeps data contiguous in memory, but this is only working for a no-generic array.
I tried to create a ValueType[], but structs are boxed. So the references are contiguous in memory but not the real data.
After many tries I don't think this is possible natively in .Net. The only possible solution I see, it's to manually managed the seralization and deserialization of structs in a byte array, but I don't think it will be performant.
Did you find a native solution? or a better solution that mine?
Edit 1:
I'm trying to implement an entity-component-system as described in the T-Machine.org blog.
No. There is no way to do Iteration 4 in C#. You can't decide where in memory a .NET struct or class will be put. There is nothing similar to Placement New of C++.
But note that even Iteration 4 seems to have more problems than solutions:
At this point, our iterations are quite good, but we’re seeing some recurring problems:
Re-allocation of arrays when Components are added/removed (I’ve not covered this above – if you’re not familiar with the problem, google “C dynamic array”)
Fragmentation (affects every iteration after Iteration 1, which doesn’t get any worse simple because it’s already as bad as it could be)
Cross-referencing (which I skipped)
but
if you have struct of around the same size, the union trick could be enough...
public enum StructType
{
Velocity = 0,
Position = 1,
Foo = 2,
Bar = 3,
}
public struct Velocity
{
public int Vx;
public int Vy;
}
public struct Position
{
public int X;
public int Y;
public int Z;
}
public struct Foo
{
public double Weight;
public double Height;
public int Age;
}
public struct Bar
{
public int ColorR;
public int ColorG;
public int ColorB;
public int Transparency;
}
[StructLayout(LayoutKind.Explicit)]
public struct SuperStruct
{
[FieldOffset(0)]
public StructType StructType;
[FieldOffset(4)]
public Velocity Velocity;
[FieldOffset(4)]
public Position Position;
[FieldOffset(4)]
public Foo Foo;
[FieldOffset(4)]
public Bar Bar;
}
"officially" in C# there are no C unions. But thorugh the use of FixedLayout and FieldOffset you can create them. Note that they are totally incompatible with reference types, and clearly the size of the SuperStruct will be the size of the biggest possible element. In this case, 32 bytes, because Foo is 20 bytes, but there is some padding needed before and after it to align to the 8 bytes boundary.
Clearly your array would be of SuperStruct types. Note that by following the Iterion 4 example, the StructType isn't strictly necessary, because the types of the elements is written in other places.
The first one in my c++ namespace is
public struct channel_vars {
int fetch_data; /* Boolean flag */
void * data; /* (malloc'd) address of data */
unsigned int lines; /* Number of lines returned */
}
I don't know what the void* is supposed to turn into when managed
The second one is
public struct hdf_call_vars_t {
struct channel_vars p_vars;
struct channel_vars s_vars;
enum FILE_VERSION file_vers; /* Set in top level sub. used in lower */
int fetch_n; /* Boolean flag */
s_line_header_t * n_addr; /* malloc'd address of ndata */
unsigned int n_lines;
csdt_file_header_t hdr;
};
In the one above, s_line_header_t is itself a struct which is made up of doubles and ints and another structure that is also made up of basic types like ints etc.
Since managed code doesn't allow pointers, how do I convert these structures to managed types? All this code is in my c++ -cli project.
thanks,
sb
You can use IntPtr class in managed code to map your field that uses pointer on unmaged code
If you are moving to c# All classes are functionally identical to pointers. They operate almost in the same way in that they can either point to a class or be null.
void* is a bit more tricky. This can point to anything. In c# all objects inherit from the base object class. So replace all void* with object and you should be fine, since you will need a cast to get back out of a both a void* and an object.
public struct ChannelVars {
public bool FetchData;
public object Data;
public uint Lines;
}
public struct HDFCallVarsT {
public ChannelVars PVars;
public ChannelVars SVars;
//enum FILE_VERSION file_vers; //you will need to get the enum set here correctly
public bool FetchN;
public SLineHeaderT NAddr; //SLineHeaderT must be a class somewhere
public uint NLines;
public CSDTFileHeaderT HDR; //CSDTFileHeaderT must be a class somewhere
};
CityHash allows us to generate 128-bit hashes, but the 128-bit representation of an integer is defined as a pair of uint64s (as seen in the header of CityHash.h):
typedef boost::uint64_t uint64;
typedef std::pair<uint64, uint64> uint128;
I have a .NET wrapper that allows me to call the 64-bit version of city hash:
public ref class CityHashDotNet
{
public:
inline static void CityHash64Batch(const char* const value, uint64* const hashes, const int numValues)
{
// I have a wrapper for CityHash that allows me to make batch calls (saves on interops)
CityHashWrapper::CityHash64Batch(value, hashes, numValues);
}
//...
}
This allows me to easily generate hashes from C# by casting the value memory pointer to sbyte* and the hash memory pointer to ulong* and calling the CityHashDotNet wrapper function:
// The MemoryPointer is an IntPtr
CityHashDotNet.CityHash64Batch((sbyte*)values.MemoryPointer, (ulong*)hashes.MemoryPointer, size);
I would like to make a wrapper around the 128-bit version of city hash, but I don't know how to Marshall the std::pair that's necessary for the hash. I defined a class that matches the std::pair and I'm planning on using it to duplicate the std::pair structure:
public class Pair<T1, T2>
{
public T1 First { get; set; }
public T2 Second { get; set; }
public Pair(T1 first, T2 second)
{
First = first;
Second = second;
}
}
The problem is that I have no way to cast an IntPtr to an std::pair<ulong,ulong>* (ulong is the same as uint64). I tried casting it to Pair<ulong,ulong>*, but I get a build error:
// Does not work!
CityHashDotNet.CityHash128Batch((sbyte*)values.MemoryPointer, (Pair<ulong,ulong>*)hashes.MemoryPointer, size);
The error I get is:
error CS1503: ... cannot convert from `Pair<ulong,ulong>*` to `std.pair<unsigned __int64,unsigned __int64>*`
How do I get around this issue?
You can't cast between pointers to these types, they don't have compatible layout. You'll have to copy the data between arrays of each type, one field at a time. Do this inside the C++/CLI code, and let the C# code see nice .NET arrays.