Develop Reference

How can I assing a certain array member to a member variable of a class which is also an array

Sorry for the confusing title! I'll try to simplify it.
I have a class which I created to immitate the behavior of typedef in C/C++. I want to use a user defined type Pixel for all pixel values because today it is byte but in the future it can be a uint16.
I am scanning all the places that pixels are handled and there are operations like this:
private void ParseImageData()
{
byte[] data = new byte[imageSize];
foreach (byte[] packet in ImagePackets)// private readonly List<byte[]> ImagePackets;
{
byte[] datapacket = SomeFunction(packet);// private byte[] SomeFunction(byte[] imagePacket)
datapacket.CopyTo(data, count);
count += PACKET_SIZE;
}
// do something with Pixel type
}
So this code block basically transfer data that is stored in a list to a array. The problematic part is the CopyTo line. If I convert data from byte[] to Pixel[], it will try to copy bytes to Pixels. Which is not right. I know that this doesn't make sense but If I could write something like this datapacket.CopyTo((pixel.Value)[], count) my problems would be solved!.
So now I added a for loop to copy the content manually. But I hated this method and I am not sure it will work if Pixel.Value will be something other than byte. I am also considering to add a new class for pixel arrays but I don't think that is an elegnat way to do it either. Here is my (hopefully)temporary solution:
private void ParseImageData()
{
Pixel[] pixels = new Pixel[imageSize];
foreach (byte[] packet in ImagePackets)
{
byte[] datapacket = SomeFunction(packet);
for(int i = 0; i< PACKET_SIZE; i++,count++)
{
pixels[i].Value = datapacket[i];
}
}
// do something with Pixel type
}
Here is the Pixel class and a potantial Pixel array class:
public class Pixel
{
public Pixel(byte val)
{
this.Value = val;
}
public byte Value { get; set; }
}
public class PixelArr
{
public PixelArr(ushort size)
{
this.Value = new byte[size];
}
public byte[] Value { get; set; }
}

You do not want a Pixel class; an object per pixel would be catastrophic. Honestly, I think you should just eat the byte => ushort conversion later if you need to, but if you really really want a Pixel struct that is basically a fancy byte, you could do something like:
public readonly struct Pixel // readonly *very* deliberate here
{
private readonly byte _value;
public Pixel(byte value) => _value = value;
public byte Value => _value;
// TODO: add equality/hash/tostring, etc
// TODO: operators?
}
Now, you can't use variance with value-type arrays, but: you can use reinterpret operations on spans. If you want to go that route: MemoryMarshal.Cast<byte, Pixel> (or the other way around) may help. It the two types are of different sizes, the output span will have a different length than the input, accordingly. But: if you're changing from byte to ushort, that then prompts questions about "endianness" and which CPUs you need this to work on!
However, and I can't emphasize this enough: I think you'd be better off just eating the change to ushort later and dealing with raw values.

Casting a managed array to an array of structs without copying

I have a managed array provided by a third party library. The type of the array does not directly reflect the data stored in the array, but instead it's the data interpreted as integers.
So int[] data = Lib.GetData(); gives me an integer array, that I would like to cast into an array of DataStructure, that could look like this.
struct DataStructure {
public int Id;
public double Value;
}
Current I use Marshal.Copy (an implementation can been seen here), but it seems a bit excessive to copy the entire thing.
Does something like this exists:
int[] data = Lib.GetData();
DataStructure[] dataStructs = InterpretAs<DataStructure>(data);
where no copying is required, but access to dataStruct elements can be done like dataStruct[1].Id?
EDIT 2:
If I just want a single DataStructure, I can use
public static T ToStruct<T>(byte[] bytes) where T : struct
{
GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
T something = Marshal.PtrToStructure<T>(handle.AddrOfPinnedObject());
handle.Free();
return something;
}
where no copying is required.
EDIT:
The answers possible duplicate are currently about 7 years old, and they consists of either copying the data or implementing a hack.

There is actually a cheat, but it is an ugly unsafe totally unsafe cheat:
[StructLayout(LayoutKind.Sequential)]
//[StructLayout(LayoutKind.Sequential, Pack = 4)]
public struct DataStructure
{
public int Id;
public double Value;
}
[StructLayout(LayoutKind.Explicit)]
public struct DataStructureConverter
{
[FieldOffset(0)]
public int[] IntArray;
[FieldOffset(0)]
public DataStructure[] DataStructureArray;
}
and then you can convert it without problems:
var myarray = new int[8];
myarray[0] = 1;
myarray[3] = 2;
//myarray[4] = 2;
DataStructure[] ds = new DataStructureConverter { IntArray = myarray }.DataStructureArray;
int i1 = ds[0].Id;
int i2 = ds[1].Id;
Note that depending on the size of DataStructure (if it is 16 bytes or 12 bytes), you have to use Pack = 4 (if it is 12 bytes) or you don't need anything (see explanation (1) later)
I'll add that this technique is undocumented and totally unsafe. It even has a problem: ds.Length isn't the length of the DataStructure[] but is the length of the int[] (so in the example given it is 8, not 2)
The "technique" is the same I described here and originally described here.
explanation (1)
The sizeof(double) is 8 bytes, so Value is normally aligned on the 8 bytes boundary, so normally there is a "gap" between Id (that has sizeof(int) == 4) and Value of 4 bytes. So normally sizeof(DataStructure) == 16. Depending on how the DataStructure is built, there could not be this gap, so the Pack = 4 that forces alignment on the 4 byte boundary.

PInvoke Pointer to struct including float array

I'm attempting to P Invoke a C library for use on a Xamarin android app.
Consider the following C structure:
typedef struct
{
bool myBool;
myOtherStruct sOtherStruct;
int myInt;
float myFloat;
float myFloatArray[1024];
float myFloatArray2[1024];
float myFloatArray3[20];
float myFloatArray4[30];
} sMyStruct;
This is called using the following function:
unsigned int initialise(sMyStruct* a_pMyStruct)
I've put this into a C# structure:
[StructLayout(LayoutKind.Sequential)]
public unsafe struct SMyStruct
{
bool myBool;
myOtherStruct sOtherStruct;
int myInt;
float myFloat;
public fixed float myFloatArray[1024];
public fixed float myFloatArray2[1024];
public fixed float myFloatArray3[20];
public fixed float myFloatArray4[30];
public unsafe float[] MyFloatArray
{
get
{
fixed (float* ptr = myFloatArray)
{
float[] array = new float[1024];
Marshal.Copy((IntPtr)ptr, array, 0, 1024 * sizeof(float));
return array;
}
}
}
public SMyStruct (bool MyBool, myOtherStruct otherStruct, int MyInt, float MyFloat)
{
myBool = MyBool;
sOtherStruct = myOtherStruct;
myInt = MyInt;
myFloat = MyFloat;
}
Here's my function in C# to invoke this:
[DllImport("libMylib")]
private static extern unsafe uint initialise(SMyStruct* a_pMyStruct);
I then call this function with:
public unsafe void init ()
{
SMyStruct initStruct;
uint result = initialise(&initStruct);
}
So what happens is the C function will return my structure with A LOT of data. I then pass the structure again into another C routine which uses these parameters for the rest of the program.
My issue is how do I get the float array data back into the correct structure variable so that I can pass it again? At current my code has been based on these questions:
Marshalling float Array to c# and
Marshalling complex struct to c#
But I've not managed to code this so I can pass the float array back to my struct without even seeing a compiler error (let alone failing when I test it!)
How do I get the float array data into my structure?
EDIT
After a couple of answers and comments, I'm adding what I was doing initially to try and add some clarity.
I get a compiler error when rather than using the "public unsafe float[]..." routine above I do this (within the struct):
public SMyStruct (bool MyBool, myOtherStruct otherStruct, int MyInt, float MyFloat, float* MyFloatArray, float* MyFloatArray2, float* MyFloatArray3, float* MyFloatArray4)
{
myBool = MyBool;
sOtherStruct = myOtherStruct;
myInt = MyInt;
myFloat = MyFloat;
myFloatArray = MyFloatArray;
myFloatArray2 = MyFloatArray2;
myFloatArray3 = MyFloatArray3;
myFloatArray4 = MyFloatArray4;
}
With this code I get the error "You cannot use fixed size buffers contained in unfixed expressions. Try using the fixed statement"
At this point I attempted to use the copy routine instead.
What I want is to ensure the fields myFloatArray, myFloatArray2 etc. to be populated with whatever the initialise function returns. FYI myBool, sOtherStruct etc. are populated as I expect.

If you don't need to access the data you can leave it as a pointer. Although it looks like you're responsible for the memory, so you'll need to alloc and later free the unmanaged memory you are using. Something like...
[DllImport("libMylib")]
private static extern uint initialise(IntPtr a_pMyStruct);
[DllImport("libMylib")]
private static extern uint anotherNativeMethod(IntPtr a_pMyStruct);
//...
//How big is myOtherStruct??
int size = 1 + ?? + 4 + 4 + (1024*4) + (1024*4) + (20*4) + (30*4);
//int size = Marhsal.SizeOf(SMyStruct);
IntPtr pMyStruct = Marshal.AllocHGlobal(size);
initialise(pMyStruct);
anotherNativeMethod(pMyStruct);
Marshal.FreeHGlobal(pMyStruct);
Note that you can still use the get the Marshaller to copy the pointer to your structure using Marshal.PtrToStructure but you no longer need to depend on it for your code.

I suspect that many of your problems are caused by you attempting to run before you can walk. You have attempted to make a complex struct with many members. Make a single mistake in one place, and nothing works anywhere.
So, how can we simplify? Well, the question you ask relates to the fixed buffers. To paraphrase the question you are asking:
How can I copy an array to and from a fixed buffer?
Let's deal with that by working with a simplified type that only contains a fixed buffer, and prove that we can copy to and from that buffer.
Your property getter is along the right lines. The biggest problem is that you pass an incorrect length. The final argument of the Marshal.Copy overload that you are calling is the number of elements. You erroneously pass the number of bytes. The setter is very similar in nature to the getter. It all looks like this:
using System;
using System.Text;
using System.Runtime.InteropServices;
namespace ConsoleApplication1
{
public unsafe struct MyStruct
{
private const int floatArrayLength = 4;
private fixed float _floatArray[floatArrayLength];
public float[] floatArray
{
get
{
float[] result = new float[floatArrayLength];
fixed (float* ptr = _floatArray)
{
Marshal.Copy((IntPtr)ptr, result, 0, floatArrayLength);
}
return result;
}
set
{
int length = Math.Min(floatArrayLength, value.Length);
fixed (float* ptr = _floatArray)
{
Marshal.Copy(value, 0, (IntPtr)ptr, length);
for (int i = length; i < floatArrayLength; i++)
ptr[i] = 0;
}
}
}
}
class Program
{
static void WriteArray(float[] arr)
{
foreach (float value in arr)
{
Console.Write(value);
Console.Write(" ");
}
Console.WriteLine();
}
static void Main(string[] args)
{
MyStruct myStruct = new MyStruct();
WriteArray(myStruct.floatArray);
myStruct.floatArray = new float[] { 1, 2, 3, 4 };
WriteArray(myStruct.floatArray);
myStruct.floatArray = new float[] { 5, 6 };
WriteArray(myStruct.floatArray);
Console.ReadLine();
}
}
}
The output of the program is:
0 0 0 0
1 2 3 4
5 6 0 0
This building block shows you how to handle your fixed buffers. You can use the code from this program and be sure that the fixed buffers are handled correctly. When you move to a more complex structure, if you have any problems you can be confident that they are not related to the fixed buffer code.

Initialise an unsafe fixed double array in struct

This question is a follow up from Marshalling C# structure to C++ Using StructureToPtr.
I have the following struct:
[StructLayout(LayoutKind.Explicit, Size = 120, CharSet = CharSet.Unicode)]
public unsafe struct DynamicState
{
[FieldOffset(0)]
public fixed double Position[3];
[FieldOffset(24)]
public fixed double Velocity[3];
[FieldOffset(48)]
public fixed double Acceleration[3];
[FieldOffset(72)]
public fixed double Attitude[3];
[FieldOffset(96)]
public fixed double AngularVelocity[3];
}
If I try and initialise an array like this:
var dynamicState = new DynamicState();
double[] array = new double[] { 1, 2, 3 };
fixed (double* pArray = array)
{
dynamicState.Acceleration = pArray;
}
I get the following error:The left-hand side of an assignment must be a variable, property or indexer.
What is the correct way to initialise an unsafe array that is part of a struct?

Well the simple approach seems to work:
for (int i = 0; i < 3; i++)
{
dynamicState.AngularVelocity[i] = array[i];
}
It may not be as far as you're looking for though. Is this a performance-critical piece of code?
This may be better:
Marshal.Copy(array, 0, new IntPtr(dynamicState.AngularVelocity), array.Length);
I can't say I've much experience with unmanaged code, but it's worth at least looking at those options...

reinterpret_cast in C#

I'm looking for a way to reinterpret an array of type byte[] as a different type, say short[]. In C++ this would be achieved by a simple cast but in C# I haven't found a way to achieve this without resorting to duplicating the entire buffer.
Any ideas?

You can achieve this but this is a relatively bad idea. Raw memory access like this is not type-safe and can only be done under a full trust security environment. You should never do this in a properly designed managed application. If your data is masquerading under two different forms, perhaps you actually have two separate data sets?
In any case, here is a quick and simple code snippet to accomplish what you asked:
byte[] bytes = new byte[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
int byteCount = bytes.Length;
unsafe
{
// By using the fixed keyword, we fix the array in a static memory location.
// Otherwise, the garbage collector might move it while we are still using it!
fixed (byte* bytePointer = bytes)
{
short* shortPointer = (short*)bytePointer;
for (int index = 0; index < byteCount / 2; index++)
{
Console.WriteLine("Short {0}: {1}", index, shortPointer[index]);
}
}
}

There are four good answers to this question. Each has different downsides. Of course, beware of endianness and realize that all of these answers are holes in the type system, just not particularly treacherous holes. In short, don't do this a lot, and only when you really need to.
Sander's answer. Use unsafe code to reinterpret pointers. This is the fastest solution, but it uses unsafe code. Not always an option.
Leonidas' answer. Use StructLayout and FieldOffset(0) to turn a struct into a union. The downsides to this are that some (rare) environments don't support StructLayout (eg Flash builds in Unity3D) and that StructLayout cannot be used with generics.
ljs' answer. Use BitConverter methods. This has the disadvantage that most of the methods allocate memory, which isn't great in low-level code. Also, there isn't a full suite of these methods, so you can't really use it generically.
Buffer.BlockCopy two arrays of different types. The only downside is that you need two buffers, which is perfect when converting arrays, but a pain when casting a single value. Just beware that length is specified in bytes, not elements. Buffer.ByteLength helps. Also, it only works on primitives, like ints, floats and bools, not structs or enums.
But you can do some neat stuff with it.
public static class Cast {
private static class ThreadLocalType<T> {
[ThreadStatic]
private static T[] buffer;
public static T[] Buffer
{
get
{
if (buffer == null) {
buffer = new T[1];
}
return buffer;
}
}
}
public static TTarget Reinterpret<TTarget, TSource>(TSource source)
{
TSource[] sourceBuffer = ThreadLocalType<TSource>.Buffer;
TTarget[] targetBuffer = ThreadLocalType<TTarget>.Buffer;
int sourceSize = Buffer.ByteLength(sourceBuffer);
int destSize = Buffer.ByteLength(targetBuffer);
if (sourceSize != destSize) {
throw new ArgumentException("Cannot convert " + typeof(TSource).FullName + " to " + typeof(TTarget).FullName + ". Data types are of different sizes.");
}
sourceBuffer[0] = source;
Buffer.BlockCopy(sourceBuffer, 0, targetBuffer, 0, sourceSize);
return targetBuffer[0];
}
}
class Program {
static void Main(string[] args)
{
Console.WriteLine("Float: " + Cast.Reinterpret<int, float>(100));
Console.ReadKey();
}
}

c# supports this so long as you are willing to use unsafe code but only on structs.
for example : (The framework provides this for you but you could extend this to int <-> uint conversion
public unsafe long DoubleToLongBits(double d)
{
return *((long*) (void*) &d);
}
Since the arrays are reference types and hold their own metadata about their type you cannot reinterpret them without overwriting the metadata header on the instance as well (an operation likely to fail).
You can howveer take a foo* from a foo[] and cast that to a bar* (via the technique above) and use that to iterate over the array. Doing this will require you pin the original array for the lifetime of the reinterpreted pointer's use.

You could wrap your shorts/bytes into a structure which allows you to access both values:
See also here: C++ union in C#
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
namespace TestShortUnion {
[StructLayout(LayoutKind.Explicit)]
public struct shortbyte {
public static implicit operator shortbyte(int input) {
if (input > short.MaxValue)
throw new ArgumentOutOfRangeException("input", "shortbyte only accepts values in the short-range");
return new shortbyte((short)input);
}
public shortbyte(byte input) {
shortval = 0;
byteval = input;
}
public shortbyte(short input) {
byteval = 0;
shortval = input;
}
[FieldOffset(0)]
public byte byteval;
[FieldOffset(0)]
public short shortval;
}
class Program {
static void Main(string[] args) {
shortbyte[] testarray = new shortbyte[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1111 };
foreach (shortbyte singleval in testarray) {
Console.WriteLine("Byte {0}: Short {1}", singleval.byteval, singleval.shortval);
}
System.Console.ReadLine();
}
}
}

You can use System.Memory to do this in a safe way.
public static TTo[] Cast<TFrom, TTo>(this TFrom[] source) where TTo : struct where TFrom : struct =>
MemoryMarshal.Cast<TFrom, TTo>(source).ToArray();
private byte[] CastToBytes(int[] foo) => foo.Cast<int, byte>(foo);

This kind of behaviour would result in C# being rather type-unsafe. You can easily achieve this in a type-safe manner, however (though of course you are copying the array in doing so).
If you want one byte to map to one short then it's simple using ConvertAll, e.g.:-
short[] shorts = Array.ConvertAll(bytes, b => (short)b);
If you want to simply map every 2 bytes to a short then the following should work:-
if (bytes.Length % 2 != 0)
{
throw new ArgumentException("Byte array must have even rank.");
}
short[] shorts = new short[bytes.Length / 2];
for (int i = 0; i < bytes.Length / 2; ++i)
{
shorts[i] = BitConverter.ToInt16(bytes, 2*i);
}
It may be possible to use the marshaller to do some weird bit-twiddling to achieve this, probably using an unsafe { ... } code block, though this would be prone to errors and make your code unverifiable.
I suspect what you're trying to do can be achieved better using a type-safe idiom rather than a type-unsafe C/C++ one!
Update: Updated to take into account comment.

Casting like this is fundamentally unsafe and not permitted in a managed language. That's also why C# doesn't support unions. Yes, the workaround is to use the Marshal class.

Wouldn't it be possible to create a collection class that implements an interface for both bytes and shorts? Maybe implement both IList< byte > and IList< short >? Then you can have your underlying collection contain bytes, but implement IList< short > functions that work on byte pairs.

I used the code from FastArraySerializer to create a type converter to get from SByte[] to Double[]
public unsafe class ConvertArrayType
{
[StructLayout(LayoutKind.Explicit)]
private struct Union
{
[FieldOffset(0)] public sbyte[] sbytes;
[FieldOffset(0)] public double[] doubles;
}
private Union _union;
public double[] doubles {
get { return _union.doubles; }
}
public sbyte[] sbytes
{
get { return _union.sbytes; }
}
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private struct ArrayHeader
{
public UIntPtr type;
public UIntPtr length;
}
private readonly UIntPtr SBYTE_ARRAY_TYPE;
private readonly UIntPtr DOUBLE_ARRAY_TYPE;
public ConvertArrayType(Array ary, Type newType)
{
fixed (void* pBytes = new sbyte[1])
fixed (void* pDoubles = new double[1])
{
SBYTE_ARRAY_TYPE = getHeader(pBytes)->type;
DOUBLE_ARRAY_TYPE = getHeader(pDoubles)->type;
}
Type typAry = ary.GetType();
if (typAry == newType)
throw new Exception("No Type change specified");
if (!(typAry == typeof(SByte[]) || typAry == typeof(double[])))
throw new Exception("Type Not supported");
if (newType == typeof(Double[]))
{
ConvertToArrayDbl((SByte[])ary);
}
else if (newType == typeof(SByte[]))
{
ConvertToArraySByte((Double[])ary);
}
else
{
throw new Exception("Type Not supported");
}
}
private void ConvertToArraySByte(double[] ary)
{
_union = new Union { doubles = ary };
toArySByte(_union.doubles);
}
private void ConvertToArrayDbl(sbyte[] ary)
{
_union = new Union { sbytes = ary };
toAryDouble(_union.sbytes);
}
private ArrayHeader* getHeader(void* pBytes)
{
return (ArrayHeader*)pBytes - 1;
}
private void toAryDouble(sbyte[] ary)
{
fixed (void* pArray = ary)
{
var pHeader = getHeader(pArray);
pHeader->type = DOUBLE_ARRAY_TYPE;
pHeader->length = (UIntPtr)(ary.Length / sizeof(double));
}
}
private void toArySByte(double[] ary)
{
fixed (void* pArray = ary)
{
var pHeader = getHeader(pArray);
pHeader->type = SBYTE_ARRAY_TYPE;
pHeader->length = (UIntPtr)(ary.Length * sizeof(double));
}
}
} // ConvertArrayType{}
Here's the VB usage:
Dim adDataYgch As Double() = Nothing
Try
Dim nGch As GCHandle = GetGch(myTag)
If GCHandle.ToIntPtr(nGch) <> IntPtr.Zero AndAlso nGch.IsAllocated Then
Dim asb As SByte()
asb = CType(nGch.Target, SByte())
Dim cvt As New ConvertArrayType(asb, GetType(Double()))
adDataYgch = cvt.doubles
End If
Catch ex As Exception
Debug.WriteLine(ex.ToString)
End Try