Develop Reference

Passing A Byte[] Segment From C# To C++ DLL Without Creating a Copy

I need to slice a byte[] and pass the sliced section to a C# DLL for processing. I am avoiding Array.Copy, because I am trying not to copy anything to hinder performance. I have been made aware of the ArraySegment class as well as Span and Memory. The confusion I am having is how to actually pass these to the DLL, as I am passing an UnmanagedType.LPArray like so:
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate int ProcessVideoFrame_t([MarshalAs(UnmanagedType.LPArray)] byte[] bytes, uint len);
Is there any way to get an underlying byte[] from these classes, or somehow pass the segment without making a copy?
My current code is:
byte[] bytes = new byte[packet.Payload.Length - headerSize];
Array.Copy(packet.Payload, headerSize, bytes, 0, bytes.Length);
helper.ProcessVideoFrame(CookieHelperDictionary[packet.Header.Cookie], bytes, (uint)bytes.Length);```

unsafe
{
fixed (byte* p = &packet.Payload[headerSize])
{
helper.ProcessVideoFrame(CookieHelperDictionary[packet.Header.Cookie], (IntPtr)p, (uint)(packet.Payload.Length - headerSize));
}
}
I also changed the function definition to this:
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
public delegate int ProcessVideoFrame_t(IntPtr rdh, IntPtr buffer, uint len);

How to copy data pointed by byte* to a buffer pointed by IntPtr? (C#)

In C#. I'm working with images obtained from unsafe context. I have an integer with the image size and image pixels pointed by a byte* variable. I would like to copy those pixels into a buffer pointed by an IntPtr. How can I do that?
byte* imgData; // image data
uint uiDataSize; // image size
...
IntPtr ptr;
buffer.GetPointer(out ptr);

I don't believe there is a framework method that will work. Marshal.Copy can copy memory into and out of an IntPtr buffer, but it doesn't work with pointers.
Instead you can P/Invoke the native MoveMemory function which copies memory between two pointers.
[DllImport("Kernel32.dll", EntryPoint="RtlMoveMemory", SetLastError=false)]
static extern void MoveMemory(IntPtr dest, IntPtr src, UIntPtr size);
...
byte* imgData; // image data
uint uiDataSize; // image size
...
IntPtr ptr;
buffer.GetPointer(out ptr);
MoveMemory(ptr, (IntPtr)imgData, (UIntPtr)uiDataSize);
Yes the size parameter of MoveMemory is UIntPtr not an int because the SIZE_T used in the native code is 32 bits on 32 bit systems and 64 bits on 64 bit systems.

Use System.Runtime.Interopservice.Marshal.Copy, if I remembered correctly. Check the function and you will know how to use it.

Use System.Runtime.Interopservice.Marshal.Copy
using System.Runtime.InteropServices;
byte* imgData; // image data
uint uiDataSize; // image size
...
IntPtr ptr;
buffer.GetPointer(out ptr);
// Copy the unmanaged array to a managed array.
byte[] managedArray = new byte[lengthOfData];
Marshal.Copy(imgData, managedArray, 0, lengthOfData);

Get an image of process memory

My goal is to create a method that will take a process handle and return an array of bytes representing that process's memory. Here's what I have:
[DllImport("Kernel32.dll")]
public static extern bool ReadProcessMemory(IntPtr hProcess, IntPtr lpBaseAddress, byte[] lpBuffer, UInt32 nSize, ref UInt32 lpNumberOfBytesRead);
public static byte[] MemRead(IntPtr handle, IntPtr address, UInt32 size, ref UInt32 bytes)
{
byte[] buffer = new byte[size];
ReadProcessMemory(handle, address, buffer, size, ref bytes);
return buffer;
}
I don't know what to pass to the wrapper method as arguments. I can find a handle and the bytes is an output variable, but what about address and size? Where can I get this data from?

Use VirtualQuery to find out if an address has actually been allocated before calling MemRead.
Start with zero as the address and 64K as the page size and then simply increment the pointer with 64K on every iteration until you reach the maximum size of memory on your system.

Directly reading large binary file in C# w/out copying

I am looking for the most efficient/direct way to do this simple C/C++ operation:
void ReadData(FILE *f, uint16 *buf, int startsamp, int nsamps)
{
fseek(f, startsamp*sizeof(uint16), SEEK_SET);
fread(buf, sizeof(uint16), nsamps, f);
}
in C#/.NET. (I'm ignoring return values for clarity - production code would check them.) Specifically, I need to read in many (potentially 10's to 100's of millions) 2-byte (16-bit) "ushort" integer data samples (fixed format, no parsing required) stored in binary in a disk file. The nice thing about the C way is that it reads the samples directly into the "uint16 *" buffer with no CPU involvement, and no copying. Yes, it is potentially "unsafe", as it uses void * pointers to buffers of unknown size, but it seems like there should be a "safe" .NET alternative.
What is the best way to accomplish this in C#? I have looked around, and come across a few hints ("unions" using FieldOffset, "unsafe" code using pointers, Marshalling), but none seem to quite work for this situation, w/out using some sort of copying/conversion. I'd like to avoid BinaryReader.ReadUInt16(), since that is very slow and CPU intensive. On my machine there is about a 25x difference in speed between a for() loop with ReadUInt16(), and reading the bytes directly into a byte[] array with a single Read(). That ratio could be even higher with non-blocking I/O (overlapping "useful" processing while waiting for the disk I/O).
Ideally, I would want to simply "disguise" a ushort[] array as a byte[] array so I could fill it directly with Read(), or somehow have Read() fill the ushort[] array directly:
// DOES NOT WORK!!
public void GetData(FileStream f, ushort [] buf, int startsamp, int nsamps)
{
f.Position = startsamp*sizeof(ushort);
f.Read(buf, 0, nsamps);
}
But there is no Read() method that takes a ushort[] array, only a byte[] array.
Can this be done directly in C#, or do I need to use unmanaged code, or a third-party library, or must I resort to CPU-intensive sample-by-sample conversion? Although "safe" is preferred, I am fine with using "unsafe" code, or some trick with Marshal, I just have not figured it out yet.
Thanks for any guidance!
[UPDATE]
I wanted to add some code as suggested by dtb, as there seem to be precious few examples of ReadArray around. This is a very simple one, w/no error checking shown.
public void ReadMap(string fname, short [] data, int startsamp, int nsamps)
{
var mmf = MemoryMappedFile.CreateFromFile(fname);
var mmacc = mmf.CreateViewAccessor();
mmacc.ReadArray(startsamp*sizeof(short), data, 0, nsamps);
}
Data is safely dumped into your passed array. You can also specify a type for more complex types. It seems able to infer simple types on its own, but with the type specifier, it would look like this:
mmacc.ReadArray<short>(startsamp*sizeof(short), data, 0, nsamps);
[UPATE2]
I wanted to add the code as suggested by Ben's winning answer, in "bare bones" form, similar to above, for comparison. This code was compiled and tested, and works, and is FAST. I used the SafeFileHandle type directly in the DllImport (instead of the more usual IntPtr) to simplify things.
[DllImport("kernel32.dll", SetLastError=true)]
[return:MarshalAs(UnmanagedType.Bool)]
static extern bool ReadFile(SafeFileHandle handle, IntPtr buffer, uint numBytesToRead, out uint numBytesRead, IntPtr overlapped);
[DllImport("kernel32.dll", SetLastError=true)]
[return:MarshalAs(UnmanagedType.Bool)]
static extern bool SetFilePointerEx(SafeFileHandle hFile, long liDistanceToMove, out long lpNewFilePointer, uint dwMoveMethod);
unsafe void ReadPINV(FileStream f, short[] buffer, int startsamp, int nsamps)
{
long unused; uint BytesRead;
SafeFileHandle nativeHandle = f.SafeFileHandle; // clears Position property
SetFilePointerEx(nativeHandle, startsamp*sizeof(short), out unused, 0);
fixed(short* pFirst = &buffer[0])
ReadFile(nativeHandle, (IntPtr)pFirst, (uint)nsamps*sizeof(short), out BytesRead, IntPtr.Zero);
}

You can use a MemoryMappedFile. After you have memory-mapped the file, you can create a view (i.e. a MemoryMappedViewAccessor) which provides a ReadArray<T> method. This method can read structs from the file without marshalling, and it works with primitive types lie ushort.

dtb's answer is an even better way (actually, it has to copy the data as well, no gain there), but I just wanted to point out that to extract ushort values from a byte array you should be using BitConverter not BinaryReader
EDIT: example code for p/invoking ReadFile:
[DllImport("kernel32.dll", SetLastError=true)]
[return:MarshalAs(UnmanagedType.Bool)]
static extern bool ReadFile(IntPtr handle, IntPtr buffer, uint numBytesToRead, out uint numBytesRead, IntPtr overlapped);
[DllImport("kernel32.dll", SetLastError=true)]
[return:MarshalAs(UnmanagedType.Bool)]
static extern bool SetFilePointerEx(IntPtr hFile, long liDistanceToMove, out long lpNewFilePointer, uint dwMoveMethod);
unsafe bool read(FileStream fs, ushort[] buffer, int offset, int count)
{
if (null == fs) throw new ArgumentNullException();
if (null == buffer) throw new ArgumentNullException();
if (offset < 0 || count < 0 || offset + count > buffer.Length) throw new ArgumentException();
uint bytesToRead = 2 * count;
if (bytesToRead < count) throw new ArgumentException(); // detect integer overflow
long offset = fs.Position;
SafeFileHandle nativeHandle = fs.SafeFileHandle; // clears Position property
try {
long unused;
if (!SetFilePositionEx(nativeHandle, offset, out unused, 0);
fixed (ushort* pFirst = &buffer[offset])
if (!ReadFile(nativeHandle, new IntPtr(pFirst), bytesToRead, out bytesToRead, IntPtr.Zero)
return false;
if (bytesToRead < 2 * count)
return false;
offset += bytesToRead;
return true;
}
finally {
fs.Position = offset; // restore Position property
}
}

I might be a bit late to the game here... but the fastest method I found was using a combination of the previous answers.
If i do the following:
MemoryMappedFile mmf = MemoryMappedFile.CreateFromFile(somePath);
Stream io = mmf.CreateViewStream();
int count;
byte[] byteBuffer = new byte[1024 << 2];
ushort[] dataBuffer = new ushort[buffer.Length >> 1];
while((count = io.Read(byteBuffer, 0, byteBuffer.Length)) > 0)
Buffer.BlockCopy(buffer, 0, dataBuffer, 0, count);
This was ~2x faster than the accepted answer.
For me, the unsafe method was the same as the Buffer.BlockCopy without the MemoryMappedFile. The MemoryMappedFile cut down on a bit of time.

low level C++ style i/o in C# for reading FAT32

I am working on reading the FAT32 entry of the hard disk and so far have been successful in reading the entries by making use of the CreateFile, ReadFile, and SetFilePointer APIs. Here is my code (written in C#) so far.
---The DLL IMPORTS-----
[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr CreateFile(string lpFileName, Int32 dwDesiredAccess,
Int32 dwShareMode, Int32 lpSecurityAttributes, Int32 dwCreationDisposition,
Int32 dwFlagsAndAttributes, IntPtr hTemplateFile);
[DllImport("kernel32.dll")]
static extern bool ReadFile(IntPtr hFile, byte[] lpBuffer,
uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, uint lpOverlapped);
[DllImport("kernel32.dll")]
extern static int SetFilePointer(IntPtr hFile, int lDistanceToMove, int lpDistanceToMoveHigh, uint dwMoveMethod);
[DllImport("kernel32.dll")]
extern static Boolean CloseHandle(IntPtr hObject);
------CODE----Will Work in any .NET Application---------
int ret, nread;
IntPtr handle;
int s = 512;
byte[] readbuffer = new byte[512];
IntPtr ptr = CreateFile(#"\\.\F:", -1073741824, 3, 0, 3, 128, IntPtr.Zero);
if (ptr != System.IntPtr.Zero)
{
int i = 100;
int ret = SetFilePointer(ptr, 0, 0, 0);
ret = SetFilePointer(ptr, 4194304, 0, 1);
while (true)
{
byte[] inp = new byte[512];
uint read = 0;
if (ret != -1)
{
ReadFile(ptr, inp, 512, out read, 0);
for (int k = 0; k < 16; k++)
{
string s = ASCIIEncoding.ASCII.GetString(inp, k*32, 11);
if (inp[k*32] == 0xE5)
{
MessageBox.Show(s);
}
}
//ret = SetFilePointer(ptr, 512, 0, 1);
}
}
}
The code above reads the F:\ drive and for trial purposes I have made it to read the first File Directory Cluster and query through each file entry and display the file name if it has been deleted.
However I want to make it to a full-blown application, for which I will have to frequently use the byte array and map it to the specified data structures according to the FAT32 Specification.
How can I efficiently use the byte array into which I am reading the data? I have tried the same code using filestream and binaryreader and it works, however now suppose I have a C Structure something like
struct bios_data
{
byte id[3];
char name[11];
byte sectorpercluster[2];
...
}
I want to have a similar data structure in C# and when I read data to a byte array I want to map it to the structure. I tried many options but didn't get a complete solution. I tried making a class and do serialization too but that also didn't work. I have around 3 more structures like theese which I will be using as I read the data from the FAT entry. How can I best achieve the desired results?

If you want to read binary data directly into structs, C-style, this article may interest you. He wrote an unmanaged wrapper around the C stdio functions and interoperates with it. I have tried it - it does work quite well. It is nice to read directly into a struct in C#, and it is fast. You can just do:
unsafe
{
fmp3.Read<MyStruct>(&myStructVar);
}

I gave an answer on how to convert between byte arrays and structs in this question.