IMFMediaBuffer from PhotoCaptureFrame - c#

I have a computer vision plugin using Hololens locatable camera on Unity 5.2. I use the class UnityEngine.XR.WSA.WebCam.PhotoCaptureFrame with photo mode enabled at highest resolution, which is 2048x1156, in NV12 color format which is camera native format.
PhotoCaptureFrame class : https://docs.unity3d.com/2017.2/Documentation/ScriptReference/XR.WSA.WebCam.PhotoCaptureFrame.html
At the moment, I do this for my plugin to process any single photo:
PhotoCaptureFrame photo; // created correctly through Unity API
List<byte> photoBuffer = new List<byte>();
photo.CopyRawImageDataIntoBuffer(photoBuffer);
// marshal photoBuffer to plugin (and it works fine for the computer vision part)
However, it copies the raw buffer each time. I want to use the raw buffer directly from my plugin. So I tried this:
IntPtr rawBufferPtr = photo.GetUnsafePointerToBuffer(); // COM pointer to IMFMediaBuffer is incremented
// send rawBufferPtr to plugin (native function described below)
Marshal.Release(rawBufferPtr);
IMFMediaBuffer interface : https://msdn.microsoft.com/en-us/library/windows/desktop/ms696261(v=vs.85).aspx
And in my C++ plugin:
#include <Mfidl.h> // for IMFMediaBuffer support
void process_photo(void *photo_buffer_wrapper, int photo_width, int photo_height)
{
// photo_buffer_wrapper = rawBufferPtr in managed environment
IMFMediaBuffer *media_buffer = reinterpret_cast<IMFMediaBuffer *>(photo_buffer_wrapper);
BYTE *photo_buffer = NULL;
HRESULT result = media_buffer->Lock(&photo_buffer, NULL, NULL);
if (SUCCEEDED(result))
{
// do image processing stuff here (with OpenCV) using photo_buffer
media_buffer->Unlock;
}
}
It appears fine to me. It does compile fine too. But at run time, I get an access violation and the applications crashes on Hololens.
Exception Code: 0xC0000005
Exception Information: The thread tried to read from or write to a virtual address for which it does not have the appropriate access.
Anyone sees the problem? Something to do with the way I pass the IMFMediaBuffer object from managed to unmanaged environment?
Thank you very much!

I will answer my own question.
The photo_buffer_wrapper is not a pointer to IMFMediaBuffer as I thought but a pointer to IUnknown. Here is the modified native function that works as intended:
// UNMANAGED ENVIRONMENT
#include <Mfidl.h> // for IMFMediaBuffer support + other COM stuff
void process_photo(void *photo_buffer_unknown, int photo_width, int photo_height)
{
// photo_buffer_unknown = photoCaptureFrame.GetUnsafePointerToBuffer() in managed environment which is an IntPtr
IMFMediaBuffer *media_buffer;
if (SUCCEEDED(reinterpret_cast<IUnknown *>(photo_buffer_unknown)->QueryInterface<IMFMediaBuffer>(&media_buffer)))
{
BYTE* photo_buffer = NULL;
if (SUCCEEDED(media_buffer->Lock(&photo_buffer, NULL, NULL)))
{
// process photo_buffer with OpenCV (wrapped in a cv::Mat)
media_buffer->Unlock();
media_buffer->Release(); // QueryInterface on IUnknown has incremented reference count by one
}
}
}
NB: The pointer returned from photoCaptureFrame.GetUnsafePointerToBuffer() still has to be released in managed environment like in my question:
// MANAGED ENVIRONMENT
IntPtr mediaBufferUnknownPtr = photoCaptureFrame.GetUnsafePointerToBuffer();
// send mediaBufferUnknownPtr to native function through extern DLL call
Marshal.Release(mediaBufferUnknownPtr)

Related

System.AccessViolationException: shared_ptr between C# .NET and C++ applications

In our project we communicate two applications, one in C# and the other in C++, via named pipes. Our intention is to pass memory pointers between them and be able to access the objects pointed by them in either application. Our current code rises a System.AccessViolationException:
System.AccessViolationException:
Attempted to read or write protected memory. This is often an
indication that other memory is corrupt.
So far we are using a shared_ptr that points to a custom struct and writting the pointer to the buffer in C++ as seen below:
typedef struct {
int one;
int a;
int two;
int b;
} DATA_STRUCT; // C++ struct
DATA_STRUCT ds;
ds.one = 10;
ds.a = 5;
ds.two = 99;
ds.b = 0;
shared_ptr<DATA_STRUCT> ptr_ds(new DATA_STRUCT);
shared_ptr<DATA_STRUCT> p(ptr_ds);
*ptr_ds = ds;
const int size = BUFFER_SIZE;
char buf[size];
memset(buf, 0xCC, 100);
while (keepReading)
{
printf("Write message:");
scanf("%s", buf);
memcpy(buf, &p, sizeof(shared_ptr<DATA_STRUCT>));
if (strcmp(buf, "quit") == 0)
keepReading = false;
else
{
WriteFile(hPipe1, buf, dwBytesToWrite, &cbWritten, NULL);
memset(buf, 0xCC, 100);
}
}
Then, in C# we read the whole buffer, we keep the bytes with the relevant information in another buffer (Rc) and convert the byte array to our custom data structure using unsafe IntPtr as you can see below:
buffer = new byte[BUFFER_SIZE];
bytesRead = clientCSharp.stream.Read(buffer, 0, BUFFER_SIZE);
public struct DATA_STRUCT
{
public int one;
public int a;
public int two;
public int b;
}; // C# struct
unsafe
{
Buffer.BlockCopy(buffer, 0, Rc, 0, ReadLength);
DATA_STRUCT ds = new DATA_STRUCT();
IntPtr aux_ptr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(IntPtr)));
IntPtr final_ptr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(IntPtr)));
Marshal.Copy(Rc, 0, aux_ptr, 4);
final_ptr = (IntPtr)Marshal.PtrToStructure(aux_ptr, typeof(IntPtr));
ds = (DATA_STRUCT)Marshal.PtrToStructure(final_ptr, typeof(IntPtr));
}
The exception rises when we try to access the final_ptr in order to load the DATA_STRUCT, the last code line presented above. Here I give some Debug images:
C++ Debug image with Pointer value written to the named pipe buffer
C# Debug image with Pointer value read from the named pipe reduced buffer (Rc)
Could it be something related to the pointer length? As it seems to me in the C++ application it has 8bytes and in the C# application it has 16bytes? Dhould we declare a safe memory location for C# and C++? If yes, then how could it be done?
Note: Our goal is to work with the unsafe IntPtr in the C# application. In this example we are loading the DATA_STRUCT object because we wanted to be sure that in the C# application we are retrieving the same object passed in the C++ application. The final applications is meant to be used in Windows.
Application data spaces are completely distinct, and have been for many years. you can't simply pass a raw pointer between applications and expect to access the same memory.
The normal approach is to serialise the content of your object, squirt it through your pipe and then reconstruct the object at the receiver.
You can set up named shared memory regions, and these are faster to share large objects (in unix, and I assume in windows), but these shared regions will probably not be located at the same address, so still only good for raw data.

Marshaling a pointer to a pointer

Problem:
I have a test application that is looking for a scanner attached to a PC. The European manufacturer supplies an SDK written in C++ and provides some header files and other information. Their C++ API populates a structure as shown below. My concern is de-referencing (Marshaling) the returned char** inside of that structure into a set of strings.
//
//The struct, as per the manufacturer's C++ header file:
//
typedef struct {
int count;
char** serialNumbers;
} ScannerList;
//
// My interpretation of the struct
//
struct ScannerList
{
public int count;
public IntPtr serialNumbers;
}
//
// C++ API Method Signature:
//
bool SCN_GetScannerList( ScannerList** scannerList );
//
// My DLLImport:
//
[DllImport(#"C:\ScannerSDK\Scanner.dll", CharSet = CharSet.Unicode)]
public static extern bool SCN_GetScannerList(out IntPtr scannerList);
Inside my C# wrapper class, I have several classes that wrap each specific set of Scanner API calls. For this method, I'm using the ScannerNative class that the DllImport resides in, along with the struct. This is the method charged with summoning the list:
private object GetScanlistFromDLL()
{
// our managed code return object - class instead of struct
ScannerNative.ScannerListSafe safeList = new ScannerNative.ScannerListSafe();
// IntPtr to give to the API
IntPtr ptr = new IntPtr();
// unmanaged code struct
ScannerNative.ScannerList scanList = new ScannerNative.ScannerList();
// Call the API with our IntPtr
bool success = ScannerNative.GetScannerList(out ptr);
if(success)
{
// Map the pointer to the type of structure from the API call
scanList = (ScannerNative.ScannerList)Marshal.PtrToStructure(ptr, typeof(ScannerNative.ScannerList));
// copy the unsafe struct members to our safe class
safeList.count = scanList.count;
//Problem - the marshaling returns a string full of garbage
//likely because char** is an array of char* ...
//Not sure how to Marshal a pointer to a pointer to a string :-\
safeList.serialNumbers = Marshal.PtrToStringUni(scanList.serialNumbers);
}
else
{
return null;
}
//
// API call to release unmanaged scanner list ..
//
return safeList;
}
Results:
I do get what appears to be a valid structure back from the API call
bool success = ScannerNative.GetScannerList(out ptr);
The subsequent casting in the if(success) block:
if(success)
{
// Map the pointer to the type of structure from the API call
scanList = (ScannerNative.ScannerList)Marshal.PtrToStructure(ptr, typeof(ScannerNative.ScannerList));
This works all the way until we get to our IntPtr stored in the scanList.serialNumbers member, which is holding a reference to the char** from the API struct.
scanList.count shows a correct value of '1' (when the scanner is attached) but I'm not able to de-ref the char** held in the scanList.serialNumbers member.
I've tried attacking this in a couple ways, but nothing is coming out and either I'm using the wrong search engines, or trying to find "c# DllImport Char** Marshal" and variants is asking the wrong question.

Access Violation while exporting an unmanaged function pointer

I have been trying for the past 4 hours to solve a very mysterious problem.
I am writing some plugin for Notepad++. To achieve syntax highlighting one has to export such a function:
//this function is exported via exports.def file
LexerFactoryFunction SCI_METHOD GetLexerFactory(unsigned int index)
{
return (index == 0) ? RTextLexer::LexerFactory : nullptr;
}
where,
LexerFactoryFunction is typedef ILexer *(*LexerFactoryFunction)();
#define SCI_METHOD __stdcall
I have managed to get this thing working perfectly with C++, however another part of the plugin is written in C#, so I tried to merge the two using Fody Costura NuGet package ( so that the CLI .dll is embedded into the main .dll ), however with no success.
What I've tried :
public ref class RTextLexerCliWrapper
{
public:
delegate ILexer * GetLexerFactoryDelegate();
IntPtr GetLexerFactory()
{
return System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate(_lexerFactoryPtr);
}
RTextLexerCliWrapper();
private:
GetLexerFactoryDelegate ^ _lexerFactoryPtr;
GCHandle gch;
~RTextLexerCliWrapper();
};
RTextLexerCliWrapper::RTextLexerCliWrapper()
{
_lexerFactoryPtr = gcnew GetLexerFactoryDelegate(&RTextLexer::LexerFactory);
gch = GCHandle::Alloc(_lexerFactoryPtr);
}
RTextLexerCliWrapper::~RTextLexerCliWrapper()
{
gch.Free();
}
This CLI wrapper, is referenced in my main .dll like this :
static RTextLexerCliWrapper _lexerWrapper = new RTextLexerCliWrapper();
[DllExport(CallingConvention = CallingConvention.Cdecl)]
static IntPtr GetLexerFactory(uint index)
{
return (index == 0) ? _lexerWrapper.GetLexerFactory() : IntPtr.Zero;
}
So what happens is, my .net function gets indeed called and the cli wrapper function is also called, and a function pointer is indeed returned. However any attempts to call that function pointer results in an access violation. Which means that either the type of the pointer is wrong or something else which I am currently missing. I have tried countless variations of the .net exported function with void *, StdCall etc. All result in the same problem.
Is there any other way to return a function pointer of a C++ class? Or well am I doing something completely wrong?
Thanks in advance!
So I have finally managed to found the solution to my problem.
First step was exporting the functions with the correct calling convention:
static RTextLexerCliWrapper _lexerWrapper = new RTextLexerCliWrapper();
[DllExport(CallingConvention = CallingConvention.StdCall)]
static IntPtr GetLexerFactory(uint index)
{
return (index == 0) ? _lexerWrapper.GetLexerFactory() : IntPtr.Zero;
}
The convention in this case had to be StdCall. Otherwise the stack pointer is invalidated, hence the exceptions.
Now in order to return a function pointer of a C++ instance things were a bit more tricky.
I am statically storing an instance of the CLI wrapper class so that it doesn't get GCed. ( _lexerWrapper ).
This instance has a function called GetLexerFactory which return a function pointer of the C++ instance ( which is then used by some other .dll to get actual instances of some object ).
The CLI Wrapper class looks like this:
public ref class RTextLexerCliWrapper
{
public:
delegate ILexer * GetLexerFactoryDelegate();
IntPtr GetLexerFactory()
{
return System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate(_lexerFactoryPtr);
}
RTextLexerCliWrapper();
private:
GetLexerFactoryDelegate ^ _lexerFactoryPtr;
GCHandle gch;
~RTextLexerCliWrapper();
};
Where ILexer * is the type of the object we shall be returning later on.
RTextLexerCliWrapper::RTextLexerCliWrapper()
{
_lexerFactoryPtr = gcnew GetLexerFactoryDelegate(&RTextLexer::LexerFactory);
gch = GCHandle::Alloc(_lexerFactoryPtr);
}
RTextLexerCliWrapper::~RTextLexerCliWrapper()
{
gch.Free();
}
So, what we have managed here is, exporting through .NET a function pointer which is able to return a pure C++ object.

Passing list of struct to C++ DLL from C# Application

I am trying to pass struct as a parameter from C# application to the C++ MFC DLL. DLL fills records in the struct object and return back to the C# application. So here I used "out" keyword in C# application to call C++ method. When I execute it, it fails with error "Attempted to read or write protected memory. This is often an indication that other memory is corrupt." I am allocating memory at C++ DLL to store the records and assign it to out parameter. I could send data in struct object and print it in C++ DLL but not able to do modification in struct object at DLL side and return back to C# application.
Could anybody please help on this.
NativeDLLHelper.cs
class NativeDLLHelper
{
const int FIELD_LENGTH = 255;
[StructLayout(LayoutKind.Sequential)]
public struct APPDATA
{
public int ID;
public int Version;
[MarshalAs(UnmanagedType.LPTStr, SizeConst = FIELD_LENGTH)]
public string AppName;
};
[DllImport("Parser.dll", EntryPoint = "?FillLstStructData#DLLWrapper##QAEXAAPAU_APPDATA#1##Z")]
public static extern void FillLstStructData(out APPDATA[] AppDataStruct);
}
Main.cs
NativeDLLHelper.APPDATA[] lstFillAppDataStruct;
NativeDLLHelper.FillLstStructData(out lstFillAppDataStruct);
for (int i = 0; i < lstFillAppDataStruct.Length; i++)
{
Console.WriteLine(" ID[{0:G}]:{1:G}", i, lstFillAppDataStruct[i].ID);
Console.WriteLine(" Version[{0:G}]:{1:G}", i, lstFillAppDataStruct[i].Version);
Console.WriteLine(" AppName[{0:G}]:{1:G}", i, lstFillAppDataStruct[i].AppName);
}
C++ DLL:
static const int FIELD_LENGTH = 128;
typedef struct _APPDATA
{
int ID;
double Version;
TCHAR AppName[FIELD_LENGTH];
}APPDATA;
extern "C" __declspec(dllexport) FillLstStructData(APPDATA* &pAppData)
{
APPDATA *localAppDataStruct = new APPDATA[2];
localAppDataStruct[0].ID = 1;
localAppDataStruct[0].Version = 1.1;
_tcscpy(localAppDataStruct[0].AppName, L"MS Visual Studio 2010");
localAppDataStruct[1].ID = 2;
localAppDataStruct[1].Version = 2.1;
_tcscpy(localAppDataStruct[1].AppName, L"MS Office 2010");
pAppData = localAppDataStruct;
}
Your exported function unmangles to:
void __thiscall DLLWrapper::FillLstStructData(struct DLLWrapper::_APPDATA * &)
That makes it an instance method of a C++ class. You cannot pinvoke such methods, they require the C++ object to be created first and you cannot do this reliably with pinvoke. Only static C++ member functions can be pinvoked. Do note that this function doesn't need to be an instance method at all, it doesn't actually use instance members of the class.
That isn't the only problem, there's also a nasty memory management issue. The function allocates memory with ::operator new and memory needs to be released by the caller. But a C# program cannot do this, it doesn't have access to the ::operator delete that the C++ code uses. This kind of function is very hard to use reliably from a C++ program for that reason, it doesn't get better when you do it from C#. The proper way to do it is to allow the caller to pass the array. In other words, APPDATA[] and an extra argument that says how long the passed array is.
If you can't rewrite this function then you must write a wrapper for this class using the C++/CLI language. Very important that this wrapper uses the exact same compiler and CRT version as the C++ DLL.

Data Read method not working properly when called through C#

I have an unmanaged dll that contains a function to read a data from a file. I wrote a CLI wrapper for it.
To test whether it is working, I wrote a simple CLI program calling the wrapper class and the data read method. It worked fine.
I used that dll in a c# program but the data is not read properly. It reads but the data read is not proper.
CLI function:
void FileReader::ReadFile()
{
int ret;
STRUCT head;
STRUCT1 *sqar;
memset ( ( void * )&head, 0, sizeof ( STRUCT) );
sqar = ( STRUCT1 * )NULL;
ret = Read_func( "somefile.someformat", &head, &sqar );
}
CLI EXE:
int main(array<System::String ^> ^args)
{
FileReader reader;
reader.ReadFile();
Console::WriteLine(L"Hello World");
return 0;
}
This works fine. When I debug, the code flows through and reads the data properly.
C# code:
{
FileReader filereader= new FileReader ();
filereader.ReadFile();
}
When I import the CLI dll in the C# project and access the code as above the data in not read properly.
STRUCT has int as members and it is read properly.STRUCT1 has int, double and structure pointers as members which is not read properly. What could be possibly wrong?
The solution was to change the Struct Member Alignment property under C/C++ > Code Generation from 1byte 1 byte /Zp1 to Default. Simple.

Categories

Resources