Marshalling ByValTStr as UTF8 - c#

I am working with a C DLL and have trouble with marshalling strings using P/Invoke.
The DLL has a struct as follows:
typedef struct
{
char sAddress[256];
BYTE byUseRtsp;
WORD wPort;
}INFO,*LPINFO;
My C# struct looks like this:
[StructLayout(LayoutKind.Sequential)]
public struct INFO
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
public string sAddress;
public byte byUseRtsp;
public short wPort;
}
The string marshalling for sAddress works with ASCII text, but the DLL uses UTF-8 encoding throughout. So as soon as multi-byte characters are used the marshalling garbles the text. Using CharSet.Unicode doesn't work here as that tells the marshaller to encode/decode strings as UTF-16 on Windows. I need a CharSet.Utf8 which unfortunately doesn't exist.
I do have a workaround, but it's ugly and want to avoid if possible. The workaround is to replace:
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
public string sAddress;
with:
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 256)]
public byte[] sAddress;
and then just re-write my code to use Encoding.UTF8.GetBytes/String() methods to get the string values. I will also need to handle null-terminators myself with this method.
Is there a better way of doing this?

Is there a better way of doing this?
Using the built-in options for marshalling what you have done is probably as good as it gets. You would want to write some helper methods to manage this for you, but I'm sure you are alive to that.
Beyond this you could use a custom marshaler. I'm not aware that there is a great body of work available on this topic. One of the more complete happens to be this question of mine: How do I write a custom marshaler which allows data to flow from native to managed?

[In case it helps others] .NET Framework 4.7 introduced UnmanagedType.LPUTF8Str which is likely just for this situation.

Related

Do I need to call Marshal.DestroyStructure when marshalling structures containing ByValTStr strings

I am doing some manual marshaling for interop from C#/.NET to unmanaged DLLs.
Consider the following struct:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
private struct LockInfo
{
ushort lockVersion;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
public string lockName;
}
I marshal this to unmanaged memory:
var lockInfo = new LockInfo();
var lockInfoPtr = Marshal.AllocHGlobal(Marshal.SizeOf(lockInfo));
Marshal.StructureToPtr(lockInfo, lockInfoPtr, false);
Once I'm done with it, do I need to call Marshal.DestroyStructure on lockInfoPtr?
I am aware of the need to call Marshal.FreeHGlobal, but prior to that, is Marshal.DestroyStructure actually required in this case?
I have found it very difficult to understand Microsoft's documentation around this. Google searching hasn't helped, possibly because I just don't quite understand marshalling properly yet (but I am learning).
Similar questions...
I have reviewed the similar question "Marshal.DestroyStructure vs Marshal.FreeHGlobal in .Net" but this question does not address the issue of the content a struct should contain that would require the use of DestroyStructure. My limited understanding is that DestroyStructure does not always need to be called, only when the structure contains certain kinds of fields. In my case I am unsure if a string, being marshalled as ByValTStr, requires the use of DestroyStructure.
Marshaling is complex stuff and a full answer could fill a whole chapter in a book, which of course is not appropriate here. So, in a very succinct nutshell:
Normally, when calling native functions from managed code, .NET marshals strings and arrays to native BSTR strings and SafeArray arrays.
To accomplish this, the marshaler calls SysAllocString and SafeArrayCreate respectively.
At some point, when these native-side strings and arrays are no longer needed, the marshaler will call SysFreeString and SafeArrayDestroy respectively to free memory.
If you take over .NET's automatic marshaling, and call methods like Marshal.StructureToPtr to manually marshal a structure, you become responsible for freeing/destroying those native-side BSTRs and SafeArrays. That's exactly what Marshal.DestroyStructure is for.
However...
By prepending the [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)] attribute to your string field, you instructed the marshaler to not marshal the string to a BSTR string, but rather to allocate a fixed-length character array within the native-side structure itself.
That being the case, there is no need to call Marshal.DestroyStructure because there is no BSTR string to free. Of course you will still need to call Marshal.FreeHGlobal, I see you are aware of that.
Credit to #SimonMourier for his comment that made it all click.

Marshalling a complex structure from C# to C

I'm reading all about structure marshalling between C and C# in 64bit environment without success.
What I need to do is to pass the structure
typedef struct DescStructTa
{
char* pszNa;
unsigned long ulTId;
char* pszT;
unsigned short usRId;
unsigned long ulOff;
unsigned long ulSi;
char szAcc[2];
unsigned char bySSize;
} DescStruct;
from a C# application to a C DLL not made by us calling the method
MT_DLL_DECL long GetAll(UINTPTR ulHandler, DescStruct **ppList, unsigned long *pulNumS);
After long search I wrote the following code:
[StructLayout(LayoutKind.Sequential, Pack = 8)]
public struct PlcSymbolDescStruct
{
[MarshalAs(UnmanagedType.LPStr)]
string pszNa;
UInt32 ulTId;
[MarshalAs(UnmanagedType.LPStr)]
string pszT;
[MarshalAs(UnmanagedType.U2)]
ushort usRId;
UInt32 ulOff;
UInt32 ulSi;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 2)]
string szAcc;
[MarshalAs(UnmanagedType.U1)]
byte bySSize;
}
[DllImport("DataSource.dll", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
[return: MarshalAs(UnmanagedType.U8)]public static extern long GetAll(ulong ulHandler, out DescStruct [] ppList,
out uint pulNumS);
Unfortunately this is still giving me a heap corruption.
If I try to pack less than 8 it gives (as expected) an access violation, but I expected this to work and I'm not able to figure out what could be wrong.
Any suggestion?
Thanks in advance!
I’m not sure you need Pack = 8. The default packing appears to be more or less compatible between C# and C++.
The szAcc field probably uses Unicode. To switch, specify CharSet=CharSet.Ansi in your [StructLayout]
Finally, if your C API has that double pointer to the array of them, you have to allocate that memory in a special way, either CoTaskMemAlloc or LocalAlloc, forgot which one.
It’s easier, also more efficient, to implement 2 functions instead. One to obtain the required size of the array. Another one to write them into caller-allocated buffer, a single pointer in C++ as opposed to double pointer, and UnmanagedType.LPArray in C#.
Update
allocation should not be necessary since the documentation states that the call returns a pointer to it's internal data structure
The runtime doesn’t know what was written in that documentation, attempts to free anyway, and crashes. You can use out IntPtr in C# API, and marshal manually.
If you have modern .NET, use unsafe, cast the IntPtr to PlcSymbolDescStruct* raw pointer, and construct ReadOnlySpan<PlcSymbolDescStruct> from that raw pointer.
if you’re stuck with old desktop .NET, call Marshal.PtrToStructure<PlcSymbolDescStruct> in a loop, incrementing the pointer by Marshal.SizeOf<PlcSymbolDescStruct>() bytes between loop iterations. No need to use unsafe code in this case.

Conversion of C/C++ Struct To C#.Net CF WinCE

Hi I am trying to convert the C/C++ Strcut to C#
C/C++ Struct looks like:
typedef struct _NDISUIO_QUERY_OID
{
NDIS_OID Oid;
PTCHAR ptcDeviceName;
UCHAR Data[sizeof(ULONG)];
} NDISUIO_QUERY_OID, *PNDISUIO_QUERY_OID;
My C# Struct is:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct _NDISUIO_QUERY_OID
{
public uint Oid;
[MarshalAs(UnmanagedType.LPWStr)]
public string ptcDeviceName;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 8*sizeof(uint))]
public string Data;
};
I was bit doubtful about the converted structure, can anyone clarify me about this conversion??
If possible can anyone please tell me any tutorials or some references that are useful for conversion of data types or structures from c/c++ to C#.Net CF.
Thanks :)
In a previous question of yours, #ctacke said that you won't be able to use MarshalAs(UnmanagedType.LPWStr) with the compact framework. He asserted that you would have to declare that field as IntPtr, and marshal it manually.
However, this MSDN document states that MarshalAs(UnmanagedType.LPWStr) works under the compact framework. I suppose I am inclined to believe the MSDN documents.
The final member is also declared incorrectly. The SizeConst must be sizeof(uint).

Using SizeConst and SizeParamIndex in Custom Marshaler?

I'm currently trying to implement a Custom Marshaler for UTF-8 C Strings.
The problem is, that the strings i'm dealing with are not neccesarily null terminated, so i need to rely on their constant size or a size parameter.
When Marshaling them as LPStr i can use the SizeParamIndex and SizeConst MarshalAs Attribute parameters, but i don't seem to have access to those inside my implementation of ICustomMarshaler.
I'd like to avoid using Byte[] and manual UTF-8 conversion on each function, but it seems like that's the only way?
Or am i missing some way to get access to the SizeParamIndex/SizeConst information?
And even if i could somehow pass this data inside the marshaler, how would i get the actual size value for a SizeParamIndex?

Using C++ native dll in .Net External component has thrown an exception

I'm trying to use a native C++ dll in C# and am getting the "External component has thrown an exception" with an error code of -2147467259.
C#:
[DllImport(#"MyDLL.dll", CharSet = CharSet.Auto)]
public static extern string MyFunction([MarshalAs(UnmanagedType.LPStr)] StringBuilder input);
C++:
__declspec(dllexport) char * __stdcall MyFunction(const char* inputPtr);
The function works just fine in C++. How can I track this error down?
I have tried using string and string builder for the parameter.
Update
I found this article http://tom-shelton.net/index.php/2008/12/11/creating-a-managed-wrapper-for-a-lib-file/
It details a way to use managed C++ to wrap an unmanaged static library in C++ which can then be used in a managed language. Would this be a good way of going about tackling this problem? Can the lib be an unmanaged dll?
try the technique from http://blog.rednael.com/2008/08/29/MarshallingUsingNativeDLLsInNET.aspx - this has saved the day several times :-)
If the parameter is in-only (and not out), stringbuilder is not required. If it's an out parameter (or ref) you should use stringbuilder and pre-allocate the buffer using stringbuilder constructor.
I can guess the problem is that you are returning an Ansi string instead of the expected unicode string. This cause the default pinvoke marshaler to read too much memory.
Try this:
[DllImport(#"MyDLL.dll", CharSet = CharSet.Auto)]
[return : MarshalAs(UnmanagedType.LPStr)]
public static extern string MyFunction([MarshalAs(UnmanagedType.LPStr)] string input);
In any case, in the vast majority of times, it does not make any sense to write Ansi C++ code. I would suggest to convert the C++ code to unicode only (not tchar but wchar_t).
CharSet = CharSet.Auto?? use CharSet.Ansi
You cannot use a char array as return, a block of memory allocated with c++ cannot be used by C#! In your case you probably need to copy it into C# managed memory.
Use StringBuilder for that or MarshalAs attribute that will copy for you the buffer into C# managed memory.
You have to change your C++ function, your C++ function must write into the destination buffer that must be allocate so it contains at least the number of characters you need plus one (for null termination character).
[DllImport(#"MyDLL.dll", CharSet = CharSet.Ansi)]
private static extern void MyFunction([MarshalAs(UnmanagedType.LPStr)] string input, StringBuilder result);
public static string MyFunctionPublic(string input)
{
StringBuilder sb = new StringBuilder(input.Length + 1);
MyFunction(input, sb);
return sb.ToString();
}
And i expect your C function to do something like this:
void __stdcall MyFunction(const char* input, char* result)
{
strcpy(result, input); // this is a dummy stupid code to show how it works.
}
Probably you will need a function that gives you idea of how much bytes you have to allocate in C#. This function can be something like this:
int __stdcall ComputeMyFunctionBytes(const char* input)
{
return strlen(input); // this is a dummy stupid code to show how it works.
}
Never return a piece of allocated memory to C#, it cannot do anything with that, nor deallocating, nor reading and nor writing, until you use unsafe code, but this is not what you are trying to do.
A good article seems to be here: http://www.devx.com/dotnet/Article/6990/1954
Also note that C# uses unicode, 16 bit per character, so it will be converted from ansi to unicode.

Categories

Resources