Develop Reference

C#: Looping through member objects of nested structs

Hi all you c# wizards!
I need to store all the memory offset values of (packed) nested structs within these respective structs.
Recusively looping through all the members works fine so far. Also, i get the appropriate memory offset values.
This struct contraption might contain several dozends of structs, and several hundreds of other members in the end.
But i do this whole thing at initialization time, so CPU performance won't be an issue here.
But:
In this iteration process, it seems i have trouble accessing the actual instances of those structs. As it turns out, when i try to store these offset values, they don't end up where i need them (of course, i need them in the instance "SomeStruct1" and its containing other struct instances, but the debugger clearly shows me the init values (-1)).
I suspect "field_info.GetValue" or "obj_type.InvokeMember" is not the proper thing to get the object reference? Is there any other way to loop through nested struct instances?
Please help! I've desperately debugged and googled for three days, but i'm so out of ideas now...
Thanks for your efforts!
-Albert
PS - the reason i do this unusual stuff:
I communicate between two embedded CPU cores via the mentioned nested struct (both are mixed c/c++ projects). This works like a charm, as both cores share the same memory, where the struct resides.
Additionally, i have to communicate between a c# host application and theses embedded cores, so i thought it could be a neat thing, if i implement a third instance of this struct. Only this time, i oviously can't use shared RAM. Instead, i implement value setters and getters for the data-holding members, find out the memory offset as well as the lenght of the data-holding members, and feed this information (along with the value itself) via USB or Ethernet down to the embedded system - so the "API" to my embedded system will simply be a struct. The only maintenance i have to do every thime i change the struct: i have to copy the holding .h file (of the embedded project) to a .cs file (host project).
I know it's crazy - but it works now.
Thanks for your interest. -Albert
This is a simplified (buggy, see below) example that should compile and execute (WinForms, c#7.3):
using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Windows.Forms;
namespace CodingExample
{
public interface Interf
{
Int32 Offset {get; set; }
}
[StructLayout (LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct sSomeStruct2 : Interf
{
public sSomeStruct2 (bool dummy)
{
Offset = -1;
SomeMember3 = 0;
}
public Int32 Offset {get; set; }
public Int32 SomeMember3;
// much more various-typed members (e. g. nested structs)...
}
[StructLayout (LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct sSomeStruct1 : Interf
{
public sSomeStruct1 (bool dummy)
{
Offset = -1;
SomeMember1 = 0;
SomeStruct2 = new sSomeStruct2 (true);
SomeMember2 = 0;
}
public Int32 Offset {get; set; }
public Int32 SomeMember1;
public sSomeStruct2 SomeStruct2;
public Int16 SomeMember2;
// much more various-typed members...
}
public partial class Form1 : Form
{
void InitializeOffsets (object obj)
{
Console.WriteLine ("obj: {0}", obj);
Type obj_type = obj.GetType ();
foreach (FieldInfo field_info in obj_type.GetFields ())
{
string field_name = field_info.Name;
Int32 offset = (Int32) Marshal.OffsetOf (obj_type, field_name);
Type field_type = field_info.FieldType;
bool is_leafe = field_type.IsPrimitive;
// none of theses three options seem to give me the right reference:
// object node_obj = field_info.GetValue (obj);
// object node_obj = field_info.GetValue (null);
object node_obj = obj_type.InvokeMember (field_name, BindingFlags.GetField, null, obj, null);
Console.WriteLine ("field: {0}; field_type: {1}; is_leafe: {2}; offset: {3}", field_name, field_type, is_leafe, offset);
if (! is_leafe)
{
// this writes not as expected:
(node_obj as Interf).Offset = offset;
InitializeOffsets (node_obj);
}
}
}
sSomeStruct1 SomeStruct1;
public Form1 ()
{
InitializeComponent ();
SomeStruct1 = new sSomeStruct1 (true);
InitializeOffsets (SomeStruct1);
}
}
}

Meanwhile i found out, what i did wrong:
i have to do boxing, so i can use "ref" when i call my initialize function:
// instead of this:
SomeStruct1 = new sSomeStruct1 (true);
// i have to do it this way:
object boxed_SomeStruct1 = new sSomeStruct1 (true);
InitializeOffsets (ref boxed_SomeStruct1);
SomeStruct1 = (sSomeStruct1) boxed_SomeStruct1;
Within the "InitializeOffsets" function, "field_info.GetValue (obj)" delivers a copy of my member object. That's why i have to copy the modified copy back at the very end of the foreach loop:
field_info.SetValue (obj, node_obj);
After these changes, the code works as intended.
Thanks for your interest. -Albert

Obtaining original variable name from within an extension method

We are currently working on a logging solution and have implemented an extension method call 'Log'. When writing to the log file, we would ideally like to write the original variable name (rather than the variable name used in the extension method).
What we are currently having to do for this is:
public void DoSomeWork()
{
String testString = "Hey look I'm a string!";
testString.Log("testString value");
}
With the extention method:
public static String Log(this String valueToStore, String name)
{
// The logging code which uses the 'name' parameter
}
The issue here is that it becomes difficult to read on longer lines of code and looks clustered. What would be ideal is this:
public void DoSomeWork()
{
String testString = "Hey look I'm a string!";
testString.Log();
}
With the extension method:
public static String Log(this String valueToStore)
{
// The logging code which is able to retrieve the
// value 'testString' from the 'valueToStore' value
}
Is this at all possible by using Reflection? I'm aware of the nameofoption, but that only returns the string 'valueToStore' when used in the extension method.

Well, short answer is no. The variable names are not guaranteed to persist after compilation in unchanged form. That information would have to be somehow persisted (for example by the use of nameof()). Also, the variable name might not exist ("test".GetVarName()).
The long answer is: yes, possibly, but it's one of the most ridiculous things I've created in my life:
using System;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
namespace Test1
{
class Program
{
static void Main(string[] args)
{
var myVarName = "test";
myVarName.Test();
Console.ReadKey();
}
}
static class Extensions
{
public static void Test(
this string str,
[System.Runtime.CompilerServices.CallerMemberName] string memberName = "",
[System.Runtime.CompilerServices.CallerFilePath] string sourceFilePath = "",
[System.Runtime.CompilerServices.CallerLineNumber] int sourceLineNumber = 0
)
{
var relevantLine = File.ReadAllLines(sourceFilePath)[sourceLineNumber-1];
var currMethodName = MethodInfo.GetCurrentMethod().Name;
var callIndex = relevantLine.IndexOf(currMethodName + "()");
var sb = new Stack<char>();
for (var i = callIndex - 2; i >= 0; --i)
{
if (Char.IsLetterOrDigit(relevantLine[i]))
{
sb.Push(relevantLine[i]);
}
}
Console.WriteLine(new String(sb.ToArray()));
}
}
}

C# 10 has CallerArgumentExpressionAttribute that will do just that
public static void PrintNameAndValue(
this object obj,
[System.Runtime.CompilerServices.CallerArgumentExpression("obj")] string callerExp = ""
)
{
Console.WriteLine(callerExp + " = " + obj.ToString());
}
It'll capture the entire expression passed:
public void TestPrintNameAndValue()
{
string mystring = "test";
int myint = 5;
mystring.PrintNameAndValue(); // mystring = test
myint.PrintNameAndValue(); // myint = 5
(myint + 10).PrintNameAndValue(); // myint + 10 = 15
mystring.ToUpper().PrintNameAndValue(); // mystring.ToUpper() = TEST
}

You can use an Expression to achieve that, but performance-wise it may not be the best option:
public static void Log<T>(Expression<Func<T>> expr)
{
var memberExpr = expr.Body as MemberExpression;
if (memberExpr == null)
return;
var varName = memberExpr.Member.Name;
var varData = expr.Compile()();
// actual logging
...
}
Usage:
var test = "Foo";
Log(() => test);
Alternatively, if you're using C# 6.0, it can get a bit better using the nameof operator:
test.Log(nameof(test));
A better solution would be one that is leveraging the compiler abilities (specifically, the "Roslyn" compiler) and provide the member name on compile time.

Not really an answer, more of a pointer, but you could try doing something with your application that you're using(e.g. visual studio) instead of doing it in code. What I mean is make it rewrite everything that looks like [variable].Log(); to [variable].Log([variable])
I am pretty sure that there has to be some weird macro or plugin which does this for you before compiling.

C# Struct Layouts and unexpected bench marking result?

I'm not really micro-managing the performance of an application, but I'm curios on the below scenario.
For Structs, by default, C# compiler generates the layout, LayoutType. Sequential. This means the fields should stay in the order defined by the programmer. I believe that this is to support interoperability with unmanaged code. However most user defined Structs have nothing to do with interoperability. I have read that for better performance, we can explicitly specify the LayoutKind.Auto, and let the CLR to decide the best possible layout. In order to test this, I thought of doing a quick benchmark on both layouts. However my result says the default layout (LayoutType.Sequnetial) is bit quicker than the explicit layout (LayoutType.Auto). I was expecting the reverse.
Below is the test I ran on my machine (x86 running .NET 4)
//uses LayoutKind.Sequence by default
public struct StructSeq
{
private readonly Byte mb;
private readonly Int16 mx;
public string a;
public string b;
public string c;
public string d;
}
[StructLayout(LayoutKind.Auto)]
public struct StructAuto
{
private readonly Byte mb;
private readonly Int16 mx;
public string a;
public string b;
public string c;
public string d;
}
public sealed class Program
{
public static void Main()
{
StructSeq sq = new StructSeq();
Stopwatch sw1 = new Stopwatch();
sw1.Start();
for (int i = 0; i < 10000; i++)
{
sq = ProcessStructSeq(sq);
}
sw1.Stop();
Console.WriteLine("Struct LayoutKind.Sequence (default) {0}", sw1.Elapsed.TotalMilliseconds);
StructAuto so = new StructAuto();
Stopwatch sw2 = new Stopwatch();
sw2.Start();
for (int i = 0; i < 10000; i++)
{
so = ProcessStructAuto(so);
}
sw2.Stop();
Console.WriteLine("Struct LayoutKind.Auto (explicit) {0}", sw2.Elapsed.TotalMilliseconds);
Console.ReadLine();
}
public static StructSeq ProcessStructSeq(StructSeq structSeq)
{
structSeq.a = "1";
structSeq.b = "2";
structSeq.c = "3";
structSeq.d = "4";
return structSeq;
}
public static StructAuto ProcessStructAuto(StructAuto structAuto)
{
structAuto.a = "1";
structAuto.b = "2";
structAuto.c = "3";
structAuto.d = "4";
return structAuto;
}
}
Below is a sample result I get on my machine (x86 running .NET 4)
Struct LayoutKind.Sequence (default) 0.7488
Struct LayoutKind.Auto (explicit) 0.7643
I ran this test multiple times and I always get Struct LayoutKind.Sequence (default) < Struct LayoutKind.Auto (explicit)
Even though it is a micro milliseconds difference, I ‘m expecting the Struct LayoutKind.Auto (explicit) to be lower than the Struct LayoutKind.Sequence (default).
Does anyone know the reason for this? Or is it my benchmarking is not accurate enough give me the right result?

I have tested your code on my system, and found that the average time taken is the same when the test is run a large number of times, with each test run slightly favoring one or the other alternative. This applies both to debug and release builds.
Also, as a quick check, I looked at the x86 code in the debugger, and I see no difference in the generated code whatsoever. So with your program as it is, the difference you observed in your measurements essentially seems to be noise.

Honestly, it's so close that it wouldn't make any sort of visible difference unless you were processing a few million of these structs. In fact, running it multiple times may yield different results. I would up the number of iterations and try to run the program without the debugger attached to see if anything changes.
Just using structs doesn't immediately make your code faster though, there are many pitfalls that make structs far slower than their class equivalents.
If you want to optimize this benchmark, you should pass the structs to the process methods as references and not return another struct (avoiding the creation of 2 additional structs for the method), which should provide a much larger speedup than the different layout kinds:
public static void ProcessStructSeq(ref StructSeq structSeq)
{
structSeq.a = "1";
structSeq.b = "2";
structSeq.c = "3";
structSeq.d = "4";
}
public static void ProcessStructAuto(ref StructAuto structAuto)
{
structAuto.a = "1";
structAuto.b = "2";
structAuto.c = "3";
structAuto.d = "4";
}
Also, there's a point where structs become slower than their class counterparts, and that's estimated to be at about 16 bytes according to this MSDN article and further explained in this StackOverflow question.

I believe there is no difference due to how your fields are laid out. The way you declared them, the padding will be the same either way. If you try interlacing the fields of different sizes, you should see a difference, at least in size, if not in speed.
Also, according to this blog post, a struct with a reference field is changed to auto layout (meaning you were benching literally the exact same thing!).
public struct MyStruct
{
private byte b1;
public long a;
private byte b2;
public long b;
private byte b3;
public long c;
private byte b4;
public long d;
}

Reading Serialized MFC CArray in C#

MFC CArray was Serialized and saved to a database. I need to read this data into a C# project. I am able to retrieve the data as byte[] from the database. I then write the byte[] to a MemoryStream. Now I need to read the data from the MemoryStream.
Someone has apparently solved this before, but did not write their solution.
http://social.msdn.microsoft.com/Forums/eu/csharpgeneral/thread/17393adc-1f1e-4e12-8975-527f42e5393e
I followed these projects in my attempt to solve the problem.
http://www.codeproject.com/Articles/32741/Implementing-MFC-Style-Serialization-in-NET-Part-1
http://www.codeproject.com/Articles/32742/Implementing-MFC-Style-Serialization-in-NET-Part-2
The first thing in the byte[] is the size of the array, and I can retrieve that with binaryReader.readInt32(). However, I cannot seem to get back the float values. If I try binaryReader.readSingle() or
public void Read(out float d) {
byte[] bytes = new byte[4];
reader.Read(bytes, m_Index, 4);
d = BitConverter.ToSingle(bytes, 0);
}
I do not get back the correct data. What am I missing?
EDIT Here is the C++ code that serializes the data
typedef CArray<float, float> FloatArray;
FloatArray floatArray;
// fill floatArray
CSharedFile memoryFile(GMEM_MOVEABLE | GMEM_ZEROINIT);
CArchive ar(&memoryFile, CArchive::store);
floatArray.Serialize(ar);
ar.Close();
EDIT 2
By reading backward, I was able to get all of the floats, and was also able to determine that the size for CArray is byte[2], or Int16. Does anyone know if this is always the case?

Using the codeproject articles above, here is a C# implementation of CArray which will allow you to deserialize a serialized MFC CArray.
// Deriving from the IMfcArchiveSerialization interface is not mandatory
public class CArray : IMfcArchiveSerialization {
public Int16 size;
public List<float> floatValues;
public CArray() {
floatValues = new List<float>();
}
virtual public void Serialize(MfcArchive ar) {
if(ar.IsStoring()) {
throw new NotImplementedException("MfcArchive can't store");
}
else {
// be sure to read in the order in which they were stored
ar.Read(out size);
for(int i = 0; i < size; i++) {
float floatValue;
ar.Read(out floatValue);
floatValues.Add(floatValue);
}
}
}
}

What is the equivalent of "ByteBuffer.flip" & "ByteBuffer.slice" in .NET?

I need to port code from Java to C#. In the Java code, the methods "ByteBuffer.flip()" and "ByteBuffer.slice" is used, and I don't know how to translate this.
I've read this question (An equivalent of javax.nio.Buffer.flip() in c#), but although an answer is given, I cannot figure how to apply it. According to Tom Hawtin, I should "Set the limit to the current position and then set the position to zero" in the underlying array. I am unsure as of how to change these values. (If you could explain the underlying logic, it would help me a lot :)
As for the ByteBuffer.slice, I have no clue on how to translate it.
EDIT: If it can be clearer with the actual code, I'll post it:
Java:
ByteBuffer buff;
buff.putShort((short) 0);
buff.put(customArray);
buff.flip();
buff.putShort((short) 0);
ByteBuffer b = buff.slice();
short size = (short) (customFunction(b) + 2);
buff.putShort(0, size);
buff.position(0).limit(size);
So far, my translation in C#.NET:
BinaryWriter b = new BinaryWriter(); //ByteBuffer buff;
b.Write((short)0); // buff.putShort((short) 0);
b.Write(paramStream.ToArray()); // buff.put(customArray);
b.BaseStream.SetLength(b.BaseStream.Position); // buff.flip; (not sure)
b.BaseStream.Position = 0; // buff.flip; too (not sure)
b.Write((short)0); // buff.putShort((short) 0)
??? // ByteBuffer b = buff.slice();
// Not done but I can do it, short size = (short) (customFunction(b) + 2);
??? // How do I write at a particular position?
??? // buff.position(0).limit(size); I don't know how to do this
Thank you!
EDIT: Changed b.BaseStream.SetLength(b.BaseStream.Length); to b.BaseStream.SetLength(b.BaseStream.Position);, based on the Java docs.

(See See http://java.sun.com/javase/6/docs/api/java/nio/ByteBuffer.html#slice%28%29 and http://java.sun.com/javase/6/docs/api/java/nio/Buffer.html#flip%28%29 for java's calls)
Flip is a quick way to reset the buffer. So for example
(pseudocode)
void flip()
{
Length = currentPos;
currentPos = 0;
}
Allows you to quickly setup the buffer you presumably just wrote to for reading from the beginning.
Update:
Splice is a bit trickier due to the requirement that "Changes to this buffer's content will be visible in the new buffer, and vice versa; the two buffers' position, limit, and mark values will be independent". There unfortunately is no concept of a shared portion of buffer (that i know of - theres always using arrays, detailed below) without making your own class. The closest thing you could do is this:
Old Code:
ByteBuffer b = buff.slice();
New Code (assuming a List)
List<Byte> b= buff;
int bStart = buffPos; // buffPos is your way of tracking your mark
the downside to the code above is that there is no way for c# to hold the new starting point of the new buffer and still share it. You'll have to manually use the new starting point whenever you do anything, from for loops (for i=bStart;...) to indexing (newList[i + bStart]...)
Your other option is to do use Byte[] arrays instead, and do something like this:
Byte[] b = &buff[buffPos];
... however that requires unsafe operations to be enabled, and I cannot vouch for its saftey, due to the garbage collector and my avoidance of the "unsafe" features.
Outside of that, theres always making your own ByteBuffer class.

Untested, but if I understand the java bits correctly, this would give you an idea on how to implement.
public class ByteBuffer {
private int _Position;
private int _Capacity;
private byte[] _Buffer;
private int _Start;
private ByteBuffer(int capacity, int position, int start, byte[] buffer) {
_Capacity = capacity;
_Position = position;
_Start = start;
_Buffer = buffer;
}
public ByteBuffer(int capacity) : this(capacity, 0 , 0, new byte[capacity]) {
}
public void Write(byte item) {
if (_Position >= _Capacity) {
throw new InvalidOperationException();
}
_Buffer[_Start + _Position++] = item;
}
public byte Read() {
if (_Position >= _Capacity) {
throw new InvalidOperationException();
}
return _Buffer[_Start + _Position++];
}
public void Flip() {
_Capacity = _Position;
_Position = _Start;
}
public ByteBuffer Slice() {
return new ByteBuffer(_Capacity-_Position, 0, _Position, _Buffer);
}
}