Is there any way to execute an array of IL codes in C# like shell codes in C/C++?
I want to create a method, convert it to IL code, obfuscate it and store in an array of bytes and finally want to execute it decrypt the array content and execute IL code.
For example this is my C# code:
static int MyMethod(string A, int B)
{
B += 10;
if (A.Equals("A"))
B = 0;
return B;
}
Now I convert it to IL code :
private static int MyMethod(string A, int B)
{
locals: int local_0,
bool local_1
/* 0000025C 00 */ nop
/* 0000025D 03 */ ldarg_1 // int B
/* 0000025E 1F 0A */ ldc_i4_s 10
/* 00000260 58 */ add
/* 00000261 10 01 */ starg_s arg_1 // int B
/* 00000263 02 */ ldarg_0 // string A
/* 00000264 72 01 00 00 70 */ ldstr "A"
/* 00000269 6F 11 00 00 0A */ callvirt System.String::Equals(string) // returns bool
/* 0000026E 16 */ ldc_i4_0
/* 0000026F FE 01 */ ceq
/* 00000271 0B */ stloc_1 // bool local_1
/* 00000272 07 */ ldloc_1 // bool local_1
/* 00000273 2D 03 */ brtrue_s loc_28
/* 00000275 16 */ ldc_i4_0
/* 00000276 10 01 */ starg_s arg_1 // int B
loc_28:
/* 00000278 03 */ ldarg_1 // int B
/* 00000279 0A */ stloc_0 // int local_0
/* 0000027A 2B 00 */ br_s loc_32
loc_32:
/* 0000027C 06 */ ldloc_0 // int local_0
/* 0000027D 2A */ ret
}
And finally this is a byte array :
private byte[] ilcode =
{
0x00, 0x03, 0x1F, 0x0A, 0x58, 0x10, 0x01, 0x02, 0x72, 0x01, 0x00, 0x00, 0x70, 0x6F, 0x11, 0x00, 0x0, 0x0A, 0x16,
0xFE, 0x01, 0x0B, 0x07, 0x2D, 0x03, 0x16, 0x10, 0x01, 0x03, 0x0A, 0x2B, 0x00, 0x06, 0x2A
};
You will need to use the infrastructure in the System.Reflection.Emit namespace. Specifically, you should look at the docs for MethodBuilder.CreateMethodBody which takes an array of bytes representing MSIL instructions. There is a full example there, but below is short snippet of its use. Here, I will create a delegate to the dynamic method as well.
I will note that this is only supported in a very limited way, which is called out in the docs:
This is currently not fully supported. The user cannot supply the location of token fix ups and exception handlers.
The issue is that metadata tokens used in IL to reference types, methods, string literals, etc are resolved at the module level. Thus IL is not completely portable in the sense that you can't take arbitrary, raw IL from a method in one module and just drop it into another method in another module. You need the proper metadata tokens in the new module. However, if you know that your IL contains no metadata tokens you can do it, but this severely limits what you can do with this. (HT: svick, Simon Svensson)
class Program
{
static void Main(string[] args)
{
// opcodes for pushing two arguments to the stack, adding, and returning the result.
byte[] ilcodes = { 0x02, 0x03, 0x58, 0x2A };
var method = CreateFromILBytes(ilcodes);
Console.WriteLine(method(2, 3));
}
private static Func<int, int, int> CreateFromILBytes(byte[] bytes)
{
var asmName = new AssemblyName("DynamicAssembly");
var asmBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(asmName, AssemblyBuilderAccess.RunAndSave);
var module = asmBuilder.DefineDynamicModule("DynamicModule");
var typeBuilder = module.DefineType("DynamicType");
var method = typeBuilder.DefineMethod("DynamicMethod",
MethodAttributes.Public | MethodAttributes.Static,
typeof(int),
new[] { typeof(int), typeof(int) });
method.CreateMethodBody(bytes, bytes.Length);
var type = typeBuilder.CreateType();
return (Func<int, int, int>)type.GetMethod("DynamicMethod").CreateDelegate(typeof(Func<int, int, int>));
}
}
Note the use of the RunAndSave option. This will save the dynamic assembly to disk in a temporary location. It may be more desirable to use RunAndCollect which will generate the assembly in memory only and allow it to be collected later when all references to it are dead. However, there are some caveats to so called collectible assemblies, detailed here.
Related
I want to share code from a C# project inside a VB.Net project.
I want to refer a public class and its variables inside VB.
So I've put both VB and C# project inside the same solution.
Here is the declaration of C# class insde C# project:
public class MyUtils
{
public static byte[] zeroArray = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
....
When I want to refer it inside VB I will have below errors:
'MyUtils' is not accessible in this context because it is 'Friend'.
I have change the accessibility of every object to public in C# but I don't know how to allow access to C# class. I should add that I have not enough familiarity with VB and its inheritance mechanisms.
I created a C# console app named "ConsoleApp2" using .NET Framework 4.8 and added a class named "MyUtils":
namespace ConsoleApp2
{
public class MyUtils
{
public static byte[] zeroArray = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
}
}
I built the project to make sure that worked.
Then I added a VB.NET console app project named "ConsoleApp1" to the same solution. I added a reference to the ConsoleApp2 project:
and used this code:
Module Module1
Sub Main()
Dim bb = ConsoleApp2.MyUtils.zeroArray
Console.WriteLine(String.Join(" ", bb.Select(Function(b) b.ToString("X2"))))
Console.ReadLine()
End Sub
End Module
and ran it to get the output:
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Assume we have an output-only Stream, which outputs bytes. The data from Stream are serialized messages, each message always starts with a byte sequence (0xAA, 0xBB, 0xCC), but the length of the message is unknown.
Currently, I created an Observable and emit each byte on the stream, then subscribe to this observable, buffer every emission, find the byte sequence, and then emit the buffer. Something like
List<byte> buffer = new List<byte>();
dataStream.subscribe(b => {
buffer.add(b);
int[] idx = SearchSequence(buffer);
if(idx.Length < 2){
// TODO: wait for more data
}
else{
messageStream.onNext(buffer.GetRange(idx[0], idx[1]));
// TODO: remove them from buffer
}
})
Is there any more elegant way to solve this issue? There are two concerns as far as I know of:
The length of message is not fixed, which invalidate Observable.buffer()
At the time when dataStream (or other observable derived from it) is subscribed, the output of Stream may in the middle of a message.
UPDATE:
How do you detect the end of a message?
There is not gap between messages, the messages are comes right next to each other. Thus, the starting sequence of a message (0xAA, 0xBB, 0xCC) is also the ending sequence of the previous message
What does your input observable look like?
My current code is like:
Observer<byte> ob = null;
var dataStream = Observable.Create<byte>(o => ob = o);
while(true){
ob.OnNext(ms.ReadByte());
}
What do you want your output observable to look like?
A observable which emits message
Observable<byte[]>
I don't know how elegant this is, but maybe it will get you (or others) started. I'm assuming you want the (0xAA, 0xBB, 0xCC) header excluded from the messages:
var s = new Subject<byte>();
IObservable<byte[]> results = s.Publish(_s => Observable.When(_s
.And(_s.Skip(1))
.And(_s.Skip(2))
.Then((a, b, c) => (a, b, c))
))
.Publish(_t => _t
.Buffer(_t.Where(t => t.a == 0xAA && t.b == 0xBB && t.c == 0xCC))
.Select(l => (l[l.Count - 1].a == 0xAA && l[l.Count - 1].b == 0xBB && l[l.Count - 1].c == 0xCC
? l.Take(l.Count - 3)
: l
)
.Select(e => e.c)
.ToArray()
)
.Skip(1)
)
;
Explanation:
We first use And/Then/When to do a double zip, so the stream (0xAA, 0xBB, 0xCC, 0x01, 0x02, 0x03, 0xAA, 0xBB, 0xCC, 0x01, 0x02, 0x03) becomes a stream of tuples that look like this:
(0xAA, 0xBB, 0xCC)
(0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
(0x02, 0x03, 0xAA)
(0x03, 0xAA, 0xBB)
(0xAA, 0xBB, 0xCC)
(0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
We then use .Where to sniff out when you have a tuple that looks like (0xAA, 0xBB, 0xCC), and use that as a buffer boundary.
Once you have your buffer boundaries, they actually cut off after the message starter occurs, so you end up for our sample stream of two messages, you end up with three lists of tuples:
List 1: (0xAA, 0xBB, 0xCC)
List 2: (0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
(0x02, 0x03, 0xAA)
(0x03, 0xAA, 0xBB)
(0xAA, 0xBB, 0xCC)
List 3: (0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
The bytes we want in each list are basically the third column, but if we want to exclude the message-introduction from the messages, then we have to do some cleanup: We have to exclude the last three elements from list 2 (and all other 'middle' lists), we have to remove list 1, and we have to preserve list 3. Removing the first list is accomplished by the .Skip(1) at the end. Stripping out the last three elements from middle lists is done by checking to see if the last element in the list is (0xAA, 0xBB, 0xCC), and if it is, taking all elements except the last three.
Given all that, I hope there's a better way to do this.
Here's some runner code:
results.Dump(); //Linqpad
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0x01);
s.OnNext(0x02);
s.OnNext(0x03);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0x01);
s.OnNext(0x02);
s.OnNext(0x03);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0xCC);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0x04);
s.OnNext(0x05);
s.OnNext(0x06);
s.OnNext(0x07);
s.OnCompleted();
Output:
01 02 03
01 02 03 AA BB
CC AA BB 04 05 06 07
Here is a custom Window operator that splits a sequence to sub-sequences, according to the supplied separator array. It is based on the built-in Window operator that accepts a IObservable<TWindowBoundary> windowBoundaries parameter.
/// <summary>Projects each element of an observable sequence into non-overlapping
/// windows that are separated by the provided separator.</summary>
public static IObservable<IObservable<T>> Window<T>(
this IObservable<T> source,
T[] separator,
IEqualityComparer<T> comparer = null)
{
return Observable.Defer(() =>
{
var boundaries = new Subject<Unit>();
var queue = new Queue<T>(separator.Length);
return source
.Do(x =>
{
if (queue.Count == separator.Length) queue.Dequeue();
queue.Enqueue(x);
if (queue.SequenceEqual(separator, comparer))
{
queue.Clear();
boundaries.OnNext(default);
}
})
.Concat(Observable.Repeat(default(T), separator.Length - 1))
.SkipLast(separator.Length - 1)
.Window(boundaries)
.Select((window, i) => i == 0 ? window : window.Skip(separator.Length));
});
}
A Subject is used for notifying the Window operator that a new boundary has been detected. The detection mechanism includes a Queue that holds the last emitted elements. This queue is compared to the separator every time a new element is emitted. The Window operator is lagging intentionally behind the detection mechanism by separator.Length - 1 elements, so that the resulting windows are aligned properly.
Usage example:
IObservable<byte> dataStream = GetDataStream();
IObservable<byte[]> messageStream = dataStream
.Window(new byte[] { 0xAA, 0xBB, 0xCC })
.SelectMany(window => window.ToArray());
I created a very simple DLL for Speck in (granted, probably inefficient) ASM. I connected to it in C# using InteropServices.
When I tested this crypto with the test vectors provided in the paper describing the algorithm, I found that the only way to get them to come out right was to "flip" the key and the plain text, and then to "flip" the crypto at the end for a match. So an endianness issue I guess. I have seen the same, for example, between a reference implementation of Serpent and TrueCrypt's version -- they produce the same result only with the bytes in the reverse order.
I will post my assembly code and my C# code for reference, though it may not be critical to see the code in order to understand my question. In the C# code is a click event handler that checks the DLL for consistency with the test vectors. As you can also see there, the program has to do a lot of array flipping in that handler to get the match.
So the question I have been working towards is this. Should I "flip" those arrays inside the DLL to account for endianness? Or should I leave it to the caller (also me, but C# side)? Or am I making mountains out of molehills and I should just ignore endianness at this point? I am not planning to sell the silly thing, so there is no worry about compatibility issues, but I am a stickler for doing things right, so I am hoping you all can guide me on the best practice here if there is one.
ASM:
.code ; the beginning of the code
; section
WinMainCRTStartup proc h:DWORD, r:DWORD, u:DWORD ; the dll entry point
mov rax, 1 ; if eax is 0, the dll won't
; start
ret ; return
WinMainCRTStartup Endp ; end of the dll entry
_DllMainCRTStartup proc h:DWORD, r:DWORD, u:DWORD ; the dll entry point
mov rax, 1 ; if eax is 0, the dll won't
; start
ret ; return
_DllMainCRTStartup Endp
SpeckEncrypt proc plaintText:QWORD, cipherText:QWORD, Key:QWORD
; Pass in 3 addresses pointing to the base of the plainText, cipherText, and Key arrays
; These come in as RCX, RDX, and R8, respectively
; I will use These, RAX, and R9 through R15 for my working space. Will do 128 bit block, 128 bit key sizes, but they will fit nicely in 64 bit registers
; simple prologue, pushing ebp and ebx and the R# registers, and moving the value of esp into ebp for the duration of the proc
push rbp
mov rbp,rsp
push rbx
push R9
push R10
push R11
push R12
push R13
push R14
push R15
; Move data into the registers for processing
mov r9,[rcx] ; rcx holds the memory location of the first 64 bits of plainText. Move this into R9. This is plainText[0]
mov r10,[rcx+8] ; put next 64 bits into R10. This is plainText[1]
;NOTE that the address of the cipherText is in RDX but we will fill r11 and r12 with values pointed at by RCX. This is per the algorithm. We will use RDX to output the final bytes
mov r11,[rcx] ; cipherText[0] = plainText[0]
mov r12,[rcx+8] ; cipherText[1] = plainText[1]
mov r13, [r8] ;First 64 bits of key. This is Key[0]
mov r14, [r8+8] ; Next 64 bits of key. This is Key[1]
push rcx ; I could get away without this and loop in another register, but I want to count my loop in rcx so I free it up for that
mov rcx, 0 ; going to count up from here to 32. Would count down but the algorithm uses the counter value in one permutation, so going to count up
EncryptRoundFunction:
ror r12,8
add r12,r11
xor r12,r13
rol r11,3
xor r11,r12
ror r14,8
add r14,r13
xor r14,rcx
rol r13,3
xor r13,r14
inc rcx
cmp rcx, 32
jne EncryptRoundFunction
pop rcx
; Move cipherText into memory pointed at by RDX. We won't bother copying the Key or plainText back out
mov [rdx],r11
mov [rdx+8],r12
; Now the epilogue, returning values from the stack into non-volatile registers.
pop R15
pop R14
pop R13
pop R12
pop R11
pop R10
pop R9
pop rbx
pop rbp
ret ; return eax
SpeckEncrypt endp ; end of the function
SpeckDecrypt proc cipherText:QWORD, plainText:QWORD, Key:QWORD
; Pass in 3 addresses pointing to the base of the cipherText, plainText, and Key arrays
; These come in as RCX, RDX, and R8, respectively
; I will use These, RAX, and R9 through R15 for my working space. Will do 128 bit block, 128 bit key sizes, but they will fit nicely in 64 bit registers
; simple prologue, pushing ebp and ebx and the R# registers, and moving the value of esp into ebp for the duration of the proc
push rbp
mov rbp,rsp
push rbx
push R9
push R10
push R11
push R12
push R13
push R14
push R15
; Move data into the registers for processing
mov r9,[rcx] ; rcx holds the memory location of the first 64 bits of cipherText. Move this into R9. This is cipherText[0]
mov r10,[rcx+8] ; put next 64 bits into R10. This is cipherText[1]
;NOTE that the address of the plainText is in RDX but we will fill r11 and r12 with values pointed at by RCX. This is per the algorithm. We will use RDX to output the final bytes
mov r11,[rcx] ; plainText[0] = cipherText[0]
mov r12,[rcx+8] ; plainText[1] = cipherText[1]
mov r13, [r8] ;First 64 bits of key. This is Key[0]
mov r14, [r8+8] ; Next 64 bits of key. This is Key[1]
push rcx ; I could get away without this and loop in another register, but I want to count my loop in rcx so I free it up for that
mov rcx, 0 ; We will count up while making the round keys
DecryptMakeRoundKeys:
; On encrypt we could make each key just as we needed it. But here we need the keys in reverse order. To undo round 31 of encryption, for example, we need round key 31.
; So we will make them all and push them on the stack, pop them off again as we need them in the main DecryptRoundFunction
; I should pull this off and call it for encrypt and decrypt to save space, but for now will have it separate
; push r13 at the beginning of the process because we need a "raw" key by the time we reach decrypt round 0
; We will not push r14 because that half of the key is only used here in the round key generation function.
; We don't need it in the decrypt rounds
push r13
ror r14,8
add r14,r13
xor r14,rcx
rol r13,3
xor r13,r14
inc rcx
cmp rcx, 32
jne DecryptMakeRoundKeys
mov rcx, 32
DecryptRoundFunction:
dec rcx
pop r13
xor r11,r12
ror r11,3
xor r12,r13
sub r12,r11
rol r12,8
cmp rcx, 0
jne DecryptRoundFunction
pop rcx
; Move cipherText into memory pointed at by RDX. We won't bother copying the Key or plainText back out
mov [rdx],r11
mov [rdx+8],r12
; Now the epilogue, returning values from the stack into non-volatile registers.
pop R15
pop R14
pop R13
pop R12
pop R11
pop R10
pop R9
pop rbx
pop rbp
ret ; return eax
SpeckDecrypt endp ; end of the function
End ; end of the dll
And the C#:
using System;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Windows.Forms;
namespace SpeckDLLTest
{
public partial class Form1 : Form
{
byte[] key = { 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 };
public Form1()
{
InitializeComponent();
Array.Reverse(key);
}
private void richTextBox1_TextChanged(object sender, EventArgs e)
{
textBox1.Text = richTextBox1.Text.Length.ToString();
if (richTextBox1.Text != "")
{
byte[] plainText = ASCIIEncoding.ASCII.GetBytes(richTextBox1.Text);
byte[] cipherText = new byte[plainText.Length];
Thread t = new Thread(() =>
{
cipherText = Encrypt(plainText);
BeginInvoke(new Action(() => richTextBox2.Text = Convert.ToBase64String(cipherText)));
});
t.Start();
t.Join();
t.Abort();
byte[] plainAgain = new byte[cipherText.Length];
t = new Thread(() =>
{
plainAgain = Decrypt(cipherText);
BeginInvoke(new Action(() => richTextBox3.Text = ASCIIEncoding.ASCII.GetString(plainAgain)));
});
t.Start();
t.Join();
t.Abort();
}
else
{
richTextBox2.Text = "";
richTextBox3.Text = "";
}
}
private byte[] Decrypt(byte[] cipherText)
{
int blockCount = cipherText.Length / 16;
if (cipherText.Length % 16 != 0) blockCount++;
Array.Resize(ref cipherText, blockCount * 16);
byte[] plainText = new byte[cipherText.Length];
unsafe
{
fixed (byte* plaintextPointer = plainText, ciphertextPointer = cipherText, keyPointer = key)
{
for (int i = 0; i < blockCount; i++)
{
for (int j = 0; j < 1; j++)
{
UnsafeMethods.SpeckDecrypt(ciphertextPointer + i * 16, plaintextPointer + i * 16, keyPointer);
}
}
}
}
return plainText;
}
private byte[] Encrypt(byte[] plainText)
{
int blockCount = plainText.Length / 16;
if (plainText.Length % 16 != 0) blockCount++;
Array.Resize(ref plainText, blockCount * 16);
byte[] cipherText = new byte[plainText.Length];
unsafe
{
fixed (byte* plaintextPointer = plainText, ciphertextPointer = cipherText, keyPointer = key)
{
for (int i = 0; i < blockCount; i++)
{
for (int j = 0; j < 1; j++)
{
UnsafeMethods.SpeckEncrypt(plaintextPointer + i * 16, ciphertextPointer + i * 16, keyPointer);
}
}
}
}
return cipherText;
}
private void button1_Click(object sender, EventArgs e)
{
byte[] plainText = { 0x6c, 0x61, 0x76, 0x69, 0x75, 0x71, 0x65, 0x20, 0x74, 0x69, 0x20, 0x65, 0x64, 0x61, 0x6d, 0x20 };
byte[] key = { 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 };
byte[] testVector = { 0xa6, 0x5d, 0x98, 0x51, 0x79, 0x78, 0x32, 0x65, 0x78, 0x60, 0xfe, 0xdf, 0x5c, 0x57, 0x0d, 0x18 };
Array.Reverse(key);
Array.Reverse(plainText);
byte[] cipherText = new byte[16];
unsafe
{
fixed (byte* plaintextPointer = plainText, ciphertextPointer = cipherText, keyPointer = key)
{
UnsafeMethods.SpeckEncrypt(plaintextPointer, ciphertextPointer, keyPointer);
Array.Reverse(cipherText);
bool testBool = true;
for (int i = 0; i < cipherText.Length; i++)
{
if (testVector[i] != cipherText[i]) testBool = false;
}
if (testBool == false) MessageBox.Show("Failed!");
else MessageBox.Show("Passed!");
}
}
}
}
public static class UnsafeMethods
{
[DllImport("Speck.dll")]
unsafe public extern static void SpeckEncrypt(byte* plainText, byte* cipherText, byte* Key);
[DllImport("Speck.dll")]
unsafe public extern static void SpeckDecrypt(byte* cipherText, byte* plainText, byte* Key);
}
}
Whether someone might like it or not, the de facto standard for byte order when it comes to networking and cryptography is big-endian (most significant byte first — the “natural” order). This applies not only to serialization of data for inter-system exchange, but to intra-system API as well and for any other case where caller is not supposed to be aware of callee internals. This convention does not have anything to do with endianness of particular hardware and popularity of such hardware. It just sets the default format for exchanged data, so that both lower-level and higher-level programs may pass data around without regard to their degree of awareness of what this data contains and how it is processed.
However, if the caller is supposed to be tightly coupled with the callee, it may be more convenient and performance-wise to pass the data in a more preprocessed form, especially if some of that data remains constant across invocations. For example, if we are dealing with asymmetric cryptography, it may be easier and faster to call the core functions with all data already translated to big integers, and for those we may prefer little-endian digit order (a “digit” or a “limb” is usually a half of largest available register) even on a big-endian byte order hardware — simply because such an order of digits is more useful for arbitrary-precision math library. But those details should not be visible to the outside world — for anyone else, we are accepting and returning big-endian bytestream.
Regarding your specific task.
As #RossRidge already pointed out, you are probably very wrong if your are simply flipping entire arrays, — you should swap bytes (BSWAP) in particular pieces being processed rather than inverting the order of those pieces besides that.
Chances are high that you are very overestimating your ability to write efficient machine code: for example, you don't interleave instructions with unrelated registers for better out-of-order execution, your loop is not aligned, you use counter increase to N instead of decrease to zero. Of course, that code will still be 10x faster than .Net anyway, but I strongly recommend you to write an implementation in C and benchmark — to get amazed of how good a compiler (MSVC, GCC) may be at optimizing even a straight-though written program (believe me, I once committed the same mistake when trying to accomplish the same task). If performance is not a big issue, do not mess with unmanaged code at all, — because it is just an external non-portable dependency that increases required trust level for you .Net application.
Use .Net functions dealing with bytes with caution, because they are very inconsistent with regard to endianness: BitConverter uses host byte order, StreamReader always sticks to little-endian, and String is all about the encoding given (of all UTF encodings, only UTF-8 is endian-agnostic).
That are the issues I noticed at first glance. There may be more of them.
My goal is to get a 64bit value hence a byte array of size 8. However my problem is that I want to set the first 20 bits myself and then have the rest to be 0s. Can this be done with the shorthand byte array initialisation?
E.g. if I wanted all 0s I would say:
byte[] test = new byte[] {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
What I've thought about/tried:
So each hexadecimal digit corresponds to 4 binary digits. Hence, if I want to specify the first 20bits, then I specify the first 5 hexadecimal digits? But I'm not sure of how to do this:
byte[] test = new byte[] {0xAF, 0x17, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00};
That would mean that I've specified the first 24 bits right? And not 20.
I could use a BitArray and do it that way but I'm just wondering whether it can be done in the above way.
How about:
byte byte1 = 0xFF;
byte byte2 = 0xFF;
byte byte3 = 0xFF;
// 8bits 8bits 4bits : total = 20 bits
// 11111111 11111111 11110000
byte[] test = new byte[] { byte1, byte2, (byte)(byte3 & 0xF0), 0x00, 0x00, 0x00, 0x00, 0x00 };
You can write your bytes backward, and use BitConverter.GetBytes(long):
var bytes = BitConverter.GetBytes(0x117AF);
Demo.
Since each hex digit corresponds to a single four-bit nibble, you can initialize data in "increments" of four bits. However, the data written in reverse will be almost certainly less clear to human readers of your code.
I have a TCP Client,which puts a packet in a structure
using System.Runtime.InteropServices;
[StructLayoutAttribute(LayoutKind.Sequential)]
public struct tPacket_5000_E
{
public Int16 size;
public Int16 opcode;
public byte securityCount;
public byte securityCRC;
public byte flag;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 8, ArraySubType = UnmanagedType.I1)]
public byte[] blowfish;
public UInt32 seedCount;
public UInt32 seedCRC;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 5, ArraySubType = UnmanagedType.I1)]
public UInt32[] seedsecurity;
}
The code I use to put the packet in the structure is:
tPacket_5000_E packet = new tPacket_5000_E();
GCHandle pin = GCHandle.Alloc(data, GCHandleType.Pinned);
packet = (tPacket_5000_E)Marshal.PtrToStructure(pin.AddrOfPinnedObject(), typeof(tPacket_5000_E));
pin.Free();
Now,before i continue I must tell you that I'm translating this project from C++ to C#.
This is the problem:
The last 3 members of tPacket_5000_E are Int32(i tried UInt32 too),which is DWORD in C++.
The values before those three members,which are NOT Int32,are equal to those in C++.(I inject same packet in both C++ and C# project).
However,those three members have different values.
in C++ the values are(correct):
seedCount:0x00000079
seedCRC:0x000000d1
SeedSecurity:
-[0]:0x548ac099
-1:0x03c4d378
-[2]:0x292e9eab
-[3]:0x4eee5ee3
-[4]:0x1071206e
in C# the values are(incorrect):
seedCount:0xd1000000
seedCRC:0x99000000
SeedSecurity:
-[0]: 0x78548ac0
-1: 0xab03c4d3
-[2]: 0xe3292e9e
-[3]: 0x6e4eee5e
-[4]: 0x00107120
The packet in both applications is equal
byte[] data = new byte[] {
0x25, 0x00, 0x00, 0x50, 0x00, 0x00, 0x0E, 0x10,
0xCE, 0xEF, 0x47, 0xDA, 0xC3, 0xFE, 0xFF, 0x79,
0x00, 0x00, 0x00, 0xD1, 0x00, 0x00, 0x00, 0x99,
0xC0, 0x8A, 0x54, 0x78, 0xD3, 0xC4, 0x03, 0xAB,
0x9E, 0x2E, 0x29, 0xE3, 0x5E, 0xEE, 0x4E, 0x6E,
0x20, 0x71, 0x10};
Click here for further information
Why the last three members in the struct are different and how to fix them?
Thanks in advance!
I'd expect that the root of your problem is that the three byte values
public byte securityCount;
public byte securityCRC;
public byte flag;
cause the next 32-bit values not to be word-aligned, and your two sets of code are adding (or not adding) internal padding differently.
I expect that the different packings look something like this:
C++ C#
================================ ================================
[size ][opcode ] [size ][opcode ]
[secCnt][secCrc][flag ][blow0 ] [secCnt][secCrc][flag ][blow0 ]
[blow1 ][blow2 ][blow3 ][blow4 ] [blow1 ][blow2 ][blow3 ][blow4 ]
[blow5 ][blow6 ][blow7 ][seedCou [blow5 ][blow6 ][blow7 ]..PAD...
nt ][seedCRC [seedCount ]
][seedSec [seedCRC ]
urity0 ][seedSec [seedSecurity0 ]
urity1 ][seedSec [seedSecurity1 ]
urity2 ][seedSec [seedSecurity2 ]
urity3 ][seedSec [seedSecurity3 ]
urity4 ] [seedSecurity4 ]
... with C# inserting a byte of padding which causes later values to be one byte off.
You can try using
[StructLayout(LayoutKind.Sequential,Pack=1)]
before your struct definition, which should use the minimum amount of space possible.
Mastering Structs in C# has some good information on how/why this happens.
I suspect that Daniel L is on the right track in his answer.
I would try adding a 4th byte after the flag. My guess is that your C++ compiler is aligning the values on word boundaries. That would "align" the C# version as well.