Develop Reference

C# x64 Disassemble byte array for make pattern array

I coding the part of the game that generates the AOB Pattern for use in the game where modding is allowed (to be precise, third-party programs are allowed).
When I read these bytes in my program,
{ 0xBA, 0x79, 0x03, 0x00, 0x00, 0x48, 0x8D, 0x4C, 0x24, 0x28, ... }
This byte array will be created, and i need to divide that byte array similar to the picture.
Like this.
{
{0xBA}, {0x79, 0x03, 0x00, 0x00}
},
{
{0x48}, {0x8D}, {0x4C}, {0x24}, {0x28}
},
{
{0xE8}, {0x9F, 0x40, 0x51, 0x00}
}, ...
I succeeded in using SharpDisasm(https://github.com/justinstenning/SharpDisasm) to divide it into the following.
var asm = new SharpDisasm.Disassembler(br.ReadBytes(2048), mode, 0, true);
var disasm = asm.Disassemble();
foreach (var inst in disasm)
{
//print 'inst.Bytes' will output like follow.
}
{0xBA, 0x79, 0x03, 0x00, 0x00}
{0x48, 0x8D, 0x4C, 0x24, 0x28}
{0xE8, 0x9F, 0x40, 0x51, 0x00}
...
The array must be divided into Opcode and Value (I don't know if it is correct) to make it look like this.
{0xBA, null, null, null, null, 0x48, 0x8D, 0x4C, 0x24, null, 0xE8, null, null, null, null}
However, SharpDisasm doesn't seem to support anything similar to that function.
Is there another library or a good way to handle it?

Share C# class and static variables in VB.NET insde the same solution

I want to share code from a C# project inside a VB.Net project.
I want to refer a public class and its variables inside VB.
So I've put both VB and C# project inside the same solution.
Here is the declaration of C# class insde C# project:
public class MyUtils
{
public static byte[] zeroArray = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
....
When I want to refer it inside VB I will have below errors:
'MyUtils' is not accessible in this context because it is 'Friend'.
I have change the accessibility of every object to public in C# but I don't know how to allow access to C# class. I should add that I have not enough familiarity with VB and its inheritance mechanisms.

I created a C# console app named "ConsoleApp2" using .NET Framework 4.8 and added a class named "MyUtils":
namespace ConsoleApp2
{
public class MyUtils
{
public static byte[] zeroArray = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
}
}
I built the project to make sure that worked.
Then I added a VB.NET console app project named "ConsoleApp1" to the same solution. I added a reference to the ConsoleApp2 project:
and used this code:
Module Module1
Sub Main()
Dim bb = ConsoleApp2.MyUtils.zeroArray
Console.WriteLine(String.Join(" ", bb.Select(Function(b) b.ToString("X2"))))
Console.ReadLine()
End Sub
End Module
and ran it to get the output:
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

How to group bytes to message in an reactive way

Assume we have an output-only Stream, which outputs bytes. The data from Stream are serialized messages, each message always starts with a byte sequence (0xAA, 0xBB, 0xCC), but the length of the message is unknown.
Currently, I created an Observable and emit each byte on the stream, then subscribe to this observable, buffer every emission, find the byte sequence, and then emit the buffer. Something like
List<byte> buffer = new List<byte>();
dataStream.subscribe(b => {
buffer.add(b);
int[] idx = SearchSequence(buffer);
if(idx.Length < 2){
// TODO: wait for more data
}
else{
messageStream.onNext(buffer.GetRange(idx[0], idx[1]));
// TODO: remove them from buffer
}
})
Is there any more elegant way to solve this issue? There are two concerns as far as I know of:
The length of message is not fixed, which invalidate Observable.buffer()
At the time when dataStream (or other observable derived from it) is subscribed, the output of Stream may in the middle of a message.
UPDATE:
How do you detect the end of a message?
There is not gap between messages, the messages are comes right next to each other. Thus, the starting sequence of a message (0xAA, 0xBB, 0xCC) is also the ending sequence of the previous message
What does your input observable look like?
My current code is like:
Observer<byte> ob = null;
var dataStream = Observable.Create<byte>(o => ob = o);
while(true){
ob.OnNext(ms.ReadByte());
}
What do you want your output observable to look like?
A observable which emits message
Observable<byte[]>

I don't know how elegant this is, but maybe it will get you (or others) started. I'm assuming you want the (0xAA, 0xBB, 0xCC) header excluded from the messages:
var s = new Subject<byte>();
IObservable<byte[]> results = s.Publish(_s => Observable.When(_s
.And(_s.Skip(1))
.And(_s.Skip(2))
.Then((a, b, c) => (a, b, c))
))
.Publish(_t => _t
.Buffer(_t.Where(t => t.a == 0xAA && t.b == 0xBB && t.c == 0xCC))
.Select(l => (l[l.Count - 1].a == 0xAA && l[l.Count - 1].b == 0xBB && l[l.Count - 1].c == 0xCC
? l.Take(l.Count - 3)
: l
)
.Select(e => e.c)
.ToArray()
)
.Skip(1)
)
;
Explanation:
We first use And/Then/When to do a double zip, so the stream (0xAA, 0xBB, 0xCC, 0x01, 0x02, 0x03, 0xAA, 0xBB, 0xCC, 0x01, 0x02, 0x03) becomes a stream of tuples that look like this:
(0xAA, 0xBB, 0xCC)
(0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
(0x02, 0x03, 0xAA)
(0x03, 0xAA, 0xBB)
(0xAA, 0xBB, 0xCC)
(0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
We then use .Where to sniff out when you have a tuple that looks like (0xAA, 0xBB, 0xCC), and use that as a buffer boundary.
Once you have your buffer boundaries, they actually cut off after the message starter occurs, so you end up for our sample stream of two messages, you end up with three lists of tuples:
List 1: (0xAA, 0xBB, 0xCC)
List 2: (0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
(0x02, 0x03, 0xAA)
(0x03, 0xAA, 0xBB)
(0xAA, 0xBB, 0xCC)
List 3: (0xBB, 0xCC, 0x01)
(0xCC, 0x01, 0x02)
(0x01, 0x02, 0x03)
The bytes we want in each list are basically the third column, but if we want to exclude the message-introduction from the messages, then we have to do some cleanup: We have to exclude the last three elements from list 2 (and all other 'middle' lists), we have to remove list 1, and we have to preserve list 3. Removing the first list is accomplished by the .Skip(1) at the end. Stripping out the last three elements from middle lists is done by checking to see if the last element in the list is (0xAA, 0xBB, 0xCC), and if it is, taking all elements except the last three.
Given all that, I hope there's a better way to do this.
Here's some runner code:
results.Dump(); //Linqpad
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0x01);
s.OnNext(0x02);
s.OnNext(0x03);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0x01);
s.OnNext(0x02);
s.OnNext(0x03);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0xCC);
s.OnNext(0xCC);
s.OnNext(0xAA);
s.OnNext(0xBB);
s.OnNext(0x04);
s.OnNext(0x05);
s.OnNext(0x06);
s.OnNext(0x07);
s.OnCompleted();
Output:
01 02 03
01 02 03 AA BB
CC AA BB 04 05 06 07

Here is a custom Window operator that splits a sequence to sub-sequences, according to the supplied separator array. It is based on the built-in Window operator that accepts a IObservable<TWindowBoundary> windowBoundaries parameter.
/// <summary>Projects each element of an observable sequence into non-overlapping
/// windows that are separated by the provided separator.</summary>
public static IObservable<IObservable<T>> Window<T>(
this IObservable<T> source,
T[] separator,
IEqualityComparer<T> comparer = null)
{
return Observable.Defer(() =>
{
var boundaries = new Subject<Unit>();
var queue = new Queue<T>(separator.Length);
return source
.Do(x =>
{
if (queue.Count == separator.Length) queue.Dequeue();
queue.Enqueue(x);
if (queue.SequenceEqual(separator, comparer))
{
queue.Clear();
boundaries.OnNext(default);
}
})
.Concat(Observable.Repeat(default(T), separator.Length - 1))
.SkipLast(separator.Length - 1)
.Window(boundaries)
.Select((window, i) => i == 0 ? window : window.Skip(separator.Length));
});
}
A Subject is used for notifying the Window operator that a new boundary has been detected. The detection mechanism includes a Queue that holds the last emitted elements. This queue is compared to the separator every time a new element is emitted. The Window operator is lagging intentionally behind the detection mechanism by separator.Length - 1 elements, so that the resulting windows are aligned properly.
Usage example:
IObservable<byte> dataStream = GetDataStream();
IObservable<byte[]> messageStream = dataStream
.Window(new byte[] { 0xAA, 0xBB, 0xCC })
.SelectMany(window => window.ToArray());

Byte array initialisation syntax - specify first 20 bits and pad rest with 0s

My goal is to get a 64bit value hence a byte array of size 8. However my problem is that I want to set the first 20 bits myself and then have the rest to be 0s. Can this be done with the shorthand byte array initialisation?
E.g. if I wanted all 0s I would say:
byte[] test = new byte[] {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
What I've thought about/tried:
So each hexadecimal digit corresponds to 4 binary digits. Hence, if I want to specify the first 20bits, then I specify the first 5 hexadecimal digits? But I'm not sure of how to do this:
byte[] test = new byte[] {0xAF, 0x17, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00};
That would mean that I've specified the first 24 bits right? And not 20.
I could use a BitArray and do it that way but I'm just wondering whether it can be done in the above way.

How about:
byte byte1 = 0xFF;
byte byte2 = 0xFF;
byte byte3 = 0xFF;
// 8bits 8bits 4bits : total = 20 bits
// 11111111 11111111 11110000
byte[] test = new byte[] { byte1, byte2, (byte)(byte3 & 0xF0), 0x00, 0x00, 0x00, 0x00, 0x00 };

You can write your bytes backward, and use BitConverter.GetBytes(long):
var bytes = BitConverter.GetBytes(0x117AF);
Demo.
Since each hex digit corresponds to a single four-bit nibble, you can initialize data in "increments" of four bits. However, the data written in reverse will be almost certainly less clear to human readers of your code.

incorrect members order in a C# structure

I have a TCP Client,which puts a packet in a structure
using System.Runtime.InteropServices;
[StructLayoutAttribute(LayoutKind.Sequential)]
public struct tPacket_5000_E
{
public Int16 size;
public Int16 opcode;
public byte securityCount;
public byte securityCRC;
public byte flag;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 8, ArraySubType = UnmanagedType.I1)]
public byte[] blowfish;
public UInt32 seedCount;
public UInt32 seedCRC;
[MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 5, ArraySubType = UnmanagedType.I1)]
public UInt32[] seedsecurity;
}
The code I use to put the packet in the structure is:
tPacket_5000_E packet = new tPacket_5000_E();
GCHandle pin = GCHandle.Alloc(data, GCHandleType.Pinned);
packet = (tPacket_5000_E)Marshal.PtrToStructure(pin.AddrOfPinnedObject(), typeof(tPacket_5000_E));
pin.Free();
Now,before i continue I must tell you that I'm translating this project from C++ to C#.
This is the problem:
The last 3 members of tPacket_5000_E are Int32(i tried UInt32 too),which is DWORD in C++.
The values before those three members,which are NOT Int32,are equal to those in C++.(I inject same packet in both C++ and C# project).
However,those three members have different values.
in C++ the values are(correct):
seedCount:0x00000079
seedCRC:0x000000d1
SeedSecurity:
-[0]:0x548ac099
-1:0x03c4d378
-[2]:0x292e9eab
-[3]:0x4eee5ee3
-[4]:0x1071206e
in C# the values are(incorrect):
seedCount:0xd1000000
seedCRC:0x99000000
SeedSecurity:
-[0]: 0x78548ac0
-1: 0xab03c4d3
-[2]: 0xe3292e9e
-[3]: 0x6e4eee5e
-[4]: 0x00107120
The packet in both applications is equal
byte[] data = new byte[] {
0x25, 0x00, 0x00, 0x50, 0x00, 0x00, 0x0E, 0x10,
0xCE, 0xEF, 0x47, 0xDA, 0xC3, 0xFE, 0xFF, 0x79,
0x00, 0x00, 0x00, 0xD1, 0x00, 0x00, 0x00, 0x99,
0xC0, 0x8A, 0x54, 0x78, 0xD3, 0xC4, 0x03, 0xAB,
0x9E, 0x2E, 0x29, 0xE3, 0x5E, 0xEE, 0x4E, 0x6E,
0x20, 0x71, 0x10};
Click here for further information
Why the last three members in the struct are different and how to fix them?
Thanks in advance!

I'd expect that the root of your problem is that the three byte values
public byte securityCount;
public byte securityCRC;
public byte flag;
cause the next 32-bit values not to be word-aligned, and your two sets of code are adding (or not adding) internal padding differently.
I expect that the different packings look something like this:
C++ C#
================================ ================================
[size ][opcode ] [size ][opcode ]
[secCnt][secCrc][flag ][blow0 ] [secCnt][secCrc][flag ][blow0 ]
[blow1 ][blow2 ][blow3 ][blow4 ] [blow1 ][blow2 ][blow3 ][blow4 ]
[blow5 ][blow6 ][blow7 ][seedCou [blow5 ][blow6 ][blow7 ]..PAD...
nt ][seedCRC [seedCount ]
][seedSec [seedCRC ]
urity0 ][seedSec [seedSecurity0 ]
urity1 ][seedSec [seedSecurity1 ]
urity2 ][seedSec [seedSecurity2 ]
urity3 ][seedSec [seedSecurity3 ]
urity4 ] [seedSecurity4 ]
... with C# inserting a byte of padding which causes later values to be one byte off.
You can try using
[StructLayout(LayoutKind.Sequential,Pack=1)]
before your struct definition, which should use the minimum amount of space possible.
Mastering Structs in C# has some good information on how/why this happens.

I suspect that Daniel L is on the right track in his answer.
I would try adding a 4th byte after the flag. My guess is that your C++ compiler is aligning the values on word boundaries. That would "align" the C# version as well.