Develop Reference

How to read a binary file quickly in c#? (ReadOnlySpan vs MemoryStream)

I'm trying to parse a binary file as fastest as possible. So this is what I first tried to do:
using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
m.Position = 0;
using (BinaryReaderBigEndian b = new BinaryReaderBigEndian(m)) {
while (b.BaseStream.Position != b.BaseStream.Length) {
UInt32 value = b.ReadUInt32();
} } } } }
Where BinaryReaderBigEndian class is implemented as it follows:
public static class BinaryReaderBigEndian {
public BinaryReaderBigEndian(Stream stream) : base(stream) { }
public override UInt32 ReadUInt32() {
var x = base.ReadBytes(4);
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }
Then, I tried to get a performance improvement using ReadOnlySpan instead of MemoryStream. So, I tried doing:
using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
int position = 0;
ReadOnlySpan<byte> stream = new ReadOnlySpan<byte>(m.ToArray());
while (position != stream.Length) {
UInt32 value = stream.ReadUInt32(position);
position += 4;
} } } }
Where BinaryReaderBigEndian class changed in:
public static class BinaryReaderBigEndian {
public override UInt32 ReadUInt32(this ReadOnlySpan<byte> stream, int start) {
var data = stream.Slice(start, 4).ToArray();
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }
But, unfortunately, I didn't notice any improvement. So, where am I doing wrong?

I did some measurement of your code on my computer (Intel Q9400, 8 GiB RAM, SSD disk, Win10 x64 Home, .NET Framework 4/7/2, tested with 15 MB (when unpacked) file) with these results:
No-Span version: 520 ms
Span version: 720 ms
So Span version is actually slower! Why? Because new ReadOnlySpan<byte>(m.ToArray()) performs additional copy of whole file and also ReadUInt32() performs many slicings of the Span (slicing is cheap, but not free). Since you performed more work, you can't expect performance to be any better just because you used Span.
So can we do better? Yes. It turns out that the slowest part of your code is actually garbage collection caused by repeatedly allocating 4-byte Arrays created by the .ToArray() calls in ReadUInt32() method. You can avoid it by implementing ReadUInt32() yourself. It's pretty easy and also eliminates need for Span slicing. You can also replace new ReadOnlySpan<byte>(m.ToArray()) with new ReadOnlySpan<byte>(m.GetBuffer()).Slice(0, (int)m.Length);, which performs cheap slicing instead of copy of whole file. So now code looks like this:
public static void Read(FileInfo path)
{
using (FileStream filestream = path.OpenRead())
{
using (var d = new GZipStream(filestream, CompressionMode.Decompress))
{
using (MemoryStream m = new MemoryStream())
{
d.CopyTo(m);
int position = 0;
ReadOnlySpan<byte> stream = new ReadOnlySpan<byte>(m.GetBuffer()).Slice(0, (int)m.Length);
while (position != stream.Length)
{
UInt32 value = stream.ReadUInt32(position);
position += 4;
}
}
}
}
}
public static class BinaryReaderBigEndian
{
public static UInt32 ReadUInt32(this ReadOnlySpan<byte> stream, int start)
{
UInt32 res = 0;
for (int i = 0; i < 4; i++)
{
res = (res << 8) | (((UInt32)stream[start + i]) & 0xff);
}
return res;
}
}
With these changes I get from 720 ms down to 165 ms (4x faster). Sounds great, doesn't it? But we can do even better. We can completely avoid MemoryStream copy and inline and further optimize ReadUInt32():
public static void Read(FileInfo path)
{
using (FileStream filestream = path.OpenRead())
{
using (var d = new GZipStream(filestream, CompressionMode.Decompress))
{
var buffer = new byte[64 * 1024];
do
{
int bufferDataLength = FillBuffer(d, buffer);
if (bufferDataLength % 4 != 0)
throw new Exception("Stream length not divisible by 4");
if (bufferDataLength == 0)
break;
for (int i = 0; i < bufferDataLength; i += 4)
{
uint value = unchecked(
(((uint)buffer[i]) << 24)
| (((uint)buffer[i + 1]) << 16)
| (((uint)buffer[i + 2]) << 8)
| (((uint)buffer[i + 3]) << 0));
}
} while (true);
}
}
}
private static int FillBuffer(Stream stream, byte[] buffer)
{
int read = 0;
int totalRead = 0;
do
{
read = stream.Read(buffer, totalRead, buffer.Length - totalRead);
totalRead += read;
} while (read > 0 && totalRead < buffer.Length);
return totalRead;
}
And now it takes less than 90 ms (8x faster then the original!). And without Span! Span is great in situations, where it allows perform slicing and avoid array copy, but it won't improve performance just by blindly using it. After all, Span is designed to have performance characteristics on par with Array, but not better (and only on runtimes that have special support for it, such as .NET Core 2.1).

In C#, what's the best way to deal with partially received messages using SocketAsyncEventArgs Buffer

In order to clean some messy code and get a better understanding of the SocketAsyncEventArgs class, I'd to know what's the most efficient technique to reassemble partially received messages from SocketAsyncEventArgs buffers.
To give you the big picture, I'm connected to a TCP server using a C# Socket client that will essentially receive data. The data received is message-based delimited by a \n character.
As you're probably already aware of, when using the ReceiveAsync method, this is almost a certitude that the last received message will be uncompleted such as you'll have to locate the index of the last complete message, copy the incomplete buffer section and keep it as start for the next received buffer and so on.
The thing is, I wish to abstract this operation from the upper layer and call the ProcessReceiveDataImpl as soon I get completed messages in the _tmpBuffer. I found that my Buffer.BlockCopy is not much readable (very old code also (-:) but anyway I wish to know what are you doing in this typical use case?
Code to reassemble messages:
public class SocketClient
{
private const int _receiveBufferSize = 8192;
private byte[] _remBuffer = new byte[2 * _receiveBufferSize];
private byte[] _tmpBuffer = new byte[2 * _receiveBufferSize];
private int _remBufferSize = 0;
private int _tmpBufferSize = 0;
private void ProcessReceiveData(SocketAsyncEventArgs e)
{
// the buffer to process
byte[] curBuffer = e.Buffer;
int curBufferSize = e.BytesTransferred;
int curBufferOffset = e.Offset;
int curBufferLastIndex = e.BytesTransferred - 1;
int curBufferLastSplitIndex = int.MinValue;
if (_remBufferSize > 0)
{
curBufferLastSplitIndex = GetLastSplitIndex(curBuffer, curBufferOffset, curBufferSize);
if (curBufferLastSplitIndex != curBufferLastIndex)
{
// copy the remain + part of the current into tmp
Buffer.BlockCopy(_remBuffer, 0, _tmpBuffer, 0, _remBufferSize);
Buffer.BlockCopy(curBuffer, curBufferOffset, _tmpBuffer, _remBufferSize, curBufferLastSplitIndex + 1);
_tmpBufferSize = _remBufferSize + curBufferLastSplitIndex + 1;
ProcessReceiveDataImpl(_tmpBuffer, _tmpBufferSize);
Buffer.BlockCopy(curBuffer, curBufferLastSplitIndex + 1, _remBuffer, 0, curBufferLastIndex - curBufferLastSplitIndex);
_remBufferSize = curBufferLastIndex - curBufferLastSplitIndex;
}
else
{
// copy the remain + entire current into tmp
Buffer.BlockCopy(_remBuffer, 0, _tmpBuffer, 0, _remBufferSize);
Buffer.BlockCopy(curBuffer, curBufferOffset, _tmpBuffer, _remBufferSize, curBufferSize);
ProcessReceiveDataImpl(_tmpBuffer, _remBufferSize + curBufferSize);
_remBufferSize = 0;
}
}
else
{
curBufferLastSplitIndex = GetLastSplitIndex(curBuffer, curBufferOffset, curBufferSize);
if (curBufferLastSplitIndex != curBufferLastIndex)
{
// we must copy the unused byte into remaining buffer
_remBufferSize = curBufferLastIndex - curBufferLastSplitIndex;
Buffer.BlockCopy(curBuffer, curBufferLastSplitIndex + 1, _remBuffer, 0, _remBufferSize);
// process the msg
ProcessReceiveDataImpl(curBuffer, curBufferLastSplitIndex + 1);
}
else
{
// we can process the entire msg
ProcessReceiveDataImpl(curBuffer, curBufferSize);
}
}
}
protected virtual void ProcessReceiveDataImpl(byte[] buffer, int bufferSize)
{
}
private int GetLastSplitIndex(byte[] buffer, int offset, int bufferSize)
{
for (int i = offset + bufferSize - 1; i >= offset; i--)
{
if (buffer[i] == '\n')
{
return i;
}
}
return -1;
}
}
Your input is very important and appreciated!
Thank you!
Updated:
Also, rather then calling the ProcessReceiveDataImpl and block further receive operations, will it be useful to queue completed messages and make them available to the consumer?

WebSocket is not able to send large data

How can I send Large data to WebSocket in Java Script?
Using below code I am able to send data for 126 characters but not more then this. (Written in C#)
public static void SendData(string text)
{
foreach (SocketClient client in ClientList)
{
if (client.Client.Connected)
{
try
{
NetworkStream l_Stream = client.Client.GetStream();
List<byte> lb = new List<byte>();
lb = new List<byte>();
lb.Add(0x81);
int size = text.Length;
lb.Add((byte)size);
lb.AddRange(Encoding.UTF8.GetBytes(text));
l_Stream.Write(lb.ToArray(), 0, size + 2);
}
catch
{
CloseClient(client);
}
}
}
}
}
Can someone please help me? I tried to use so many things but none of them are working for me.
I am using Chrome 25 for the same.

This is how a websocket frame looks according to RFC 6455:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-------+-+-------------+-------------------------------+
|F|R|R|R| opcode|M| Payload len | Extended payload length |
|I|S|S|S| (4) |A| (7) | (16/64) |
|N|V|V|V| |S| | (if payload len==126/127) |
| |1|2|3| |K| | |
+-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
| Extended payload length continued, if payload len == 127 |
+ - - - - - - - - - - - - - - - +-------------------------------+
| |Masking-key, if MASK set to 1 |
+-------------------------------+-------------------------------+
| Masking-key (continued) | Payload Data |
+-------------------------------- - - - - - - - - - - - - - - - +
: Payload Data continued ... :
+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
| Payload Data continued ... |
+---------------------------------------------------------------+
You are setting the payload length of the websocket frame in byte 1 (the 2nd byte - first byte is byte 0). A byte can only have 256 states. But the first bit of byte 1 is used for the masking flag. So you can only represent values from 0 to 127.
When the payload length is larger than 125, you have to set byte 1 to 126 and put the length into byte 2 and 3 ("Extended payload length"). When your payload is even longer than 65535 bytes, you have to set byte 1 to 127 and put the payload length into the bytes 2-9 ("Extended payload length" and "Extended payload length continued"). When your payload length is even larger than 64bit (16 Exabyte, or about 16 million Terabyte), then... you should rather send a bunch of trucks filled with hard drives.
More details about that can be found in the official Websocket RFC.

You can send any size of data to WebSocket but large data will be fragmented.
Check this: https://datatracker.ietf.org/doc/html/draft-ietf-hybi-thewebsocketprotocol-17#section-5 and 5.4
and
pastebin.com/TxGZaHTY
and
http://buildnewgames.com/websockets/
for more info.

Referred from this blog. There could be a limit for single payload size to send and receive in the WebSocket server. For instance, in tomcat, by default it is 8192 bytes, checkout org.apache.tomcat.websocket.textBufferSize in tomcat's doc.
So, make sure if there is any payload size limit in your WebSocket server, if yes then receive it as partial payloads.

package GoodExample;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;
import java.util.HashMap;
import sun.misc.BASE64Encoder;
/**
*
* #author
* Anders, Improved by Christopher Price
*/
public class WebRTCSignal {
public static final int MASK_SIZE = 4;
public static final int SINGLE_FRAME_UNMASKED = 0x81;
private ServerSocket serverSocket;
private Socket socket;
public WebRTCSignal() throws IOException {
serverSocket = new ServerSocket(1337);
connect();
}
private void connect() throws IOException {
System.out.println("Listening");
socket = serverSocket.accept();
System.out.println("Got connection");
if(handshake()) {
listenerThread();
}
}
private boolean handshake() throws IOException {
PrintWriter out = new PrintWriter(socket.getOutputStream());
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
HashMap<String, String> keys = new HashMap<>();
String str;
//Reading client handshake
while (!(str = in.readLine()).equals("")) {
String[] s = str.split(": ");
System.out.println();
System.out.println(str);
if (s.length == 2) {
keys.put(s[0], s[1]);
}
}
//Do what you want with the keys here, we will just use "Sec-WebSocket-Key"
String hash;
try {
hash = new BASE64Encoder().encode(MessageDigest.getInstance("SHA- 1").digest((keys.get("Sec-WebSocket-Key") + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").getBytes()));
} catch (NoSuchAlgorithmException ex) {
ex.printStackTrace();
return false;
}
//Write handshake response
out.write("HTTP/1.1 101 Switching Protocols\r\n"
+ "Upgrade: websocket\r\n"
+ "Connection: Upgrade\r\n"
+ "Sec-WebSocket-Accept: " + hash + "\r\n"
+ "Origin: http://face2fame.com\r\n"
+ "\r\n");
out.flush();
return true;
}
private byte[] readBytes(int numOfBytes) throws IOException {
byte[] b = new byte[numOfBytes];
socket.getInputStream().read(b);
return b;
}
public void sendMessage(byte[] msg) throws IOException {
System.out.println("Sending to client");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
BufferedOutputStream os = new BufferedOutputStream(socket.getOutputStream());
baos.write(SINGLE_FRAME_UNMASKED);
baos.write(msg.length);
baos.write(msg);
baos.flush();
baos.close();
convertAndPrint(baos.toByteArray());
os.write(baos.toByteArray(), 0, baos.size());
os.flush();
}
public void listenerThread() {
Thread t = new Thread(new Runnable() {
#Override
public void run() {
try {
while (true) {
System.out.println("Recieved from client: " + reiceveMessage());
System.out.println("Enter data to send");
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
});
t.start();
}
public String reiceveMessage() throws IOException {
byte[] buf = readBytes(2);
//dont use this byte[] buf2 = readBytes(4);
int extendedsize = 0;
System.out.println("Headers:");
if(!convertAndPrintHeader(buf)){ // This means we detected an extended message
String CaryOverDetectiona = new String("");
byte[] bufadder1 = buf.clone();
byte[] bufadder2 = readBytes(2);
byte[] array1and2 = new byte[bufadder1.length + bufadder2.length];
System.arraycopy(bufadder1, 0, array1and2, 0, bufadder1.length);
System.arraycopy(bufadder2, 0, array1and2, bufadder1.length, bufadder2.length);
for (byte b : array1and2) {
CaryOverDetectiona = (String.format("%02X ", b));
System.out.print(CaryOverDetectiona);
}
int i = ((array1and2[1] & 0xFF) - 0x80);
//int c = ((array1and2[2] & 0xFF) - 0x80);
//System.out.println(c+128);
int j = ((array1and2[3] & 0xFF) - 0x80)+2;
//System.out.println("The size of this uber message is" + j);
extendedsize = i+j;
System.out.println("Extended Size is" + extendedsize);
//System.exit(0);
}
//convertAndPrint(buf2);// Check out the byte sizes
int opcode = buf[0] & 0x0F;
if (opcode == 8) {
//Client want to close connection!
System.out.println("Client closed!");
socket.close();
System.exit(0);
return null;
} else {
int payloadSize = getSizeOfPayload(buf[1]);
if (extendedsize>=126){
payloadSize = extendedsize;}
System.out.println("Payloadsize: " + payloadSize);
buf = readBytes(MASK_SIZE + payloadSize);
System.out.println("Payload:");
convertAndPrint(buf);
buf = unMask(Arrays.copyOfRange(buf, 0, 4), Arrays.copyOfRange(buf, 4, buf.length));
String message = new String(buf);
return message;
}
}
private int getSizeOfPayload(byte b) {
//Must subtract 0x80 from masked frames
int a = b & 0xff;
//System.out.println("PAYLOAD SIZE INT" + a);
return ((b & 0xFF) - 0x80);
}
private byte[] unMask(byte[] mask, byte[] data) {
for (int i = 0; i < data.length; i++) {
data[i] = (byte) (data[i] ^ mask[i % mask.length]);
}
return data;
}
private boolean convertAndPrintHeader(byte[] bytes) {
StringBuilder sb = new StringBuilder();
String CaryOverDetection = new String();
// We must test byte 2 specifically for this. In the next step we add length bytes perhaps?
//for(int i = 0; i < bytes.length; i++) {
//}
for (byte b : bytes) {
CaryOverDetection = (String.format("%02X ", b));
if (CaryOverDetection.contains("FE")){
return false;
}
sb.append(String.format("%02X ", b));
}
System.out.println(sb.toString());
return true;
}
private void convertAndPrint(byte[] bytes) {
StringBuilder sb = new StringBuilder();
for (byte b : bytes) {
sb.append(String.format("%02X ", b));
}
System.out.println(sb.toString());
}
public static void main(String[] args) throws IOException, InterruptedException, NoSuchAlgorithmException {
WebRTCSignal j = new WebRTCSignal();
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
while (true) {
System.out.println("Write something to the client!");
j.sendMessage(br.readLine().getBytes());
}
}
}

Byte array cryptography in C#

I want to create a nice cryptography using bitwise operators.
However I fail to do so.
I want it to have bitwise operators using a byte array to encrypt and decrypt my byte array.
public class Cryptographer
{
private byte[] Keys { get; set; }
public Cryptographer(string password)
{
Keys = Encoding.ASCII.GetBytes(password);
}
public void Encrypt(byte[] data)
{
for(int i = 0; i < data.Length; i++)
{
data[i] = (byte) (data[i] & Keys[i]);
}
}
public void Decrypt(byte[] data)
{
for (int i = 0; i < data.Length; i++)
{
data[i] = (byte)(Keys[i] & data[i]);
}
}
}
I know this is wrong, thats why I need help.
I simply want it to use 1 string to encrypt and decrypt all data.

This is what is sometimes known as 'craptography', because it provides the illusion of security while being functionally useless in protecting anything. Use the framework classes if you want to do cryptography right, because it's extremely difficult to roll your own.
Take a look at this for advice on what you are trying to do (encrypt/decrypt) - http://msdn.microsoft.com/en-us/library/e970bs09.aspx. Really your requirements should determine what classes you decide to use. This has good background: http://msdn.microsoft.com/en-us/library/92f9ye3s.aspx
For simple encrypt/decrypt (if this is what you need) DPAPI may be the simplest way.

You seem to be trying to implement the XOR cipher. XOR is ^ in C#:
public void Crypt(byte[] data)
{
for(int i = 0; i < data.Length; i++)
{
data[i] = (byte) (data[i] ^ Keys[i]);
} ↑
}
Since the Encrypt and Decrypt method do exactly the same, you need only one method.
Note, however, that this is just a toy and not suitable to secure data in real-world scenarios. Have a look at the System.Security.Cryptography Namespace which provides many implementations of proven algorithms. Using these correctly is still hard to get right though.

Use Xor ^ operator and not And &. Also you should not assume that data and key are the same length.
public class Cryptographer
{
private byte[] Keys { get; set; }
public Cryptographer(string password)
{
Keys = Encoding.ASCII.GetBytes(password);
}
public void Encrypt(byte[] data)
{
for(int i = 0; i < data.Length; i++)
{
data[i] = (byte) (data[i] ^ Keys[i % Keys.Length]);
}
}
public void Decrypt(byte[] data)
{
for (int i = 0; i < data.Length; i++)
{
data[i] = (byte)(Keys[i % Keys.Length] ^ data[i]);
}
}
}

static void Main(string[] args)
{
Int32 a = 138;
Console.WriteLine("first int: " + a.ToString());
byte[] bytes = BitConverter.GetBytes(a);
var bits = new BitArray(bytes);
String lol = ToBitString(bits);
Console.WriteLine("bit int: " + lol);
lol = lol.Substring(1, lol.Length - 1) + lol[0];
Console.WriteLine("left : " + lol);
byte[] bytes_new = GetBytes(lol);
byte[] key = { 12, 13, 24, 85 };
var bits2 = new BitArray(key);
String lol2 = ToBitString(bits2);
Console.WriteLine("key : " + lol2);
byte[] cryptedBytes = Crypt(bytes_new, key);
var bits3 = new BitArray(cryptedBytes);
String lol3 = ToBitString(bits3);
Console.WriteLine(" XOR: " + lol3);
byte[] deCryptedBytes = Crypt(cryptedBytes, key);
var bits4 = new BitArray(cryptedBytes);
String lol4 = ToBitString(bits4);
Console.WriteLine(" DEXOR: " + lol4);
int a_new = BitConverter.ToInt32(bytes_new, 0);
Console.WriteLine("and int: " + a_new.ToString());
Console.ReadLine();
}
public static byte[] Crypt(byte[] data, byte[] key)
{
byte[] toCrypt = data;
for (int i = 0; i < toCrypt.Length; i++)
{
toCrypt[i] = (byte)(toCrypt[i] ^ key[i]);
}
return toCrypt;
}
private static String ToBitString(BitArray bits)
{
var sb = new StringBuilder();
for (int i = bits.Count - 1; i >= 0; i--)
{
char c = bits[i] ? '1' : '0';
sb.Append(c);
}
return sb.ToString();
}
private static byte[] GetBytes(string bitString)
{
byte[] result = Enumerable.Range(0, bitString.Length / 8).
Select(pos => Convert.ToByte(
bitString.Substring(pos * 8, 8),
2)
).ToArray();
List<byte> mahByteArray = new List<byte>();
for (int i = result.Length - 1; i >= 0; i--)
{
mahByteArray.Add(result[i]);
}
return mahByteArray.ToArray();
}

Remember, there is no such thing as a 'secure' cipher. Any encryption method that can be written can be broken.
With that being said, using simple bitwise techniques for encryption is inviting a not too bright hacker to break your encryption. There are guys/gals that sit around all day long with nothing better to do.
Use one of the encryption libraries that uses a large key and do something 'unusual' to that key before using it. Even so, remember, there are people employed and not employed to do nothing but break cryptographic messages all around the world; 24 by 7.
The Germans thought they had an un-breakable system in WW II. They called it Enigma. Do some reading on it and you will discover that it was broken even before the war broke out!

Need help in translating C++ to C#

I don't know C++ very well especially the IO part. Can anyone please help me to translate the following C++ code into C#?
unsigned *PostingOffset, *PostingLength, NumTerms;
void LoadSubIndex(char *subindex) {
FILE *in = fopen(subindex, "rb");
if (in == 0) {
printf("Error opening sub-index file '%s'!\n", subindex);
exit(EXIT_FAILURE);
}
int len=0;
// Array of terms
char **Term;
char *TermList;
fread(&NumTerms, sizeof(unsigned), 1, in);
PostingOffset = (unsigned*)malloc(sizeof(unsigned) * NumTerms);
PostingLength = (unsigned*)malloc(sizeof(unsigned) * NumTerms);
Term = (char**)malloc(sizeof(char*) * NumTerms);
Term = (char**)malloc(sizeof(char*) * NumTerms);
// Offset of each posting
fread(PostingOffset, sizeof(unsigned), NumTerms, in);
// Length of each posting in bytes
fread(PostingLength, sizeof(unsigned), NumTerms, in);
// Number of bytes in the posting terms array
fread(&len, sizeof(unsigned), 1, in);
TermList = (char*)malloc(sizeof(char) * len);
fread(TermList, sizeof(unsigned)*len, 1, in);
unsigned k=1;
Term[0] = &TermList[0];
for (int i=1; i<len; i++) {
if (TermList[i-1] == '\0') {
Term[k] = &TermList[i];
k++;
}
}
fclose(in);
}
Thanks in advance.

I'll give you a headstart.
using(var reader = new BinaryReader(new FileStream(subindex, FileMode.Open)) {
int numTerms = reader.ReadUInt32();
postingOffset = new UInt32[numTerms];
postingLength = new UInt32[numTerms];
var term = new byte[numTerms];
for(int i=0;i<numTerms;i++)
postingOffset[i] = reader.ReadUInt32();
for(int i=0;i<numTerms;i++)
postingLength[i] = reader.ReadUInt32();
var len = reader.ReadInt32();
var termList = new ... // byte[] or uint32[] ??
//etc
}
There's no need to close the file handle here - it will close when the using { } block loses scope.
I didn't finish it because there are some flaws in your code. With TermList you are reading in 4 times as much data as you've allocated. You shouldn't be allocating Term twice either - that will result in leaking memory.
To turn Term back into a string, use Encoding.ASCII.GetString(term).TrimEnd('\0');