Develop Reference

Byte conversion to INT64, under the hood

Good day. For a current project I need to know how datatypes are represented as bytes. For example, if I use :
long three = 500;var bytes = BitConverter.GetBytes(three);
I get the values 244,1,0,0,0,0,0,0. I get that it is a 64 bit value, and 8 bits go int a bit, thus are there 8 bytes. But how does 244 and 1 makeup 500? I tried Googling it, but all I get is use BitConverter. I need to know how the bitconverter works under the hood. If anybody can perhaps point me to an article or explain how this stuff works, it would be appreciated.

It's quite simple.
BitConverter.GetBytes((long)1); // {1,0,0,0,0,0,0,0};
BitConverter.GetBytes((long)10); // {10,0,0,0,0,0,0,0};
BitConverter.GetBytes((long)100); // {100,0,0,0,0,0,0,0};
BitConverter.GetBytes((long)255); // {255,0,0,0,0,0,0,0};
BitConverter.GetBytes((long)256); // {0,1,0,0,0,0,0,0}; this 1 is 256
BitConverter.GetBytes((long)500); // {244,1,0,0,0,0,0,0}; this is yours 500 = 244 + 1 * 256
If you need source code you should check Microsoft GitHub since implementation is open source :)
https://github.com/dotnet

From the source code:
// Converts a long into an array of bytes with length
// eight.
[System.Security.SecuritySafeCritical] // auto-generated
public unsafe static byte[] GetBytes(long value)
{
Contract.Ensures(Contract.Result<byte[]>() != null);
Contract.Ensures(Contract.Result<byte[]>().Length == 8);
byte[] bytes = new byte[8];
fixed(byte* b = bytes)
*((long*)b) = value;
return bytes;
}

Byte[] to BitArray and back to Byte[]

As the title states, i'm trying to convert a byte array to bit array back to byte array again.
I am aware that Array.CopyTo() takes care of that but the byte array received is not the same as the original one due to how BitArray stores values in LSB.
How do you go about it in C#?

This should do it
static byte[] ConvertToByte(BitArray bits) {
// Make sure we have enough space allocated even when number of bits is not a multiple of 8
var bytes = new byte[(bits.Length - 1) / 8 + 1];
bits.CopyTo(bytes, 0);
return bytes;
}
You can verify it using a simple driver program like below
// test to make sure it works
static void Main(string[] args) {
var bytes = new byte[] { 10, 12, 200, 255, 0 };
var bits = new BitArray(bytes);
var newBytes = ConvertToByte(bits);
if (bytes.SequenceEqual(newBytes))
Console.WriteLine("Successfully converted byte[] to bits and then back to byte[]");
else
Console.WriteLine("Conversion Problem");
}
I know that the OP is aware of the Array.CopyTo solution (which is similar to what I have here), but I don't see why it's causing any Bit order issues. FYI, I am using .NET 4.5.2 to verify it. And hence I have provided the test case to confirm the results

To get a BitArray of byte[] you can simply use the constructor of BitArray:
BitArray bits = new BitArray(bytes);
To get the byte[] of the BitArray there are many possible solutions. I think a very elegant solution is to use the BitArray.CopyTo method. Just create a new array and copy the bits into:
byte[]resultBytes = new byte[(bits.Length - 1) / 8 + 1];
bits.CopyTo(resultBytes, 0);

Read and write more than 8 bit symbols

I am trying to write an Encoded file.The file has 9 to 12 bit symbols. While writing a file I guess that it is not written correctly the 9 bit symbols because I am unable to decode that file. Although when file has only 8 bit symbols in it. Everything works fine. This is the way I am writing a file
File.AppendAllText(outputFileName, WriteBackContent, ASCIIEncoding.Default);
Same goes for reading with ReadAllText function call.
What is the way to go here?
I am using ZXing library to encode my file using RS encoder.
ReedSolomonEncoder enc = new ReedSolomonEncoder(GenericGF.AZTEC_DATA_12);//if i use AZTEC_DATA_8 it works fine beacuse symbol size is 8 bit
int[] bytesAsInts = Array.ConvertAll(toBytes.ToArray(), c => (int)c);
enc.encode(bytesAsInts, parity);
byte[] bytes = bytesAsInts.Select(x => (byte)x).ToArray();
string contentWithParity = (ASCIIEncoding.Default.GetString(bytes.ToArray()));
WriteBackContent += contentWithParity;
File.AppendAllText(outputFileName, WriteBackContent, ASCIIEncoding.Default);
Like in the code I am initializing my Encoder with AZTEC_DATA_12 which means 12 bit symbol. Because RS Encoder requires int array so I am converting it to int array. And writing to file like here.But it works well with AZTEC_DATA_8 beacue of 8 bit symbol but not with AZTEC_DATA_12.

Main problem is here:
byte[] bytes = bytesAsInts.Select(x => (byte)x).ToArray();
You are basically throwing away part of the result when converting the single integers to single bytes.
If you look at the array after the call to encode(), you can see that some of the array elements have a value higher than 255, so they cannot be represented as bytes. However, in your code quoted above, you cast every single element in the integer array to byte, changing the element when it has a value greater than 255.
So to store the result of encode(), you have to convert the integer array to a byte array in a way that the values are not lost or modified.
In order to make this kind of conversion between byte arrays and integer arrays, you can use the function Buffer.BlockCopy(). An example on how to use this function is in this answer.
Use the samples from the answer and the one from the comment to the answer for both conversions: Turning a byte array to an integer array to pass to the encode() function and to turn the integer array returned from the encode() function back into a byte array.
Here are the sample codes from the linked answer:
// Convert byte array to integer array
byte[] result = new byte[intArray.Length * sizeof(int)];
Buffer.BlockCopy(intArray, 0, result, 0, result.Length);
// Convert integer array to byte array (with bugs fixed)
int bytesCount = byteArray.Length;
int intsCount = bytesCount / sizeof(int);
if (bytesCount % sizeof(int) != 0) intsCount++;
int[] result = new int[intsCount];
Buffer.BlockCopy(byteArray, 0, result, 0, byteArray.Length);
Now about storing the data into files: Do not turn the data into a string directly via Encoding.GetString(). Not all bit sequences are valid representations of characters in any given character set. So, converting a random sequence of random bytes into a string will sometimes fail.
Instead, either store/read the byte array directly into a file via File.WriteAllBytes() / File.ReadAllBytes() or use Convert.ToBase64() and Convert.FromBase64() to work with a base64 encoded string representation of the byte array.
Combined here is some sample code:
ReedSolomonEncoder enc = new ReedSolomonEncoder(GenericGF.AZTEC_DATA_12);//if i use AZTEC_DATA_8 it works fine beacuse symbol size is 8 bit
int[] bytesAsInts = Array.ConvertAll(toBytes.ToArray(), c => (int)c);
enc.encode(bytesAsInts, parity);
// Turn int array to byte array without loosing value
byte[] bytes = new byte[bytesAsInts.Length * sizeof(int)];
Buffer.BlockCopy(bytesAsInts, 0, bytes, 0, bytes.Length);
// Write to file
File.WriteAllBytes(outputFileName, bytes);
// Read from file
bytes = File.ReadAllBytes(outputFileName);
// Turn byte array to int array
int bytesCount = bytes.Length * 40;
int intsCount = bytesCount / sizeof(int);
if (bytesCount % sizeof(int) != 0) intsCount++;
int[] dataAsInts = new int[intsCount];
Buffer.BlockCopy(bytes, 0, dataAsInts, 0, bytes.Length);
// Decoding
ReedSolomonDecoder dec = new ReedSolomonDecoder(GenericGF.AZTEC_DATA_12);
dec.decode(dataAsInts, parity);

Most efficient way to save binary code to file

I have a string that only contains 1 and 0 and I need to save this to a .txt-File.
I also want it to be as small as possible. Since I have binary code, I can turn it into pretty much everything. Saving it as binary is not an option, since apparently every character will be a whole byte, even if it's a 1 or a 0.
I thought about turning my string into an Array of Byte but trying to convert "11111111" to Byte gave me a System.OverflowException.
My next thought was using an ASCII Codepage or something. But I don't know how reliable that is. Alternatively I could turn all of the 8-Bit pieces of my string into the corresponding numbers. 8 characters would turn into a maximum of 3 (255), which seems pretty nice to me. And since I know the highest individual number will be 255 I don't even need any delimiter for decoding.
But I'm sure there's a better way.
So:
What exactly is the best/most efficient way to store a string that only contains 1 and 0?

You could represent all your data as 64 bit integers and then write them to a binary file:
// The string we are working with.
string str = #"1010101010010100010101101";
// The number of bits in a 64 bit integer!
int size = 64;
// Pad the end of the string with zeros so the length of the string is divisible by 64.
str += new string('0', str.Length % size);
// Convert each 64 character segment into a 64 bit integer.
long[] binary = new long[str.Length / size]
.Select((x, idx) => Convert.ToInt64(str.Substring(idx * size, size), 2)).ToArray();
// Copy the result to a byte array.
byte[] bytes = new byte[binary.Length * sizeof(long)];
Buffer.BlockCopy(binary, 0, bytes, 0, bytes.Length);
// Write the result to file.
File.WriteAllBytes("MyFile.bin", bytes);
EDIT:
If you're only writing 64 bits then it's a one-liner:
File.WriteAllBytes("MyFile.bin", BitConverter.GetBytes(Convert.ToUInt64(str, 2)));

I would suggest using BinaryWriter. Like this:
BinaryWriter writer = new BinaryWriter(File.Open(fileName, FileMode.Create));

What is the Fastest way to convert byte[] to float[] and vice versa?

Which is the fastest way to convert a byte[] to float[] and vice versa (without a loop of course).
I'm using BlockCopy now, but then I need the double memory. I would like some kind of cast.
I need to do this conversion just to send the data through a socket and reconstruct the array in the other end.

Surely msarchet's proposal makes copies too. You are talking about just changing the way .NET thinks about a memory area, if you dont' want to copy.
But, I don't think what you want is possible, as bytes and floats are represented totally different in memory. A byte uses exactly a byte in memory, but a float uses 4 bytes (32 bits).
If you don't have the memory requirements to store your data, just represent the data as the data type you will be using the most in memory, and convert the values you actually use, when you use them.
How do you want to convert a float (which can represent a value between ±1.5 × 10−45 and±3.4 × 10^38) into a byte (which can represent a value between 0 and 255) anyway?
(see more info her about:
byte: http://msdn.microsoft.com/en-us/library/5bdb6693(v=VS.100).aspx
float: http://msdn.microsoft.com/en-us/library/b1e65aza.aspx
More about floating types in .NET here: http://csharpindepth.com/Articles/General/FloatingPoint.aspx

You can use StructLayout to achieve this (from Stack Overflow question C# unsafe value type array to byte array conversions):
[StructLayout(LayoutKind.Explicit)]
struct UnionArray
{
[FieldOffset(0)]
public Byte[] Bytes;
[FieldOffset(0)]
public float[] Floats;
}
static void Main(string[] args)
{
// From bytes to floats - works
byte[] bytes = { 0, 1, 2, 4, 8, 16, 32, 64 };
UnionArray arry = new UnionArray { Bytes = bytes };
for (int i = 0; i < arry.Bytes.Length / 4; i++)
Console.WriteLine(arry.Floats[i]);
}

IEnumerable<float> ToFloats(byte[] bytes)
{
for(int i = 0; i < bytes.Length; i+=4)
yield return BitConverter.ToSingle(bytes, i);
}

Two ways if you have access to LINQ:
var floatarray = ByteArry.AsEnumerable.Cast<float>().ToArray();
or just using Array Functions
var floatarray = Array.ConvertAll(ByteArray, item => (float)item);