Develop Reference

What's the fastest way to convert a float[] to byte[]?

OpenCV uses floats to store SIFT descriptors, where actually it is integer values from 0 to 255.
I am using OpenCvSharp, which is a C# wrapper for OpenCV and I would like to convert the float-based descriptors to byte[] because it takes only 1/4 of the space.
Before I realized that, I was converting the float[] to byte[] like this to store the descriptors in a database:
float[] floatDescriptor = ...;
byte[] byteDescriptor = new byte[floatDescriptor.Length * sizeof(float)];
Buffer.BlockCopy(floatDescriptor, 0, byteDescriptor, 0, byteDescriptor.Length);
This was very fast, because I could copy the whole float array without any transformation into a byte array. But it takes 4 times more space than this:
float[] floatDescriptor = ...;
byte[] byteDescriptor = new byte[floatDescriptor.Length];
for (int i = 0; i < floatDescriptor.Length; ++i)
{
byteDescriptor [i] = (byte)floatDescriptor[i];
}
But this is a lot slower. Is there a faster way to do it?
Edit
I am aware of the fact, that two different things are happening there. I'm just wondering if there is some kind of faster way for batch processing casts in arrays. Like Buffer.BlockCopy() is faster than BitConverter.GetBytes(float) for every float in an array.

Understanding the Size of Byte Arrays

I was reading this popular stack overflow question Creating a byte array from a stream and wanted to get some clarification on how byte arrays work.
in this chunk of code here:
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = PictureStream.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
Here's what I'm not understanding:
I'm getting lost on the size that this array is set to. For example, I use that code chunk to convert an image stream to a byte array, but i'm usually reading images that are larger than 2 megabytes, which is far larger than the size of the array that's reading in the picture- 16*1024 bytes. However, the above code converts the image from a stream to a byte array totally fine, no "out of bounds index" errors to be had.
So how is my array a smaller size than the photo I'm reading in, yet still manages to read it totally fine?

The array you pass is just a buffer. When you read from the stream it returns the number of bytes read and populates the buffer array with that many elements (it is not always fully filled). Then you write that many bytes to the memory stream. This process is repeated until there are no more bytes to read from the file.
You will notice that the array produced by ToArray is much larger than your buffer size.

As already mentioned in the comments.
The function read of Picture stream only reads a chunk of data,
actually exactly the amount which the transport buffer has.
The we read this amount we write it to the output stream from the transport buffer.
I tried to write some code snipped to demonstrates what is going on:
int inputBufferSizeInByte = 1024 * 1000 * 5; // 5 MiB = 5000 KiB
// AmountKiloByte * factor MiB * factorWhatWeWant
Byte[] inputBuffer = new Byte[inputBufferSizeInByte];
//we fill our inputBuffer with random numbers
Random rnd = new Random();
rnd.NextBytes(inputBuffer);
//we define our streams
MemoryStream inputMemoryStream = new MemoryStream(inputBuffer);
MemoryStream outPutMemoryStream = new MemoryStream();
//we define a smaller buffer for reading
int transportBufferSizeInByte = 1024 * 16; // 16 KiB
byte[] transportBufferFor = new byte[transportBufferSizeInByte];
int amountTotalWeReadInByte = 0;
int tempReadAmountInByte = 0;
int callWriteCounter = 0;
do
{
tempReadAmountInByte = inputMemoryStream.Read(transportBufferFor, 0, transportBufferSizeInByte);
//we write what we got to the output
if(tempReadAmountInByte>0)
{
outPutMemoryStream.Write(transportBufferFor, 0, tempReadAmountInByte);
callWriteCounter++;
}
//we calc how the total amout
amountTotalWeReadInByte += tempReadAmountInByte;
} while (tempReadAmountInByte > 0);
//we sum up
Console.WriteLine("input buffer size: \t" + inputBufferSizeInByte + " \t in Byte");
Console.WriteLine("total amount read \t" + amountTotalWeReadInByte + " \t in Byte");
Console.WriteLine("output stream size: \t" + outPutMemoryStream.Length + " \t in Byte");
Console.WriteLine("called strean write \t" + callWriteCounter + "\t\t times");
output:
input buffer size: 5120000 in Byte
total amount read 5120000 in Byte
output stream size: 5120000 in Byte
called strean write 313 times
So we call 313 times the stream write function and everthing behaves like it should.
That's brings me to key question:
why is there in size difference between the picture in memory and in hard disk ?
I do think the picture encoding is the reason.
The difference of the size of a picture on the hard disk and its memory representation belongs often
to the picture encoding. I know this fact from working with the cpp library opencv.
I rather guess the c# implementation behaves similar.
See some Q/A about this topic:
[question]: JPEG image memory byte size from OpenCV imread doesn't seem right

How to get file type from encrypted file?

How do I get the file type using C# from an encrypted file (i.e. file.enc)?
Encryption Method: Shift Cipher Z256
Shift Cipher Encryption:
Yi = (Xi + k) % 256
Xi = (Yi - k) % 256
Where:
Xi, i = 1 : n, is the input in plain bytes.
Yi, i = 1 : n, is the output cipher bytes.
k is the shift key which is a secret byte between 1 and 255.
If I have to decrypt the file first, how could I decrypt it without using exhaustive search to find the shift key?
I'm not talking about getting .enc as I can already easily do that. I'm not able to determine how the file was before encryption such as .doc, .xls, .pdf, .jpg, or .wav file types.
What I have tried:
byte[] byteArray = File.ReadAllBytes(openFileDialog1.FileName);
// Mean
double mean = 0;
for (int i = 0; i < byteArray.Length; i++)
{
mean += byteArray[i];
}
mean = mean / byteArray.Length;
txtMean.Text = mean.ToString("#.000");
// Median
byteArray.ToList().Sort();
int median = byteArray[(int)Math.Floor((decimal)(byteArray.Length / 2))];
txtMedian.Text = median.ToString();
// Mode
var groups = byteArray.GroupBy(g => g);
int maxCount = groups.Max(g => g.Count());
int mode = groups.First(g => g.Count() == maxCount).Key;
txtMode.Text = mode.ToString();
// Standard Deviation
double standardDeviation = byteArray.Select(value => (value - mean) * (value - mean)).Sum();
standardDeviation = Math.Sqrt(standardDeviation / byteArray.Length);
txtStandardDeviation.Text = standardDeviation.ToString("#.000");
// Entropy (I don't know how to get this part.)
int entropy = 0;
txtEntropy.Text = entropy.ToString();
So, from this you can see I take the file, read all bytes of the file and find the mean, median, mode, standard deviation, and entropy values.
By the way, I don't know how to find the entropy value for the file, is there some formula for this or maybe a C# built-in method? I have searched, but found nothing.
I thought using the mode value would be able to determine the file type, but rather it only determines .pdf files as .pdf files have a mode of 48.
.doc, .xls, .docx, .xlsx, .jpg, and .wav files all give me a mode of 0.
I have also tried reading the bytes using the following page(s):
ASCII Character Codes Chart 1 - https://msdn.microsoft.com/en-us/library/60ecse8t(v=vs.80).aspx
ASCII Character Codes Chart 2 - https://msdn.microsoft.com/en-us/library/9hxt0028(v=vs.80).aspx
using this code:
string str = Encoding.ASCII.GetString(byteArray).Substring(0, 256);
but it just returns gibberish in which I am unable to determine the difference in file types.

If it is a 'Caesar Shift', then you just run down the alphabet, trying each possible shift, there are only 25 of them.
NBCM CM UH YRUGJFY
nbcm cm uh yrugjfy
ocdn dn vi zsvhkgz
pdeo eo wj atwilha
qefp fp xk buxjmib
rfgq gq yl cvyknjc
sghr hr zm dwzlokd
this is an example
uijt jt bo fybnqmf
vjku ku cp gzcorng
wklv lv dq hadpsoh
xlmw mw er ibeqtpi
ymnx nx fs jcfruqj
znoy oy gt kdgsvrk
aopz pz hu lehtwsl
bpqa qa iv mfiuxtm
cqrb rb jw ngjvyun
drsc sc kx ohkwzvo
estd td ly pilxawp
ftue ue mz qjmybxq
guvf vf na rknzcyr
hvwg wg ob sloadzs
iwxh xh pc tmpbeat
jxyi yi qd unqcfbu
kyzj zj re vordgcv
lzak ak sf wpsehdw
mabl bl tg xqtfiex
nbcm cm uh yrugjfy

Fastest way to convert float to bytes and then save byte array in memory?

I am currently writing code that converts an audio clip into a float array and then want to convert that float array into bytes, and finally convert that byte array to hexadecimal.
Everything works but we are attempting to save arrays of data that are hundreds of thousands of elements long when this data is converted to bytes and once we try to save this data as a hexadecimal string it is a bit much or takes too long for the mobile devices we are testing on to handle.
So my question is are there any ways to optimize / speed up this process?
Here is my code for Convert our float array to bytes:
public byte[] ConvertFloatsToBytes(float[] audioData){
byte[] bytes = new byte[audioData.Length * 4];
//*** This function converts our current float array elements to the same exact place in byte data
Buffer.BlockCopy(audioData,0,bytes,0,bytes.Length);
return bytes;
}
Here we convert that data into a hex string :
public static string ByteArrayToString(byte[] ba)
{
string hex = BitConverter.ToString(ba);
//Debug.Log("ba.length = " + ba.Length.ToString() +"hex string = " + hex);
return hex.Replace("-","");
}
Ultimately at the end we save the string out and convert it from the hex string to a float array .
Like I said that code is slow but it is working I am just trying to find the best ways to optimize / speed up this process to improve performance

Do you know which part is costing you? I strongly suspect that the conversion to a hexadecimal array isn't the bottleneck in your program.
The final part, where you remove the hyphens ends up copying the string. You can probably do better by writing your own method that duplicates what BitArray.ToString does, without the hyphens. That is:
const string chars = "0123456789ABCDEF";
public string ByteArrayToString(byte[] ba)
{
var sb = new StringBuilder(ba.Length*2);
for (int i = 0; i < ba.Length; ++i)
{
var b = ba[i];
sb.Append(chars[b >> 4]);
sb.Append(chars[b & 0x0F]);
}
return sb.ToString();
}
That will avoid one string copy.
If you're willing to use unsafe code (don't know if you can on the devices you're working with), you can speed that even further by not even copying to the array of bytes. Rather, you fix the array of floats in memory and then address it with a byte pointer See Unsafe Code and Pointers if you're interested in that.

That sounds really convoluted - are audio samples not normally integers?
Anyway, StreamWriter supports writing of single and double natively, so you could use that to build a memory stream that you then convert to hex.

Creating a .wav File in C#

As an excuse to learn C#, I have been trying to code a simple project: creating audio files. To start, I want to make sure that I can write files that meet the WAVE format. I have researched the format online (for example, here), but whenever I try to play back a file, it won't open correctly. Here is my code. Is something missing or incorrect?
uint numsamples = 44100;
ushort numchannels = 1;
ushort samplelength = 1; // in bytes
uint samplerate = 22050;
FileStream f = new FileStream("a.wav", FileMode.Create);
BinaryWriter wr = new BinaryWriter(f);
wr.Write("RIFF");
wr.Write(36 + numsamples * numchannels * samplelength);
wr.Write("WAVEfmt ");
wr.Write(16);
wr.Write((ushort)1);
wr.Write(numchannels);
wr.Write(samplerate);
wr.Write(samplerate * samplelength * numchannels);
wr.Write(samplelength * numchannels);
wr.Write((ushort)(8 * samplelength));
wr.Write("data");
wr.Write(numsamples * samplelength);
// for now, just a square wave
Waveform a = new Waveform(440, 50);
double t = 0.0;
for (int i = 0; i < numsamples; i++, t += 1.0 / samplerate)
{
wr.Write((byte)((a.sample(t) + (samplelength == 1 ? 128 : 0)) & 0xff));
}

The major problem is:
BinaryWriter.Write(string) writes a string that is prefixed with it's length for BinaryReader to read it back. It is not intended to be used like your case. You need to write the bytes directly instead of using BinaryWriter.Write(string).
What you should do:
Convert the string into bytes and then write the bytes directly.
byte[] data = System.Text.Encoding.ASCII.GetBytes("RIFF");
binaryWriter.Write(data);
or make it one line:
binaryWriter.Write(System.Text.Encoding.ASCII.GetBytes("RIFF"));
There may also be other problems, like the integers you are writing may not be of the same size as required. You should check them carefully.
As for endianess, the link you put states that data are in little-endian and BinaryWriter uses little-endian, so this should not be a problem.

The simplest way possible, you can simply change:
wr.Write("RIFF");
to:
wr.Write("RIFF".ToArray());
Writing a string in a binary file, it will include the length of the string so that it can be deserialized back into a string later. In this case you just want the four bytes to be written as four bytes, and converting it to a char array will do just that.

I lack the proper WAV data, but try replacing the part of your code where you generate the header with this code (replace appropriately):
wr.Write(Encoding.ASCII.GetBytes("RIFF"));
wr.Write(0);
wr.Write(Encoding.ASCII.GetBytes("WAVE"));
wr.Write(Encoding.ASCII.GetBytes("fmt "));
wr.Write(18 + (int)(numsamples * samplelength));
wr.Write((short)1); // Encoding
wr.Write((short)numchannels); // Channels
wr.Write((int)(samplerate)); // Sample rate
wr.Write((int)(samplerate * samplelength * numchannels)); // Average bytes per second
wr.Write((short)(samplelength * numchannels)); // block align
wr.Write((short)(8 * samplelength)); // bits per sample
wr.Write((short)(numsamples * samplelength)); // Extra size
wr.Write("data");

#Alvin-wong answer works perfect. Just wanted to add another suggestion although a few more lines is:
binaryWriter.Write('R');
binaryWriter.Write('I');
binaryWriter.Write('F');
binaryWriter.Write('F');