Develop Reference

Most efficient way to save binary code to file

I have a string that only contains 1 and 0 and I need to save this to a .txt-File.
I also want it to be as small as possible. Since I have binary code, I can turn it into pretty much everything. Saving it as binary is not an option, since apparently every character will be a whole byte, even if it's a 1 or a 0.
I thought about turning my string into an Array of Byte but trying to convert "11111111" to Byte gave me a System.OverflowException.
My next thought was using an ASCII Codepage or something. But I don't know how reliable that is. Alternatively I could turn all of the 8-Bit pieces of my string into the corresponding numbers. 8 characters would turn into a maximum of 3 (255), which seems pretty nice to me. And since I know the highest individual number will be 255 I don't even need any delimiter for decoding.
But I'm sure there's a better way.
So:
What exactly is the best/most efficient way to store a string that only contains 1 and 0?

You could represent all your data as 64 bit integers and then write them to a binary file:
// The string we are working with.
string str = #"1010101010010100010101101";
// The number of bits in a 64 bit integer!
int size = 64;
// Pad the end of the string with zeros so the length of the string is divisible by 64.
str += new string('0', str.Length % size);
// Convert each 64 character segment into a 64 bit integer.
long[] binary = new long[str.Length / size]
.Select((x, idx) => Convert.ToInt64(str.Substring(idx * size, size), 2)).ToArray();
// Copy the result to a byte array.
byte[] bytes = new byte[binary.Length * sizeof(long)];
Buffer.BlockCopy(binary, 0, bytes, 0, bytes.Length);
// Write the result to file.
File.WriteAllBytes("MyFile.bin", bytes);
EDIT:
If you're only writing 64 bits then it's a one-liner:
File.WriteAllBytes("MyFile.bin", BitConverter.GetBytes(Convert.ToUInt64(str, 2)));

I would suggest using BinaryWriter. Like this:
BinaryWriter writer = new BinaryWriter(File.Open(fileName, FileMode.Create));

When using Itext or itextsharp, how do determine the length to set a digital cert buffer length?

I am working with itextsharp (but this question is probably applies to itext also) and I am adding digital signatures to PDFs. I have done reading and I understand that the length of a digital signature will vary based on LTV, and a hole host of other factors, so you typically will allocate an overly large buffer to hold the certificate information when you add the signature to a document to insure that it will have sufficient space.
The thing that is puzzling to me is I see examples all over the net where a sig stamper is set up like this:
Dictionary<PdfName, int> exc = new Dictionary<PdfName, int>();
exc.Add(PdfName.CONTENTS, BUFFER_SIZE * 2 + 2);
sap.PreClose(exc);
and then later is zeroed out
byte[] signature_buffer = new byte[BUFFER_SIZE];
int index = 0;
while (index < signature_buffer.Length)
signature_buffer[index++] = 0x20;
PdfDictionary dic2 = new PdfDictionary();
dic2.Put(PdfName.CONTENTS, new PdfString(signature_buffer).SetHexWriting(true));
Why do we create the initial dictionary entry with a buffer length * 2 + 2? Why isn't it the same size as the one used by the PdfDictionary? Is a case where sloppy code has just been copied all over the place, or is there a deeper reason?

Why do we create the initial dictionary entry with a buffer length * 2 + 2?
This is explained in the method comment of PdfSignatureAppearance.preClose in iText (it likely is identical in iTextSharp):
/**
* This is the first method to be called when using external signatures. The general sequence is:
* preClose(), getDocumentBytes() and close().
* <p>
* If calling preClose() <B>dont't</B> call PdfStamper.close().
* <p>
* <CODE>exclusionSizes</CODE> must contain at least
* the <CODE>PdfName.CONTENTS</CODE> key with the size that it will take in the
* document. Note that due to the hex string coding this size should be
* byte_size*2+2.
* #param exclusionSizes a <CODE>HashMap</CODE> with names and sizes to be excluded in the signature
* calculation. The key is a <CODE>PdfName</CODE> and the value an
* <CODE>Integer</CODE>. At least the <CODE>PdfName.CONTENTS</CODE> must be present
* #throws IOException on error
* #throws DocumentException on error
*/
public void preClose(HashMap<PdfName, Integer> exclusionSizes) throws IOException, DocumentException
As explained here, due to the hex string coding this size should be byte_size*2+2 because the size that the signature container will take in the document is
twice the length in bytes (due to hex encoding)
plus 2 (for the opening and closing angled brackets enclosing hex encoded strings in PDF).

Fastest way to convert float to bytes and then save byte array in memory?

I am currently writing code that converts an audio clip into a float array and then want to convert that float array into bytes, and finally convert that byte array to hexadecimal.
Everything works but we are attempting to save arrays of data that are hundreds of thousands of elements long when this data is converted to bytes and once we try to save this data as a hexadecimal string it is a bit much or takes too long for the mobile devices we are testing on to handle.
So my question is are there any ways to optimize / speed up this process?
Here is my code for Convert our float array to bytes:
public byte[] ConvertFloatsToBytes(float[] audioData){
byte[] bytes = new byte[audioData.Length * 4];
//*** This function converts our current float array elements to the same exact place in byte data
Buffer.BlockCopy(audioData,0,bytes,0,bytes.Length);
return bytes;
}
Here we convert that data into a hex string :
public static string ByteArrayToString(byte[] ba)
{
string hex = BitConverter.ToString(ba);
//Debug.Log("ba.length = " + ba.Length.ToString() +"hex string = " + hex);
return hex.Replace("-","");
}
Ultimately at the end we save the string out and convert it from the hex string to a float array .
Like I said that code is slow but it is working I am just trying to find the best ways to optimize / speed up this process to improve performance

Do you know which part is costing you? I strongly suspect that the conversion to a hexadecimal array isn't the bottleneck in your program.
The final part, where you remove the hyphens ends up copying the string. You can probably do better by writing your own method that duplicates what BitArray.ToString does, without the hyphens. That is:
const string chars = "0123456789ABCDEF";
public string ByteArrayToString(byte[] ba)
{
var sb = new StringBuilder(ba.Length*2);
for (int i = 0; i < ba.Length; ++i)
{
var b = ba[i];
sb.Append(chars[b >> 4]);
sb.Append(chars[b & 0x0F]);
}
return sb.ToString();
}
That will avoid one string copy.
If you're willing to use unsafe code (don't know if you can on the devices you're working with), you can speed that even further by not even copying to the array of bytes. Rather, you fix the array of floats in memory and then address it with a byte pointer See Unsafe Code and Pointers if you're interested in that.

That sounds really convoluted - are audio samples not normally integers?
Anyway, StreamWriter supports writing of single and double natively, so you could use that to build a memory stream that you then convert to hex.

How can i convert stereo pcm samples to mono samples using naudio

How can i convert stereo pcm samples to mono samples using naudio?
or converting an stereo mp3 file to mono raw samples!
i try this befor :
for (int u = 0; u < output.Length; u+=4)
{
byte[] Lbuffer = new byte[2];
byte[] Rbuffer = new byte[2];
Lbuffer[0] = output[u + 0];
Lbuffer[1] = output[u + 1];
Rbuffer[0] = output[u + 2];
Rbuffer[1] = output[u + 3];
Int16 leftSample = BitConverter.ToInt16(Lbuffer, 0);
Int16 rightSample = BitConverter.ToInt16(Rbuffer, 0);
Int16 mixedMono = (Int16)(0.5f * (float)leftSample + (float)rightSample);
Byte[] mixedMonoBytes = BitConverter.GetBytes(mixedMono);
mono[counter] = mixedMonoBytes[0];
mono[counter+1] = mixedMonoBytes[1];
//mono[counter] = Convert.ToByte((Convert.ToInt16(buffer[0]) + Convert.ToInt16(buffer[2]))/2);
//mono[counter+1] = Convert.ToByte((Convert.ToInt16(buffer[0]) + Convert.ToInt16(buffer[2]))/2);
counter += 2;
}
but it does not work currently! it result has noises!
output is an array that contains raw samples!

As #daniel-s pointed out, to convert PCM samples from stereo (2 channels) to mono (1 channel), you can simply take the average of the two channels: for every sample, take the value from the left channel, add the value from the right channel, and divide by 2. Use saturation arithmetic to avoid overflows.
To convert an MP3 file to raw (PCM) samples, you need to run the MP3 file through a file parser and MP3 bitstream decoder. There are many libraries and applications out there to do this; for example, see FFmpeg.
[Edit] I forgot to mention, more importantly, NAudio supports decoding MP3 files through either ACM or DMO codec; see NAudio - MP3 for examples.

CRC-16 and CRC-32 Checks

I need help trying to verify CRC-16 values (also need help with CRC-32 values). I tried to sit down and understand how CRC works but I am drawing a blank.
My first problem is when trying to use an online calculator for calculating the message "BD001325E032091B94C412AC" into CRC16 = 12AC. The documentation states that the last two octets are the CRC16 value, so I am inputting "BD001325E032091B94C4" into the site http://www.lammertbies.nl/comm/info/crc-calculation.html and receive 5A90 as the result instead of 12AC.
Does anybody know why these values are different and where I can find code for how to calculate CRC16 and CRC32 values (I plan to later learn how to do this but times doesn't allow right now)?
Some more messages are as following:
16000040FFFFFFFF00015FCB
3C00003144010405E57022C7
BA00001144010101B970F0ED
3900010101390401B3049FF1
09900C800000000000008CF3
8590000000000000000035F7
00900259025902590259EBC9
0200002B00080191014BF5A2
BB0000BEE0014401B970E51E
3D000322D0320A2510A263A0
2C0001440000D60000D65E54
--Edit--
I have included more information. The documentation I was referencing is TIA-102.BAAA-A (from the TIA standard). The following is what the documentation states (trying to avoid copyright infringement as much as possible):
The Last Block in a packet comprises several octets of user information and / or
pad octets, followed by a 4-octet CRC parity check. This is referred to as the
packet CRC.
The packet CRC is a 4-octet cyclic redundancy check coded over all of the data
octets included in the Intermediate Blocks and the octets of user information of
the Last Block. The specific calculation is as follows.
Let k be the total number of user information and pad bits over which the packet
CRC is to be calculated. Consider the k message bits as the coefficients of a
polynomial M(x) of degree k–1, associating the MSB of the zero-th message
octet with x^k–1 and the LSB of the last message octet with x^0. Define the
generator polynomial, GM(x), and the inversion polynomial, IM(x).
GM(x) = x^32 + x^26 + x^23 + x^22 + x^16 + x^12 + x^11 + x^10 + x^8 + x^7 + x^5 +
x^4 + x^2 + x + 1
IM(x) = x^31 + x^30 + x^29 + ... + x^2 + x +1
The packet CRC polynomial, FM(x), is then computed from the following formula.
FM(x) = ( x^32 M(x) mod GM(x) ) + IM(x) modulo 2, i.e., in GF(2)
The coefficients of FM(x) are placed in the CRC field with the MSB of the zero-th
octet of the CRC corresponding to x^31 and the LSB of the third octet of the CRC
corresponding to x^0.
In the above quote, I have put ^ to show powers as the formatting didn't stay the same when quoted. I'm not sure what goes to what but does this help?

I have a class I converted from a C++ I found in internet, it uses a long to calculate a CRC32. It adhere to the standard and is the one use by PKZIP, WinZip and Ethernet. To test it, use Winzip and compress a file then calculate the same file with this class, it should return the same CRC. It does for me.
public class CRC32
{
private int[] iTable;
public CRC32() {
this.iTable = new int[256];
Init();
}
/**
* Initialize the iTable aplying the polynomial used by PKZIP, WINZIP and Ethernet.
*/
private void Init()
{
// 0x04C11DB7 is the official polynomial used by PKZip, WinZip and Ethernet.
int iPolynomial = 0x04C11DB7;
// 256 values representing ASCII character codes.
for (int iAscii = 0; iAscii <= 0xFF; iAscii++)
{
this.iTable[iAscii] = this.Reflect(iAscii, (byte) 8) << 24;
for (int i = 0; i <= 7; i++)
{
if ((this.iTable[iAscii] & 0x80000000L) == 0) this.iTable[iAscii] = (this.iTable[iAscii] << 1) ^ 0;
else this.iTable[iAscii] = (this.iTable[iAscii] << 1) ^ iPolynomial;
}
this.iTable[iAscii] = this.Reflect(this.iTable[iAscii], (byte) 32);
}
}
/**
* Reflection is a requirement for the official CRC-32 standard. Note that you can create CRC without it,
* but it won't conform to the standard.
*
* #param iReflect
* value to apply the reflection
* #param iValue
* #return the calculated value
*/
private int Reflect(int iReflect, int iValue)
{
int iReturned = 0;
// Swap bit 0 for bit 7, bit 1 For bit 6, etc....
for (int i = 1; i < (iValue + 1); i++)
{
if ((iReflect & 1) != 0)
{
iReturned |= (1 << (iValue - i));
}
iReflect >>= 1;
}
return iReturned;
}
/**
* PartialCRC caculates the CRC32 by looping through each byte in sData
*
* #param lCRC
* the variable to hold the CRC. It must have been initialize.
* <p>
* See fullCRC for an example
* </p>
* #param sData
* array of byte to calculate the CRC
* #param iDataLength
* the length of the data
* #return the new caculated CRC
*/
public long CalculateCRC(long lCRC, byte[] sData, int iDataLength)
{
for (int i = 0; i < iDataLength; i++)
{
lCRC = (lCRC >> 8) ^ (long) (this.iTable[(int) (lCRC & 0xFF) ^ (int) (sData[i] & 0xff)] & 0xffffffffL);
}
return lCRC;
}
/**
* Caculates the CRC32 for the given Data
*
* #param sData
* the data to calculate the CRC
* #param iDataLength
* then length of the data
* #return the calculated CRC32
*/
public long FullCRC(byte[] sData, int iDataLength)
{
long lCRC = 0xffffffffL;
lCRC = this.CalculateCRC(lCRC, sData, iDataLength);
return (lCRC /*& 0xffffffffL)*/^ 0xffffffffL);
}
/**
* Calculates the CRC32 of a file
*
* #param sFileName
* The complete file path
* #param context
* The context to open the files.
* #return the calculated CRC32 or -1 if an error occurs (file not found).
*/
long FileCRC(String sFileName, Context context)
{
long iOutCRC = 0xffffffffL; // Initilaize the CRC.
int iBytesRead = 0;
int buffSize = 32 * 1024;
FileInputStream isFile = null;
try
{
byte[] data = new byte[buffSize]; // buffer de 32Kb
isFile = context.openFileInput(sFileName);
try
{
while ((iBytesRead = isFile.read(data, 0, buffSize)) > 0)
{
iOutCRC = this.CalculateCRC(iOutCRC, data, iBytesRead);
}
return (iOutCRC ^ 0xffffffffL); // Finalize the CRC.
}
catch (Exception e)
{
// Error reading file
}
finally
{
isFile.close();
}
}
catch (Exception e)
{
// file not found
}
return -1l;
}
}

Read Ross Williams tutorial on CRCs to get a better understanding of CRC's, what defines a particular CRC, and their implementations.
The reveng website has an excellent catalog of known CRCs, and for each the CRC of a test string (nine bytes: "123456789" in ASCII/UTF-8). Note that there are 22 different 16-bit CRCs defined there.
The reveng software on that same site can be used to reverse engineer the polynomial, initialization, post-processing, and bit reversal given several examples as you have for the 16-bit CRC. (Hence the name "reveng".) I ran your data through and got:
./reveng -w 16 -s 16000040FFFFFFFF00015FCB 3C00003144010405E57022C7 BA00001144010101B970F0ED 3900010101390401B3049FF1 09900C800000000000008CF3 8590000000000000000035F7 00900259025902590259EBC9 0200002B00080191014BF5A2 BB0000BEE0014401B970E51E 3D000322D0320A2510A263A0 2C0001440000D60000D65E54
width=16 poly=0x1021 init=0xc921 refin=false refout=false xorout=0x0000 check=0x2fcf name=(none)
As indicated by the "(none)", that 16-bit CRC is not any of the 22 listed on reveng, though it is similar to several of them, differing only in the initialization.
The additional information you provided is for a 32-bit CRC, either CRC-32 or CRC-32/BZIP in the reveng catalog, depending on whether the bits are reversed or not.

There are quite a few parameters to CRC calculations: Polynomial, initial value, final XOR... see Wikipedia for details. Your CRC does not seem to fit the ones on the site you used, but you can try to find the right parameters from your documentation and use a different calculator, e.g. this one (though I'm afraid it doesn't support HEX input).
One thing to keep in mind is that CRC-16 is usually calculated over the data that is supposed to be checksummed plus two zero-bytes, e.g. you are probably looking for a CRC16 function where CRC16(BD001325E032091B94C40000) == 12AC. With checksums calculated in this way, the CRC of the data with checksum appended will work out to 0, which makes checking easier, e.g. CRC16(BD001325E032091B94C412AC) == 0000