Develop Reference

C# Converting base64 string to 16-bit words stored in little-endian byte order

I am trying to upload a base64 of a signature but I need it to be a base64 encoding of an array of 16-bit words stored in little-endian byte order. Can anyone help me convert the base64 to 16-bit array in little-endian byte and then convert it again to base64?

To do this you can create arrays of the correct type (byte[] and short[]) and use Buffer.BlockCopy() to copy the bytes between them, thus converting the data.
This does not account for little-endian/big-endian differences, but since you state that this only needs to run on little-endian systems, we don't need to worry about it.
Here's a sample console app that demonstrates how to do the conversion. It does the following:
Create an array of shorts 0..99 inclusive.
Convert array of shorts to array of bytes (preserving endianness).
Convert array of bytes to base 64 string.
Convert base 64 string back into array of bytes.
Convert array of bytes back into array of shorts (preserving endianness).
Compare converted array of shorts with original array to prove correctness.
Here's the code:
using System;
using System.Linq;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
// Create demo array of shorts 0..99 inclusive.
short[] sourceShorts = Enumerable.Range(0, 100).Select(i => (short)i).ToArray();
// Convert array of shorts to array of bytes. (Will be little-endian on Intel.)
int byteCount = sizeof(short) * sourceShorts.Length;
byte[] dataAsByteArray = new byte[byteCount];
Buffer.BlockCopy(sourceShorts, 0, dataAsByteArray, 0, byteCount);
// Convert array of bytes to base 64 string.
var asBase64 = Convert.ToBase64String(dataAsByteArray);
Console.WriteLine(asBase64);
// Convert base 64 string back to array of bytes.
byte[] fromBase64 = Convert.FromBase64String(asBase64);
// Convert array of bytes back to array of shorts.
if (fromBase64.Length % sizeof(short) != 0)
throw new InvalidOperationException("Byte array size must be multiple of sizeof(short) to be convertable to shorts");
short[] destShorts = new short[fromBase64.Length/sizeof(short)];
Buffer.BlockCopy(fromBase64, 0, destShorts, 0, fromBase64.Length);
// Prove that the unconverted shorts match the source shorts.
if (destShorts.SequenceEqual(sourceShorts))
Console.WriteLine("Converted and unconverted successfully");
else
Console.WriteLine("Error: conversion was unsuccessful");
}
}
}

Large byte array - any benefit of storing length within the byte array?

Q: Is there any benefit of storing the length of a large array within the array itself?
Explanation:
Let's say we compress some large binary serialized object by using the GZipStream class of the System.IO.Compression namespace.
The output will be a Base64 string of some compressed byte array.
At some later point the Base64 string gets converted back to a byte array and the data needs to be decompressed.
While compressing the data we create a new byte array with the size of the compressed byte array + 4.
In the first 4 bytes we store the length/size of the compressed byte array and we then BlockCopy the length and the data to the new array. This new array gets converted into a Base64 string.
While decompressing we convert the Base64 string into a byte array.
Now we can extract the length of the actual compressed data by using the BitConverter class which will extract a Int32 from the first 4 bytes.
We then allocate a byte array with the length that we got from the first 4 bytes and let the Stream write the decompressed bytes to the byte array.
I can't image that something like this actually has any benefit at all.
It adds more complexity to the code and more operations need to be executed.
Readability is reduced too.
The BlockCopy operations alone should consume so much resources that this just cannot have a benefit, right?
Compression example code:
byte[] buffer = new byte[0xffff] // Some large binary serialized object
// Compress in-memory.
using (var mem = new MemoryStream())
{
// The actual compression takes place here.
using (var zipStream = new GZipStream(mem, CompressionMode.Compress, true)) {
zipStream.Write(buffer, 0, buffer.Length);
}
// Store compressed byte data here.
var compressedData = new byte[mem.Length];
mem.Position = 0;
mem.Read(compressedData, 0, compressedData.Length);
/* Increase the size by 4 to accommadate for an Int32 that
** will store the total length of the compressed data. */
var zipBuffer = new byte[compressedData.Length + 4];
// Store length of compressedData array in the first 4 bytes.
Buffer.BlockCopy(compressedData, 0, zipBuffer, 4, compressedData.Length);
// Store the compressedData array after the first 4 bytes which store the length.
Buffer.BlockCopy(BitConverter.GetBytes(buffer.Length), 0, zipBuffer, 0, 4);
return Convert.ToBase64String(zipBuffer);
}
Decompression example code:
byte[] zipBuffer = Convert.FromBase64String("some base64 string");
using (var inStream = new MemoryStream())
{
// The length of the array that was stored in the first 4 bytes.
int dataLength = BitConverter.ToInt32(zipBuffer, 0);
// Allocate array with specific size.
byte[] buffer = new byte[dataLength];
// Writer data to buffer array.
inStream.Write(zipBuffer, 4, zipBuffer.Length - 4);
inStream.Position = 0;
// Decompress data.
using (var zipStream = new GZipStream(inStream, CompressionMode.Decompress)) {
zipStream.Read(buffer, 0, buffer.Length);
}
... code
... code
... code
}

You tagged the question as C#, wich means .NET, so the question is irrelevant:
The Framework already store the length with the Array. It is how the array classes do the sanity checks on Indexers. It how it prevents overflow attacks in managed code. That help alone is worth any minor inefficiency (note that the JiT is actually able to prune most of the checks. With a loop for example, it will simply look at the running variable once per loop).
You would have to go all the way into unmanaged code and handling naked pointers to have a hope to get rid of it. But why would you? The difference is so small, it falls under the speed rant. If it maters, you propably got a realtime programming case. And starting those with .NET was a bad idea.

Read and write more than 8 bit symbols

I am trying to write an Encoded file.The file has 9 to 12 bit symbols. While writing a file I guess that it is not written correctly the 9 bit symbols because I am unable to decode that file. Although when file has only 8 bit symbols in it. Everything works fine. This is the way I am writing a file
File.AppendAllText(outputFileName, WriteBackContent, ASCIIEncoding.Default);
Same goes for reading with ReadAllText function call.
What is the way to go here?
I am using ZXing library to encode my file using RS encoder.
ReedSolomonEncoder enc = new ReedSolomonEncoder(GenericGF.AZTEC_DATA_12);//if i use AZTEC_DATA_8 it works fine beacuse symbol size is 8 bit
int[] bytesAsInts = Array.ConvertAll(toBytes.ToArray(), c => (int)c);
enc.encode(bytesAsInts, parity);
byte[] bytes = bytesAsInts.Select(x => (byte)x).ToArray();
string contentWithParity = (ASCIIEncoding.Default.GetString(bytes.ToArray()));
WriteBackContent += contentWithParity;
File.AppendAllText(outputFileName, WriteBackContent, ASCIIEncoding.Default);
Like in the code I am initializing my Encoder with AZTEC_DATA_12 which means 12 bit symbol. Because RS Encoder requires int array so I am converting it to int array. And writing to file like here.But it works well with AZTEC_DATA_8 beacue of 8 bit symbol but not with AZTEC_DATA_12.

Main problem is here:
byte[] bytes = bytesAsInts.Select(x => (byte)x).ToArray();
You are basically throwing away part of the result when converting the single integers to single bytes.
If you look at the array after the call to encode(), you can see that some of the array elements have a value higher than 255, so they cannot be represented as bytes. However, in your code quoted above, you cast every single element in the integer array to byte, changing the element when it has a value greater than 255.
So to store the result of encode(), you have to convert the integer array to a byte array in a way that the values are not lost or modified.
In order to make this kind of conversion between byte arrays and integer arrays, you can use the function Buffer.BlockCopy(). An example on how to use this function is in this answer.
Use the samples from the answer and the one from the comment to the answer for both conversions: Turning a byte array to an integer array to pass to the encode() function and to turn the integer array returned from the encode() function back into a byte array.
Here are the sample codes from the linked answer:
// Convert byte array to integer array
byte[] result = new byte[intArray.Length * sizeof(int)];
Buffer.BlockCopy(intArray, 0, result, 0, result.Length);
// Convert integer array to byte array (with bugs fixed)
int bytesCount = byteArray.Length;
int intsCount = bytesCount / sizeof(int);
if (bytesCount % sizeof(int) != 0) intsCount++;
int[] result = new int[intsCount];
Buffer.BlockCopy(byteArray, 0, result, 0, byteArray.Length);
Now about storing the data into files: Do not turn the data into a string directly via Encoding.GetString(). Not all bit sequences are valid representations of characters in any given character set. So, converting a random sequence of random bytes into a string will sometimes fail.
Instead, either store/read the byte array directly into a file via File.WriteAllBytes() / File.ReadAllBytes() or use Convert.ToBase64() and Convert.FromBase64() to work with a base64 encoded string representation of the byte array.
Combined here is some sample code:
ReedSolomonEncoder enc = new ReedSolomonEncoder(GenericGF.AZTEC_DATA_12);//if i use AZTEC_DATA_8 it works fine beacuse symbol size is 8 bit
int[] bytesAsInts = Array.ConvertAll(toBytes.ToArray(), c => (int)c);
enc.encode(bytesAsInts, parity);
// Turn int array to byte array without loosing value
byte[] bytes = new byte[bytesAsInts.Length * sizeof(int)];
Buffer.BlockCopy(bytesAsInts, 0, bytes, 0, bytes.Length);
// Write to file
File.WriteAllBytes(outputFileName, bytes);
// Read from file
bytes = File.ReadAllBytes(outputFileName);
// Turn byte array to int array
int bytesCount = bytes.Length * 40;
int intsCount = bytesCount / sizeof(int);
if (bytesCount % sizeof(int) != 0) intsCount++;
int[] dataAsInts = new int[intsCount];
Buffer.BlockCopy(bytes, 0, dataAsInts, 0, bytes.Length);
// Decoding
ReedSolomonDecoder dec = new ReedSolomonDecoder(GenericGF.AZTEC_DATA_12);
dec.decode(dataAsInts, parity);

Add values to byte array [duplicate]

This question already has answers here:
c# how to add byte to byte array
(7 answers)
Closed 7 years ago.
I have a byte array which contains an image; I converted the image to byte array using this method:
public byte[] imageToByteArray(System.Drawing.Image imageIn)
{
MemoryStream ms = new MemoryStream();
imageIn.Save(ms,System.Drawing.Imaging.ImageFormat.Gif);
return ms.ToArray();
}
This is the byte array:
Bitmap bmp = Image.FromFile("xxx");
byte[] buffer = ImageToByteArray(bmp);
Now I would like to add some minor minor information about the image in the byte array,such as the position it should be drawn to,etc.
How could it be done? Lets say i want to add these 2 values:1209,540.

First I don't think you can use these values 1209,540 since byte's max val is 255.
But if your values are between that range 0-255, why don't you add two bytes at the end of the array and when the conversion is to be made remove the last two values?

Convert into to byte array and insert into other array

I work in a c# wpf application in which I want to do several things. I'm working with byte arrays to compose MIDI Show Control messages (specified in the MSC Specification 1.0).
The structure of this message is that a 0x00 byte is like a comma between all the parts of the message. I compose a message like this:
byte[] data =
{(byte)0xF0, // SysEx
(byte)0x7F, // Realtime
(byte)0x7F, // Device id
(byte)0x02, // Constant
(byte)0x01, // Lighting format
(commandbyte), // GO
(qnumber), // qnumber
(byte)0x00, // comma
(qlist), // qlist
(byte)0x00, // comma
(byte)0xF7, // End of SysEx
};
I want the user to fill in unsigned integers (like 215.5) and I want to convert these numbers to bytes (without 0x00 bytes because then the message is interpreted wrong).
What is the best way to convert the numbers and place the byte array in the places mentioned above?

You might want to take a look at the BitConverter class, which is designed to convert base types into byte arrays.
http://msdn.microsoft.com/en-us/library/system.bitconverter.aspx
But I'm not sure what guidance you are seeking for placing the items into your array. Array.Copy can work to simply copy the bytes in, but maybe I am misunderstanding.

Found it out like this:
Used someone else's converter code like this:
static byte[] VlqEncode(int value)
{
uint uvalue = (uint)value;
if (uvalue < 128)
return new byte[] { (byte)uvalue };
// simplest case
// calculate length of buffer required
int len = 0;
do
{
uvalue >>= 7;
} while (uvalue != 0);
// encode
uvalue = (uint)value;
byte[] buffer = new byte[len];
int offset = 0;
do { buffer[offset] = (byte)(uvalue & 127);
// only the last 7 bits
uvalue >>= 7; if(uvalue != 0) buffer[offset++] |= 128;
// continuation bit
} while (uvalue != 0);
return buffer;
}
Then I use this to convert the integer:
byte[] mybytearray = VlqEncode(integer);
I then make a new arraylist in which I add each item in sequence:
ArrayList mymessage = new ArrayList();
foreach(byte uvalue in mymessage)
{
mymessage.Add((byte)uvalue);
}
mymessage.Add((byte)0x00);
`
and so on until I have the correct message. I then only have to convert this a byte array like this:
byte[] data = new byte[mymessage.count];
data = (byte[])mymessage.ToArray(typeof(byte));`