Develop Reference

Indicating the end of a raw data chunk in an RLE algorithm that can contain all byte values

I'm writing an RLE algorithm in C# that can work on any file as input. The approach to encoding I'm taking is as follows:
An RLE packet contains 1 byte for the length and 1 byte for the value. For example, if the byte 0xFF appeared 3 times in a row, 0x03 0xFF would be written to the file.
If representing the data as raw data would be more efficient, I use 0x00 as a terminator. This works because the length of a packet can never be zero. If I wanted to add the bytes 0x53 0x2C 0x01 to my compressed file it would look like this:
0x03 0xFF 0x00 0x53 0x2C 0x01
However a problem arises when trying to switch back to RLE packets. I can't use a byte as a terminator like I did for switching onto raw data because any byte value from 0x00 to 0xFF can be in the input data, and when decoding the bytes the decoder would misinterpret the byte as a terminator and ruin everything.
What can I do to indicate that I have to switch back to RLE packets when it can't be written as data in the file?
Here is my code if it helps:
private static void RunLengthEncode(ref byte[] bytes)
{
// Create a list to store the bytes
List<byte> output = new List<byte>();
byte runLengthByte;
int runLengthCounter = 0;
// Set the RLE byte to the first byte in the array and increment the RLE counter
runLengthByte = bytes[0];
// For each byte in the input array...
for (int i = 0; i < bytes.Length; i++)
{
if (runLengthByte == bytes[i] || runLengthCounter == 255)
{
runLengthCounter++;
}
else
{
// RLE packets under 3 should be written as raw data to avoid increasing the file size
if (runLengthCounter < 3)
{
// Add a 0x00 to indicate raw data
output.Add(0x00);
// Add the bytes that were skipped while counting the run length
for (int j = i - runLengthCounter; j < i; j++)
{
output.Add(bytes[j]);
}
}
else
{
// Add 2 bytes, one for the number of bytes and one for the value
output.Add((byte)runLengthCounter);
output.Add(runLengthByte);
}
runLengthCounter = 1;
runLengthByte = bytes[i];
}
// Add the last bytes to the list when finishing
if (i == bytes.Length - 1)
{
// Add 2 bytes, one for the number of bytes and one for the value
output.Add((byte)runLengthCounter);
output.Add(runLengthByte);
}
}
// Set the bytes to the RLE encoded data
bytes = output.ToArray();
}
Also if you want to comment and say that RLE isn't very efficient for binary data, I know it isn't. This is a project I'm doing to implement many kinds of compression to learn about them, not for an actual product.
Any help would be appreciated! Thanks!

There are many ways to unambiguously encode run-lengths. One simple way is, when decoding: if you see two equal bytes in a row, then the next byte is a a count of repeats of that byte after those first two. I.e. 0..255 additional repeats, so encoding runs of 2..257. (There's no point in encoding runs of 0 or 1.)

c# convert 2 bytes into int value

First, I had read many posts and tried BitConverter methods for the conversion, but I haven't got the desired result.
From a 2 byte array of:
byte[] dateArray = new byte[] { 0x07 , 0xE4 };
Y need to get an integer with value 2020. So, the decimal of 0x7E4.
Following method does not returning the desired value,
int i1 = BitConverter.ToInt16(dateArray, 0);

The endianess tells you how numbers are stored on your computer. There are two possibilities: Little endian and big endian.
Big endian means the biggest byte is stored first, i.e. 2020 would become 0x07, 0xE4.
Little endian means the lowest byte is stored first, i.e. 2020 would become 0xE4, 0x07.
Most computers are little endian, hence the other way round a human would expect. With BitConverter.IsLittleEndian, you can check which type of endianess your computer has. Your code would become:
byte[] dateArray = new byte[] { 0x07 , 0xE4 };
if(BitConverter.IsLittleEndian)
{
Array.Reverse(dataArray);
}
int i1 = BitConverter.ToInt16(dateArray, 0);

dateArray[0] << 8 | dateArray[1]

Converting non digit text to double and then back to string

I have a unique situation where I have to write code on top of an already establish platform so I am trying to figure out a hack to make something work.
The problem I have is I have a user defined string. Basically naming a signal. I need to get this into another program but the only method available is within a double value. Below is what I have tried but not been able to get it to work. I tried converting the string to byte array and then creating a new string by looping the bytes. Then I convert this string to a Double. Then use BitCoverter to get it back to byte array and then try to get the string.
Not sure if this can even be achieve. Any ideas?
string signal = "R3MEXA";
string newId = "1";
byte[] asciiBytes = System.Text.Encoding.ASCII.GetBytes(signal);
foreach (byte b in asciiBytes)
newId += b.ToString();
double signalInt = Double.Parse(newId);
byte[] bytes = BitConverter.GetBytes(signalInt);
string result = System.Text.Encoding.ASCII.GetString(bytes);

Asuming your string consists of ASCII characters (7Bit):
Convert your string into a bit-Array, seven bits per character.
Convert this bit-array into a string of digits, using 3 bits for each digit. (there are digits 0..7)
Convert this string of digits to a double number.

You initially set newId to "1", which means when you're doing later conversion, you're not going to get the right output unless to account for the "1" again.

It doesn't work, because if you convert it back you don't know the length of a byte.
So I made every byte to a length of 3.
string signal = "R3MEXA";
string newId = "1";
byte[] asciiBytes = System.Text.Encoding.ASCII.GetBytes(signal);
foreach (byte b in asciiBytes)
newId += b.ToString().PadLeft(3,'0'); //Add Zero, if the byte has less than 3 digits
double signalInt = Double.Parse(newId);
//Convert it back
List<byte> bytes = new List<byte>(); //Create a list, we don't know how many bytes will come (Or you calc it: maximum is _signal / 3)
//string _signal = signalInt.ToString("F0"); //Maybe you know a better way to get the double to string without scientific
//This is my workaround to get the integer part from the double:
//It's not perfect, but I don't know another way at the moment without losing information
string _signal = "";
while (signalInt > 1)
{
int _int = (int)(signalInt % 10);
_signal += (_int).ToString();
signalInt /= 10;
}
_signal = String.Join("",_signal.Reverse());
for (int i = 1; i < _signal.Length; i+=3)
{
byte b = Convert.ToByte(_signal.Substring(i, 3)); //Make 3 digits to one byte
if(b!=0) //With the ToString("F0") it is possible that empty bytes are at the end
bytes.Add(b);
}
string result = System.Text.Encoding.ASCII.GetString(bytes.ToArray()); //Yeah "R3MEX" The "A" is lost, because double can't hold that much.
What can improved?
Not every PadLeft is necessary. Work from back to front and if the third digit of a byte is greater than 2, you know, that the byte has only two digits. (Sorry for my english, I write an example).
Example
194 | 68 | 75 | 13
194687513
Reverse:
315786491
31 //5 is too big 13
57 //8 is too big 75
86 //4 is too big 68
491 //1 is ok 194

c# byte and bit casting - could this be done better?

I'm building arrays of bytes to be communicated over Bluetooth. These bytes are partly built from enumerated types, such as the following :
public enum Motor
{
A = 0x00,
B = 0x01,
C = 0x02,
AB = 0x03,
AC = 0x04,
BC = 0x05,
}
Later in my code I create a variable called MyMotor of type MyMotor.B. I then pass this variable to a method in which I build my byte array.
My issue is that the software I'm communicating with via Bluetooth expects the hex value of the enumerated value as a string, ie MyMotor.B = byte 0x01 = dec 1 = hex 31. However casting MyMotor directly to a char would result in it evaluating to it's enumerated value ie MyMotor = B = hex 42.
For various reasons I can't change my enurated list, so I've settled on what feels like a very hacked together two line section of code :
String motorchar = Convert.ToString(Convert.ToInt32(MyMotor)); // convert to temp var
command[5] = (byte)(motorchar[0]); // store hex value of var
It works as I'd like ie command[5] = hex31
I wonder if there's a better way. All the articles I've found talk about dealing with entire byte arrays rather than individual bytes and chars.

chars[0] = (char)('0' + ((byte)myMotor & 0x0F));
chars[1] = (char)('0' + (((byte)myMotor & 0xF0) >> 4));
This needs a little more tweaking for hexadecimal, though.

If your other app expects a string then provide one.
Make an array of strings to hold the values (which you know) and use the int value of the enum as an index into that array.

Unless I am missing something your two line code is equivalent to just calling;
BitConverter.ToString(MyMotor);
No?

If you know that your program's values and the values the API expects always differ by some fixed amount (for example, AC = 0x04, but the API wants "4", then you can write a simple conversion:
char c = (char)((int)Motor + '0');
That gets kind of ugly when there are more than 10 values, though. You can special case it for hexadecimal digits, but after that it's pretty bad.
You're better off creating a lookup table, the most general being a dictionary:
Dictionary<Motor, string> MotorLookup = new Dictionary<Motor, string>() {
{ Motor.A, "0" },
{ Motor.B, "1" },
// etc, etc.
};
That's going to be the most flexible and most maintainable.

Why not use:
//If you want the ASCII representation.
// e.g. myMotor == B, then the Byte Decimal value will be 49 or 0x31.
command[5] = (byte)((byte)myMotor).ToString()[0];
or
//If you want the numeric representation:
// e.g. myMotor == B, then the Byte Decimal value will be 1 or 0x01.
command[5] = (byte)myMotor;
I see you using values like "0x01" and "0x05". The "0x" prefix means it's a hexadecimal number, but you never go past 5, so it's exactly the same as using integer values "1" and "5".
I don't see how you're even getting Decimal 1 == Hex 31 or Hex 42 that you mention in your post. The ASCII equivalent of Char '1' is Decimal 49 or Hex 31.

How to convert a byte array with hexadecimal contents to string with decimal in c#

Language : C#
Basically i have a Byte array which contains hexadecimal contents.
I want to convert it into a String and the hexadecimal contents should also be converted into Decimal contents.
My final string should contain Equivalent Decimal Values of the Hexa Decimal values contained in the initial Byte Array.
I converted byte array to string using
System.Text.Encoding.GetEncoding(1251).GetString
But how to convert the Hex to Decimal ?
Even if we can do it in multiple steps it is not a problem.
sorry to ask silly doubts , please Spare.
Thanks in Advance!

It's not entirely clear what you mean.
Byte arrays just contain byte values - they're just numbers. In other words:
byte x = 0x20;
byte y = 32;
are exactly the same - they both just set the value to be 32.
Now, if you want to convert a byte array into a number, look at BitConverter and its methods like BitConverter.ToInt32. That will convert the byte array into a number (an int in that particular case) - you can then just call ToString() on the number to get a decimal representation as a string.
How many bytes is your original data? That will be a key factor in determining which BitConverter method to call. You will also need to know the endianness of the data - if BitConverter in "normal" .NET is little endian; you might be interested in the EndianBitConverter class in my MiscUtil library if your data is really big-endian.

If your array really contains the character codes of the hexadecimal representation of numbers, then you don't have to bother with decoding. As the character codes for those characters are the same in all regular encodings, you can just cast the bytes to characters.
I wrote an extension that parses a stream of characters into a stream of bytes:
static class Hex {
public static IEnumerable<byte> ParseHex(this IEnumerable<char> chars) {
int buffer = 0;
bool first = true;
foreach (char c in chars) {
int b = (c - '0') % 32;
if (b > 9) b -= 7;
if (first) {
buffer |= b << 4;
} else {
yield return (byte)(buffer | b);
buffer = 0;
}
first = !first;
}
if (!first) {
yield return (byte)buffer;
}
}
}
(If someone recognises part of the code, it's based on my code in this answer.)
Usage:
byte[] data = { 48, 70, 51, 67, 70, 56, 48, 55, 57, 49 };
string result = string.Join(",",
data.Cast<char>().ParseHex().Select(b => b.ToString()).ToArray()
);

I'm a little unsure what you're trying to do. Are you trying to create a comma separated string of the decimal numbers - e.g:
byte[] hexValues = { 0x9, 0xA, 0xB, 0xC, 0xD, 0xE, 0xF };
string decimalValuesAsStringList = string.Join(",", hexValues.Select(i => i.ToString()).ToArray());
...or like this:
char[] hexCharacters = { '9', 'A', 'B', 'C', 'D', 'E', 'F' };
byte[] hexValues = hexCharacters.Select(c => Convert.ToByte(c)).ToArray();
string decimalValuesAsStringList = string.Join(",", hexValues.Select(i => Convert.ToInt32(((char)i).ToString(), 16).ToString()).ToArray());