Develop Reference

What's the importance of the Offset variable in this algorithm?

What's the meaning of this variable named Offset in this algorithm ?
It's declared in the second calcCrc16 parameter.
For me it's useless bcause it's aways zero and it's used in a sum.
this algorithm generates a crc-16. I'm trying to understand this algorithm bcause a have to create a algorithm that verify crc-16, and i want to use this code as base.
public sealed class CRC
{
private readonly int _polynom;
public static readonly CRC Default = new CRC(0xA001);
public CRC(int polynom)
{
_polynom = polynom;
}
public int CalcCrc16(byte[] buffer)
{
return CalcCrc16(buffer, 0, buffer.Length, _polynom, 0);
}
public static int CalcCrc16(byte[] buffer, int offset, int bufLen, int polynom, int preset)
{
preset &= 0xFFFF;
polynom &= 0xFFFF;
var crc = preset;
for (var i = 0; i < (bufLen + 2); i++)
{
var data = buffer[(i + offset) % buffer.Length] & 0xFF;
crc ^= data;
for (var j = 0; j < 8; j++)
{
if ((crc & 0x0001) != 0)
{
crc = (crc >> 1) ^ polynom;
}
else
{
crc = crc >> 1;
}
}
}
return crc & 0xFFFF;
}
}

I created a simple example, using a small 4 byte message (in a 6 byte buffer):
using System;
namespace crc16
{
class Program
{
private static ushort Crc16(byte[] bfr, int bfrlen)
{
ushort crc = 0;
for (int i = 0; i < bfrlen; i++)
{
crc ^= bfr[i];
for (int j = 0; j < 8; j++)
// assumes twos complement math
crc = (ushort)((crc >> 1)^((0 - (crc&1)) & 0xa001));
}
return crc;
}
static void Main(string[] args)
{
ushort crc;
byte[] data = new byte[6] {0x11, 0x22, 0x33, 0x44, 0x00, 0x00};
crc = Crc16(data, 4); // generate crc
data[4] = (byte)(crc & 0xff); // append crc (lsb first)
data[5] = (byte)(crc >> 8);
crc = Crc16(data, 6); // verify crc;
Console.WriteLine("{0:X4}", crc);
return;
}
}
}

It's part of the signature of a public method, suitable whenever you want to calculate a CRC, but not on your entire buffer.
Sure, most of the time you may just use the simple version of the method, and in that case the parameter is always zero, but typically hashing and CRC implementations are built with an API like this, allowing you to calculate your CRC in chunks if you'd like.

Optimize bit reader for ReadInt on datastream

Could anyone help me optimize this piece of code? Its currently a large bottleneck as it gets called very often. Even a 25% speed improvement would be significant.
public int ReadInt(int length)
{
if (Position + length > Length)
throw new BitBufferException("Not enough bits remaining.");
int result = 0;
while (length > 0)
{
int off = Position & 7;
int count = 8 - off;
if (count > length)
count = length;
int mask = (1 << count) - 1;
int bits = (Data[Position >> 3] >> off);
result |= (bits & mask) << (length - count);
length -= count;
Position += count;
}
return result;
}
Best answer would go to fastest solution. Benchmarks done with dottrace. Currently this block of code takes up about 15% of the total cpu time. Lowest number wins best answer.
EDIT: Sample usage:
public class Auth : Packet
{
int Field0;
int ProtocolHash;
int Field1;
public override void Parse(buffer)
{
Field0 = buffer.ReadInt(9);
ProtocolHash = buffer.ReadInt(32);
Field1 = buffer.ReadInt(8);
}
}
Size of Data is variable but in most cases 512 bytes;

How about using pointers and unsafe context? You didn't say anything about your input data, method context, etc. so I tried to deduct all of these by myself.
public class BitTest
{
private int[] _data;
public BitTest(int[] data)
{
Length = data.Length * 4 * 8;
// +2, because we use byte* and long* later
// and don't want to read outside the array memory
_data = new int[data.Length + 2];
Array.Copy(data, _data, data.Length);
}
public int Position { get; private set; }
public int Length { get; private set; }
and ReadInt method. Hope comments give a little light on the solution:
public unsafe int ReadInt(int length)
{
if (Position + length > Length)
throw new ArgumentException("Not enough bits remaining.");
// method returns int, so getting more then 32 bits is pointless
if (length > 4 * 8)
throw new ArgumentException();
//
int bytePosition = Position / 8;
int bitPosition = Position % 8;
Position += length;
// get int* on array to start with
fixed (int* array = _data)
{
// change pointer to byte*
byte* bt = (byte*)array;
// skip already read bytes and change pointer type to long*
long* ptr = (long*)(bt + bytePosition);
// read value from current pointer position
long value = *ptr;
// take only necessary bits
value &= (1L << (length + bitPosition)) - 1;
value >>= bitPosition;
// cast value to int before returning
return (int)value;
}
}
}
I didn't test the method, but would bet it's much faster then your approach.
My simple test code:
var data = new[] { 1 | (1 << 8 + 1) | (1 << 16 + 2) | (1 << 24 + 3) };
var test = new BitTest(data);
var bytes = Enumerable.Range(0, 4)
.Select(x => test.ReadInt(8))
.ToArray();
bytes contains { 1, 2, 4, 8}, as expected.

I Don't know if this give you a significant improvements but it should give you some numbers.
Instead of creating new int variables inside the loop (this requires a time to create) let reserved those variables before entering the loop.
public int ReadInt(int length)
{
if (Position + length > Length)
throw new BitBufferException("Not enough bits remaining.");
int result = 0;
int off = 0;
int count = 0;
int mask = 0;
int bits = 0
while (length > 0)
{
off = Position & 7;
count = 8 - off;
if (count > length)
count = length;
mask = (1 << count) - 1;
bits = (Data[Position >> 3] >> off);
result |= (bits & mask) << (length - count);
length -= count;
Position += count;
}
return result;
}
HOPE THIS increase your performance even a bit

Most efficient way to reverse the order of a BitArray?

I've been wondering what the most efficient way to reverse the order of a BitArray in C#. To be clear, I don't want to inverse the Bitarray by calling .Not(), I want to reverse the order of the bits in the array.
Cheers,
Chris

public void Reverse(BitArray array)
{
int length = array.Length;
int mid = (length / 2);
for (int i = 0; i < mid; i++)
{
bool bit = array[i];
array[i] = array[length - i - 1];
array[length - i - 1] = bit;
}
}

For a long array and relative few uses, just wrap it:
class BitArrayReverse
{
private BitArray _ba;
public BitArrayReverse(BitArray ba) { _ba = ba; }
public bool this[int index]
{
get { return _ba[_ba.Length - 1 - index]; }
set { _ba[_ba.Length - 1 - index] = value; }
}
}

This will be the best way
to reverse MSB <-> LSB of any length using XOR in the for loop
public static BitArray BitsReverse(BitArray bits)
{
int len = bits.Count;
BitArray a = new BitArray(bits);
BitArray b = new BitArray(bits);
for (int i = 0, j = len-1; i < len; ++i, --j)
{
a[i] = a[i] ^ b[j];
b[j] = a[i] ^ b[j];
a[i] = a[i] ^ b[j];
}
return a;
}
// in 010000011010000011100b
// out 001110000010110000010b

Dim myBA As New BitArray(4)
myBA(0) = True
myBA(1) = False
myBA(2) = True
myBA(3) = True
Dim myBoolArray1(3) As Boolean
myBA.CopyTo(myBoolArray1, 0)
Array.Reverse(myBoolArray1)
myBA = New BitArray(myBoolArray1)

For a short but inefficient answer:
using System.Linq;
var reversedBa = new BitArray(myBa.Cast<bool>().Reverse().ToArray())

Because the size if fixed at 8-bits just the "table" lookup from below is sufficient -- when dealing with a plain byte a look-up is likely the quickest way. The extra overhead of BitSet to get/set the data may, however, nullify the look-up benefit. Also the initial build cost and persistent overhead need to be considered (but the values could be coded into an array literal ... ick!)
On the other hand, if the data is only 8 bit (ever), and "performance is important", why use a BitArray at all? A BitArray could always be used for the nice features, such as "exploding" to an Enumerable while C# already has decent byte bit manipulation built-in.
Assuming a more general case that the data is 8-bit aligned... but of some undetermined length
Is this actually better (faster, more efficient, etc) than just doing it "per item" in the BitArray? I have no idea but suspect not. I would definitely start with the "simple" methods -- this is here as just a proof-of-concept and may (or may not be) interesting to compare in a benchmark. Anyway, write for clarity first ... and the below is not it! (There is at least one bug in it -- I blame the extra complexity ;-)
byte reverse (byte b) {
byte o = 0;
for (var i = 0; i < 8; i++) {
o <<= 1;
o |= (byte)(b & 1);
b >>= 1;
}
return o;
}
byte[] table;
BitArray reverse8 (BitArray ar) {
if (ar.Count % 8 != 0) {
throw new Exception("no!");
}
byte[] d = new byte[ar.Count / 8];
ar.CopyTo(d, 0);
// this only works if the bit array is
// a multiple of 8. we swap bytes and
// then reverse bits in each byte
int mid = d.Length / 2;
for (int i = 0, j = d.Length - 1; i < mid; i++, j--) {
byte t = d[i];
d[i] = table[d[j]];
d[j] = table[t];
}
return new BitArray(d);
}
string tostr (BitArray x) {
return string.Join("",
x.OfType<bool>().Select(i => i ? "1" : "0").ToArray());
}
void Main()
{
table = Enumerable.Range(0,256).Select(v => reverse((byte)v)).ToArray();
{
byte[] s = new byte[] { 1, 0xff };
BitArray ar = new BitArray(s);
// linqpad :)
tostr(ar).Dump();
tostr(reverse8(ar)).Dump();
}
"--".Dump();
{
byte[] s = new byte[] { 3, 42, 19 };
BitArray ar = new BitArray(s);
// linqpad :)
tostr(ar).Dump();
tostr(reverse8(ar)).Dump();
}
}
Output:
1000000011111111
1111111100000001
--
110000000101010011001000
000100110101010000000011
The expr.Dump() is a LINQPad feature.

Adapted the answer from #TimLoyd and turned it into an extension for easier use.
public static BitArray Reverse(this BitArray array)
{
int length = array.Length;
int mid = (length / 2);
for (int i = 0; i < mid; i++)
{
bool bit = array[i];
array[i] = array[length - i - 1];
array[length - i - 1] = bit;
}
return new BitArray(array);
}
Usage:
var bits = new BitArray(some_bytes).Reverse();

Encode integer as variable-length big-endian byte-array

I need to write an integer to a byte array such that leading zeros are omitted and the bytes are written in big endian order.
Example:
int original = 0x00123456;
byte[] encoded = Encode(original); // == new byte[] { 0x12, 0x34, 0x56 };
int decoded = Decode(encoded); // == 0x123456
My Decode method:
private static int Decode(byte[] buffer, int index, int length)
{
int result = 0;
while (length > 0)
{
result = (result << 8) | buffer[index];
index++;
length--;
}
return result;
}
I'm struggling to come up with an Encode method that doesn't require a temporary buffer or reverses the bytes after writing them in little endian order. Can anyone help?
private static int Encode(int value, byte[] buffer, int index)
{
}

As per OP's request, here is a version without loops for a 32-bit number:
private static int Encode(int value, byte[] buffer, int index)
{
byte temp;
bool leading = true;
temp = (value >> 24) & 0xFF;
if (temp > 0) {
buffer[index++] = temp;
leading = false;
}
temp = (value >> 16) & 0xFF;
if (temp > 0 || leading == false) {
buffer[index++] = temp;
leading = false;
}
temp = (value >> 8) & 0xFF;
if (temp > 0 || leading == false) {
buffer[index++] = temp;
leading = false;
}
temp = value & 0xFF;
buffer[index++] = temp;
return index;
}
Version using a loop for 32-bit numbers:
private static int Encode(int value, byte[] buffer, int index)
{
int length = 0;
for (int i = 3; i >= 0; i++) {
byte temp = (byte)(value >> (8 * i));
if (temp > 0 || length > 0) {
buffer[index++] = temp;
length++;
}
}
return length;
}
Note that this version doesn't write anything if the input is just 0.

private static int Encode(int value, byte[] buffer, int index)
{
int length = 0;
int valueCopy = value;
while (valueCopy != 0)
{
valueCopy >>= 8;
length++;
}
for (int i = 0; i < length; i++)
{
buffer[index + length - i - 1] = (byte)value;
value >>= 8;
}
return length;
}

Note that you are saving the value to a variable Length byte array. If you save these byteArray, you need save also the length.
You can see the protected Functions Write7BitEncodedInt from BinaryWriter and Read7BitEncodedInt from BinaryReader.
These functions save storage on disk for positive numbers. A number 0-128 needs only one byte.
Microsoft uses these functions to store/retrieve the String length prefix when save to Stream.
To use these Functions, you can create own Class derived from BinaryReader / BinaryWriter.

Convert Byte Array to Bit Array?

How would I go about converting a bytearray to a bit array?

The obvious way; using the constructor that takes a byte array:
BitArray bits = new BitArray(arrayOfBytes);

It depends on what you mean by "bit array"... If you mean an instance of the BitArray class, Guffa's answer should work fine.
If you actually want an array of bits, in the form of a bool[] for instance, you could do something like that :
byte[] bytes = ...
bool[] bits = bytes.SelectMany(GetBits).ToArray();
...
IEnumerable<bool> GetBits(byte b)
{
for(int i = 0; i < 8; i++)
{
yield return (b & 0x80) != 0;
b *= 2;
}
}

public static byte[] ToByteArray(this BitArray bits)
{
int numBytes = bits.Count / 8;
if (bits.Count % 8 != 0) numBytes++;
byte[] bytes = new byte[numBytes];
int byteIndex = 0, bitIndex = 0;
for (int i = 0; i < bits.Count; i++) {
if (bits[i])
bytes[byteIndex] |= (byte)(1 << (7 - bitIndex));
bitIndex++;
if (bitIndex == 8) {
bitIndex = 0;
byteIndex++;
}
}
return bytes;
}

You can use BitArray to create a stream of bits from a byte array. Here an example:
string testMessage = "This is a test message";
byte[] messageBytes = Encoding.ASCII.GetBytes(testMessage);
BitArray messageBits = new BitArray(messageBytes);

byte number = 128;
Convert.ToString(number, 2);
=> out: 10000000

public static byte[] ToByteArray(bool[] byteArray)
{
return = byteArray
.Select(
(val1, idx1) => new { Index = idx1 / 8, Val = (byte)(val1 ? Math.Pow(2, idx1 % 8) : 0) }
)
.GroupBy(gb => gb.Index)
.Select(val2 => (byte)val2.Sum(s => (byte)s.Val))
.ToArray();
}