Develop Reference

Confirmation of Reverse Reciprocal CRC-8 Value?

I've spent quite a bit of time trying to confirm the type of CRC-8 algorithm used in ASCII data communications between two devices. I have confirmed that the CRC is calculated on the 0x02 Start of text byte + the next byte of data. An Interface Design Document that I have describing one device specifies the use of a 0xEA polynomial with an initial value of 0xFF. An example of one captured message is below:
Input Bytes: 0x02 0x41
CRC Result: b10011011 or 0x9B
Going into the problem, I had little to no knowledge of the inner working of a typical CRC algorithm. Initially, I tried hand calculation against the input bytes to confirm my understanding of the algo before attempting a code solution. This involved XORing the 1st input byte with my 0xFF initial value and then skipping to the second input byte to continue the XOR operations.
Having tried multiple times to confirm the CRC through typical XOR operations while shifting the MSB left out of the register during each step, I could never get the results I wanted. Today, I realized that the 0xEA polynomial is also considered to be a reversed reciprocal of the 0xD5 poly with an implied 1+x^8 that is commonly used in CRC-8 algos. How does this fact change how I would go about manually calculating the CRC? I've read that in some instances a reverse leads to the algo right shifting bits instead of left shifting?

The polynomial is x^8+x^7+x^5+x^3+x^2+x+1 => 01AF bit reversed to x^8+x^7+x^6+x^5+x^3+x+1 => 0x1EB. Example code where the conditional XOR is done after the shift, so the XOR value is 0x1EB>>1 = 0xF5. A 256 byte table lookup could be used to replace the inner loop.
using System;
namespace crc8r
{
class Program
{
private static byte crc8r(byte[] bfr, int bfrlen)
{
byte crc = 0xff;
for (int j = 0; j < bfrlen; j++)
{
crc ^= bfr[j];
for (int i = 0; i < 8; i++)
// assumes twos complement math
crc = (byte)((crc>>1)^((0-(crc&1))&0xf5));
}
return crc;
}
static void Main(string[] args)
{
byte[] data = new byte[3] {0x02, 0x41, 0x00};
byte crc;
crc = crc8r(data, 2); // crc == 0x9b
Console.WriteLine("{0:X2}", crc);
data[2] = crc;
crc = crc8r(data, 3); // crc == 0x00
Console.WriteLine("{0:X2}", crc);
return;
}
}
}
Regarding "EA", if the polynomial is XOR'ed before the shift, 0x1EB (or 0x1EA since bit 0 will be shifted off and doesn't matter) is used. XOR'ing before the shift requires 9 bits, or a post shift OR or XOR of 0x80, while XOR'ing after the shift only requires 8 bits.
Example line of code using 0x1eb before the shift:
crc = (byte)((crc^((0-(crc&1))&0x1eb))>>1);

Concatenating bytes into integer or short preserving sign

I know I can extract bytes from int like this:
bytes[] IntToBytes(int i)
{
return new byte [] {(byte) ((i >> 8) & 0xff), (byte) (i & 0xff)};
}
which I subsequently send as part of a serial transmission. But can I do the reverse, after receiving a sequence of bytes, reconstruct the original data, preserving the sign. Currently, I do this, but it feels a bit over the top:
int BytesToInt( byte hi, byte lo)
{
return ((hi << 24) | (lo << 16)) >> 16;
}
Is there another way or a better way? Does it make a difference if I know I am ultimately dealing with signed 16-bit data only?

You're working with signed 16-bit data only. So why are you passing (and returning) an int and not a short? You're throwing the sign information away, so it will not actually work for negative numbers. Instead, use short and you'll be fine - and the extra type information will make your code safer.
byte[] ShortToBytes(short i)
{
return new byte [] {(byte) ((i >> 8) & 0xff), (byte) (i & 0xff)};
}
short BytesToShort(byte hi, byte lo)
{
return unchecked((short)((hi << 8) | lo));
}
The main benefit (apart from being clearer and actually working) is that you can no longer pass an invalid value to the method. That's always good :)
Oh, and I'd recommend keeping the interface symmetric - BytesToShort should also take a byte[] (or some other structure that has the two bytes).

CRC-16 and CRC-32 Checks

I need help trying to verify CRC-16 values (also need help with CRC-32 values). I tried to sit down and understand how CRC works but I am drawing a blank.
My first problem is when trying to use an online calculator for calculating the message "BD001325E032091B94C412AC" into CRC16 = 12AC. The documentation states that the last two octets are the CRC16 value, so I am inputting "BD001325E032091B94C4" into the site http://www.lammertbies.nl/comm/info/crc-calculation.html and receive 5A90 as the result instead of 12AC.
Does anybody know why these values are different and where I can find code for how to calculate CRC16 and CRC32 values (I plan to later learn how to do this but times doesn't allow right now)?
Some more messages are as following:
16000040FFFFFFFF00015FCB
3C00003144010405E57022C7
BA00001144010101B970F0ED
3900010101390401B3049FF1
09900C800000000000008CF3
8590000000000000000035F7
00900259025902590259EBC9
0200002B00080191014BF5A2
BB0000BEE0014401B970E51E
3D000322D0320A2510A263A0
2C0001440000D60000D65E54
--Edit--
I have included more information. The documentation I was referencing is TIA-102.BAAA-A (from the TIA standard). The following is what the documentation states (trying to avoid copyright infringement as much as possible):
The Last Block in a packet comprises several octets of user information and / or
pad octets, followed by a 4-octet CRC parity check. This is referred to as the
packet CRC.
The packet CRC is a 4-octet cyclic redundancy check coded over all of the data
octets included in the Intermediate Blocks and the octets of user information of
the Last Block. The specific calculation is as follows.
Let k be the total number of user information and pad bits over which the packet
CRC is to be calculated. Consider the k message bits as the coefficients of a
polynomial M(x) of degree k–1, associating the MSB of the zero-th message
octet with x^k–1 and the LSB of the last message octet with x^0. Define the
generator polynomial, GM(x), and the inversion polynomial, IM(x).
GM(x) = x^32 + x^26 + x^23 + x^22 + x^16 + x^12 + x^11 + x^10 + x^8 + x^7 + x^5 +
x^4 + x^2 + x + 1
IM(x) = x^31 + x^30 + x^29 + ... + x^2 + x +1
The packet CRC polynomial, FM(x), is then computed from the following formula.
FM(x) = ( x^32 M(x) mod GM(x) ) + IM(x) modulo 2, i.e., in GF(2)
The coefficients of FM(x) are placed in the CRC field with the MSB of the zero-th
octet of the CRC corresponding to x^31 and the LSB of the third octet of the CRC
corresponding to x^0.
In the above quote, I have put ^ to show powers as the formatting didn't stay the same when quoted. I'm not sure what goes to what but does this help?

I have a class I converted from a C++ I found in internet, it uses a long to calculate a CRC32. It adhere to the standard and is the one use by PKZIP, WinZip and Ethernet. To test it, use Winzip and compress a file then calculate the same file with this class, it should return the same CRC. It does for me.
public class CRC32
{
private int[] iTable;
public CRC32() {
this.iTable = new int[256];
Init();
}
/**
* Initialize the iTable aplying the polynomial used by PKZIP, WINZIP and Ethernet.
*/
private void Init()
{
// 0x04C11DB7 is the official polynomial used by PKZip, WinZip and Ethernet.
int iPolynomial = 0x04C11DB7;
// 256 values representing ASCII character codes.
for (int iAscii = 0; iAscii <= 0xFF; iAscii++)
{
this.iTable[iAscii] = this.Reflect(iAscii, (byte) 8) << 24;
for (int i = 0; i <= 7; i++)
{
if ((this.iTable[iAscii] & 0x80000000L) == 0) this.iTable[iAscii] = (this.iTable[iAscii] << 1) ^ 0;
else this.iTable[iAscii] = (this.iTable[iAscii] << 1) ^ iPolynomial;
}
this.iTable[iAscii] = this.Reflect(this.iTable[iAscii], (byte) 32);
}
}
/**
* Reflection is a requirement for the official CRC-32 standard. Note that you can create CRC without it,
* but it won't conform to the standard.
*
* #param iReflect
* value to apply the reflection
* #param iValue
* #return the calculated value
*/
private int Reflect(int iReflect, int iValue)
{
int iReturned = 0;
// Swap bit 0 for bit 7, bit 1 For bit 6, etc....
for (int i = 1; i < (iValue + 1); i++)
{
if ((iReflect & 1) != 0)
{
iReturned |= (1 << (iValue - i));
}
iReflect >>= 1;
}
return iReturned;
}
/**
* PartialCRC caculates the CRC32 by looping through each byte in sData
*
* #param lCRC
* the variable to hold the CRC. It must have been initialize.
* <p>
* See fullCRC for an example
* </p>
* #param sData
* array of byte to calculate the CRC
* #param iDataLength
* the length of the data
* #return the new caculated CRC
*/
public long CalculateCRC(long lCRC, byte[] sData, int iDataLength)
{
for (int i = 0; i < iDataLength; i++)
{
lCRC = (lCRC >> 8) ^ (long) (this.iTable[(int) (lCRC & 0xFF) ^ (int) (sData[i] & 0xff)] & 0xffffffffL);
}
return lCRC;
}
/**
* Caculates the CRC32 for the given Data
*
* #param sData
* the data to calculate the CRC
* #param iDataLength
* then length of the data
* #return the calculated CRC32
*/
public long FullCRC(byte[] sData, int iDataLength)
{
long lCRC = 0xffffffffL;
lCRC = this.CalculateCRC(lCRC, sData, iDataLength);
return (lCRC /*& 0xffffffffL)*/^ 0xffffffffL);
}
/**
* Calculates the CRC32 of a file
*
* #param sFileName
* The complete file path
* #param context
* The context to open the files.
* #return the calculated CRC32 or -1 if an error occurs (file not found).
*/
long FileCRC(String sFileName, Context context)
{
long iOutCRC = 0xffffffffL; // Initilaize the CRC.
int iBytesRead = 0;
int buffSize = 32 * 1024;
FileInputStream isFile = null;
try
{
byte[] data = new byte[buffSize]; // buffer de 32Kb
isFile = context.openFileInput(sFileName);
try
{
while ((iBytesRead = isFile.read(data, 0, buffSize)) > 0)
{
iOutCRC = this.CalculateCRC(iOutCRC, data, iBytesRead);
}
return (iOutCRC ^ 0xffffffffL); // Finalize the CRC.
}
catch (Exception e)
{
// Error reading file
}
finally
{
isFile.close();
}
}
catch (Exception e)
{
// file not found
}
return -1l;
}
}

Read Ross Williams tutorial on CRCs to get a better understanding of CRC's, what defines a particular CRC, and their implementations.
The reveng website has an excellent catalog of known CRCs, and for each the CRC of a test string (nine bytes: "123456789" in ASCII/UTF-8). Note that there are 22 different 16-bit CRCs defined there.
The reveng software on that same site can be used to reverse engineer the polynomial, initialization, post-processing, and bit reversal given several examples as you have for the 16-bit CRC. (Hence the name "reveng".) I ran your data through and got:
./reveng -w 16 -s 16000040FFFFFFFF00015FCB 3C00003144010405E57022C7 BA00001144010101B970F0ED 3900010101390401B3049FF1 09900C800000000000008CF3 8590000000000000000035F7 00900259025902590259EBC9 0200002B00080191014BF5A2 BB0000BEE0014401B970E51E 3D000322D0320A2510A263A0 2C0001440000D60000D65E54
width=16 poly=0x1021 init=0xc921 refin=false refout=false xorout=0x0000 check=0x2fcf name=(none)
As indicated by the "(none)", that 16-bit CRC is not any of the 22 listed on reveng, though it is similar to several of them, differing only in the initialization.
The additional information you provided is for a 32-bit CRC, either CRC-32 or CRC-32/BZIP in the reveng catalog, depending on whether the bits are reversed or not.

There are quite a few parameters to CRC calculations: Polynomial, initial value, final XOR... see Wikipedia for details. Your CRC does not seem to fit the ones on the site you used, but you can try to find the right parameters from your documentation and use a different calculator, e.g. this one (though I'm afraid it doesn't support HEX input).
One thing to keep in mind is that CRC-16 is usually calculated over the data that is supposed to be checksummed plus two zero-bytes, e.g. you are probably looking for a CRC16 function where CRC16(BD001325E032091B94C40000) == 12AC. With checksums calculated in this way, the CRC of the data with checksum appended will work out to 0, which makes checking easier, e.g. CRC16(BD001325E032091B94C412AC) == 0000

CRC programming help needed, CRC32 conversion from the .NET class to C

Code(written in C):
unsigned long chksum_crc32 (unsigned char *block, unsigned int length)
{
register unsigned long crc;
unsigned long i;
crc = 0xFFFFFFFF;
for (i = 0; i < length; i++)
{
crc = ((crc >> 8) & 0x00FFFFFF) ^ crc_tab[(crc ^ *block++) & 0xFF];
}
return (crc ^ 0xFFFFFFFF);
}
/* chksum_crc32gentab() -- to a global crc_tab[256], this one will
* calculate the crcTable for crc32-checksums.
* it is generated to the polynom [..]
*/
void chksum_crc32gentab ()
{
unsigned long crc, poly;
int i, j;
poly = 0xEDB88320L;
for (i = 0; i < 256; i++)
{
crc = i;
for (j = 8; j > 0; j--)
{
if (crc & 1)
{
crc = (crc >> 1) ^ poly;
}
else
{
crc >>= 1;
}
}
crc_tab[i] = crc;
}
}
For starters; I know how CRC works, first the divisor is calculated with a specified polynomial, then this FCS(frame check sequence) is appended to the data set and sent to the end users system. Once the transfer is finished, the FCS is checked with the same polynomial used to calculate the FCS, and if the remainder of the data with that divisor is zero, then you know the data is correct.
I do not understand the implementation of these two functions. From what I have learned, the function chksum_crc32gentab() generates all the possible hex values the checksum could take with the 32 bit CRC polynomial. One thing I dont get is how poly = 0xEDB88320L; is equivelent to a polynomial. I don't understand the logic in the bottom of this function either. For example, the conditional if (crc & 1), does this mean that for every bit in crc that is 1, compute, otherwise shift right one bit?
I also do not understand chksum_crc32(unsigned char *block, unsigned int length);. Does this function just take in a string of bytes and convert them to the proper crc value computed with the table?. I guess I am confused about the logic it uses within the for loop.
If anyone understands this code an explanation would be great; this does work for the crc32 conversion from the .net class, an example of how data is converted then used by these functions would be something like:
(C# source)
MemoryStream ms = new MemoryStream(System.Text.Encoding.Default.GetBytes(input));
foreach (byte b in crc32.ComputeHash(ms))
hash += b.ToString("x2").ToLower();
Here is the original site and project the C code was taken from. http://www.codeproject.com/Articles/35134/How-to-calculate-CRC-in-C
Any explanation would help

Or just google it... Second hit is: http://www.opensource.apple.com/source/xnu/xnu-1456.1.26/bsd/libkern/crc32.c
Backporting it from C#'s the hard way to do it, most of these algorithms are already in C.

In CRC calculations, binary polynomials, which are sums of x^n with either a 0 or 1 coefficient, are represented simply as binary words where the position of the 0 or 1 indicates which power of x it is a coefficient of.
0xEDB88320L represents the coefficients of the CRC32 polynomial as 1's where there is an x^n term (except for the x^32 term, which is left out). The CRC32 polynomial (why oh why doesn't stackoverflow have TeX equations like math.stackexchange -- I can't write decent equations here! sigh, sorry for the rant ...) is:
x^32 + x^26 + x^23 + x^22 + x^16 + x^12 + x^11 + x^10 + x^8 + x^7 + x^5 + x^4 + x^2 + x + 1
Because of how this CRC is defined with respect to bit-ordering, the lowest coefficients are in the highest bits. So the first E in the hex constant above is 1110 representing (in order from left to right in the bits), 1 + x + x^2.
You can find the construction in the crc32.c source file of zlib, from which a snippet is shown here:
static const unsigned char p[] = {0,1,2,4,5,7,8,10,11,12,16,22,23,26};
/* make exclusive-or pattern from polynomial (0xedb88320UL) */
poly = 0;
for (n = 0; n < (int)(sizeof(p)/sizeof(unsigned char)); n++)
poly |= (z_crc_t)1 << (31 - p[n]);
/* generate a crc for every 8-bit value */
for (n = 0; n < 256; n++) {
c = (z_crc_t)n;
for (k = 0; k < 8; k++)
c = c & 1 ? poly ^ (c >> 1) : c >> 1;
crc_table[0][n] = c;
}
The if (crc & 1) or c & 1 ? above looks at the low bit of the CRC at each step before it is shifted away. That is effectively a carry bit for the polynomial subtraction operation, so if it is a one, the polynomial is subtracted (exclusive-ored) from the shifted down polynomial in the CRC (multiplied by x). The CRC is shifted down whether the low bit is 1 or not.
The chksum_crc32() function that you show indeed computes the CRC on the provided block of data. It is the standard table-based approach for CRC calculations on strings of bytes, which indexes the table by the exclusive-or of the data byte and the low byte of the CRC. This does the same thing as shifting in a bit at a time and applying the polynomial for 1 bits, but does it in one step instead of eight. The CRC is effectively multiplied by x^8 (the >> 8), and is exclusive-ored with the effect of exclusive-oring with the polynomial 0 to 8 times at various shifted locations depending on the index value. It is simply a speed trick using a pre-computed table.
You can find even more extreme speed tricks used in zlib's crc32.c that uses larger tables and processes more data a time.

CRC-4 implementation in C#

I've been searching the net for a C# implementation of the 4-bit cyclic redundancy check (CRC-4-ITU) but so far I've been unsuccessful.
Is there anyone who's able to give me a reference implementation of CRC-4-ITU? Preferrably with the standard polynomial if there is a standard polynomial (I've read the spec pointed to by wikipedia as the CRC4 spec without finding a definition of the polynomial).
I'd also really appreciate some sort of test suite or test data to verify a CRC4 implementation.
Thanks!

The Cyclic Redundancy Check article at Wikipedia says the polynomial is x^4 + x + 1. There is also a pretty good description of how the checksum is computed.
Here is an algorithm for CRC16. I know it's not what you asked for, but it should be relatively straightforward to adapt it for 4 bits.
public ushort calculate(byte[] bytes)
{
int crc = 0xFFFF; // initial value
// loop, calculating CRC for each byte of the string
for (int byteIndex = 0; byteIndex < bytes.Length; byteIndex++)
{
ushort bit = 0x80; // initialize bit currently being tested
for (int bitIndex = 0; bitIndex < 8; bitIndex++)
{
bool xorFlag = ((crc & 0x8000) == 0x8000);
crc <<= 1;
if (((bytes[byteIndex] & bit) ^ (ushort)0xff) != (ushort)0xff)
{
crc = crc + 1;
}
if (xorFlag)
{
crc = crc ^ 0x1021;
}
bit >>= 1;
}
}
return (ushort)crc;
}
http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_24775723.html
Also, there is this guide to computing checksums:
http://www.ross.net/crc/download/crc_v3.txt
"Everything you wanted to know about CRC algorithms, but were afraid
to ask for fear that errors in your understanding might be detected."