Is there any difference in calculating CRC32 checksum in C# and Java? - c#

I have to calculate CRC32 checksum for a string in C# and send it to an external application.
On the other end they will calculate it using Java.
But my checksum does not match on the their end.
e.g. CRC32 checksum of the following string
43HLV109520DAP10072la19z6
is 1269993351 on their end.
And 2947932745 at my end using C#
Please tell me what's going wrong in my code.
I am using this 0xffffffff default seed and following crc table
readonly static uint[] CRCTable = new uint[] {
0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419,
0x706AF48F, 0xE963A535, 0x9E6495A3, 0x0EDB8832, 0x79DCB8A4,
0xE0D5E91E, 0x97D2D988, 0x09B64C2B, 0x7EB17CBD, 0xE7B82D07,
0x90BF1D91, 0x1DB71064, 0x6AB020F2, 0xF3B97148, 0x84BE41DE,
0x1ADAD47D, 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7, 0x136C9856,
0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, 0x14015C4F, 0x63066CD9,
0xFA0F3D63, 0x8D080DF5, 0x3B6E20C8, 0x4C69105E, 0xD56041E4,
0xA2677172, 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B,
0x35B5A8FA, 0x42B2986C, 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3,
0x45DF5C75, 0xDCD60DCF, 0xABD13D59, 0x26D930AC, 0x51DE003A,
0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, 0xCFBA9599,
0xB8BDA50F, 0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924,
0x2F6F7C87, 0x58684C11, 0xC1611DAB, 0xB6662D3D, 0x76DC4190,
0x01DB7106, 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F,
0x9FBFE4A5, 0xE8B8D433, 0x7807C9A2, 0x0F00F934, 0x9609A88E,
0xE10E9818, 0x7F6A0DBB, 0x086D3D2D, 0x91646C97, 0xE6635C01,
0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, 0x6C0695ED,
0x1B01A57B, 0x8208F4C1, 0xF50FC457, 0x65B0D9C6, 0x12B7E950,
0x8BBEB8EA, 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3,
0xFBD44C65, 0x4DB26158, 0x3AB551CE, 0xA3BC0074, 0xD4BB30E2,
0x4ADFA541, 0x3DD895D7, 0xA4D1C46D, 0xD3D6F4FB, 0x4369E96A,
0x346ED9FC, 0xAD678846, 0xDA60B8D0, 0x44042D73, 0x33031DE5,
0xAA0A4C5F, 0xDD0D7CC9, 0x5005713C, 0x270241AA, 0xBE0B1010,
0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, 0xCE61E49F,
0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17,
0x2EB40D81, 0xB7BD5C3B, 0xC0BA6CAD, 0xEDB88320, 0x9ABFB3B6,
0x03B6E20C, 0x74B1D29A, 0xEAD54739, 0x9DD277AF, 0x04DB2615,
0x73DC1683, 0xE3630B12, 0x94643B84, 0x0D6D6A3E, 0x7A6A5AA8,
0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1, 0xF00F9344,
0x8708A3D2, 0x1E01F268, 0x6906C2FE, 0xF762575D, 0x806567CB,
0x196C3671, 0x6E6B06E7, 0xFED41B76, 0x89D32BE0, 0x10DA7A5A,
0x67DD4ACC, 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5,
0xD6D6A3E8, 0xA1D1937E, 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1,
0xA6BC5767, 0x3FB506DD, 0x48B2364B, 0xD80D2BDA, 0xAF0A1B4C,
0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, 0x316E8EEF,
0x4669BE79, 0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236,
0xCC0C7795, 0xBB0B4703, 0x220216B9, 0x5505262F, 0xC5BA3BBE,
0xB2BD0B28, 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, 0xB5D0CF31,
0x2CD99E8B, 0x5BDEAE1D, 0x9B64C2B0, 0xEC63F226, 0x756AA39C,
0x026D930A, 0x9C0906A9, 0xEB0E363F, 0x72076785, 0x05005713,
0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, 0x0CB61B38, 0x92D28E9B,
0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21, 0x86D3D2D4, 0xF1D4E242,
0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1,
0x18B74777, 0x88085AE6, 0xFF0F6A70, 0x66063BCA, 0x11010B5C,
0x8F659EFF, 0xF862AE69, 0x616BFFD3, 0x166CCF45, 0xA00AE278,
0xD70DD2EE, 0x4E048354, 0x3903B3C2, 0xA7672661, 0xD06016F7,
0x4969474D, 0x3E6E77DB, 0xAED16A4A, 0xD9D65ADC, 0x40DF0B66,
0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, 0x47B2CF7F, 0x30B5FFE9,
0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605,
0xCDD70693, 0x54DE5729, 0x23D967BF, 0xB3667A2E, 0xC4614AB8,
0x5D681B02, 0x2A6F2B94, 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B,
0x2D02EF8D
};

CRC32 is calculated over a sequence of bytes and not over a string. So to calculate CRC32 you need to transform the string into bytes first. If you use a different encoding to transform a string to a sequence of bytes the result will be different.
Thus you need to use the same encoding on both sides. I recommend using UTF-8 without BOM.

I have calculated CRC32 with Java and got the same you got in C#. I.e. CRC32(43HLV109520DAP10072la19z6)=2947932745. This means that either they have a bug in java, or you have a bug during transmission.
Code follows.
I suggest you try to send simple data to java application, like zeros or ones, and try to deduce how do they compute CRC.
public static void main(String[] args) {
CRC32 crc32 = new CRC32();
String data = "43HLV109520DAP10072la19z6";
String[] cs = new String[] {"utf8" /*, "cp1252", "cp866" */};
byte[] array;
byte b;
for(int i=0; i<cs.length; ++i) {
array = data.getBytes(Charset.forName(cs[i]));
crc32.reset();
crc32.update(array);
System.out.println(String.format("%s: %d", cs[i], crc32.getValue()));
/*
for(int j=0; j<array.length/2; j++) {
b = array[i];
array[i] = array[array.length-1-i];
array[array.length-1-i] = b;
}
*/
for(int j=0; j<array.length; j+=2) {
b = array[i];
array[i] = array[i+2];
array[i+1] = b;
}
crc32.reset();
crc32.update(array);
System.out.println(String.format("of modified: %d", crc32.getValue()));
}
}
UPDATE
Endiannes reverse also not help
for(int j=0; j<array.length; j+=4) {
b = array[i];
array[i] = array[i+3];
array[i+3] = b;
b = array[i+1];
array[i+1] = array[i+2];
array[i+2] = b;
}

Without delving into any detail, the problem can be related to Java's lack of unsigned integer types. The problem could happen at the int level, but also at the byte level. This is one avenue of investigation.

CRC is calculated over a sequence of bytes and not over a string.
Whichever CRC in java looks different due unavailability of Unsigned int in java.
Convert calculated Int CRC into Hex String and take last 2 Bytes (length 4)
That is your actual CRC unsigned Int.
String hexCrc = Integer.toHexString(crcCalculated);
hexCrc = hexCrc.substring(hexCrc.length()-4);
compare hex CRC of c# and Java both should be same.

Related

Byte to value error

So in c#, I have needed a random below given number generator and I found one on StackOverFlow. But near the end, it converts the byte array into a BigInteger. I tried doing the same, though I am using the Deveel-Math lib as it allows me to us BigDeciamals. But I have tried to the array change into a value, and that into a String but I keep getting a "Could not find any recognizable digits." error and as of now I am stumped.
public static BigInteger RandomIntegerBelow1(BigInteger N)
{
byte[] bytes = N.ToByteArray();
BigInteger R;
Random random = new Random();
do
{
random.NextBytes(bytes);
bytes[bytes.Length - 1] &= (byte)0x7F; //force sign bit to positive
R = BigInteger.Parse(BytesToStringConverted(bytes)) ;
//the Param needs a String value, exp: BigInteger.Parse("100")
} while (R >= N);
return R;
}
static string BytesToStringConverted(byte[] bytes)
{
using (var stream = new MemoryStream(bytes))
{
using (var streamReader = new StreamReader(stream))
{
return streamReader.ReadToEnd();
}
}
}
Deveel-Math
Wrong string conversion
You are converting your byte array to a string of characters based on UTF encoding. I'm pretty sure this is not what you want.
If you want to convert a byte array to a string that contains a number expressed in decimal, try this answer using BitConverter.
if (BitConverter.IsLittleEndian)
Array.Reverse(array); //need the bytes in the reverse order
int value = BitConverter.ToInt32(array, 0);
This is way easier
On the other hand, I notice that Deveel-Math's BigInteger has a constructor that takes a byte array as input (see line 226). So you should be able to greatly simplify your code by doing this:
R = new Deveel.Math.BigInteger(1, bytes) ;
However, since Deveel.Math appears to be BigEndian, you may need to reverse the array first:
System.Array.Reverse(bytes);
R = new Deveel.Math.BigInteger(1, bytes);

C# Array.Value differs from created array in c++

What is the best way to handle the following situation in C#?
I have a server application written in C/C++.
For example
It creates a unsigned char buffer with length 256.
In this buffer the server stores the data the client sends to it. After storing, there are some cryptography checks with the received buffer.
I'm writing a client for this server in C#.
The problem is the buffer the server is expecting at fixed length of 256.
When I create a byte[] array with content and total length of 256 the cryptography checks are failing.
The reason I found out is simple. The server expects a buffer of 256 bytes long. For example if the message is "Hello World", the rest of the buffer has to be 0-padded.
Or, better explained: the bytes need to be (unsigned) "204" or (signed) "-52".
I think this is a C/C++ concept issue/problem.
To solve my problem, I am setting that value explicitly.
public static byte[] GetCBuffer(this byte[] data, int len)
{
byte[] tmp = new byte[len];
for(int i = 0; i < len; i++)
if(i < data.Length)
tmp[i] = data[i];
else
tmp[i] = 204;
return tmp;
}
Are there better ways to work with these art expected bytes? Am I not seeing something essential?
If you don't like if in your code, you can try LINQ:
public static byte[] GetCBuffer(this byte[] data, int len)
{
byte[] tmp = Enumerable.Repeat((byte)204, len).ToArray();
for(int i = 0; i < data.Length; i++)
{
tmp[i] = data[i];
}
return ret;
}
But conceptually, it's the same, just looks (arguably) a bit nicer.
Can you 'memset' the entire array to the desired fill character and then insert the actual data?

Abstractly How To Check If A Decryption Attempt Was Successful

I have been working on the problem sets at Project Euler for some time and have enjoyed the challenges presented. I am currently on Problem 59 which involves the process of encryption and decryption.
The problem is, by any reasonable standard, a very simple decryption problem.
I have been told that the encryption key consists of 3 lowercase letters.
I have been given a description of the encryption/decryption process.
I have been given an encrypted text file which contains only encrypted common English words
I fully understand the process of importing the data, cycling through all possible keys, and attempting to decrypt the file data with each possible key. My trouble is, after a decryption attempt with one of the keys, how can I tell if that key was the correct decryption key? As far as the computer is concerned, every decryption key just converts the data from one value to another. The meaning of that value is purely subjective/interpreted. How can I tell if a decryption key has decrypted the data into something meaningful (ie. common English words)
Here is the code I have so far (C#):
static void Main(string[] args)
{
/* Get the data from the file and convert to byte array*/
StreamReader inData = new StreamReader(#"C:\Users\thantos\Desktop\cipher1.txt");
string[] strData = inData.ReadLine().Split(new char[] {','});
byte[] fileData = new byte[strData.Length];
foreach (string x in strData) { byte.Parse(x); }
/* for each three character lowercase password */
for (uint i = 0; i < 26; i++) {
for (uint j = 0; j < 26; j++){
for (uint k = 0; k < 26; k++) {
/* create a key */
byte[] key = new byte[3];
key[0] = (byte)(i + 97);
key[1] = (byte)(j + 97);
key[2] = (byte)(k + 97);
/* create temp copy of data */
byte[] dataCopy = new byte[fileData.Length];
fileData.CopyTo(dataCopy, 0);
/* decrypt using key */
for (uint l = 0; l < dataCopy.Length; l++) {
dataCopy[l] = (byte)(dataCopy[l] ^ key[l%key.Length]);
}
/* cannot figure out how to check
* if data is meaningfully decrypted
*/
bool dataIsMeaningful = isMeaningful(dataCopy);
if(dataIsMeaningful) {
/* do stuff with data if correct
* decryption key was found
*/
}
}
}
}
}
I have tried this method for isMeaningful():
public static isMeaningful(byte[] inData) {
bool isGood = true;
for (uint i = 0; good && i < inData.Length; i++) {
isGood &= (char.IsLower((char)inData[i]));;
}
return isGood;
}
But it returns true for all 17576 possible keys.
How can I determine if the decryption key has decrypted the file data into meaningful data? I'm not looking for solution code or the answer to the Project Euler problem, I am looking for an explanation of how to check that your decryption attempt was successful.
Try different keys and score them against the letter frequency of normal English: space, followed by E, T, A, O, I, N, S, H, R, D, L, U. Pick the highest scoring keys first for a trial.
Well, you can assume only valid ASCII values are permitted (since it should be plaintext). So for every letter you decode, just check that the resulting XOR results in a valid value: if(dataCopy[l] < 32 || dataCopy[l] > 122) continue; This should help to eliminate a vast majority of the possible key combinations.
Actually, this technique could even be used to narrow down your keyset to begin with. So instead of iterating through the loop 26^3 times over the entire 1200 character string, initially iterate 26 times and keep track of which letter at which position is still valid.
var letters = Enumerable.Repeat(Enumerable.Range((int)'a', 'z'-'a' + 1).Select(e => (char)e), 3).Select (e => e.ToList()).ToList();
for(int i = 0, j = 0; i < passwordBytes.Length; i++)
{
j = i % 3;
for(int k = 0; k < letters[j].Count; k++)
{
byte r = (byte)(letters[j][k] ^ passwordBytes[i]);
if(r < 32 || r > 122) letters[j].RemoveAt(k--);
}
}
That should get your valid characters down to almost nothing, at which point you can just iterate over your remaining values (which shouldn't take too long) and look for a few valid sequences (I hear looking for " the " is a good option).
When you have used a key to decrypt the data, it should have given you a text document filled with text. To check for legitimate data, would it be possible to just split the text into a text array using a space character as a delimiter, and then check each of the "words" in the array against an English dictionary?
Can you send a checksum of the original message along with the encrypted message? After decryption, compute the checksum and see if they match.
If that is not available, what about a test message? You could send a test message first such as "This is my test message" and check to see if the decrypted message matches verbatim. It's not fool-proof, but it may cover you.

Converting very large decimal numbers to hexadecimal (1.67119535743*10^33)

I'm curious as to if it'd be possible to convert a very, very large decimal number such as 1.67119535743*10^33/1.67119535743E+33 to hexadecimal via PHP or C#. All my previous attempts have failed, unfortunately. Thanks to all in advance!
Do you mean, convert it to a hex string? You might look at bigint libraries, like this one on CodeProject.
BigInteger bi = new BigInteger("12345678901234567890");
string s = bi.ToHexString();
I presume you're storing the number as a byte array, and want to output the hex number as a string?
This should do the job in C#:
public static string ConvertToHex(byte[] value)
{
var sb = new System.Text.StringBuilder();
for (int i = 0; i < sb.Length; i++)
sb.Append(value[i].ToString("X"));
return sb.ToString();
}

C# ushort[] to string conversion; is this possible?

I have a very painful library which, at the moment, is accepting a C# string as a way to get arrays of data; apparently, this makes marshalling for pinvokes easier.
So how do I make a ushort array into a string by bytes? I've tried:
int i;
String theOutData = "";
ushort[] theImageData = inImageData.DataArray;
//this is as slow like molasses in January
for (i = 0; i < theImageData.Length; i++) {
byte[] theBytes = System.BitConverter.GetBytes(theImageData[i]);
theOutData += String.Format("{0:d}{1:d}", theBytes[0], theBytes[1]);
}
I can do it this way, but it doesn't finish in anything remotely close to a sane amount of time.
What should I do here? Go unsafe? Go through some kind of IntPtr intermediate?
If it were a char* in C++, this would be significantly easier...
edit: the function call is
DataElement.SetByteValue(string inArray, VL Length);
where VL is a 'Value Length', a DICOM type, and the function itself is generated as a wrapper to a C++ library by SWIG. It seems that the representation chosen is string, because that can cross managed/unmanaged boundaries relatively easily, but throughout the C++ code in the project (this is GDCM), the char* is simply used as a byte buffer. So, when you want to set your image buffer pointer, in C++ it's fairly simple, but in C#, I'm stuck with this weird problem.
This is hackeration, and I know that probably the best thing is to make the SWIG library work right. I really don't know how to do that, and would rather a quick workaround on the C# side, if such exists.
P/Invoke can actually handle what you're after most of the time using StringBuilder to create writable buffers, for example see pinvoke.net on GetWindowText and related functions.
However, that aside, with data as ushort, I assume that it is encoded in UTF-16LE. If that is the case you can use Encoding.Unicode.GetString(), but that will exepect a byte array rather than a ushort array. To turn your ushorts into bytes, you can allocate a separate byte array and use Buffer.BlockCopy, something like this:
ushort[] data = new ushort[10];
for (int i = 0; i < data.Length; ++i)
data[i] = (char) ('A' + i);
string asString;
byte[] asBytes = new byte[data.Length * sizeof(ushort)];
Buffer.BlockCopy(data, 0, asBytes, 0, asBytes.Length);
asString = Encoding.Unicode.GetString(asBytes);
However, if unsafe code is OK, you have another option. Get the start of the array as a ushort*, and hard-cast it to char*, and then pass it to the string constructor, like so:
string asString;
unsafe
{
fixed (ushort *dataPtr = &data[0])
asString = new string((char *) dataPtr, 0, data.Length);
}
One thing you can do is switch from using a string to a stringBuilder it will help performance tremendously.
If you are willing to use unsafe code you can use pointers and implement the your c# code just like your c++. Or you could write a small c++\cli dll that implements this functionality.
Look into the Buffer class:
ushort[] theImageData = inImageData.DataArray;
byte[] buf = new byte[Buffer.ByteLength(theImageData)]; // 2 bytes per short
Buffer.BlockCopy(theImageData, 0, buf, 0, Buffer.ByteLength(theImageData));
string theOutData = System.Text.Encoding.ASCII.GetString(buf);
Just FYI, this has been fixed in later revision (gdcm 2.0.10). Look here:
http://gdcm.sourceforge.net/
-> http://apps.sourceforge.net/mediawiki/gdcm/index.php?title=GDCM_Release_2.0
I don't like this much, but it seems to work given the following assumptions:
1. Each ushort is an ASCII char between 0 and 127
2. (Ok, I guess there is just one assumption)
ushort[] data = inData; // The ushort array source
Byte[] bytes = new Byte[data.Length]; // Assumption - only need one byte per ushort
int i = 0;
foreach(ushort x in data) {
byte[] tmp = System.BitConverter.GetBytes(x);
bytes[i++] = tmp[0];
// Note: not using tmp[1] as all characters in 0 < x < 127 use one byte.
}
String str = Encoding.ASCII.GetString(bytes);
I'm sure there are better ways to do this, but it's all I could come up with quickly.
You can avoid unnecessary copying this way :
public static class Helpers
{
public static string ConvertToString(this ushort[] uSpan)
{
byte[] bytes = new byte[sizeof(ushort) * uSpan.Length];
for (int i = 0; i < uSpan.Length; i++)
{
Unsafe.As<byte, ushort>(ref bytes[i * 2]) = uSpan[i];
}
return Encoding.Unicode.GetString(bytes);
}
}

Categories

Resources