Develop Reference

Make GetHashCode method behave the same for strings for different processes

If I run this:
Console.WriteLine("Foo".GetHashCode());
Console.WriteLine("Foo".GetHashCode());
it will print the same number twice but if I run the program again it will print a different number.
According to Microsoft and other places on the internet we cannot rely on GetHashCode function to return the same value. But if I plan on using it on strings only how can I make use of it and expect to always return the same value for the same string? I love how fast it is. It will be great if I could get the source code of it and use it on my application.
Reason why I need it (you may skip this part)
I have a lot of complex objects that I need to serialize and send them between inter process communication. As you know BinaryFormatter is now obsolete so I then tried System.Text.Json to serialize my objects. That was very fast but because I have a lot of complex objects deserialization did not work well because I am making heavy use of polymorphism. Then I tried Newtonsoft (json.net) and that work great with this example: https://stackoverflow.com/a/71398251/637142. But it was very slow. I then decided I will use the best option and that is ProtoBuffers. So I was using protobuf-net and that worked great but the problem is that I have some objects that are very complex and it was a pain to place thousands of attributes. For example I have a base class that was being used by 70 other classes I had to place an attribute of inheritance for every single one it was not practical. So lastly I decided to implement my own algorithm it was not that complicated. I just have to traverse the properties of each object and if one property was not a value type then traverse them again recursively. But in order for this custom serialization that I build to be fast I needed to store all reflection objects in memory. So I have a dictionary with the types and propertyInfos. So the first time I serialize it will be slow but then it is even faster than ProtoBuf! So yes this approach is fast but every process must have the same exact object otherwise it will not work. Another tradeoff is that it's size is larger than protobuf because every time I serialize a property I include the full name of that property before. As a result I want to hash the full name of the property into an integer (4 bytes) and the GetHashCode() function does exactly that!
A lot of people may suggest that I should use MD5 or a different alternative but take a look at the performance difference:
// generate 1 million random GUIDS
List<string> randomGuids = new List<string>();
for (int i = 0; i < 1_000_000; i++)
randomGuids.Add(Guid.NewGuid().ToString());
// needed to measure time
var sw = new Stopwatch();
sw.Start();
// using md5 (takes aprox 260 ms)
using (var md5 = MD5.Create())
{
sw.Restart();
foreach (var guid in randomGuids)
{
byte[] inputBytes = System.Text.Encoding.ASCII.GetBytes(guid);
byte[] hashBytes = md5.ComputeHash(inputBytes);
// make use of hashBytes to make sure code is compiled
if (hashBytes.Length == 44)
throw new Exception();
}
var elapsed = sw.Elapsed.TotalMilliseconds;
Console.WriteLine($"md5: {elapsed}");
}
// using .net framework 4.7 source code (takes aprox 65 ms)
{
[System.Security.SecuritySafeCritical] // auto-generated
[ReliabilityContract(Consistency.WillNotCorruptState, Cer.MayFail)]
static int GetHashCodeDotNetFramework4_7(string str)
{
#if FEATURE_RANDOMIZED_STRING_HASHING
if(HashHelpers.s_UseRandomizedStringHashing)
{
return InternalMarvin32HashString(this, this.Length, 0);
}
#endif // FEATURE_RANDOMIZED_STRING_HASHING
unsafe
{
fixed (char* src = str)
{
#if WIN32
int hash1 = (5381<<16) + 5381;
#else
int hash1 = 5381;
#endif
int hash2 = hash1;
#if WIN32
// 32 bit machines.
int* pint = (int *)src;
int len = this.Length;
while (len > 2)
{
hash1 = ((hash1 << 5) + hash1 + (hash1 >> 27)) ^ pint[0];
hash2 = ((hash2 << 5) + hash2 + (hash2 >> 27)) ^ pint[1];
pint += 2;
len -= 4;
}
if (len > 0)
{
hash1 = ((hash1 << 5) + hash1 + (hash1 >> 27)) ^ pint[0];
}
#else
int c;
char* s = src;
while ((c = s[0]) != 0)
{
hash1 = ((hash1 << 5) + hash1) ^ c;
c = s[1];
if (c == 0)
break;
hash2 = ((hash2 << 5) + hash2) ^ c;
s += 2;
}
#endif
#if DEBUG
// We want to ensure we can change our hash function daily.
// This is perfectly fine as long as you don't persist the
// value from GetHashCode to disk or count on String A
// hashing before string B. Those are bugs in your code.
hash1 ^= -484733382;
#endif
return hash1 + (hash2 * 1566083941);
}
}
}
sw.Restart();
foreach (var guid in randomGuids)
if (GetHashCodeDotNetFramework4_7(guid) == 1234567)
throw new Exception("this will probably never happen");
var elapsed = sw.Elapsed.TotalMilliseconds;
Console.WriteLine($".NetFramework4.7SourceCode: {elapsed}");
}
// using .net 6 built in GetHashCode function (takes aprox: 22 ms)
{
sw.Restart();
foreach (var guid in randomGuids)
if (guid.GetHashCode() == 1234567)
throw new Exception("this will probably never happen");
var elapsed = sw.Elapsed.TotalMilliseconds;
Console.WriteLine($".net6: {elapsed}");
}
Running this in release mode these where my results:
md5: 254.7139
.NetFramework4.7SourceCode: 74.2588
.net6: 23.274
I got the source code from .NET Framework 4.8 from this link: https://referencesource.microsoft.com/#mscorlib/system/string.cs,8281103e6f23cb5c
Anyways searching on the internet I have found this helpful article:
https://andrewlock.net/why-is-string-gethashcode-different-each-time-i-run-my-program-in-net-core/
and I have done exactly what it tells you to do and I have added:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<runtime>
<UseRandomizedStringHashAlgorithm enabled="1" />
</runtime>
</configuration>
to my app.config file and still I get different values for "foo".GetHashCode() every time I run my application.
How can I make the GetHashcode() method return always the same value for the string "foo" in .net 6?
Edit
I will just use the solution of .net framework 4.8 source code that took 73ms to execute and move on. I was just curios to understand why was the build in hashcode so much faster.
At least I understand now why the hash is different every time. By looking at the source code of .net 6 the reason why it has a different hash every time is because of this:
namespace System
{
internal static partial class Marvin
{
... .net source code
....
public static ulong DefaultSeed { get; } = GenerateSeed();
private static unsafe ulong GenerateSeed()
{
ulong seed;
Interop.GetRandomBytes((byte*)&seed, sizeof(ulong));
return seed;
}
}
}
As a result I have tried this just for fun and still did not work:
var ass = typeof(string).Assembly;
var marvin = ass.GetType("System.Marvin");
var defaultSeed = marvin.GetProperty("DefaultSeed");
var value = defaultSeed.GetValue(null); // returns 3644491462759144438
var field = marvin.GetField("<DefaultSeed>k__BackingField", BindingFlags.NonPublic | BindingFlags.Static);
ulong v = 3644491462759144438;
field.SetValue(null, v);
but on the last line I get the exception: System.FieldAccessException: 'Cannot set initonly static field '<DefaultSeed>k__BackingField' after type 'System.Marvin' is initialized.'
But still even if this worked it would be very unsfafe. I rader have something execute 3 times slower and move on.

Why not to use the implementation suggested on the article you shared?
I'm copying it for reference:
static int GetDeterministicHashCode(this string str)
{
unchecked
{
int hash1 = (5381 << 16) + 5381;
int hash2 = hash1;
for (int i = 0; i < str.Length; i += 2)
{
hash1 = ((hash1 << 5) + hash1) ^ str[i];
if (i == str.Length - 1)
break;
hash2 = ((hash2 << 5) + hash2) ^ str[i + 1];
}
return hash1 + (hash2 * 1566083941);
}
}

How to simplify these methods to avoid confusion?

I've come here today to ask a question about these methods. I've taken lead on a personal project as a hobby and unfortunately I can't contact the old developer to ask what these methods even do. I'm pretty new to C# so I was asking if anyone could help me in simplifying them, to avoid the confusion I'm having? If anyone could actually tell me what they do also that would really help.
I'm just a little confused about them as of now... They were in the utilities folder. The project is an emulation server for a game, sending and receiving packets is the main focus.
public static int DecodeInt32(byte[] v)
{
if ((v[0] | v[1] | v[2] | v[3]) < 0)
{
return -1;
}
return (v[0] << 0x18) + (v[1] << 0x10) + (v[2] << 8) + v[3];
}
public static int DecodeInt16(byte[] v)
{
if ((v[0] | v[1]) < 0)
{
return -1;
}
return (v[0] << 8) + v[1];
}
Here is a part of code that uses them, might help in finding out?
using (BinaryReader Reader = new BinaryReader(new MemoryStream(Data)))
{
if (Data.Length < 4)
return;
int MsgLen = Utilities.DecodeInt32(Reader.ReadBytes(4));
if ((Reader.BaseStream.Length - 4) < MsgLen)
{
this._halfData = Data;
this._halfDataRecieved = true;
return;
}
else if (MsgLen < 0 || MsgLen > 5120)//TODO: Const somewhere.
return;
byte[] Packet = Reader.ReadBytes(MsgLen);
using (BinaryReader R = new BinaryReader(new MemoryStream(Packet)))
{
int Header = Utilities.DecodeInt16(R.ReadBytes(2));
byte[] Content = new byte[Packet.Length - 2];
Buffer.BlockCopy(Packet, 2, Content, 0, Packet.Length - 2);
ClientPacket Message = new ClientPacket(Header, Content);
try
{
Server.GetGame().GetPacketManager().TryExecutePacket(this, Message);
}
catch (Exception e)
{
ExceptionLogger.LogException(e);
}
this._deciphered = false;
}
if (Reader.BaseStream.Length - 4 > MsgLen)
{
byte[] Extra = new byte[Reader.BaseStream.Length - Reader.BaseStream.Position];
Buffer.BlockCopy(Data, (int)Reader.BaseStream.Position, Extra, 0, (int)(Reader.BaseStream.Length - Reader.BaseStream.Position));
this._deciphered = true;
HandleMoreData(Extra);
}
}

The BinaryReader has the methods ReadInt16 and ReadInt32 (and many others). Therefore you could replace the decoding methods.
int MsgLen = Utilities.DecodeInt32(Reader.ReadBytes(4));
becomes
int MsgLen = Reader.ReadInt32();
I assume that the Endianness of the bytes is right for the BinaryReader methods.

No CRC64 implementation equal to CommonCrypto?

I am porting some code from C on OSX to C# that uses CommonCrypto with the kCN_CRC_64_ECMA_182 CRC64 implementation. For example, using CommonCrypto the CRC would be computed with:
CNCRC(kCN_CRC_64_ECMA_182, bytes, bytesLen, &crcResult)
This outputs the correct value. When using the C# library HashLib (or any other code), the output is completely different, for example, the equivalent to the above using HashLib would be:
var checksum = HashFactory.Checksum.CreateCRC64(0x42F0E1EBA9EA3693UL); // ECMA 182
var result = checksum.ComputeBytes(bytes);
Any ideas? Is there an implementation in C# that is equivalent to Apple's CommonCrypto in terms of output?

Here is some simple C code to compute the ECMA-182 CRC:
#include <stddef.h>
#include <stdint.h>
#define POLY UINT64_C(0x42f0e1eba9ea3693)
#define TOP UINT64_C(0x8000000000000000)
/* Return crc updated with buf[0..len-1]. If buf is NULL, return the initial
crc. So, initialize with crc = crc64_ecma182(0, NULL, 0); and follow with
one or more applications of crc = crc64_ecma182(crc, buf, len); */
int64_t crc64_ecma182(int64_t crc, unsigned char *buf, size_t len)
{
int k;
if (buf == NULL)
return 0;
while (len--) {
crc ^= (uint64_t)(*buf++) << 56;
for (k = 0; k < 8; k++)
crc = crc & TOP ? (crc << 1) ^ POLY : crc << 1;
}
return crc;
}
I think HashLib is simply wrong, judging by what I found on github. It is doing the CRC reflected, whereas the CRC64 defined in ECMA-182 is not reflected.

To spare you all the endless search for a working CRC64 algorithm that returns the correct result, while in reality only finding unreadable code and god-object implementations which still yield the wrong result...
Here is the basic algorithm.
private static readonly ulong[] Crc64Table =
{
0x0000000000000000, 0x42f0e1eba9ea3693, 0x85e1c3d753d46d26, 0xc711223cfa3e5bb5, 0x493366450e42ecdf, 0x0bc387aea7a8da4c, 0xccd2a5925d9681f9, 0x8e224479f47cb76a,
0x9266cc8a1c85d9be, 0xd0962d61b56fef2d, 0x17870f5d4f51b498, 0x5577eeb6e6bb820b, 0xdb55aacf12c73561, 0x99a54b24bb2d03f2, 0x5eb4691841135847, 0x1c4488f3e8f96ed4,
0x663d78ff90e185ef, 0x24cd9914390bb37c, 0xe3dcbb28c335e8c9, 0xa12c5ac36adfde5a, 0x2f0e1eba9ea36930, 0x6dfeff5137495fa3, 0xaaefdd6dcd770416, 0xe81f3c86649d3285,
0xf45bb4758c645c51, 0xb6ab559e258e6ac2, 0x71ba77a2dfb03177, 0x334a9649765a07e4, 0xbd68d2308226b08e, 0xff9833db2bcc861d, 0x388911e7d1f2dda8, 0x7a79f00c7818eb3b,
0xcc7af1ff21c30bde, 0x8e8a101488293d4d, 0x499b3228721766f8, 0x0b6bd3c3dbfd506b, 0x854997ba2f81e701, 0xc7b97651866bd192, 0x00a8546d7c558a27, 0x4258b586d5bfbcb4,
0x5e1c3d753d46d260, 0x1cecdc9e94ace4f3, 0xdbfdfea26e92bf46, 0x990d1f49c77889d5, 0x172f5b3033043ebf, 0x55dfbadb9aee082c, 0x92ce98e760d05399, 0xd03e790cc93a650a,
0xaa478900b1228e31, 0xe8b768eb18c8b8a2, 0x2fa64ad7e2f6e317, 0x6d56ab3c4b1cd584, 0xe374ef45bf6062ee, 0xa1840eae168a547d, 0x66952c92ecb40fc8, 0x2465cd79455e395b,
0x3821458aada7578f, 0x7ad1a461044d611c, 0xbdc0865dfe733aa9, 0xff3067b657990c3a, 0x711223cfa3e5bb50, 0x33e2c2240a0f8dc3, 0xf4f3e018f031d676, 0xb60301f359dbe0e5,
0xda050215ea6c212f, 0x98f5e3fe438617bc, 0x5fe4c1c2b9b84c09, 0x1d14202910527a9a, 0x93366450e42ecdf0, 0xd1c685bb4dc4fb63, 0x16d7a787b7faa0d6, 0x5427466c1e109645,
0x4863ce9ff6e9f891, 0x0a932f745f03ce02, 0xcd820d48a53d95b7, 0x8f72eca30cd7a324, 0x0150a8daf8ab144e, 0x43a04931514122dd, 0x84b16b0dab7f7968, 0xc6418ae602954ffb,
0xbc387aea7a8da4c0, 0xfec89b01d3679253, 0x39d9b93d2959c9e6, 0x7b2958d680b3ff75, 0xf50b1caf74cf481f, 0xb7fbfd44dd257e8c, 0x70eadf78271b2539, 0x321a3e938ef113aa,
0x2e5eb66066087d7e, 0x6cae578bcfe24bed, 0xabbf75b735dc1058, 0xe94f945c9c3626cb, 0x676dd025684a91a1, 0x259d31cec1a0a732, 0xe28c13f23b9efc87, 0xa07cf2199274ca14,
0x167ff3eacbaf2af1, 0x548f120162451c62, 0x939e303d987b47d7, 0xd16ed1d631917144, 0x5f4c95afc5edc62e, 0x1dbc74446c07f0bd, 0xdaad56789639ab08, 0x985db7933fd39d9b,
0x84193f60d72af34f, 0xc6e9de8b7ec0c5dc, 0x01f8fcb784fe9e69, 0x43081d5c2d14a8fa, 0xcd2a5925d9681f90, 0x8fdab8ce70822903, 0x48cb9af28abc72b6, 0x0a3b7b1923564425,
0x70428b155b4eaf1e, 0x32b26afef2a4998d, 0xf5a348c2089ac238, 0xb753a929a170f4ab, 0x3971ed50550c43c1, 0x7b810cbbfce67552, 0xbc902e8706d82ee7, 0xfe60cf6caf321874,
0xe224479f47cb76a0, 0xa0d4a674ee214033, 0x67c58448141f1b86, 0x253565a3bdf52d15, 0xab1721da49899a7f, 0xe9e7c031e063acec, 0x2ef6e20d1a5df759, 0x6c0603e6b3b7c1ca,
0xf6fae5c07d3274cd, 0xb40a042bd4d8425e, 0x731b26172ee619eb, 0x31ebc7fc870c2f78, 0xbfc9838573709812, 0xfd39626eda9aae81, 0x3a28405220a4f534, 0x78d8a1b9894ec3a7,
0x649c294a61b7ad73, 0x266cc8a1c85d9be0, 0xe17dea9d3263c055, 0xa38d0b769b89f6c6, 0x2daf4f0f6ff541ac, 0x6f5faee4c61f773f, 0xa84e8cd83c212c8a, 0xeabe6d3395cb1a19,
0x90c79d3fedd3f122, 0xd2377cd44439c7b1, 0x15265ee8be079c04, 0x57d6bf0317edaa97, 0xd9f4fb7ae3911dfd, 0x9b041a914a7b2b6e, 0x5c1538adb04570db, 0x1ee5d94619af4648,
0x02a151b5f156289c, 0x4051b05e58bc1e0f, 0x87409262a28245ba, 0xc5b073890b687329, 0x4b9237f0ff14c443, 0x0962d61b56fef2d0, 0xce73f427acc0a965, 0x8c8315cc052a9ff6,
0x3a80143f5cf17f13, 0x7870f5d4f51b4980, 0xbf61d7e80f251235, 0xfd913603a6cf24a6, 0x73b3727a52b393cc, 0x31439391fb59a55f, 0xf652b1ad0167feea, 0xb4a25046a88dc879,
0xa8e6d8b54074a6ad, 0xea16395ee99e903e, 0x2d071b6213a0cb8b, 0x6ff7fa89ba4afd18, 0xe1d5bef04e364a72, 0xa3255f1be7dc7ce1, 0x64347d271de22754, 0x26c49cccb40811c7,
0x5cbd6cc0cc10fafc, 0x1e4d8d2b65facc6f, 0xd95caf179fc497da, 0x9bac4efc362ea149, 0x158e0a85c2521623, 0x577eeb6e6bb820b0, 0x906fc95291867b05, 0xd29f28b9386c4d96,
0xcedba04ad0952342, 0x8c2b41a1797f15d1, 0x4b3a639d83414e64, 0x09ca82762aab78f7, 0x87e8c60fded7cf9d, 0xc51827e4773df90e, 0x020905d88d03a2bb, 0x40f9e43324e99428,
0x2cffe7d5975e55e2, 0x6e0f063e3eb46371, 0xa91e2402c48a38c4, 0xebeec5e96d600e57, 0x65cc8190991cb93d, 0x273c607b30f68fae, 0xe02d4247cac8d41b, 0xa2dda3ac6322e288,
0xbe992b5f8bdb8c5c, 0xfc69cab42231bacf, 0x3b78e888d80fe17a, 0x7988096371e5d7e9, 0xf7aa4d1a85996083, 0xb55aacf12c735610, 0x724b8ecdd64d0da5, 0x30bb6f267fa73b36,
0x4ac29f2a07bfd00d, 0x08327ec1ae55e69e, 0xcf235cfd546bbd2b, 0x8dd3bd16fd818bb8, 0x03f1f96f09fd3cd2, 0x41011884a0170a41, 0x86103ab85a2951f4, 0xc4e0db53f3c36767,
0xd8a453a01b3a09b3, 0x9a54b24bb2d03f20, 0x5d45907748ee6495, 0x1fb5719ce1045206, 0x919735e51578e56c, 0xd367d40ebc92d3ff, 0x1476f63246ac884a, 0x568617d9ef46bed9,
0xe085162ab69d5e3c, 0xa275f7c11f7768af, 0x6564d5fde549331a, 0x279434164ca30589, 0xa9b6706fb8dfb2e3, 0xeb46918411358470, 0x2c57b3b8eb0bdfc5, 0x6ea7525342e1e956,
0x72e3daa0aa188782, 0x30133b4b03f2b111, 0xf7021977f9cceaa4, 0xb5f2f89c5026dc37, 0x3bd0bce5a45a6b5d, 0x79205d0e0db05dce, 0xbe317f32f78e067b, 0xfcc19ed95e6430e8,
0x86b86ed5267cdbd3, 0xc4488f3e8f96ed40, 0x0359ad0275a8b6f5, 0x41a94ce9dc428066, 0xcf8b0890283e370c, 0x8d7be97b81d4019f, 0x4a6acb477bea5a2a, 0x089a2aacd2006cb9,
0x14dea25f3af9026d, 0x562e43b4931334fe, 0x913f6188692d6f4b, 0xd3cf8063c0c759d8, 0x5dedc41a34bbeeb2, 0x1f1d25f19d51d821, 0xd80c07cd676f8394, 0x9afce626ce85b507
};
public static ulong Calculate(byte[] data)
{
ulong crc64 = ulong.MaxValue;
int length = data.Length;
for (int i = 0; i < length; i++)
{
crc64 = Crc64Table[((uint)(crc64 >> 56) ^ data[i]) & 0xff] ^ (crc64 << 8);
}
return ~crc64;
}
Looking for a byte[] instead?
Replace return ~crc64; with
crc64 = ~crc64;
return new[]
{
(byte)((crc64 & 0xff00000000000000) >> 56),
(byte)((crc64 & 0xff000000000000) >> 48),
(byte)((crc64 & 0xff0000000000) >> 40),
(byte)((crc64 & 0xff00000000) >> 32),
(byte)((crc64 & 0xff000000) >> 24),
(byte)((crc64 & 0xff0000) >> 16),
(byte)((crc64 & 0xff00) >> 8),
(byte)(crc64 & 0xff)
};

Android base64 hash mismatch with server side hash using C# script

I am creating base64 hash using HMAC SHA256 in my Android application. and send it on server for match with server side hash.
Following this tutorial.
Working Android code:
public String getHash(String data,String key)
{
try
{
String secret = key;
String message = data;
Mac sha256_HMAC = Mac.getInstance("HmacMD5");
SecretKeySpec secret_key = new SecretKeySpec(secret.getBytes(), "HmacMD5");
sha256_HMAC.init(secret_key);
String hash = Base64.encodeBase64String(sha256_HMAC.doFinal(message.getBytes()));
System.out.println(hash);
return hash;
}
catch (Exception e){
System.out.println("Error");
}
}
server code is in C# script and its as per below
using System.Security.Cryptography;
namespace Test
{
public class MyHmac
{
private string CreateToken(string message, string secret)
{
secret = secret ?? "";
var encoding = new System.Text.ASCIIEncoding();
byte[] keyByte = encoding.GetBytes(secret);
byte[] messageBytes = encoding.GetBytes(message);
using (var hmacsha256 = new HMACSHA256(keyByte))
{
byte[] hashmessage = hmacsha256.ComputeHash(messageBytes);
return Convert.ToBase64String(hashmessage);
}
}
}
}
but hash key generated at android side is not match with server side and below is objective c code which generate same as C# code
objective c code:
#import "AppDelegate.h"
#import <CommonCrypto/CommonHMAC.h>
#implementation AppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
NSString* key = #"secret";
NSString* data = #"Message";
const char *cKey = [key cStringUsingEncoding:NSASCIIStringEncoding];
const char *cData = [data cStringUsingEncoding:NSASCIIStringEncoding];
unsigned char cHMAC[CC_SHA256_DIGEST_LENGTH];
CCHmac(kCCHmacAlgSHA256, cKey, strlen(cKey), cData, strlen(cData), cHMAC);
NSData *hash = [[NSData alloc] initWithBytes:cHMAC length:sizeof(cHMAC)];
NSLog(#"%#", hash);
NSString* s = [AppDelegate base64forData:hash];
NSLog(s);
}
+ (NSString*)base64forData:(NSData*)theData
{
const uint8_t* input = (const uint8_t*)[theData bytes];
NSInteger length = [theData length];
static char table[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
NSMutableData* data = [NSMutableData dataWithLength:((length + 2) / 3) * 4];
uint8_t* output = (uint8_t*)data.mutableBytes;
NSInteger i;
for (i=0; i < length; i += 3) {
NSInteger value = 0;
NSInteger j;
for (j = i; j < (i + 3); j++) {
value <<= 8;
if (j < length) { value |= (0xFF & input[j]);
}
}
NSInteger theIndex = (i / 3) * 4; output[theIndex + 0] = table[(value >> 18) & 0x3F];
output[theIndex + 1] = table[(value >> 12) & 0x3F];
output[theIndex + 2] = (i + 1) < length ? table[(value >> 6) & 0x3F] : '=';
output[theIndex + 3] = (i + 2) < length ? table[(value >> 0) & 0x3F] : '=';
}
return [[NSString alloc] initWithData:data encoding:NSASCIIStringEncoding];
}
#end
please help me to sove out this issue,
Thanks in advance.
I have solved this issue by changing HmacSHA256 to HmacMD5 and its give same hash value as given by C# code.
I have updated my question with working code. check it

I suspect this is an encoding issue.
In one sample you specify the string should be encoded using ASCII when converting the string to a byte array. In the other sample you do not specify an encoding.
If the default encoding is anything other than ASCII that means the byte arrays will be different, leading to different hash results.

In android secret.getBytes may get UTF-16 bytes, check the length of the result. In general separate such functions out into separate statements for easier debugging.
Not the answer, rather a demonstration of a simpler Obj-C implementation and provides the hash and Base64 vaules:
NSString* key = #"secret";
NSString* data = #"Message";
NSData *keyData = [key dataUsingEncoding:NSASCIIStringEncoding];
NSData *dataData = [data dataUsingEncoding:NSASCIIStringEncoding];
NSMutableData *hash = [NSMutableData dataWithLength:CC_SHA256_DIGEST_LENGTH];
CCHmac(kCCHmacAlgSHA256, keyData.bytes, keyData.length , dataData.bytes, dataData.length, hash.mutableBytes);
NSLog(#"hash: %#", hash);
NSString* s = [hash base64EncodedStringWithOptions:0];
NSLog(#"s: %#", s);
Output:
hash: <aa747c50 2a898200 f9e4fa21 bac68136 f886a0e2 7aec70ba 06daf2e2 a5cb5597>
s: qnR8UCqJggD55PohusaBNviGoOJ67HC6Btry4qXLVZc=

C# equalent to perl `pack("v",value)` while packing some values into `byte[]`

I am trying to replicate behavior of a perl script in my c# code. When we convert any value into the Byte[] it should look same irrespective of the language used. SO
I have this function call which looks like this in perl:
$diag_cmd = pack("V", length($s_part)) . $s_part;
where $s_par is defined in following function. It is taking the .pds file at the location C:\Users\c_desaik\Desktop\DIAG\PwrDB\offtarget\data\get_8084_gpio.pds
$s_part =
sub read_pds
{
my $bin_s;
my $input_pds_file = $_[0];
open(my $fh, '<', $input_pds_file) or die "cannot open file $input_pds_file";
{
local $/;
$bin_s = <$fh>;
}
close($fh);
return $bin_s;
}
My best guess is that this function is reading the .pds file and turning it into a Byte array.
Now, I tried to replicate the behavior into c# code like following
static byte[] ConstructPacket()
{
List<byte> retval = new List<byte>();
retval.AddRange(System.IO.File.ReadAllBytes(#"C:\Users\c_desaik\Desktop\DIAG\PwrDB\offtarget\data\get_8084_gpio.pds"));
return retval.ToArray();
}
But the resulting byte array does not look same. Is there any special mechanism that I have to follow to replicate the behavior of pack("V", length($s_part)) . $s_part ?

As Simon Whitehead mentioned the template character V tells pack to pack your values into unsigned long (32-bit) integers (in little endian order). So you need to convert your bytes to a list (or array) of unsigned integers.
For example:
static uint[] UnpackUint32(string filename)
{
var retval = new List<uint>();
using (var filestream = System.IO.File.Open(filename, System.IO.FileMode.Open))
{
using (var binaryStream = new System.IO.BinaryReader(filestream))
{
var pos = 0;
while (pos < binaryStream.BaseStream.Length)
{
retval.Add(binaryStream.ReadUInt32());
pos += 4;
}
}
}
return retval.ToArray();
}
And call this function:
var list = UnpackUint32(#"C:\Users\c_desaik\Desktop\DIAG\PwrDB\offtarget\data\get_8084_gpio.pds");
Update
If you wanna read one length-prefixed string or a list of them, you can use this function:
private string[] UnpackStrings(string filename)
{
var retval = new List<string>();
using (var filestream = System.IO.File.Open(filename, System.IO.FileMode.Open))
{
using (var binaryStream = new System.IO.BinaryReader(filestream))
{
var pos = 0;
while ((pos + 4) <= binaryStream.BaseStream.Length)
{
// read the length of the string
var len = binaryStream.ReadUInt32();
// read the bytes of the string
var byteArr = binaryStream.ReadBytes((int) len);
// cast this bytes to a char and append them to a stringbuilder
var sb = new StringBuilder();
foreach (var b in byteArr)
sb.Append((char)b);
// add the new string to our collection of strings
retval.Add(sb.ToString());
// calculate start position of next value
pos += 4 + (int) len;
}
}
}
return retval.ToArray();
}

pack("V", length($s_part)) . $s_part
which can also be written as
pack("V/a*", $s_part)
creates a length-prefixed string. The length is stored as a 32-bit unsigned little-endian number.
+----------+----------+----------+----------+-------- ...
| Length | Length | Length | Length | Bytes
| ( 7.. 0) | (15.. 8) | (23..16) | (31..24) |
+----------+----------+----------+----------+-------- ...
This is how you recreate the original string from the bytes:
Read 4 bytes
If using a machine other than a little-endian machine,
Rearrange the bytes into the native order.
Cast those bytes into an 32-bit unsigned integer.
Read a number of bytes equal to that number.
Convert that sequences of bytes into a string.
Some languages provide tools that perform more than one of these steps.
I don't know C#, so I can't write the code for you, but I can give you an example in two other languages.
In Perl, this would be written as follows:
sub read_bytes {
my ($fh, $num_bytes_to_read) = #_;
my $buf = '';
while ($num_bytes_to_read) {
my $num_bytes_read = read($fh, $buf, $num_bytes_to_read, length($buf));
if (!$num_bytes_read) {
die "$!\n" if !defined($num_bytes_read);
die "Premature EOF\n";
}
$num_bytes_to_read -= $num_bytes_read;
}
return $buf;
}
sub read_uint32le { unpack('V', read_bytes($_[0], 4)) }
sub read_pstr { read_bytes($_[0], read_uint32le($_[0])) }
my $str = read_pstr($fh);
In C,
int read_bytes(FILE* fh, void* buf, size_t num_bytes_to_read) {
while (num_bytes_to_read) {
size_t num_bytes_read = fread(buf, 1, num_bytes_to_read, fh);
if (!num_bytes_read)
return 0;
num_bytes_to_read -= num_bytes_read;
buf += num_bytes_read;
}
return 1;
}
int read_uint32le(FILE* fh, uint32_t* p_i) {
int ok = read_bytes(fh, p_i, sizeof(*p_i));
if (!ok)
return 0;
{ /* Rearrange bytes on non-LE machines */
const char* p = (char*)p_i;
*p_i = ((((p[3] << 8) | p[2]) << 8) | p[1]) << 8) | p[0];
}
return 1;
}
char* read_pstr(FILE* fh) {
uint32_t len;
char* buf = NULL;
int ok;
ok = read_uint32le(fh, &len);
if (!ok)
goto ERROR;
buf = malloc(len+1);
if (!buf)
goto ERROR;
ok = read_bytes(fh, buf, len);
if (!ok)
goto ERROR;
buf[len] = '\0';
return buf;
ERROR:
if (p)
free(p);
return NULL;
}
char* str = read_pstr(fh);