Convert from hex string (from UCS-2) into UTF-8 - c#

I'm using a third party SMS provider and have hit an issue with converting from UCS-2 messages back into readable text.
Their API documentation has this code sample which converts UCS-2 messges into what I'm picking up on the API.
string message = "Это тестовое сообщение юникода";
byte[] ba = Encoding.BigEndianUnicode.GetBytes (message);
var hexString = BitConverter.ToString (ba);
Console.WriteLine ("#U" + hexString.Replace("-",""));
Which converts the message string into
#U042D0442043E00200442043504410442043E0432043E043500200441043E043E043104490435043D043804350020044E043D0438043A043E04340430
This looks like the UCS-2 messages I'm picking up from their API.
Unfortunately they don't give any code samples of how to convert the messages back into a readable form.
I'm sure its not there in the docs because its something simple - but I just seem to figure out how to do it.

To reverse what you have (the string of hex prefixed with #U)
var message = "Это тестовое сообщение юникода";
var ba = Encoding.BigEndianUnicode.GetBytes(message);
var hexString = BitConverter.ToString(ba);
var encoded = "#U" + hexString.Replace("-", "");
Console.WriteLine(encoded);
// reverse
var bytes = Enumerable.Range(2, encoded.Length-2)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(encoded.Substring(x, 2), 16))
.ToArray();
var result = Encoding.BigEndianUnicode.GetString(bytes);
Console.WriteLine(result);
Output
#U042D0442043E00200442043504410442043E0432043E043500200441043E043E043104490435043D043804350020044E043D0438043A043E04340430
Это тестовое сообщение юникода
Demo here

looks like this would be the reverse:
string message = Encoding.BigEndianUnicode.GetString(ba);
The extraction of bytes could be done by such a method:
private IEnumerable<byte> GetTheBytes(string uc2Message)
{
string bytesOnly = uc2Message.Trim('#', 'U');
for (int i = 0; i < bytesOnly.Length-2; i+=2)
{
yield return Convert.ToByte($"{bytesOnly[i]}{bytesOnly[i+1]}", 16);
}
}
Console.WriteLine(Encoding.BigEndianUnicode.GetString(GetTheBytes(uc2Message).ToArray()));

Related

Convert emoji to hex number

I'm trying to convert an emoji to an hex number or a string.
there is any way to convert this 👱 in this : 0x00000000D83DDC71L or D83DDC71
Edit
my code is this:
var bytes = Encoding.UTF8.GetBytes(emoji.ToString()); //emoji is 👱
var number = BitConverter.ToUInt32(bytes, 0); //number is 2610470896
var emojiCode = unicode.ToString("X"); // emojiCode is 9B989FF0
the problem is that i need my emojiCode to be D83DDC71
i hope is more clear now.
You have to do something like:
var str = "\uD83D\uDC71";
string res = BitConverter.ToString(Encoding.BigEndianUnicode.GetBytes(str)).Replace("-", "");
Note that you want your Unicode string to be in "big endian" mode (so Encoding.BigEndianUnicode)
Probably easier without going through the Encoding conversion:
string res = string.Concat(str.Select(x => ((ushort)x).ToString("X4")));
(ushort and char are nearly the same thing, but ushort is built to be formatted as a number, while char is built to be formatted as a character)
Emoji Unicode is not a single hex number, and it only encoding by UTF32.
So you could split it, like this:
byte[] utfBytes = System.Text.Encoding.UTF32.GetBytes("👱");
print(utfBytes.Length);
for (int i = 0; i < utfBytes.Length; i += 4)
{
if (i != 0) result += '-';
result += System.BitConverter.ToInt32(utfBytes, i).ToString("x2").ToUpper();
}

Converting PHP function to C#

I've almost finished converting a PHP script to a C# one to use in ASP.net.
I've already converted this to match the the correct result so far:-
function sign_url($url, $key, $secret)
{
if (strpos($url,'?') !== false)
{
$url .= "&";
}
else
{
$url .= "?";
}
$url .= "ApplicationKey=" . $key;
$signature = hash_hmac("sha1", urldecode($url), $secret);
$url .= "&Signature=" . hex_to_base64($signature);
return $url;
}
Here is the part i'm struggling with:-
function hex_to_base64($hex){
$return = '';
foreach(str_split($hex, 2) as $pair){
$return .= chr(hexdec($pair));
}
return base64_encode($return);
}
From what i can gather it splits the input string ($hex) into sets of 2 characters.
It then converts these hexadecimal strings to decimal numbers before getting the chr() value of this number.
Storing all these conversions in a $return string.
Finally it does the base 64 conversion.
I'm struggling to find the best way to do this in C#, particularly the splitting into 2 characters while doing the other conversions.
Old, good, C-style code that do not use LINQ and megabytes of other libraries:
static string HexToBase64(string hex)
{
byte[] buf = new byte[hex.Length / 2];
for (int i = 0; i < hex.Length / 2; i++)
{
buf[i] = Convert.ToByte(hex.Substring(i*2,2), 16);
}
return Convert.ToBase64String(buf);
}
You can do it using two amazing .Net technologies - LINQ and Rx:
private static string HexStringToBase64String(string hexString)
{
var byteArray = hexString
.ToObservable()
.Buffer(2)
.Select(pair => new string(pair.ToArray()))
.Select(numberString => Byte.Parse(numberString, NumberStyles.HexNumber))
.ToEnumerable()
.ToArray();
return Convert.ToBase64String(byteArray);
}
There's no need to split each hex byte into two characters, or loop through each character pair for that matter.
Take a look at the following questions:
How to convert numbers between hexadecimal and decimal in C#?
How can I convert a hex string to a byte array?
I would do something like this
public static string HexToBase64String(string hex)
{
if (string.IsNullOrEmpty(hex) || hex.Length % 2 == 1)
throw new ArgumentException("Invalid hex value", "hex");
var bytes = Enumerable.Range(0, hex.Length)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(hex.Substring(x, 2), 16))
.ToArray();
return System.Convert.ToBase64String(bytes);
}
Try these:
str_split = String.Split() (http://msdn.microsoft.com/en-us/library/system.string.split(v=vs.110).aspx)
base64_encode = Convert.ToBase64String() (http://msdn.microsoft.com/en-us/library/system.convert.tobase64string(v=vs.110).aspx)
chr = Convert.ToChar() (http://msdn.microsoft.com/en-us/library/system.convert.tochar(v=vs.110).aspx)
hexdec = .ToString("X") (http://msdn.microsoft.com/en-us/library/8wch342y(v=vs.110).aspx)
hash_hmac = SHA1.ComputeHash() (http://msdn.microsoft.com/en-us/library/s02tk69a(v=vs.110).aspx)

Calculating SHA1 From Hex/Binary String in C#

After 24 hours worth of programming I finally cracked, I want to make a C# WindowsForm application that shows you the value of a hashed hexstring input like in hashcalc.
I could only make it for textstring input even after googling it.
To demonstrate, the input 060201080808040602040909080909003583150369840500 should output d8f6b336a4df3336bf7de58a38b1189f6c5ce1e8
and not a6879cb4510b18e8f41b3491ce474fd2ff9e2979
Also this is for SHA1 Hashing so keep it only at that, Thanks!
For this 090505050509050009080003000605003569190380108300
I have 3b8d562adb792985a7393a6ab228aa6e7526410a, not 3b8d562adb792985a7393a6ab228aa6e752641a
I think that the last byte is wrong.
I think I don't really understand your problem. You can hash any input and output it any way you want. To achieve this, you are likely to use the Encoding class with an encoding of your choice and call the GetBytes() method. Then you take the SHA1 class and let it calculate the hash value. And for user text you tell the string class to use hex formatting for numbers. And this not only applies to the SHA1 class ;)
You need to import the namespace:
using System.Security.Cryptography
and call
var hash = new SHA1CryptoServiceProvider().ComputeHash(inputBytes);
to produce the hash.
If your problem is about converting the hex string to bytes, here's a full sample that shows how to parse the input and format the output:
var input = "060201080808040602040909080909003583150369840500";
// parse the input into a byte[]
var inputBytes = Enumerable.Range(0, input.Length/2)
.Select(i => input.Substring(i*2, 2))
.Select(s => byte.Parse(s, NumberStyles.HexNumber))
.ToArray();
var hash = new SHA1CryptoServiceProvider().ComputeHash(inputBytes);
var outputHexString = string.Join(" ",
hash.Select(b => b.ToString("X")).ToArray());
Console.WriteLine(outputHexString);
Here's how it works: http://ideone.com/BE7ecU
private void button1_Click(object sender, EventArgs e)
{
string input= "060201080808040602040909080909003583150369840500";
SHA1CryptoServiceProvider sha1 = new SHA1CryptoServiceProvider();
byte[] hash = sha1.ComputeHash(ConvertHexStringToByteArray(input));
string delimitedHexHash = BitConverter.ToString(hash);
string hexHash = delimitedHexHash.Replace("-", "");
MessageBox.Show(hexHash);
}
public static byte[] ConvertHexStringToByteArray(string hexString)
{
if (hexString.Length % 2 != 0)
{
throw new ArgumentException(String.Format(CultureInfo.InvariantCulture, "The binary key cannot have an odd number of digits: {0}", hexString));
}
byte[] HexAsBytes = new byte[hexString.Length / 2];
for (int index = 0; index < HexAsBytes.Length; index++)
{
string byteValue = hexString.Substring(index * 2, 2);
HexAsBytes[index] = byte.Parse(byteValue, NumberStyles.HexNumber, CultureInfo.InvariantCulture);
}
return HexAsBytes;
}

Converting string to byte array in C# (CSV)

I wrote a function to convert byte[] to string, and I add ";" after each byte. Now I want to convert this string to byte[] by splitting the string (similar to a CSV string).
public string ByteArrayToString(byte[] byteArray,string s)
{
for (int i = 0; i < byteArray.Length; i++)
{
s += byteArray[i].ToString() + ";";
}
s = s.Substring(0, s.Length - 1);
return s;
}
How could I write a function to convert this string to that byte array again?
try this
var byteArray = new byte[] {123, 11, 111};
var stringBytes = string.Join(";", byteArray.Select(b => b.ToString()));
var newByteArray = stringBytes.Split(';').Select(s => byte.Parse(s)).ToArray();
I guess that you want to get rid of the ; when converting also. I think you want to do something like this:
byte[] result = Encoding.UTF8.GetBytes(s.Replace(";",""));
This will fail if the original byte array actually contains a ;that is valid data, but in that case you will have lots of problems anyway since your "CSV" file will be wrongly formatted.
Consider using Split String
StringBuilder will be useful instead of String (Performance wise).
With StringBuilder:
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(objStringBuilder.ToString());
with String:
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(objString);
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
return encoding.GetBytes(yourStringVariable);
I know you already know the answer by now... but this code solves the problem, i hope it helps someone else.
int counter= 0;
string cadena = "8,5,6,3,4,6,3"
string[] foto = cadena.Split(',');
byte[] fotoFinal = new byte[foto.Length];
foreach (string s in foto)
{
fotoFinal[contador] = Convert.ToByte(s);
counter++;
}
str.Split(new char[]{';'},
StringSplitOptions.RemoveEmptyEntries).Select(s => byte.Parse(s)).ToArray();
Simply :)
public static byte[] Bytes ( this string Key )
{
return Enumerable.Range(0, Key.Binary().Length / 8 )
.Select(Index => Convert.ToByte(
Key.Binary().Substring(Index * 8, 8), 2))
.ToArray();
}
string[] sbytes = sl.Split(',');
byte[] b = new byte[sbytes.Length];
for (int j = 0; j < sbytes.Length; j++)
{
byte newByte = byte.Parse(sbytes[j], System.Globalization.NumberStyles.HexNumber);
b[j] = newByte;
}
I like using number styles hex number.

Calculating info_hash from UrlEncoded query string

Basically, I'm building a small tracker for experimental purposes. I've gotten quite far, and am now working on the announce part.
What I really can't figure out is how I should decode the info_hash query string provided.
From the specification, it is a urlencoded 20-byte SHA1 hash, which made me write this code,
byte[] foo = Encoding.Default.GetBytes(HttpUtility.UrlDecode(infoHash));
string temp = "";
foreach (byte b in foo)
{
temp += b.ToString("X");
}
Which gives 'temp' the following value,
5D3F3F3F3F5E3F3F3F153FE4033683F55693468
The first and last few characters are correct. This is the raw info_hash,
%5d%96%b6%f6%84%5e%ea%da%c5%15%c4%0e%403h%b9Ui4h
And this is what both uTorrent and my own tracker gives me as info_hash when generating it from the torrent file,
5D96B6F6845EEADAC515C40E403368B955693468
What am I doing wrong?
UrlDecode returns a string, but a SHA1 hash doesn't make sense if interpreted as (ANSI) string.
You need to decode the input string directly to an byte array, without the roundtrip to a string.
var s = "%5d%96%b6%f6%84%5e%ea%da%c5%15%c4%0e%403h%b9Ui4h";
var ms = new MemoryStream();
for (var i = 0; i < s.Length; i++)
{
if (s[i] == '%')
{
ms.WriteByte(
byte.Parse(s.Substring(i + 1, 2), NumberStyles.AllowHexSpecifier));
i += 2;
}
else if (s[i] < 128)
{
ms.WriteByte((byte)s[i]);
}
}
byte[] infoHash = ms.ToArray();
string temp = BitConverter.ToString(infoHash);
// "5D-96-B6-F6-84-5E-EA-DA-C5-15-C4-0E-40-33-68-B9-55-69-34-68"
HttpUtility.UrlDecodeToBytes

Categories

Resources