byte[] newBytes = new Byte[] { 169 };
string string1 = System.Text.Encoding.UTF8.GetString(newBytes, 0, newBytes.Length);
In the above program, I expected string1 to have the value of copyright symbol ©.
But I get some other value (possibly some junk) as shown below
Where did I go wrong?
UTF8 requires multiple bytes to encode character points greater than 127. If you run the reverse, you'll see what it expects:
System.Text.Encoding.UTF8.GetBytes("©"); // { 194, 169 }
Try this:
byte[] newBytes = new Byte[] { 194, 169 };
string string1 = System.Text.Encoding.UTF8.GetString(newBytes, 0, newBytes.Length);
If you absolutely have to use that original byte array, you'll need to pick a different encoding. For example, the Windows-1252 encoding uses a single byte to encode the copyright symbol:
byte[] newBytes = new Byte[] { 169 };
var encoding = Encoding.GetEncoding(1252);
string string1 = encoding.GetString(newBytes, 0, newBytes.Length); // "©"
Related
I'm using the .net port of libsodium. The hash generation function has two forms, one that accepts byte arrays and one that accepts strings:
public static byte[] ArgonHashBinary(string password, string salt, long opsLimit, int memLimit, long outputLength = ARGON_SALTBYTES)
public static byte[] ArgonHashBinary(byte[] password, byte[] salt, long opsLimit, int memLimit, long outputLength = ARGON_SALTBYTES)
What i'm having an issue with is both forms producing the same hash when the input values are identical.
var saltAsBytes = PasswordHash.ArgonGenerateSalt();
var saltAsString = Encoding.UTF8.GetString(saltAsBytes);
var tmp = Encoding.UTF8.GetBytes(saltAsString);
var hash1 = PasswordHash.ArgonHashBinary(password, saltAsString, 6, 134217728, 16);
var hash2 = PasswordHash.ArgonHashBinary( Encoding.UTF8.GetBytes(password), saltAsBytes, 6, 134217728, 16);
Anything with "PasswordHash." is libsodium and not my code.
From the code above when i convert it from a string and then back to a byte array the byte array. The byte array array is always a different length. ArgonGenerateSalt() produces a byte array with a length of 16. When i convert it back from a string above its generally ~30 (different every time because of different salts produced).
Why am i converting to UTF8? Because thats what they are doing internally:
https://github.com/adamcaudill/libsodium-net/blob/master/libsodium-net/PasswordHash.cs
public static byte[] ArgonHashBinary(string password, string salt, StrengthArgon limit = StrengthArgon.Interactive, long outputLength = ARGON_SALTBYTES)
{
return ArgonHashBinary(Encoding.UTF8.GetBytes(password), Encoding.UTF8.GetBytes(salt), limit, outputLength);
}
When i convert the salt to a UTF8 string the hashing function will fail because they are checking the length of the byte array to make sure its 16 bytes. If i convert it to a ASCII string it works but produces a different hash (which is expected).
To clarify the hashing piece in this code is not the issue. Figuring out why tmp is different then saltAsBytes is the key.
I think the problem here is that the ArgonGenerateSalt method doesn't return a UTF8 encoded string, it returns completely random bytes.
You can't decode random bytes as a UTF8 string and expect it to round trip. A trivial example to see where this blows up is to do the following:
var data = new byte[] { 128 };
var dataAsString = Encoding.UTF8.GetString( data );
var dataAsBytes = Encoding.UTF8.GetBytes( dataAsString );
After this, dataAsBytes will be 3 bytes (specifically 239, 191, 189).
Converting a byte array to string and then back again produced different results
A binary data may not be converted to string and then back to byte array
using Encoding.[AnyEncoding].GetBytes and Encoding.[AnyEncoding].GetString
Instead use Convert.ToBase64String and Convert.FromBase64String
You can easily test...
var bytes = new byte[] { 255, 255, 255 };
var buf = Encoding.UTF8.GetString(bytes);
var newbytes = Encoding.UTF8.GetBytes(buf);
newbytes's length will be 9.....
Edit: This is the test case for #Theo
var bytes = new byte[] { 0, 216 }; //any new byte[] { X, 216 };
var buf = Encoding.Unicode.GetString(bytes);
var newbytes = Encoding.Unicode.GetBytes(buf); //253,255
I have a string which contains unicode data.
I want to write it in a file . When the data is written in file it gives me simple unicode value instead of languages other than english.
string originalString = ((char)(buffer[index])).ToString();
//sb.Append(DecodeEncodedNonAsciiCharacters(originalString.ToString()));
foreach (char c1 in originalString)
{
// test if char is ascii, otherwise convert to Unicode Code Point
int cint = Convert.ToInt32(c1);
if (cint <= 127 && cint >= 0)
asAscii.Append(c1.ToString());
else
{
//String s = Char.ConvertFromUtf32(cint);
asAscii.Append(String.Format("\\u{0:x4} ", cint).Trim());
// asAscii.Append(s);
}
}
sb.Append((asAscii));
Console.WriteLine();
when i see the output file the data shows like this
1 00:00:27,709-->00:00:32,959
1.2 \u00e0\u00a4\u0085\u00e0\u00a4\u00b0\u00e0\u00a4\u00ac \u00e0\u00a4\u00b2\u00e0\u00a5\u008b\u00e0\u00a4\u0097 28
\u00e0\u00a4\u00b0\u00e0\u00a4\u00be\u00e0\u00a4\u009c\u00e0\u00a5\u008d\u00e0\u00a4\u00af
\u00e0\u00a4\u0094\u00e0\u00a4\u00b0
\u00e0\u00a4\u00b8\u00e0\u00a4\u00be\u00e0\u00a4\u00a4
\u00e0\u00a4\u0095\u00e0\u00a5\u0087\u00e0\u00a4\u0082\u00e0\u00a4\u00a6\u00e0\u00a5\u008d\u00e0\u00a4\u00b0
\u00e0\u00a4\u00b6\u00e0\u00a4\u00be\u00e0\u00a4\u00b8\u00e0\u00a4\u00bf\u00e0\u00a4\u00a4
\u00e0\u00a4\u00aa\u00e0\u00a5\u008d\u00e0\u00a4\u00b0\u00e0\u00a4\u00a6\u00e0\u00a5\u0087\u00e0\u00a4\u00b6
but it should look like this
1 00:00:27,400 --> 00:00:32,760
1.2 अरब लोग 28 राज्य और सात केंद्र शासित प्रदेश
I have tried many things but none has done my job.
string unicodeString = "This string contains the unicode character Pi(\u03a0)";
// Create two different encodings.
Encoding ascii = Encoding.ASCII;
Encoding unicode = Encoding.Unicode;
// Convert the string into a byte[].
byte[] unicodeBytes = unicode.GetBytes(unicodeString);
// Perform the conversion from one encoding to the other.
byte[] asciiBytes = Encoding.Convert(unicode, ascii, unicodeBytes);
// Convert the new byte[] into a char[] and then into a string.
// This is a slightly different approach to converting to illustrate
// the use of GetCharCount/GetChars.
char[] asciiChars = new char[ascii.GetCharCount(asciiBytes, 0, asciiBytes.Length)];
ascii.GetChars(asciiBytes, 0, asciiBytes.Length, asciiChars, 0);
string asciiString = new string(asciiChars);
// Display the strings created before and after the conversion.
Console.WriteLine("Original string: {0}", unicodeString);
Console.WriteLine("Ascii converted string: {0}", asciiString);
I am reading a file into a byte array and converting the byte array into a string to pass into a method(I cant pass the byte array itself) and in the function definition I am reconverting the string to byte array. but both the byte arrays( before and after conversion are different)
I am using the following pilot code to test if byte arrays are same.
byte[] bytes = File.ReadAllBytes(#"C:\a.jpg");
string encoded = Convert.ToBase64String(bytes);
byte[] bytes1 = Encoding.ASCII.GetBytes(encoded);
When I use bytes in the api call, it succeds and when I use bytes1 it throws an exception. Please tell me how can I safely convert the byte array to string and back such that both arrays reman same.
Use this:
byte[] bytes = File.ReadAllBytes(#"C:\a.jpg");
string encoded = Convert.ToBase64String(bytes);
byte[] bytes1 = Convert.FromBase64String(encoded);
I'll post a response from another thread:
static byte[] GetBytes(string str)
{
byte[] bytes = new byte[str.Length * sizeof(char)];
System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
return bytes;
}
static string GetString(byte[] bytes)
{
char[] chars = new char[bytes.Length / sizeof(char)];
System.Buffer.BlockCopy(bytes, 0, chars, 0, bytes.Length);
return new string(chars);
}
full thread here: How do I get a consistent byte representation of strings in C# without manually specifying an encoding?
I have a string and want to convert it to a byte array of hex value using C#.
for eg, "Hello World!" to byte[] val=new byte[] {0x48, 0x65, 0x6C, 0x6C, 0x6F, 0x20, 0x57, 0x6F, 0x72, 0x6C, 0x64, 0x21};,
I see the following code in Converting string value to hex decimal
string input = "Hello World!";
char[] values = input.ToCharArray();
foreach (char letter in values)
{
// Get the integral value of the character.
int value = Convert.ToInt32(letter);
// Convert the decimal value to a hexadecimal value in string form.
string hexOutput = String.Format("0x{0:X}", value);
Console.WriteLine("Hexadecimal value of {0} is {1}", letter, hexOutput);
}
I want this value into byte array but can't write like this
byte[] yy = new byte[values.Length];
yy[i] = Convert.ToByte(Convert.ToInt32(hexOutput));
I try this code referenced from How to convert a String to a Hex Byte Array? where I passed the hex value 48656C6C6F20576F726C6421 but I got the decimal value not hex.
public byte[] ToByteArray(String HexString)
{
int NumberChars = HexString.Length;
byte[] bytes = new byte[NumberChars / 2];
for (int i = 0; i < NumberChars; i += 2)
{
bytes[i / 2] = Convert.ToByte(HexString.Substring(i, 2), 16);
}
return bytes;
}
and I also try code from How can I convert a hex string to a byte array?
But once I used Convert.ToByte or byte.Parse , the value change to decimal value.
How should I do?
Thanks in advance
I want to send 0x80 (i.e, 128) to serial port but when I copy and paste the character equivalent to 128 to the variable 'input' and convert to byte, I got 63 (0x3F). So I think I need to send hex array. I think I got the wrong idea. Pls see screen shot.
For now, I solve this to combine byte arrays.
string input = "Hello World!";
byte[] header = new byte[] { 2, 48, 128 };
byte[] body = Encoding.ASCII.GetBytes(input);
Hexadecimal has nothing to do with this, your desired result is nothing more nor less than an array of bytes containing the ASCII codes.
Try Encoding.ASCII.GetBytes(s)
There's something strange with your requirement:
I have a string and want to convert it to a byte array of hex value
using C#.
An byte is just an 8-bit value. You can present it as decimal (e.g. 16) or hexidecimal (e.g. 0x10).
So, what do you realy want?
In case you are really wanting to get a string which contains the hex representation of an array of bytes, here's how you can do that:
public static string BytesAsString(byte[] bytes)
{
string hex = BitConverter.ToString(bytes); // This puts "-" between each value.
return hex.Replace("-",""); // So we remove "-" here.
}
It seems like you’re mixing converting to array and displaying array data.
When you have array of bytes it’s just array of bytes and you can represent it in any possible way binary, decimal, hexadecimal, octal, whatever… but that is only valid if you want to visually represent these.
Here is a code that manually converts string to byte array and then to array of strings in hex format.
string s1 = "Stack Overflow :)";
byte[] bytes = new byte[s1.Length];
for (int i = 0; i < s1.Length; i++)
{
bytes[i] = Convert.ToByte(s1[i]);
}
List<string> hexStrings = new List<string>();
foreach (byte b in bytes)
{
hexStrings.Add(Convert.ToInt32(b).ToString("X"));
}
This is my code:
byte[] base64String = //this is being set properly
var base64CharArray = new char[base64String.Length];
Convert.ToBase64CharArray(base64String,
0,
base64String.Length,
base64CharArray,
0);
var Base64String = new string(base64CharArray);
When i run this, I get the following error when calling Convert.ToBase64CharArray:
Either offset did not refer to a position in the string, or there is an insufficient length of destination character array. Parameter name: offsetOut
How do i fix this, so i can convert my byte array to a string, or is there a better way to convert a byte array to a string?
Why do you need the char array? Just convert your byte[] directly to a Base64 string:
string base64String = Convert.ToBase64String(myByteArray);
base64 encoding needs 4 characters to encode 3 bytes of input. you have to enlarge your output array.
here is one way you can convert byte array to string
static byte[] GetBytes(string str)
{
byte[] bytes = new byte[str.Length * sizeof(char)];
System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
return bytes;
}
static string GetString(byte[] bytes)
{
char[] chars = new char[bytes.Length / sizeof(char)];
System.Buffer.BlockCopy(bytes, 0, chars, 0, bytes.Length);
return new string(chars);
}
you don't really need to worry about encoding.
more details can be found here
This is a simple form of doing it
string System.Text.Encoding.UTF8.GetString(YourbyteArray)