Develop Reference

Why do I get a different value after turning an integer into ASCII and then back to an integer?

Why, when I turn INT value to bytes and to ASCII and back, I get another value?
Example:
var asciiStr = new string(Encoding.ASCII.GetChars(BitConverter.GetBytes(2000)));
var intVal = BitConverter.ToInt32(Encoding.ASCII.GetBytes(asciiStr), 0);
Console.WriteLine(intVal);
// Result: 1855

ASCII is only 7-bit - code points above 127 are unsupported. Unsupported characters are converted to ? per the docs on Encoding.ASCII:
The ASCIIEncoding object that is returned by this property might not have the appropriate behavior for your app. It uses replacement fallback to replace each string that it cannot encode and each byte that it cannot decode with a question mark ("?") character.
So 2000 decimal = D0 07 00 00 hexadecimal (little endian) = [unsupported character] [BEL character] [NUL character] [NUL character] = ? [BEL character] [NUL character] [NUL character] = 3F 07 00 00 hexadecimal (little endian) = 1855 decimal.

TL;DR: Everything's fine. But you're a victim of character replacement.
We start with 2000. Let's acknowledge, first, that this number can be represented in hexadecimal as 0x000007d0.
BitConverter.GetBytes
BitConverter.GetBytes(2000) is an array of 4 bytes, Because 2000 is a 32-bit integer literal. So the 32-bit integer representation, in little endian (least significant byte first), is given by the following byte sequence { 0xd0, 0x07, 0x00, 0x00 }. In decimal, those same bytes are { 208, 7, 0, 0 }
Encoding.ASCII.GetChars
Uh oh! Problem. Here's where things likely took an unexpected turn for you.
You're asking the system to interpret those bytes as ASCII-encoded data. The problem is that ASCII uses codes from 0-127. The byte with value 208 (0xd0) doesn't correspond to any character encodable by ASCII. So what actually happens?
When decoding ASCII, if it encounters a byte that is out of the range 0-127 then it decodes that byte to a replacement character and moves to the next byte. This replacement character is a question mark ?. So the 4 chars you get back from Encoding.ASCII.GetChars are ?, BEL (bell), NUL (null) and NUL (null).
BEL is the ASCII name of the character with code 7, which traditionally elicits a beep when presented on a capable terminal. NUL (code 0) is a null character traditionally used for representing the end of a string.
new string
Now you create a string from that array of chars. In C# a string is perfectly capable of representing a NUL character within the body of a string, so your string will have two NUL chars in it. They can be represented in C# string literals with "\0", in case you want to try that yourself. A C# string literal that represents the string you have would be "?\a\0\0" Did you know that the BEL character can be represented with the escape sequence \a? Many people don't.
Encoding.ASCII.GetBytes
Now you begin the reverse journey. Your string is comprised entirely of characters in the ASCII range. The encoding of a question mark is code 63 (0x3F). And the BEL is 7, and the NUL is 0. so the bytes are { 0x3f, 0x07, 0x00, 0x00 }. Surprised? Well, you're encoding a question mark now where before you provided a 208 (0xd0) byte that was not representable with ASCII encoding.
BitConverter.ToInt32
Converting these four bytes back to a 32-bit integer gives the integer 0x0000073f, which, in decimal, is 1855.

String encoding (ASCII, UTF8, SHIFT_JIS, etc.) is designed to pigeonhole human language into a binary (byte) form. It isn't designed to store arbitrary binary data, such as the binary form of an integer.
While your binary data will be interpreted as a string, some of the information will be lost, meaning that storing binary data in this way will fail in the general case. You can see the point where this fails using the following code:
for (int i = 0; i < 255; ++i)
{
var byteData = new byte[] { (byte)i };
var stringData = System.Text.Encoding.ASCII.GetString(byteData);
var encodedAsBytes = System.Text.Encoding.ASCII.GetBytes(stringData);
Console.WriteLine("{0} vs {1}", i, (int)encodedAsBytes[0]);
}
Try it online
As you can see it starts off well because all of the character codes correspond to ASCII characters, but once we get up in the numbers (i.e. 128 and beyond), we start to require a more than 7 bits to store the binary value. At this point it ceases to be decoded correctly, and we start seeing 63 come back instead of the input value.
Ultimately you will have this problem encoding binary data using any string encoding. You need to choose an encoding method specifically meant for storing binary data as a string.
Two popular methods are:
Hexadecimal
Base64 using ToBase64String and FromBase64String
Hexadecimal example (using the hex methods here):
int initialValue = 2000;
Console.WriteLine(initialValue);
// Convert from int to bytes and then to hex
byte[] bytesValue = BitConverter.GetBytes(initialValue);
string stringValue = ByteArrayToString(bytesValue);
Console.WriteLine("As hex: {0}", stringValue); // outputs D0070000
// Convert form hex to bytes and then to int
byte[] decodedBytesValue = StringToByteArray(stringValue);
int intValue = BitConverter.ToInt32(decodedBytesValue, 0);
Console.WriteLine(intValue);
Try it online
Base64 example:
int initialValue = 2000;
Console.WriteLine(initialValue);
// Convert from int to bytes and then to base64
byte[] bytesValue = BitConverter.GetBytes(initialValue);
string stringValue = Convert.ToBase64String(bytesValue);
Console.WriteLine("As base64: {0}", stringValue); // outputs 0AcAAA==
// Convert form base64 to bytes and then to int
byte[] decodedBytesValue = Convert.FromBase64String(stringValue);
int intValue = BitConverter.ToInt32(decodedBytesValue, 0);
Console.WriteLine(intValue);
Try it online
P.S. If you simply wanted to convert your integer to a string (e.g. "2000") then you can simply use .ToString():
int initialValue = 2000;
string stringValue = initialValue.ToString();

C# Convert Hex String Array to Byte Array

I have a String[] of hex values "10" "0F" "3E" "42" stored.
I found this method to convert to a Byte[]
public static byte[] ToByteArray(String HexString)
{
int NumberChars = HexString.Length;
byte[] bytes = new byte[NumberChars / 2];
for (int i = 0; i < NumberChars; i += 2)
{
bytes[i / 2] = Convert.ToByte(HexString.Substring(i, 2), 16);
}
return bytes;
}
However this converts the values to the hex equivalent. But the values are already in the hex equivalent!
For example this makes "10" "0F" "3E" "42" into "16" "15" "62" "66".
I want it to directly copy the values as they are already the correct hex value.
Edit:
Basically...
I want a byte array with the literal characters in the String[] So say the second value in String[] is 0F. I want the first byte in Byte[] to be 0F and not 16
Any ideas?
Edit2
Let me clarify. I don't want to convert my String[] values into Hexadecimal, as they are already Hexadecimal. I want to directly copy them to a Byte[]
The problem is my string of values "10" "0F" "3E" 42" already has the hexadecimal value I want. I want the byte array to contain those exact values and not convert them, they are already hexadecimal form.

You have to convert (or parse) string in order to get byte since string and byte are different types:
// 10 == 10d
byte b = Convert.ToByte("10"); // if "10" is a decimal representation
// 16 == 0x10
byte b = Convert.ToByte("10", 16); // if "10" is a hexadecimal representation
If you want to process an array, you can try a simple Linq:
using System.Linq;
...
string[] hexValues = new string[] {
"10", "0F", "3E", "42"};
byte[] result = hexValues
.Select(value => Convert.ToByte(value, 16))
.ToArray();
If you want to print out result as hexadecimal, use formatting ("X2" format string - at least 2 hexadecimal digits, use captital letters):
// 10, 0F, 3E, 42
Console.Write(string.Join(", ", result.Select(b => b.ToString("X2"))));
Compare with same array but in a different format ("d2" - at least 2 decimal digits)
// 16, 15, 62, 66
Console.Write(string.Join(", ", result.Select(b => b.ToString("d2"))));
If no format provided, .Net uses default one and represents byte in decimal:
// 16, 15, 62, 66
Console.Write(string.Join(", ", result));

You're really confusing representation and numbers here.
A string like "0F" can be seen as a representation of a number in base 16, that is, in decimal representation, 16.
Which is the exact same thing as representing 16 as F or 0F or XVI or
IIIIIIIIIIIIIIII or whatever other representation you choose.
The string "0F" actually looks in memory like this
Hexadecimal representation:
0x30 0x46 0x00
Decimal representation:
48 70 0
Binary representation:
0b00110000 0b01000110 0b00000000

Byte is simply a data type which is infact a subset of an integer.
Byte takes interger values ranging from -2^7(-128) to 2^7-1$(127)
Calling Convert.ToByte(string, 16) simply converts your string to an equivalent hex value and then to an equivalent value in byte.
Note the byte data type is always an integer data but used in place of an integer just to save space in memory. As referenced above the byte datatype takes values from -128 to 127 thereby saving you more space in memory than the integer data type would.
Please Note that you are likely to run into an error if the hexadecimal value you wish to convert to byte is less than -128 or greater than 127
The link below shows an instance of this error when I try converting a string whose value when converted to hexadecimal is greater than 127.
Error when converting to Byte
You get an error whenever you do this.
I hope my answer and Dmitry Bychenko's sheds more light into your problem.
Please feel free to comment if it doesnt.

Why are ASCII values of a byte different when cast as Int32?

I'm in the process of creating a program that will scrub extended ASCII characters from text documents. I'm trying to understand how C# is interpreting the different character sets and codes, and am noticing some oddities.
Consider:
namespace ASCIITest
{
class Program
{
static void Main(string[] args)
{
string value = "Slide™1½”C4®";
byte[] asciiValue = Encoding.ASCII.GetBytes(value); // byte array
char[] array = value.ToCharArray(); // char array
Console.WriteLine("CHAR\tBYTE\tINT32");
for (int i = 0; i < array.Length; i++)
{
char letter = array[i];
byte byteValue = asciiValue[i];
Int32 int32Value = array[i];
//
Console.WriteLine("{0}\t{1}\t{2}", letter, byteValue, int32Value);
}
Console.ReadLine();
}
}
}
Output from program
CHAR BYTE INT32
S 83 83
l 108 108
i 105 105
d 100 100
e 101 101
T 63 8482 <- trademark symbol
1 49 49
½ 63 189 <- fraction
" 63 8221 <- smartquotes
C 67 67
4 52 52
r 63 174 <- registered trademark symbol
In particular, I'm trying to understand why the extended ASCII characters (the ones with my notes added to the right of the third column) show up with the correct value when cast as int32, but all show up as 63 when cast as the byte value. What's going on here?

ASCII.GetBytes conversion replaces all characters outside of ASCII range (0-127) with question mark (code 63).
So since your string contains characters outside of that range your asciiValue have ? instead of all interesting symbols like ™ - its Char (Unicode) repesentation is 8482 which is indeed outside of 0-127 range.
Converting string to char array does not modify values of characters and you still have original Unicode codes (char is essentially Int16) - casting it to longer integer type Int32 does not change the value.
Below are possible conversion of that character into byte/integers:
var value = "™";
var ascii = Encoding.ASCII.GetBytes(value)[0]; // 63(`?`) - outside 0-127 range
var castToByte = (byte)(value[0]); // 34 = 8482 % 256
var Int16 = (Int16)value[0]; // 8482
var Int32 = (Int16)value[0]; // 8482
Details available at ASCIIEncoding Class
ASCIIEncoding corresponds to the Windows code page 20127. Because ASCII is a 7-bit encoding, ASCII characters are limited to the lowest 128 Unicode characters, from U+0000 to U+007F. If you use the default encoder returned by the Encoding.ASCII property or the ASCIIEncoding constructor, characters outside that range are replaced with a question mark (?) before the encoding operation is performed.

Convert hexadecimal string to its numerical values in C# [duplicate]

This question already has answers here:
Convert integer to hexadecimal and back again
(11 answers)
Closed 9 years ago.
i have a text box on my form. I want to write "0x31" as a string to my textbox and then when i clicked a button, i want to convert this string to 0x31 as a hexadecimal value.
How can i convert this string to hexadecimal value?

int i = Convert.ToInt32("0x31", 16);
Console.WriteLine("0x" + i.ToString("X2"))

string hexValues = "48 65 6C 6C 6F 20 57 6F 72 6C 64 21";
string[] hexValuesSplit = hexValues.Split(' ');
foreach (String hex in hexValuesSplit)
{
// Convert the number expressed in base-16 to an integer.
int value = Convert.ToInt32(hex, 16);
// Get the character corresponding to the integral value.
string stringValue = Char.ConvertFromUtf32(value);
char charValue = (char)value;
Console.WriteLine("hexadecimal value = {0}, int value = {1}, char value = {2} or {3}",
hex, value, stringValue, charValue);
}
Example From: http://msdn.microsoft.com/en-us/library/bb311038.aspx

Hexadecimal is just a representation of an value, it is not a value itself.
This page will tell you everything you need to know about parsing and displaying hex in C#
http://msdn.microsoft.com/en-us/library/bb311038.aspx

First to clear up: The string is in hexadecimal format, when you convert it to a value it's just a numeric value, it's not hexadecimal.
Use the Int32.Parse method with the NumberStyle.HexNumber specifier:
string input = "0x31";
int n;
if (input.StartsWith("0x")) {
n = Int32.Parse(input.Substring(2), NumberStyles.HexNumber);
} else {
n = Int32.Parse(input);
}

The string hex value is a representation of a value. The actual string value can be converted to whatever you like(float, int etc.)
There are several ways to do the conversion. Simple example:
// convert to int from base 16
int value = Convert.ToInt32(hex, 16);

Note that hex is just a representation of a value - so what you are really asking is how you can parse a value from the string - do it like so:
int val = int.Parse("0x31", NumberStyles.HexNumber);
val now contains an int with the hex value 0x31.

convert Hex UTF-8 bytes to Hex code point

how can i convert
Hex UTF-8 bytes -E0 A4 A4 to hex code point - 0924
ref: http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=e0+a4+a4&mode=bytes
I need this because when i read Unicode data in c# it is taking it as single byte sequence and displaying 3 characters instead of 1,but i need 3 byte sequence(read 3 bytes and display single character),I tried many solutions but didn't get the result.
If I can display or store a 3-byte sequence utf-8 character then I don't need conversion.
senario is like this:
string str=getivrresult();
in str I have a word with each character as 3 byte utf-8 sequence.
Edited:
string str="त";
//i want it as "त" in str.
Character त
Character name DEVANAGARI LETTER TA
Hex code point 0924
Decimal code point 2340
Hex UTF-8 bytes E0 A4 A4
Octal UTF-8 bytes 340 244 244
UTF-8 bytes as Latin-1 characters bytes à ¤ ¤
Thank You.

Use the GetString methdod in the Encoding class:
byte[] data = { 0xE0, 0xA4, 0xA4 };
string str = Encoding.UTF8.GetString(data);
The string now contains one character with the character code 0x924.

//utf-8 Single Byte Sequence input
string str = "त";
int i = 0;
byte[] data=new byte[3];
foreach (char c in str)
{
string tmpstr = String.Format("{0:x2}", (int)c);
data[i] = Convert.ToByte(int.Parse(tmpstr, System.Globalization.NumberStyles.HexNumber));
i++;
}
//utf-8 3-Byte Sequence Output now stp contains "त".
string stp = Encoding.UTF8.GetString(data);