Develop Reference

Best way to transform string to valid encoding in C#

Sorry in advance if you have a duplicate or a simple question! I can't find the answer.
I'm working with a dll made in Delphi. Data can be sent to the device using a DLL. However, at the time the data is sent, some strings are not accepted or are written blank. The data sent to the device is stored in a txt file. It was generated using txt file third party program.
That is, I think the string is in an indefinite format. If I send in utf-8 format, it receives all the information. But some strings at the time ???? ???? remains.
Many of my texts are in the Cyrillic alphabet.
What I did:
// string that send to device
[MarshalAsAttribute(UnmanagedType.LPStr, SizeConst = 36)]
public string Name;
When I did this, the device received only 10 out of 100 data.
If i encoding with UTF-8:
byte[] bytes = Encoding.Default.GetBytes(getDvsName[1].ToString());
string res = Encoding.UTF8.GetString(bytes);
Got all the data this way but too many strings are became as ??? ????.
Also i tried like this:
static private string Win1251ToUTF8(string source)
{
Encoding utf8 = Encoding.GetEncoding(«utf-8»);
Encoding win1251 = Encoding.GetEncoding(«windows-1251»);
byte[] utf8Bytes = win1251.GetBytes(source);
byte[] win1251Bytes = Encoding.Convert(win1251, utf8, utf8Bytes);
source = win1251.GetString(win1251Bytes);
return source;
}
All of the above methods did not help. How can I receive incoming information in the correct format? Are there other ways?

hi there here is what went wrong you did encode the string to default instead of utf8.
string tom = "ටොම් හැන්ක්ස්";
byte[] bytes = Encoding.UTF8.GetBytes(tom);
string res = Encoding.UTF8.GetString(bytes);

Equivalent of System.IO.File.ReadAllBytes(filePath)

Im trying to get all bytes from a file but the issue is that i only have access to the file once so what im doing is saving the file´s string in a txt file so when i need to use it i read the saved file´s string and convert the string to a byte array but something is not working,
basicaly what i need is the equivalent of System.IO.File.ReadAllBytes(filePath) what is actually working good but is not what i need in this case.
i've tried this out so far
public byte[] getByteArray(string fileString)
{
Encoding utf8 = Encoding.utf8;
Encoding ascii = Encoding.ASCII;
Encoding unicode = Encoding.Unicode;
return utf8.GetBytes(fileString);
}
i have tried every encoding of that class but didn't work

how can I safely convert byte array to string in C# on Linux (under mono)?

Currently I am using something like this:
private static ASCEncoding = new Encoding();
...
...
and my method:
...
public object some_method(object BinaryRequest)
{
byte[] byteRequest = (byte[])BinaryRequest;
string strRequest = ASCEncoding.GetString(byteRequest);
...
}
some characters when checked under Windows are different when checked Under Linux
9I9T (win)
98T (linux)

When you are communicating between systems, it's a good idea to use a specific and documented encoding for your text. For text written in the English language (including programming languages which use English for keywords/etc), the UTF-8 encoding is likely to use the fewest overall number of bytes in the encoded representation.
byte[] byteRequest = (byte[])BinaryRequest;
string strRequest = Encoding.UTF8.GetString(byteRequest);
Obviously to use this, you are expected to produce your requests using the same encoding.
string strRequest = ...
byte[] byteRequest = Encoding.UTF8.GetBytes(strRequest);

string stringValue = Encoding.Default.GetString(byteArray);

Sending a string containing special characters through a TcpClient (byte[])

I'm trying to send a string containing special characters through a TcpClient (byte[]). Here's an example:
Client enters "amé" in a textbox
Client converts string to byte[] using a certain encoding (I've tried all the predefined ones plus some like "iso-8859-1")
Client sends byte[] through TCP
Server receives and outputs the string reconverted with the same encoding (to a listbox)
Edit :
I forgot to mention that the resulting string was "am?".
Edit-2 (as requested, here's some code):
#DJKRAZE here's a bit of code :
byte[] buffer = Encoding.ASCII.GetBytes("amé");
(TcpClient)server.Client.Send(buffer);
On the server side:
byte[] buffer = new byte[1024];
Client.Recieve(buffer);
string message = Encoding.ASCII.GetString(buffer);
ListBox1.Items.Add(message);
The string that appears in the listbox is "am?"
=== Solution ===
Encoding encoding = Encoding.GetEncoding("iso-8859-1");
byte[] message = encoding.GetBytes("babé");
Update:
Simply using Encoding.Utf8.GetBytes("ééé"); works like a charm.

Never too late to answer a question I think, hope someone will find answers here.
C# uses 16 bit chars, and ASCII truncates them to 8 bit, to fit in a byte. After some research, I found UTF-8 to be the best encoding for special characters.
//data to send via TCP or any stream/file
byte[] string_to_send = UTF8Encoding.UTF8.GetBytes("amé");
//when receiving, pass the array in this to get the string back
string received_string = UTF8Encoding.UTF8.GetString(message_to_send);

Your problem appears to be the Encoding.ASCII.GetBytes("amé"); and Encoding.ASCII.GetString(buffer); calls, as hinted at by '500 - Internal Server Error' in his comments.
The é character is a multi-byte character which is encoded in UTF-8 with the byte sequence C3 A9. When you use the Encoding.ASCII class to encode and decode, the é character is converted to a question mark since it does not have a direct ASCII encoding. This is true of any character that has no direct coding in ASCII.
Change your code to use Encoding.UTF8.GetBytes() and Encoding.UTF8.GetString() and it should work for you.

Your question and your error is not clear to me but using Base64String may solve the problem
Something like this
static public string EncodeTo64(string toEncode)
{
byte[] toEncodeAsBytes
= System.Text.ASCIIEncoding.ASCII.GetBytes(toEncode);
string returnValue
= System.Convert.ToBase64String(toEncodeAsBytes);
return returnValue;
}
static public string DecodeFrom64(string encodedData)
{
byte[] encodedDataAsBytes
= System.Convert.FromBase64String(encodedData);
string returnValue =
System.Text.ASCIIEncoding.ASCII.GetString(encodedDataAsBytes);
return returnValue;
}

Vietnamese character in .NET Console Application (UTF-8)

I'm trying to write down a UTF-8 string (Vietnamese) into C# Console but no success. I'm running on Windows 7.
I tried to use the Encoding class that convert string to char[] to byte[] and then to String, but no help, the string is input directly from the database.
Here is some example
Tôi tên là Đức, cuộc sống thật vui vẻ
tuyệt vời
It does not show the special character like Đ or ứ... instead it show up ?, much worse than with the Encoding class.
Does anyone can try this out or know about this problem?
My code
static void Main(string[] args)
{
XDataContext _new = new XDataContext();
Console.OutputEncoding = Encoding.GetEncoding("UTF-8");
string srcString = _new.Posts.First().TITLE;
Console.WriteLine(srcString);
// Convert the UTF-16 encoded source string to UTF-8 and ASCII.
byte[] utf8String = Encoding.UTF8.GetBytes(srcString);
byte[] asciiString = Encoding.ASCII.GetBytes(srcString);
// Write the UTF-8 and ASCII encoded byte arrays.
Console.WriteLine("UTF-8 Bytes: {0}", BitConverter.ToString(utf8String));
Console.WriteLine("ASCII Bytes: {0}", BitConverter.ToString(asciiString));
// Convert UTF-8 and ASCII encoded bytes back to UTF-16 encoded
// string and write.
Console.WriteLine("UTF-8 Text : {0}", Encoding.UTF8.GetString(utf8String));
Console.WriteLine("ASCII Text : {0}", Encoding.ASCII.GetString(asciiString));
Console.WriteLine(Encoding.UTF8.GetString(utf8String));
Console.WriteLine(Encoding.ASCII.GetString(asciiString));
}
and here is the outstanding output
Nhà báo đi hội báo Xuân
UTF-8 Bytes: 4E-68-C3-A0-20-62-C3-A1-6F-20-C4-91-69-20-68-E1-BB-99-69-20-62-C3-
A1-6F-20-58-75-C3-A2-6E
ASCII Bytes: 4E-68-3F-20-62-3F-6F-20-3F-69-20-68-3F-69-20-62-3F-6F-20-58-75-3F-
6E
UTF-8 Text : Nhà báo đi hội báo Xuân
ASCII Text : Nh? b?o ?i h?i b?o Xu?n
Nhà báo đi hội báo Xuân
Nh? b?o ?i h?i b?o Xu?n
Press any key to continue . . .

class Program
{
[DllImport("kernel32.dll")]
static extern bool SetConsoleOutputCP(uint wCodePageID);
static void Main(string[] args)
{
SetConsoleOutputCP(65001);
Console.OutputEncoding = Encoding.UTF8;
Console.WriteLine("tést, тест, τεστ, ←↑→↓∏∑√∞①②③④, Bài viết chọn lọc");
Console.ReadLine();
}
}
Screenshot of the output (use Consolas or another font that has all the above characters):

You will need to set Console.OutputEncoding to match UTF-8.
Probably something like:
Console.OutputEncoding = System.Text.Encoding.UTF8;

Does the font you use in the Console window support the characters you are trying to display?

it is the problem with cmd.exe console. It doesn't support unicode. [Nothing to do with C#/.NET]
Try changing it to a GUI app if you can or write to a file.