I am using C# and .NET to encode and decode base64 string. The following are snippets of my code:
Base64 encoding:
using (var stream = new MemoryStream())
…...
return Convert.ToBase64String(stream.ToArray());
}
Base64 decoding
byte[] bytes = Convert.FromBase64String(messageBody);
My code fails 99% of the time with 1% chance to succeed though. The stack trace is as follows:
5xx Error Returned:System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters. at System.Convert.FromBase64_ComputeResultLength(Char inputPtr, Int32 inputLength) at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength) at System.Convert.FromBase64String(String s)*
Does anyone know what can cause base64 decoding to fail? My encoding and decoding methods are symmetric and I am really confused about what can be the root cause for this issue?
Thanks for all your replies.
It turned out there were still some old messages in Json format that previously failed in getting delivered and kept retrying in our system; however the new code change of our receiving side got deployed and our receiving side starts to expect messages in protobuf format which results in Deserialization failure when receiving old Json format messages.
In order to debug an issue like this I usually write some tests or create a console app to watch the variables as they change from function to function.
One of the possible scenario's for base64 decoding to fail is if the decoder input is HTMLEncoded. This is common when you pass an encrypted string into a URL for example. It will automatically be HTML encoded and then it sometimes can and sometimes can't be decoded depending on the characters that the encoded output has.
Here's a simple console app to demonstrate this.
class Program
{
static void Main(string[] args)
{
string input = "testaa";
TestEncodeDecode("test");
TestEncodeDecode("testa");
TestEncodeDecode("testaa");
Console.ReadLine();
}
private static void TestEncodeDecode(string input)
{
string encoded = Encode(input);
Console.WriteLine($"Encoded: {encoded}");
string htmlEncoded = WebUtility.UrlEncode(encoded);
Console.WriteLine($"htmlEncoded: {htmlEncoded}");
string decodedString = Decode(htmlEncoded);
Console.WriteLine($"Decoded: {decodedString}");
Console.WriteLine();
}
private static string Decode(string htmlEncoded)
{
try
{
byte[] decoded = Convert.FromBase64String(htmlEncoded);
return Encoding.ASCII.GetString(decoded);
}
catch(Exception)
{
return "Decoding failed";
}
}
private static string Encode(string input)
{
byte[] bytes = Encoding.ASCII.GetBytes(input);
using (var stream = new MemoryStream())
{
stream.Write(bytes);
return Convert.ToBase64String(stream.ToArray());
}
}
}
You'll see that the first two arguments ("test" and "testa") fail to decode, but the third ("testaa") will succeed.
In order to "fix" this, change the Decode method as follows:
private static string Decode(string htmlEncoded)
{
try
{
string regularEncodedString = WebUtility.UrlDecode(htmlEncoded);
byte[] decoded = Convert.FromBase64String(regularEncodedString);
return Encoding.ASCII.GetString(decoded);
}
catch(Exception)
{
return "Decoding failed";
}
}
Related
Sorry in advance if you have a duplicate or a simple question! I can't find the answer.
I'm working with a dll made in Delphi. Data can be sent to the device using a DLL. However, at the time the data is sent, some strings are not accepted or are written blank. The data sent to the device is stored in a txt file. It was generated using txt file third party program.
That is, I think the string is in an indefinite format. If I send in utf-8 format, it receives all the information. But some strings at the time ???? ???? remains.
Many of my texts are in the Cyrillic alphabet.
What I did:
// string that send to device
[MarshalAsAttribute(UnmanagedType.LPStr, SizeConst = 36)]
public string Name;
When I did this, the device received only 10 out of 100 data.
If i encoding with UTF-8:
byte[] bytes = Encoding.Default.GetBytes(getDvsName[1].ToString());
string res = Encoding.UTF8.GetString(bytes);
Got all the data this way but too many strings are became as ??? ????.
Also i tried like this:
static private string Win1251ToUTF8(string source)
{
Encoding utf8 = Encoding.GetEncoding(«utf-8»);
Encoding win1251 = Encoding.GetEncoding(«windows-1251»);
byte[] utf8Bytes = win1251.GetBytes(source);
byte[] win1251Bytes = Encoding.Convert(win1251, utf8, utf8Bytes);
source = win1251.GetString(win1251Bytes);
return source;
}
All of the above methods did not help. How can I receive incoming information in the correct format? Are there other ways?
hi there here is what went wrong you did encode the string to default instead of utf8.
string tom = "ටොම් හැන්ක්ස්";
byte[] bytes = Encoding.UTF8.GetBytes(tom);
string res = Encoding.UTF8.GetString(bytes);
I was given a url that, in the documentation says the following:
The response is a binary encoded octet stream of the raw 52 byte
archive records.
I created this code to test it out:
public static void GetHistory(string url)
{
var response = WebRequestClasses.GetResponse(url);
var nres = DeserializeObject(response);
Console.WriteLine(nres);
}
public static object DeserializeObject(string str)
{
byte[] bytes = Convert.FromBase64String(str);
using (MemoryStream stream = new MemoryStream(bytes))
{
return new BinaryFormatter().Deserialize(stream);
}
}
It is returning the following error:
An unhandled exception of type 'System.FormatException' occurred in
mscorlib.dll
Additional information: The input is not a valid Base-64 string as it
contains a non-base 64 character, more than two padding characters, or
an illegal character among the padding characters.
I am getting a response as you can see from the print screen:
I've tried searching on octet parsing but I'm not getting anything. I'm not very familiar with encoding but I would assume it's a problem with the Base=64 vs octet stream?
I'm trying to encrypt and decrypt Id with MachineKey.
Here is my code that calls the encrypt and decrypt functions:
var encryptedId = Encryption.Protect(profileId.ToString(), UserId);
var decryptedId = Encryption.UnProtect(encryptedId, UserId);
Here is the functions:
public static string Protect(string text, string purpose)
{
if(string.IsNullOrEmpty(text))
{
return string.Empty;
}
byte[] stream = Encoding.Unicode.GetBytes(text);
byte[] encodedValues = MachineKey.Protect(stream, purpose);
return HttpServerUtility.UrlTokenEncode(encodedValues);
}
public static string UnProtect(string text, string purpose)
{
if(string.IsNullOrEmpty(text))
{
return string.Empty;
}
byte[] stream = HttpServerUtility.UrlTokenDecode(text);
byte[] decodedValues = MachineKey.Unprotect(stream, purpose);
return Encoding.UTF8.GetString(decodedValues);
}
The input to the Protect method is 15. This results that the encryptedId variable holds the following string: 6wOttbtJoVBV7PxhVWXGz4AQVYcuyHvTyJhAoPu4Okd2aKhhCGbKlK_T4q3GgirotfOZYZXke0pMdgwSmC5vxg2
To encrypt this, I send this string as a parameter to the UnProtect method.
The result of the decryption should be 15, but is instead: 1\05\0
I can't understand why. I have tried to use only integers in this function, but I still have the same problem. The output of the decrypt differs.
You have an encoding mismatch, you encode a buffer containing the UTF-16 (Encoding.Unicode) representation of the string (which will interleave \0 as you see given that it uses 2 bytes per character for that string) but you decode it as UTF-8 (Encoding.UTF8). You need to be consistent in both methods.
Here is my method for sending data:
// this method serializes an object and is returned as a string (Base64 encoded)
public void Send(Packet packetData)
{
try
{
StreamWriter w = new StreamWriter(cli.GetStream());
string s = SerializeObject(packetData);
w.WriteLine(s + "\n");
w.Flush();
}
catch (ObjectDisposedException) { ShutdownClient(-2); }
}
cli is TcpClient object
Packet is a serializable object
Here's my receive method:
private string GetMessage(StreamReader r)
{
try
{
string s = r.ReadLine();
s = s.Replace(" ", "");
// this string is returned and then deserialized
return s;
}
catch (Exception e) { System.Windows.Forms.MessageBox.Show(e.Message); return null; }
}
When I use this, 50% of the time it works. If it doesn't work, it's because of this:
"The input stream is not a valid binary format. The starting contents (in bytes) are: 6D-2E-44-72-61-77-69-6E-67-2E-43-6F-6C-6F-72-0F-00 ..."
I have tried using Encoding.Default.GetString/GetBytes in replacement of Base64, but then I get this:
"Binary stream '0' does not contain a valid BinaryHeader. Possible causes are invalid stream or object version change between serialization and deserialization."
If I print out the length of this (Default encoded) string, it is 183. But if I print out the actual string, nothing is printed.
How else can I send a byte[] as a string over StreamWriter?
How else can I send a byte[] as a string
Not the way you do it now, the byte[] content will get corrupted when the string is normalized. A string should only ever be used to store Unicode characters. Not every Unicode codepoint is valid.
If using a string is paramount then you should use Convert.ToBase64String() at the transmitter, Convert.FromBase64String() at the receiving end.
Do keep in mind that TCP is entirely capable of transferring binary data. You possibly fell into this hole because TCP implements a stream, it doesn't do anything to help you transmit messages. The simple way to transfer a binary 'message' is to first write the Length of the byte[]. The receiver first read that length, then knows what it should pass to the Read() call to recover the byte[] back from the TCP stream.
I'm trying to send a string containing special characters through a TcpClient (byte[]). Here's an example:
Client enters "amé" in a textbox
Client converts string to byte[] using a certain encoding (I've tried all the predefined ones plus some like "iso-8859-1")
Client sends byte[] through TCP
Server receives and outputs the string reconverted with the same encoding (to a listbox)
Edit :
I forgot to mention that the resulting string was "am?".
Edit-2 (as requested, here's some code):
#DJKRAZE here's a bit of code :
byte[] buffer = Encoding.ASCII.GetBytes("amé");
(TcpClient)server.Client.Send(buffer);
On the server side:
byte[] buffer = new byte[1024];
Client.Recieve(buffer);
string message = Encoding.ASCII.GetString(buffer);
ListBox1.Items.Add(message);
The string that appears in the listbox is "am?"
=== Solution ===
Encoding encoding = Encoding.GetEncoding("iso-8859-1");
byte[] message = encoding.GetBytes("babé");
Update:
Simply using Encoding.Utf8.GetBytes("ééé"); works like a charm.
Never too late to answer a question I think, hope someone will find answers here.
C# uses 16 bit chars, and ASCII truncates them to 8 bit, to fit in a byte. After some research, I found UTF-8 to be the best encoding for special characters.
//data to send via TCP or any stream/file
byte[] string_to_send = UTF8Encoding.UTF8.GetBytes("amé");
//when receiving, pass the array in this to get the string back
string received_string = UTF8Encoding.UTF8.GetString(message_to_send);
Your problem appears to be the Encoding.ASCII.GetBytes("amé"); and Encoding.ASCII.GetString(buffer); calls, as hinted at by '500 - Internal Server Error' in his comments.
The é character is a multi-byte character which is encoded in UTF-8 with the byte sequence C3 A9. When you use the Encoding.ASCII class to encode and decode, the é character is converted to a question mark since it does not have a direct ASCII encoding. This is true of any character that has no direct coding in ASCII.
Change your code to use Encoding.UTF8.GetBytes() and Encoding.UTF8.GetString() and it should work for you.
Your question and your error is not clear to me but using Base64String may solve the problem
Something like this
static public string EncodeTo64(string toEncode)
{
byte[] toEncodeAsBytes
= System.Text.ASCIIEncoding.ASCII.GetBytes(toEncode);
string returnValue
= System.Convert.ToBase64String(toEncodeAsBytes);
return returnValue;
}
static public string DecodeFrom64(string encodedData)
{
byte[] encodedDataAsBytes
= System.Convert.FromBase64String(encodedData);
string returnValue =
System.Text.ASCIIEncoding.ASCII.GetString(encodedDataAsBytes);
return returnValue;
}