Develop Reference

Encoding, decoding and re-encoding not producing original results

I suspect that my decoding is not working properly. That is why I am testing it by encoding, decoding and the re-encoding to see if I am getting the same result. That is however not the case.
I encoded a byte[] named model.PDF to a base64 string.
Now, for decoding, I converted model.PDF to a decoded base64 string. However the output looks faulty or corrupted upon debugging and I suspect this is where something is going wrong.
To encode again, the decoded data is turned into byte[] again and then into an encoded base64 string. However base64EncodedData does not match plainTextEncodedData. Please help me create a flawless encode to decode to re-encode flow.
// ENCODING - Byte array -> base64 encoded string
string base64EncodedData = Convert.ToBase64String(model.PDF);
// DECODING - Byte array -> base64 decoded string
var base64DecodedData = Encoding.UTF8.GetString(model.PDF);
// ENCODING AGAIN
byte[] plainTextBytes = Encoding.UTF8.GetBytes(base64DecodedData);
var plainTextEncodedData = Convert.ToBase64String(plainTextBytes);
To elaborate, the re-encoding matches the initial encoding perfectly if executed like this.
var PDF = System.Text.Encoding.UTF8.GetBytes("redgreenblue");
string base64EncodedData = Convert.ToBase64String(PDF);
// DECODING - Byte array -> base64 decoded string
var base64DecodedData = Encoding.UTF8.GetString(PDF);
// ...
But, my model.PDF is fetched from the database as shown below, in which case the re-encoding does not match.
while (reader.Read()) {
model.PDF = reader["PDF"] == DBNull.Value ? null : (byte[])reader["PDF"];
}
On an online base64 decoder (https://www.base64decode.org/), decoding an example value of base64EncodedData shows the ideal and correct value.
%PDF-1.5
%
1 0 obj
<</Type/Catalog/Pages 2 0 R/Lang(en-IN) /StructTreeRoot 8 0 R/MarkInfo<</Marked true>>>>
endobj
2 0 obj
<</Type/Pages/Count 1/Kids[ 4 0 R] >>
endobj
3 0 obj
<</Author(admin) /CreationDate(D:20190724114817+05'30')
/ModDate(D:20190724114817+05'30') /Producer(Microsoft Excel 2013) /Creator(Microsoft Excel 2013) >>
endobj
4 0 obj
<</Type/Page/Parent 2 0 R/Resources<</Font<</F1 6 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] >>/MediaBox[ 0 0 612 792] /Contents 5 0 R/Group<</Type/Group/S/Transparency/CS/DeviceRGB>>/Tabs/S/StructParents 0>>
endobj
5 0 obj
<</Filter/FlateDecode/Length 171>>
stream
...
However, in my program, the value of base64DecodedData shows up in its entirety as:
%PDF-1.5
%����
1 0 obj
<</Type/Catalog/Pages 2 0 R/Lang(en-IN) /StructTreeRoot 8 0 R/MarkInfo<</Marked true>>>>
endobj
2 0 obj
<</Type/Pages/Count 1/Kids[ 4 0 R] >>
endobj
3 0 obj
<</Author(admin) /CreationDate(D:20190724114817+05'30')
/ModDate(D:20190724114817+05'30') /Producer(��
The 2 look similar in ways but my program seems to be producing a corrupt version of what the actual base64 decoded string should be.

A PDF is an ASCII file that can contain binary data (including strings in other encodings).
So you cannot read it as plain text.
If a PDF file contains binary data, as most do [...] the header line
shall be immediately followed by a comment line containing at least
four binary characters—that is, characters whose codes are 128 or
greater.
Taken from this answer, which has some more infos
You see exactly these four characters in your own output.

C# Encoding.Default.GetBytes() method and Integer representation

I saw this code example:
using (FileStream fStream = File.Open(#"C:\myMessage.dat", FileMode.Create))
{
string msg = "Helloo";
byte[] msgAsByteArray = Encoding.Default.GetBytes(msg);
foreach (var a in msgAsByteArray)
{
Console.WriteLine($"a: {a}");
}
// Write byte[] to file.
fStream.Write(msgAsByteArray, 0, msgAsByteArray.Length);
// Reset internal position of stream.
fStream.Position = 0;
// Read the types from file and display to console.
Console.Write("Your message as an array of bytes: ");
byte[] bytesFromFile = new byte[msgAsByteArray.Length];
for (int i = 0; i < msgAsByteArray.Length; i++)
{
bytesFromFile[i] = (byte)fStream.ReadByte();
Console.Write(bytesFromFile[i]);
}
// Display decoded messages.
Console.Write("\nDecoded Message: ");
Console.WriteLine(Encoding.Default.GetString(bytesFromFile));
And the result of Console.WriteLine($"a: {a}") is this:
a: 72
a: 101
a: 108
a: 108
a: 111
a: 111
1.
I thought byte[] is composed of many each unit of byte.
But each byte is represented in integer number.
That numbers must be corresponding ASCII characters.
In C#, byte array means data represented in ASCII?
2.
Is the file myMessage.dat composed of binary data composed of only 0 and 1?
But when I open myMessage.dat with the text editor, it's showing Helloo text string. What's the reason for this?

A byte is a 8bit integer with values from 0 to 255. The output to console outputs the normal number, by providing a format string (https://learn.microsoft.com/en-us/dotnet/standard/base-types/standard-numeric-format-strings) you can output as hex. You can use this answer to get the binary representation.
You explicitly converted the "Halloo" to bytes with Encoding.Default.GetBytes() - that is kindof like converting it to its ascii value but heeding the default encoding on your system.
Your texteditor interpretes the data of the file and displays it as it can. If you put a byte[] myBytes = new [] {0,7,12,3,9,30} into a file and open that with your textedit you will get nonreadable texts as "normal text" starts around 32 , before are f.e. tabs, bells, line feeds and other special non printable characters. See f.e. NonPrintableAscii

c# Convert.To/FromBase64String confusion

Assuming I have this Method.
private static void Example(string data)
{
Console.WriteLine("Initial : {0}", data);
data = data.PadRight(data.Length + 1, '0');
Console.WriteLine("Step 1 : {0}", data);
data = data.PadRight(data.Length + 4 - data.Length % 4, '=');
Console.WriteLine("Step 2 : {0}", data);
byte[] byteArray = Convert.FromBase64String(data);
string newData = Convert.ToBase64String(byteArray);
Console.WriteLine("Step 3 : {0}", newData);
}
I expect the output given the input string "1" to be as follows
Initial : 1
Step 1 : 10
Step 2 : 10==
Step 3 : 10==
Instead the output is this.
Initial : 1
Step 1 : 10
Step 2 : 10==
Step 3 : 1w==
And I have no idea why. I would expect the output to be the same as the input but it isn't.
I have tried replacing
data = data.PadRight(data.Length + 1, '0');
with
data = data + "0";
It appears with longer input strings too, for example strings with a length of 5 or 9. It works fine if I add "=" but then I exceed my padding limit with Convert.FromBase64String()
So my question is really what is going on and how can I get my expected output,?
What am I doing wrong?
Edit: For those confused as to why I'm using bas64 it is related to this PHP decrypting data with RSA Private Key

Basically, there's no byte array which would be encoded to 10==.
If a base64 string ends with ==, that means that the final 4 characters only represent a single byte. So only the first character and the first 2 bits of the second character are relevant. Looking at the Wikipedia table, 10 means values of:
'1' = 53 '0' = 52
110101 110100
So that's encoding a byte of 1101 0111, and then the final four bits (0100) are ignored. When you re-encode the data, it's using 0s for the final four bits instead, giving:
'1' = 53 'w' = 48
110101 110000
Fundamentally, it's not clear what you're trying to do - but if your input is part of a base64-encoded value, that's pretty odd. The code is behaving the way I'd expect it to - it's just not useful code...

Stream, string and null character

I have a stream which contains several \0 inside it. I have to replace textual parts of this stream, but when I do
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd();
text only contains the beginning of the stream (because of the \0 character). So
text = text.Replace(search, replace);
StreamWriter writer = new StreamWriter(stream);
writer.Write(text);
will not do the expected job since I don't parse the "full" stream. Any idea on how to get access to the full data and replace some textual parts ?
EDIT : An example of what I see on notepad
stream
H‰­—[oã6…ÿÛe)Rêq%ÙrlËñE±“-úàÝE[,’íKÿþŽDjxÉ6ŒÅ"XkÏáGqF að÷óð!SN>¿¿‰È†/$ËÙpñ<^HVÀHuñ'¹¿à»U?`äŸ?
¾fØø(Ç,ükøéàâ+ùõ7øø2ÜTJ«¶Ïäd×SÿgªŸF_ß8ÜU#<Q¨|œp6åâ-ªÕ]³®7Ûn¹ÚÝ｜‰,¨¹^ãI©…Ë<UIÐI‡Û©* Ǽ,,ý¬5O->qä›Ü
endstream 
endobj
8 0 obj
<<
/Type /FontDescriptor
/FontName /Verdana
/Ascent 765
/Descent -207
/CapHeight 1489
/Flags 32
/ItalicAngle 0
/StemV 86
/StemH 0
/FontBBox [ -560 -303 1523 1051 ]
/FontFile2 31 0 R
>>
endobj
9 0 obj
And I want to replace /FontName /Verdana by /FontName /Arial on the fly, for example.

Ah, now we're getting to it...
This file a pdf
Then it's not a text file. That's a binary file, and should be treated as a binary file. Using StreamReader on it will lose data. You'll need to use a different API to access the data in it - one which understands the PDF format. Have a look at iTextSharp or PDFTron.

I can't duplicate your results. The code below creates a string with a \0 in it, writes to file, and then reads it back. The resulting string has the \0 in it:
string s = "hello\x0world";
File.WriteAllText("foo.txt", s);
string t;
using (var f = new StreamReader("foo.txt"))
{
t = f.ReadToEnd();
}
Console.WriteLine(t == s); // prints "True"
I get the same results if I do var t = File.ReadAllText("foo.txt");

Read .dat file in c#

i want to read a .dat file that have following conditions:-
First name offset 21
First name format ASCIIz 15 chars + \0
Middle initials offset 37
ID offset -8
ID format/length Unsigned int (4 bytes)
so help me for sorting this issue in c#.
Thanks in advance.
Gurpreet
.dat file
( ÿ / rE ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ XÙþÞ¦d e e Mr. Sam Ascott Sam 9209 Sandpiper Lane 21204 410 5558987 410 5556700 275 MM229399098 (¬ Þ e ܤ•Þ„ œÔ£ÝáØáØ ’Þ[Þ €–˜ ä–˜ [Þ ¶ Norman Eaton Friend of Dr. Shultz Removal of #1,16,17 & 32 öÜÝ)Ý Ä d 01 21 21 21 e 101 22099 XÙþÞ¦d e . Mrs. Patty Baxter Patty 3838 Tommytrue Court 21234 410 2929290 410 3929209 FM218798127 HAY FEVER Þ . „¤¢Þè   _ÐÍÝBÒBÒ ’ÞÝ €–˜ ä–˜ ÍÝ f Joanne Abbey

Here is a tutorial how to use BinaryReader for this purpose:
http://dotnetperls.com/binaryreader

You can use Jet OleDB to query .dat files:
var query = "select * from file.dat";
var connection = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\\file.dat;Extended Properties=\"text;HDR=NO;FMT=FixedLength\"");
See this link:
Code Project. Read Text File (txt, csv, log, tab, fixed length)
And check these:
Reading a sequential-access file
DAT files in C#
Read a file in C#
BinaryReader Class
Jon Skeet. Reading binary data in C#

As said, have a look at the BinaryReader
//Example...
BinaryReader reader = new BinaryReader(stream);
string name = Encoding.ASCII.GetString(reader.ReadBytes(8));
int number = reader.ReadInt32();

my .dat file content - " €U§µ­PÕ „ÕG¬u "
click here to see the content of my .dat file in Notepad and hexa editor
string fileName = #"W:\yourfilename.dat";
//Read the binary file as byte array
byte[] bHex = File.ReadAllBytes(fileName);
//Create string builder for extracting the HEX values
StringBuilder st = new StringBuilder();
//initialize the int for 0
int i = 0;
// check it worked
//Reverse the HEX array for readability
foreach (char c in bHex.Reverse())
{
i++;
// 12 to 21 byte in the reverse order for interseted value in ticks"
if (i > 12 && i < 21)
st.Append(Convert.ToInt32(c).ToString("X2"));
}
// Convert HEX to Deciamal
long Output = Convert.ToInt64(st.ToString(), 16);
//Convert ticks to date time
DateTime dt = new DateTime(Output);
//Write the output date to console
Console.Write(dt);
Final de-crypt binary content to data-time.
final output of the program