Develop Reference

How to determine UUDecoding method needed?

I'm communicating to a device that returns uuencoded data:
ASCII: EZQAEgETAhMQIBwIAUkAAABj
HEX: 45-5A-51-41-45-67-45-54-41-68-4D-51-49-42-77-49-41-55-6B-41-41-41-42-6A
The documentation for this device states the above is uuencoded but I can't figure out how to decode it. The final result won't be a human readable string but the first byte reveals the number of bytes for the following product data. (Which would be 23 or 24?)
I've tried using Crypt2 to decode it; it doesn't seem to match 644, 666, 744 modes.
I've tried to hand write it out following the Wiki: https://en.wikipedia.org/wiki/Uuencoding#Formatting_mechanism
Doesn't make sense! How do I decode this uuencoded data?

I agree with #canton7 that it looks like it's base64 encoded. You can decode it like this
byte[] decoded = Convert.FromBase64String("EZQAEgETAhMQIBwIAUkAAABj");
and if you want, you can print the hex values like this
Console.WriteLine(BitConverter.ToString(decoded));
which prints
11-94-00-12-01-13-02-13-10-20-1C-08-01-49-00-00-00-63

As #HansKilian says in the comments, this is not uuencoded.
If you base64-decode it you get (in hex):
11 94 00 12 01 13 02 13 10 20 1c 08 01 49 00 00 00 63
The first number, 17 in decimal, is the same as the number of bytes following it, which matches:
The final result won't be a human readable string but the first byte reveals the number of bytes for the following product data.
(#HansKilian made the original call that it was base64-encoded. This answer provides confirmation of that by looking at the first decoded byte, but please accept his answer)

How would I reverse this simple-looking algorithm?

I got some old LED board to which you'd send some text and hang it up somewhere... it was manufactured in 1994/95 and it communicates over a serial port, with a 16-bit MS-DOS application in which you can type in some text.
So, because you probably couldn't run it anywhere except by using DOSBox or similar tricks, I decided to rewrite it in C#.
After port-monitoring the original dos-exe I've found that it's really not interested in you rebuilding it - requests must be answered suitable, varying bytes, pre-sent "ping" messages, etc...
Maybe you know a similar checksum routine/pattern as my dos-exe uses or you could give any tips in trying to reverse-engineer this... Additionally, because I am only familiar with programming and didn't spend much time on reversing methods and/or analyzing protocols, please don't judge me if this topic is a bit of a stupid idea - I'll be glad about any help I get...
The message really containing the text that should be displayed is 143 bytes long (just that long because it puts filler bytes if you don't use up all the space with your text), and in that msg I noticed the following patterns:
The fourth byte (which still belongs to the msg header) varies from a list of 6 or 7 repeating values (in my examples, that byte will always be 0F).
The two last bytes function as a checksum
Some examples:
displayed text: "123" (hex: "31 32 33"), checksum hex: "45 52"
text: "132" ("31 33 32"), checksum hex: "55 FF"
text: "122" ("31 32 32"), checksum hex: "95 F4"
text: "133" ("31 33 33"), checksum hex: "85 59"
text: "112" ("31 31 32"), checksum hex: "C5 C8"
text: "124" ("31 32 34"), checksum hex: "56 62"
text: "134" ("31 33 34"), checksum hex: "96 69"
text: "211" ("32 31 31"), checksum hex: "5D 63"
text: "212" ("32 31 32"), checksum hex: "3C A8"
text: {empty}, checksum hex: "DB BA"
text: "1" ("31"), checksum hex: "AE 5F"
So far I am completely sure that the checksum really does depend on this fourth byte in the header, because if it changes, the checksums will be completely different for the same text to be displayed.
Here's an an example of a full 143 bytes-string displaying "123", just for giving you a better orientation:
02 86 04 0F 05 03 01 03 01 03 01 03 00 01 03 00 ...............
00 31 00 32 00 33 00 20 00 20 00 20 00 20 00 20 .1.2.3. . . . .
00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 . . . . . . . .
00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 . . . . . . . .
00 20 00 20 00 20 00 20 00 20 00 FE 03 01 03 01 . . . . . .þ....
04 01 03 00 01 03 00 00 20 00 20 00 20 00 20 00 ........ . . . .
20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . .
20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 . . . . . . . .
20 00 20 00 20 00 20 00 20 00 20 00 20 45 52
(the text information starts with 2nd byte in 2nd line "31 00 32 00 33 00 (...)"
Unfortunately on the whole web, there are no user manuals, documentations, not even a real evidence that this info board-device ever existed.

I'll write F(s) for the checksum you get when feeding in string s.
Observe that:
F("122") xor F("123") = 95 F4 xor 45 52 = D0 A6
F("132") xor F("133") = 55 FF xor 85 59 = D0 A6
F("123") xor F("124") = 45 52 xor 56 62 = 13 30
F("133") xor F("134") = 85 59 xor 96 69 = 13 30
all of which is consistent with the checksum having the following property, which checksums not infrequently have: changing a given bit in the input always XORs the output with the same thing.
I predict, e.g., that F("210") = F("211") xor D0 A6 = 8D C5, and similarly that F("222") = 3C A8 xor C5 C8 xor 95 F4 = 6C 94.
If this is true, then the following gives you a brute-force-y way to figure out the checksum in general, provided you have a black box that computes checksums for you (which apparently you have):
Find the checksum of an input all of whose bits are 0. Call this a.
For each bit position k, find the checksum of an input all of whose bits are 0 except for bit k which is 1. Call this a XOR b(k).
Now the checksum of an arbitrary input is a XOR each b(k) where bit k is set in the input.
Usually the b(k) will be closely related to one another -- the usual pattern is that you're feeding bits into a shift register -- so the above is more brute-force-y than you'd need given an understanding of the algorithm. But I expect it works, if you are able to feed in arbitrarily chosen bit patterns as input.
If not, you may still be able to do it. E.g., suppose all you actually get to choose is 29 7-bit ASCII character values, at positions 17,19,...73 of your input. Then you can first of all feed in all spaces (0x20) and then XOR each in turn with 1-bits in positions 0..6. That won't give you all the b(k) but it will give you enough for arbitrary 29-ASCII-character inputs.

Which encoding to use for reading a string from a file?

I'm parsing a file (which I don't generate) that contains a string. The string is always preceded by 2 bytes which tell me the length of the string that follows.
For example:
05 00 53 70 6F 72 74
would be:
Sport
Using a C# BinaryReader, I read the string using:
string s = new string(binaryReader.ReadChars(size));
Sometimes there's the odd funky character which seems to push the position of the stream on further than it should. For example:
0D 00 63 6F 6F 6B 20 E2 80 94 20 62 6F 6F 6B
Should be:
cook - book
and although it reads fine the stream ends up two bytes further along than it should?! (Which then messes up the rest of the parsing.)
I'm guessing it has something to do with the 0xE2 in the middle, but I'm not really sure why or how to deal with it.
Any suggestions greatly appreciated!

My guess is that the string is encoded in UTF-8. The 3-byte sequence E2 80 94 corresponds to the single Unicode character U+2014 (EM DASH).

In your first example
05 00 53 70 6F 72 74
none of the bytes are over 0x7F and that happens to be the limit for 7 bit ASCII. UTF-8 retains compability with ASCII by using the 8th bit to indicate that there will be more information to come.
0D 00 63 6F 6F 6B 20 E2 80 94 20 62 6F 6F 6B
Just as Ted noticed your "problems" starts with 0xE2 because that is not a 7 bit ASCII character.
The first byte 0x0D tells us there should be 11 characters but there are 13 bytes.
0xE2 tells us that we've found the beginning of a UTF-8 sequence since the most significant bit is set (it's over 127). In this case a sequence that represents — (EM Dash).

As you did correctly state the E2 character is the problem. BinaryReader.ReadChars(n) does not read n-bytes but n UTF-8 encoded Unicode characters. See Wikipedia for Unicode Encodings. The term you are after are Surrogate Characters. In UTF-8 characters in the range of 000080 – 00009F are represented by two bytes. This is the reason for your offset mismatch.
You need to use BinaryReader.ReadBytes to fix the offset issue and the pass it to an Encoding instance.
To make it work you need to read the bytes with BinaryReader and then decode it with the correct encoding. Assuming you are dealing with UTF-8 then you need to pass the byte array to
Encoding.UTF8.GetString(byte [] rawData)
to get your correctly encoded string back.
Yours,
Alois Kraus

How to edit a binary file's hex value using C#

So here's my issue. I have a binary file that I want to edit. I can use a hex editor to edit it of course, but I need to make a program to edit this particular file. Say that I know a certain hex I want to edit, I know it's address etc. Let's say that it's a 16-bit binary, and the address is 00000000, it's on row 04 and it has a value of 02. How could I create a program that would change the value of that hex, and only that hex with the click of a button?
I've found resources that talk about similar things, but I can't for the life of me find help with the exact issue.
Any help would be appreciated, and please, don't just tell me the answer if there is one but try and explain a bit.

I think this is best explained with a specific example. Here are the first 32 bytes of an executable file as shown in Visual Studio's hex editor:
00000000 4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00
00000010 B8 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00
Now a file is really just a linear sequence of bytes. The rows that you see in a hex editor are just there to make things easier to read. When you want to manipulate the bytes in a file using code, you need to identify the bytes by their 0-based positions. In the above example, the positions of the non-zero bytes are as follows:
Position Value
-------- ------
0 0x4D
1 0x5A
2 0x90
4 0x03
8 0x04
12 0xFF
13 0xFF
16 0xB8
24 0x40
In the hex editor representation shown above, the numbers on the left represent the positions of the first byte in the corresponding line. The editor is showing 16 bytes per line, so they increment by 16 (0x10) at each line.
If you simply want to take one of the bytes in the file and change its value, the most efficient approach that I see would be to open the file using a FileStream, seek to the appropriate position, and overwrite the byte. For example, the following will change the 0x40 at position 24 to 0x04:
using (var stream = new FileStream(path, FileMode.Open, FileAccess.ReadWrite)) {
stream.Position = 24;
stream.WriteByte(0x04);
}

Well the first thing would probably be to understand the conversions. Hex to decimal probably isn't as important (unless of course you need to change the value from a decimal first, but that's a simple conversion formula), but hex to binary will be important seeing as each hex character (0-9,A-F) corresponds to a specific binary output.
After understanding that stuff, the next step is to figure out exactly what you are searching for, make the proper conversion, and replace that exact string. I would recommend (if the buffer wouldn't be too large) to take the entire hex dump and replace whatever you're searching for in there to avoid overwriting a duplicate binary sequence.
Hope that helps!
Regards,
Dennis M.

Write all bits in C#

How can i write all bits of a file using c#?
For example writing 0 to all bits
Please provide me with a sample

I'm not sure why you'd want to do this, but this will overwrite a file with data that is the same length but contains byte values of zero:
File.WriteAllBytes(filePath, new byte[new FileInfo(filePath).Length]);

Definitely has the foul stench of homework to it.
Hint - Think why someone might want to do this. Just deleting the file and replacing with a file of 0s of the correct length might not be what you're after.

Have a look at System.IO.FileInfo; you'll need to open a writable stream for the file you're interested in and then write however many bytes (with value 0 in your example) to it as there are in the file already (which you can ascertain via FileInfo.Length). Be sure to dispose of the stream once you're done with it – using constructs are useful for this purpose.

Consider using the BinaryWriter available in the .NET framework
using(BinaryWriter binWriter =
new BinaryWriter(File.Open(fileName, FileMode.Create)))
{
binWriter.Write("Hello world");
}

When you say write all bits to a file I'll assume you mean bits as in nyble, bit, byte. That's just writing an integer to a file. You can't have a 4 bit file as far as I know so the smallest denomination will be a byte.
You probably don't want to be responsible for serializing yourself, so your easiest option would be to use the BinaryReader and BinaryWriter classes, and then manipulate the bits inside your C#.
The BinaryWriter class uses a 4 byte integer as minimum however. For example
writer.Write( 1 ); // 01
writer.Write( 10 ); // 0a
writer.Write( 100 ); // 64
writer.Write( 1000 ); // 3e8
writer.Write( 10000 ); // 2710
//writer.Write( 123456789 ); // 75BCD15
is written to file as
01 00 00 00 0a 00 00 00 64 00 00 00 e8 03 00 00 10 27 00 00 15 cd 5b 07

read into a byte and then test against >= powers of 2 to get each of the bits in that byte