Develop Reference

Receive special char from serial port

I need to receive and transmit data with a serial port. I have no problem receiving and transmitting, but I do not see the received data correctly.
If I use a program, ComTestSerial, I see these correct data:
{STX}1H|\^&|||cobas6000^1|||||host|RSUPL^BATCH|P|1 P|1|||||||U||||||^
O|1| IANNETTA M
BIS|0^5016^1^^S1^SC|^^^480^|R||||||N||||1|||||||20191018113556|||F
C|1|I| ^ ^^^|G
R|1|^^^480/|11|U/L{ETB}A6
But if I use my program in c #, with RicheditText or Texbox, I see this wrong data:
2497212492943812412412499111989711554484848944912412412412412410411111511612482838580769466658467721248012449138012449124124124124124124124851241241241241241249413791244912432323232323232327365787869848465327732667383124489453484954944994948349948367124949494525648941248212412412412412412478124124124124491241241241241241241245048495749484956494951535354124124124701367124491247312432323232323232323232323232323232323232323232323232323232323294323232323232323232323232323232323232323232323232329494941247113821244912494949452564847124494912485477623655413104.
I use this simple code (written by a colleague) to receive:
string cMsg = "";
while (this.ComPort.BytesToRead > 0)
{
int nChar = this.ComPort.ReadChar();
cMsg += nChar.ToString();
}
Thread.Sleep(100);
return cMsg;
Which reads data from a serial connection that works perfectly.
What could be the problem?

You're converting a number to a string, so, say, when nChar is 2, the output will be a string "2", and when nChar is 49, the output will be "49".
So, the message begins with {STX}1. {STX} is an ASCII control code 2, and 1 is ASCII code 49. Thus the "wrong data" begins with "249".
Thus, the data isn't wrong, and the code does exactly what you told it to, except that your colleague didn't mean what you intended :)
Instead of converting ASCII codes to strings, convert them to characters, and also use a stringbuilder to minimize the number of times the string is resized.
StringBuilder message(ComPort.BytesToRead);
while (ComPort.BytesToRead > 0)
{
message.Append((char)ComPort.ReadChar());
}
return message.ToString();
But you don't need to do any of it! SerialPort.ReadExisting does what you want:
return ComPort.ReadExisting();.
Stylistic note: C# is not Java, and littering the code with this. is not idiomatic nor necessary. Don't do it unless there's a good reason to.

From your code, it seems that you are getting an integer from your port, and when you are using ToString() you just write a number to string
int nChar = this.ComPort.ReadChar();
cMsg += nChar.ToString();
This integer should be A 21-bit Unicode code point.
So you just can use Char.ConvertFromUtf32(Int32) Method, it will convert integer to the actual character:
https://learn.microsoft.com/en-us/dotnet/api/system.char.convertfromutf32?view=netframework-4.8
Your full code should then look like this:
string cMsg = "";
while (this.ComPort.BytesToRead > 0)
{
int nChar = this.ComPort.ReadChar();
cMsg += Char.ConvertFromUtf32(nChar);
}
Thread.Sleep(100);
return cMsg;

Fixing a mis-encoded string after the fact

Main problem and question:
Given a garbled string for which the actual text is known, is it possible to consistently repair the garbled string?
According to Nyerguds comment on this answer:
If the string is an incorrect decoding done with a simply 8-bit
Encoding and you have the Encoding used to decode it, you can
usually get the bytes back without any corruption, though.
(emphases mine)
Which suggests that there are cases when it is not possible to derive the original bytes back. This leads me to the following question: are there cases when (mis)encoding an array of bytes is a lossy and irreversible operation?
Background:
I am calling an external C++ library that calls a web API somewhere. Sometimes this library gives me slightly garbled text. In my C# project, I am trying to find a way to consistently reverse the miscoding, but I only seem to be able to do so part of the time.
What I've tried:
It seems clear that the C++ library is wrongly encoding the original bytes, which it later passes to me as a string. My approach has been to guess at the encoding that the C++ library used to interpret the original source bytes. Then, I iterate through all possible encodings, reinterpreting the hopefully "original" bytes with another encoding.
class TestCase
{
public string Original { get; set; }
public string Actual { get; set; }
public List<string> Matches { get;} = new List<string>();
}
void Main()
{
var testCases = new List<TestCase>()
{
new TestCase {Original = "窶弑-shaped", Actual = "“U-shaped"},
new TestCase {Original = "窶廡窶・Type", Actual = "“F” Type"},
new TestCase {Original = "Ko窶冩lau", Actual = "Ko’olau"},
new TestCase {Original = "窶彗s is", Actual = "“as is"},
new TestCase {Original = "窶從ew", Actual = "“new"},
new TestCase {Original = "faﾃｧade", Actual = "façade"}
};
var encodings = Encoding.GetEncodings().Select(x => x.GetEncoding()).ToList();
foreach (var testCase in testCases)
{
foreach (var from in encodings)
{
foreach (var to in encodings)
{
// Guess the original bytes of the string
var guessedSourceBytes = from.GetBytes(testCase.Original);
// Guess what the bytes should have been interpreted as
var guessedActualString = to.GetString(guessedSourceBytes);
if (guessedActualString == testCase.Actual)
{
testCase.Matches.Add($"Reversed using \"{from.CodePage} {from.EncodingName}\", reinterpreted as: \"{to.CodePage} {to.EncodingName}\"");
}
}
}
}
}
As we can see above, out of the six test cases, all but one (窶廡窶・) was successful. In the successful cases, Shift-JIS (codepage 932) seemed to result in the correct "original" byte sequence for UTF8.
Getting the Shift-JIS bytes for 窶廡窶・ yields: E2 80 9C 46 E2 80 81 45.
E2 80 9C coincides with the UTF8 bytes for left double quotation mark, which is correct. However, E2 80 81 is em quad in UTF8, not the right double quotation mark I am expecting. Reinterpreting the whole byte sequence in UTF8 results in “F EType
No matter which encoding I use to derive the "original" bytes, and no matter what encoding I use to reinterpret said bytes, no combination seems to be able to successfully convert 窶廡窶・ to “F”.
Interestingly if I derive the UTF8 bytes for “F” Type, and purposely misinterpret those bytes as Shift-JIS, I get back 窶廡窶・Type
Encoding.GetEncoding(932).GetString(Encoding.UTF8.GetBytes("“F” Type"))
This leads me to believe that encoding can actually lead to data loss. I'm not well versed on encoding though, so could someone confirm whether my conclusion is correct, and if so, why this data loss occurs?

Yes, there are encodings that don't support all characters. One most common example is ASCIIEncoding that replaces all characters outside of standard ASCII range with ?.
...Because ASCII is a 7-bit encoding, ASCII characters are limited to the lowest 128 Unicode characters, from U+0000 to U+007F. … characters outside that range are replaced with a question mark (?) before the encoding operation is performed.

Chars above 127 won't convert properly to Byte

I noticed a strange behavior, chars above 127 will not convert properly into Byte. It's a well known problem but I can't understand why it happens. I found out about it when I was working on a client-server app. The thing is chars are unsigned and so are Bytes so where is the loss of data
public class Constants
{
public constant char TOP3_REQUEST_CODE = (char)148;
}
public void printTopThree()
{
string request = Constants.TOP3_REQUEST_CODE.ToString();
string response = SendAndRecive (Constants.PORT, Constants.IP, request, Globals.SOCKET);
//The rest isn't relevant.
public string SendAndRecive(string port, string ip, string request, Socket socket)
{
Byte[] bytesSend = Encoding.ASCII.GetBytes(request);
Console.WriteLine(request [0]);
Console.WriteLine(bytesSend[0]);
//The program continues but its not relevant.
}
The code afterward doesn't change the byte array or string so it can't affect the results.
The output is:
148
63
The first char in the request is the code of the message (happens to be 148) but after converting, the first byte is 63.
My questions are:
1. How I can fix this? Is there some kind of another encoding that may solve my problem?
2. Why does this thing happen anyways?
EDIT: The request looks like this (in general):
1st byte: (char) Code (20,100,148 etc...)
2nd - 4th bytes: (int) Length (the length of the JSON object, The length can be 1, 2, or 3 bytes long)
5th - X bytes: (char) JsonObject (It's converted to char[])
Thanks for your time and attention
- Anthon

Strings and characters in C# don't use single-byte encodings. You should use the unicode encoding the CLR uses under the hood:
Byte[] bytesSend = Encoding.Unicode.GetBytes(request);

Thanks to all of you,
#Lasse Vågsæther Karlsen and #Marc Gravell♦ where right,
Use UTF-8.

leading zero dynamically gets removed

I have strange validation error
public long mobile { get; set; }
[RegularExpression(#"^([09]{2}[0-9]{8})$", ErrorMessage = "mobile number is not correct")]
if the user entered 094532415678 it is correct but validation error appears due to the dynamic removal of leading zeroes in asp c# , model will receive the above number as 94532415678 and wont be saved to database throwing error for mobile validation
I have tried to save it to string and add the leading zero to mobile and save it but still when the string gets converted to "long" directly it removes the leading zero , any solution for this problem

Store telephone numbers (and simular information) as a string. Leading 0's are removed when you save phonenumbers as a number (long).

try
[RegularExpression(#"^([09]{2}[0-9]{8})$", ErrorMessage = "mobile number is not correct")]
public string mobile { get; set; }

How to Transform the data in a .txt file for Electronic Fund Transfers

So at the moment our ERP/PSA software produces an EFT (Electronic Fund Transfer) .txt file which contains Bank and employee bank information which is then sent to the bank.
Problem is as follows the format to which the EFT File is currently being produced is US standard and not suitable to Canadian bank standards. But I have the required canadian bank standard format.
The format of the file is all about number of columns in a file and the number of characters they contain (if the data for the column doesnt reach the number of characters it is filled with spaces).
So I.e.
1011234567Joe,Bloggs 1234567
And for example lets say I try transform to Canadian Standard
A101Joe,Bloggs 1234567 1234567
Where for example "A" needs to be added to first line in the record.
I'm just wondering how to go about a task like this in C#
I.e.
Read in text file.
Line by Line parse data in terms of start and end of characters
Assign values to variables
Rebuild new file with these variables with different ordering and additional data

I don't have my IDE open so my syntax might be a tad off, but I'll try to point you in the right direction. Anyways, what fun would it be to give you the solution outright?
First you're going to want to get a list of lines:
IEnumerable<string> lines = text.Split('\n');
You said that the columns don't have delimiters but rather are of fixed widths, but you didn't mention where the columns sizes are defined. Generally, you're going to want to pull out the text of each column with
colText = line.Substring(startOfColumn, lengthOfColumn);
For each column you'll have to calculate startOfColumn and lengthOfColumn, depending on the positions and lengths of the columns.
Hopefully that's a good enough foundation for you to get started.

I think that your best bet is to create a class to hold the logical data that is present in the file and have methods in this class for parsing the data from a given format and saving it back to a given format.
For example, assume the following class:
public class EFTData
{
public string Name { get; set; }
public string RoutingNumber { get; set; }
public string AccountNumber { get; set; }
public string Id { get; set; }
public void FromUSFormat(string sLine)
{
this.Id = sLine.Substring(0, 3);
this.RoutingNumber = sLine.Substring(3, 7);
this.Name = sLine.Substring(10, 20);
this.AccountNumber = sLine.Substring(30, 7);
}
public string ToCanadianFormat()
{
var sbText = new System.Text.StringBuilder(100);
// Note that you can pad or trim fields as needed here
sbText.Append("A");
sbText.Append(this.Id);
sbText.Append(this.RoutingNumber);
sbText.Append(this.AccountNumber);
return sbText.ToString();
}
}
You can then read from a US file and write to a Canadian file as follows:
// Assume there is only a single line in the file
string sLineToProcess = System.IO.File.ReadAllText("usin.txt");
var oData = new EFTData();
// Parse the us data
oData.FromUSFormat(sLineToProcess);
// Write the canadian data
using (var oWriter = new StreamWriter("canout.txt"))
{
oWriter.Write(oData.ToCanadianFormat());
}

var lines = File.ReadAllLines(inputPath);
var results = new List<string>();
foreach (var line in lines)
{
results.Add(string.Format("A{0}", line));
}
File.WriteAllLines(outputPath, results.ToArray());