Develop Reference

Csv File Doesn't Show German Characters After Downloading via .Net Web Api

I have to return a csv file template which has some columns with german characters in my one of my api enpoint.
This is the simplified version of endpoint:
[HttpGet]
[ProducesResponseType(typeof(Stream), StatusCodes.Status200OK)]
[Route("template")]
public async Task GetTemplate()
{
Response.StatusCode = StatusCodes.Status200OK;
Response.ContentType = "text/csv";
Response.Headers.AddContentDispositionAttachment("Template.csv");
CsvConfiguration csvConfiguration = new (new CultureInfo("de-de"))
{
Delimiter = ";",
HasHeaderRecord = false
};
string header = $"Url;Beschreibung;Code;ID;Löschen;Straße;Pünklitch";
using var streamWriter = new StreamWriter(Response.Body, Encoding.UTF8, 65536);
await using var csvWriter = new CsvWriter(streamWriter, csvConfiguration);
await streamWriter.WriteAsync(header);
}
It is pretty straight forward.
The problem is; when I call endpoint via swagger in local or on server and download the file it doesn't show german characters properly. Also when I entagrate endpoint to my Front end and download file via ui it also behave as same way and doesn't show german characters. But when I call the endpoint from browser directly everything looks ok.
This is how csv file looks after download:
Any idea ? How can I fix encoding problem?
I am using .net framwork 6

The problem is when you open a CSV file by double clicking on it to open it in Excel, Excel assumes the file is in an older Windows encoding, rather than UTF8. You have two options:
Force a Byte Order Mark (BOM) at the beginning of the file for UTF8. Most newer versions of Excel will recognize the BOM and open it as UTF8 instead of the older encoding. There are some versions of Excel, however, that will ignore the BOM. (FYI your code is not using the csvWriter, but this should still work when you do output with csvWriter rather than streamWriter directly.)
using var streamWriter = new StreamWriter(Response.Body, Encoding.UTF8, 65536);
using var csvWriter = new CsvWriter(streamWriter, csvConfiguration);
var preamble = Encoding.UTF8.GetPreamble();
await streamWriter.BaseStream.WriteAsync(preamble, 0, preamble.Length);
await streamWriter.WriteAsync(header);
Use the older Windows encoding instead of UTF8. This should work as long as you only have German characters and not other UTF8 characters that won't convert. I'm not certain what it would be in Germany, but I'm going to guess it is Windows-1252.
using var streamWriter = new StreamWriter(Response.Body, Encoding.GetEncoding(1252), 65536);
// Or you might need to use iso-8859-1
using var streamWriter = new StreamWriter(Response.Body, Encoding.GetEncoding("iso-8859-1"), 65536);

You can make a custom Ascii converter which will replace all your chars with an ascii symbol. for example you could use Dictionary<string, string> and map each char to it's ascii
private Dictionary<string, string> CharToASCII = new Dictionary<string, string>() {
{"Ö", #"\'d6"},
{"ö", #"\'f6"}
}
then you can just call a function with your string as an input parameter and as a result you will have an ascii written instead of German letters
public string ConvertToAscii(string myString)
{
var res = CharToASCII.Aggregate(myString, (current, value) =>
current.Replace(value.Key, value.Value));
return res;
}

How can i create correct line breaks in a sns sms message?

I'm using the AWS .NET-SDK for sending SMS messages with the AWS SNS service. So far, so good; but when I use line breaks, I see the ? char at this point before the line break begins in the SMS. After that character, the line break is added as expected. Is there any possibility to get a line break without this ? character?
I have also tried following:
StringBuilder.AppendLine,
"\\n",
"\\r\\n",
#"\n",
#"\r\n",
Environment.NewLine
And encoding the string into UTF-8.
Example which doesn't work:
// Create message string
var sb = new StringBuilder();
sb.AppendLine("Line1.");
sb.Append("Line2.\\n");
sb.AppendLine(Environment.NewLine);
sb.Append(#"Line4\n");
// Encode into UTF-8
var utf8 = UTF8Encoding.UTF8;
var stringBytes = Encoding.Default.GetBytes(sb.ToString());
var decodedString = utf8.GetString(stringBytes);
var message = decodedString;
// Create request
var publishRequest = new PublishRequest
{
PhoneNumber = "+491234567890",
Message = message,
Subject = "subject",
MessageAttributes = "Promotional"
};
// Send SMS
var response = await snsClient.PublishAsync("topic", message, "subject");

Simply remove all attempts to encode the string. .NET strings are Unicode, specifically UTF16 already. PublishAsync expects a .NET string, not UTF8 bytes.
As for why this error occurs, it's because the code converts the string into bytes using the local machine's codepage and then tries to read those bytes as if they were UTF8, which they aren't - using UTF8 as a system codepage is a beta feature on Windows 10 which breaks a lot of applications.
The newline character for SMS is \n. Environment.NewLine returns \r\n unless you use .NET Core on Linux. StringBuilder.AppendLine uses Environment.NewLine so you can't use it.
You shouldn't need anything more than String.Join to combine multiple lines into a single message:
var message=String.Join("\n",lines);
If you need to use a StringBuilder, use AppendFormat to append a line with the \n character at the end, eg :
builder.AppendFormat("{0}\n",line);
Update
I was able to send an SMS containing newlines with this code:
var region = Amazon.RegionEndpoint.EUWest1;
var snsClient = new AmazonSimpleNotificationServiceClient(region);
var sb = new StringBuilder()
.Append("Line1.\n")
.Append("Line2.\n")
.Append("Line4\n");
var message = sb.ToString();
// Create request
var publishRequest = new PublishRequest
{
PhoneNumber = phone,
Message = message,
};
// Send SMS
var response = await snsClient.PublishAsync(publishRequest);
The message I received contained :
Line1.
Line2.
Line4.
I decided to get fancy and changed the last line to :
.Append("Line4ΑΒΓ£§¶\n");
I received this text without problems too

Equivalent of C# Encoding.UTF8.GetBytes in php

I have an example in C# and have to write the same in PHP.
request = request.Replace(sign, string.Empty);
byte[] sha1Request;
using (var shaM = new SHA1Managed())
{
sha1Request = shaM.ComputeHash(Encoding.UTF8.GetBytes(request));
}
log.InfoFormat($"request={request}. sha1Request={Convert.ToBase64String(sha1Request)}. Sign={sign}", request, Convert.ToBase64String(sha1Request));
var pubKey = (RSACryptoServiceProvider)FrontInterface.GetCertificate(checkFrontCertificateCod.Value).PublicKey.Key;
var isValid = pubKey.VerifyData(Encoding.UTF8.GetBytes(Convert.ToBase64String(sha1Request)), new SHA1CryptoServiceProvider(), Convert.FromBase64String(sign));
if (!isValid)
{
throw new Exception("Wrong digital sign");
}
So, I may not convert string to bytes in php and line sha1Request = shaM.ComputeHash(Encoding.UTF8.GetBytes(request));
will be in PHP: sha1Request =sha1(request, true);
Am I rigth? If not, please help me to convert in PHP this line.
Thanks a lot.

Note that sha1 should not really be used any more for security relevant applications, it is out of date.
C# Version:
string text = "<Hällo World>";
byte[] sha1;
using (var shaM = new SHA1Managed())
{
sha1 = shaM.ComputeHash(Encoding.UTF8.GetBytes(text));
}
string encoded = Convert.ToBase64String(sha1);
Console.Write(encoded);
PHP Version:
$text = "<Hällo World>";
// Encode as UTF8 if necessary (May not be necessary if string is already utf-8)
$text = utf8_encode($text);
// Calculate SHA1
$sha1 = sha1($text, TRUE);
// Convert to Base64
$encoded = base64_encode($sha1);
echo($encoded);
Both versions should output
1nSiStZRa/quRru7Sqe+ejupqfs=
Note that the call to utf8_encode should only be there if the string you work with is not actually already encoded in utf8.
If the string is a literal in a *.php file, this depends on how the file is stored on the disk. (What character set it uses).
If the string is retrieved from a web request or from a database or from reading a file, this also depends on what character set the web form, the database or the external file use.

Encoding speech input for Amazon Lex?

I'm having problem with the encode of speech input for Amazon Lex.
If i assign InputStream as null, it works, i receive the default voice answer from Lex : "How can i help you"
var amazonLexClient = new AmazonLexClient("APPID", "APPSECRET", Amazon.RegionEndpoint.USEast1);
var amazonPostRequest = new Amazon.Lex.Model.PostContentRequest();
var amazonPostResponse = new Amazon.Lex.Model.PostContentResponse();
amazonPostRequest.BotAlias = "BookTrip";
amazonPostRequest.BotName = "BookTrip";
amazonPostRequest.ContentType = "audio/l16; rate=16000; channels=1";
amazonPostRequest.UserId = "user";
amazonPostRequest.InputStream = null;
amazonPostResponse = await amazonLexClient.PostContentAsync(amazonPostRequest);
If i try to send a recorded voice "How are you" using the encode (required by Lex : 16KHz, 8bits, 1 channel) below
var amazonLexClient = new AmazonLexClient("APPID", "APPSECRET", Amazon.RegionEndpoint.USEast1);
var amazonPostRequest = new Amazon.Lex.Model.PostContentRequest();
var amazonPostResponse = new Amazon.Lex.Model.PostContentResponse();
amazonPostRequest.BotAlias = "BookTrip";
amazonPostRequest.BotName = "BookTrip";
amazonPostRequest.ContentType = "audio/l16; rate=16000; channels=1";
amazonPostRequest.UserId = "user";
amazonPostRequest.InputStream = new MemoryStream();
WaveFormat target = new WaveFormat(16000, 8, 1);
WaveStream streamIn = new WaveFileReader("F:\\Whatever.wav");
WaveFormatConversionStream str = new WaveFormatConversionStream(target, streamIn);
WaveFileWriter.WriteWavFileToStream(amazonPostRequest.InputStream, str);
amazonPostResponse = await amazonLexClient.PostContentAsync(amazonPostRequest);
Then it doesn't work, after about 20~25s Lex server will return null.
Amazon.Runtime.AmazonUnmarshallingException: 'Error unmarshalling response back from AWS.'
NullReferenceException: Object reference not set to an instance of an object.
Can anyone tell me how to encode a wav file to make it work with Amazon Lex?
Btw im using Visual Studio 2017, C# with NAudio library.

Amazon Lex expects the audio to be in PCM or Opus format (read this documentation for more details). You can refer this AI blog post from Amazon to get more information on how to PCM encode your wav audio.

There seems to be some kind of problem inside the AWSSDk for c# - what's happened is that the Lex service has returned a plain-text error message, and the SDK is trying to parse it as JSON. Sometimes you can dig into the exception details and find that raw response, or just use Fiddler.

Ruby base64 to C# base64

I'm writing a Ruby/Rhomobile application that takes an image, encodes it in base64 and sends to the server(that is running C# on ASP.NET), as part of a bigger XML.
However, once I try to decode the base64 and save it to the hard disk, the resulting file does not work as an image.
Here's the relevant ruby code:
image_element = REXML::Element.new("image")
image_element.text = Base64.encode64(open(Rho::RhoApplication::get_blob_path(self.image_uri)) { |io| io.read })
form_element.add_element(image_element)
And here is my C# code:
var doc = new XmlDocument();
doc.LoadXml(Server.UrlDecode(Request.Form[0]));
var imageBase64 = doc.SelectNodes("//image")[0];
var imageBytes = imageBase64.InnerText;
using(var imgWriter = new FileStream(#"c:\img.jpg",FileMode.Create))
{
imgWriter.Write(imageBytes,0,imageBytes.Length);
}

I would investigate your call to Server.UrlDecode. It seems like that could corrupt your data.
It seems like the "+" sign is of specific concern, per this SO question. Server.UrlDecode uses HttpServerUtility.UrlDecode, and here's the documentation for it.