I developed a .net program to retrieve data through an api and import the data into postgresql database with utf8 encode hosted on AWS RDS.
The problem is that I developed the code on a windows machine with gb2312(active page code: 936) but I deployed the program on a windows server with utf8.en_us(active page code:437) on AWS. The application running on my machine can correctly store Chinese characters into database and displays well. But it turns out to import Gibberish data into database when running on server.
I tried to do a conversion directly on string within the code, like this:
private string Utf8y(string a)
{
return Encoding.UTF8.GetString(Encoding.Default.GetBytes(a));
}
But it's in vain.
Any idea or solution on this?
Thank you!
Finally find out a solution:
Go to control panel -> language, time, location blabla, change your non-unicode encoding to Chinese(Simplified).
Should work with other languages too.
Related
I am retrieving data from a Firebird database and I'm having trouble to get things to work properly.
I've set the database encoding to None and update some values with accents in my DB editor (Flamerobin or other) and store this string:
1_éöüäà
Then I load the data in my Blazor app using default Firebird client and Dapper. If I use a connection string with Charset=None encoding, the loaded string now is
1_�����
Ok so let's change the charset to Charset=ISO8859_1 and relaunch the application. I now get:
1_éöüäà
GREAT - not all is lost, it seems it is stored in DB with ISO8859_1 encoding...
So now, let's edit the value in the Blazor app and store it in the DB still with Dapper and ISO8859_1 charset. I use the SAME connection string for loading and storing data - here is what I actually save (I copy-pasted Dappers' query parameters here):
3_éöüäà
When I check what I have in my DB now (Flamerobin) I get the following strings:
3_éöüäÃ
Ok so the encoding was changed somewhere... After I reload the data in my App, I get the SAME result as above - so encoding was somehow messed up while saving and reloading from DB, using the SAME encoding?! How could that be?!
So well, why not try using Charset=None again to load the data in the app? Guess what, the result is now
3_éöüäà (in App) and 3_éöüäà (in DB)
What? Why would I see the string CORRECTLY by setting Charset=NONE in the connection string but, as it was saved with Charset=ISO8859_1 from the App?
Note that on a WINFORMS app, the exact same manip works the same whether I load/save the data with Charset=None OR Charset=ISO8859_1... Result is alwas as desired : 1_éöüäà.
EDIT: I see the SAME result on my Blazor page AND in the VS debugger console everywhere. I therefore doubt that would be a rendering issue on the browser side - it really is on the .Net / server side that things are happening!
I'm developing a web app and trying to store some input data from a post request using asp.net core 3.1.
at first I should say that I'm using non-English language for inputs.
In local state everything goes fine, however when I publish it and try to store data in MySQL database, values are unexpected(in form of some question marks). the first thing came to my mind was maybe I used an inappropriate charset and encoding, so I change the encoding to the closest thing that I have in my local. didn't work.
I even analyzed the HTTP request and there were no problems.
then I tried to insert directly in phpMyAdmin with insert method. worked fine.
local encoding: utf8mb4_0900_ai_ci
remote encoding: utf8mb4_unicode_ci
any help would be appreciated.
The connection parameters determine the character set of the bytes in the client. To correctly store and retrieve data you must know the charset and announce it when connecting.
While transferring data between the client and server, MySQL will covert from that charset to the charset defined for the columns you are storing into / fetching from.
It sounds like you established the charset correctly for one client and incorrectly for the other.
If you can provide the hex of the characters while in the client, I can further explain.
utf8mb4_0900_ai_ci and utf8mb4_unicode_ci are different Collations for the same character set (encoding) utf8mb4. The characters will go between client and server without any transcoding. However, comparisons may cause trouble due to the different collations.
I am working on integrating my company's product with Jira so users can log bug reports directly from the application. Everything was wqorking smoothly so i decided to import the data from our live Jira system into my development system so that i had some real world data to play with. Since then when i try to get an authtication token it throws the following exception "com.atlassian.crowd.exception.PasswordEncoderNotFoundException: The PasswordEncoder 'atlassian-security' was not found in the Internal Encoders list by the PasswordEncoderFactory". I have checked and the Atlassian Security Password Encoder is enabled in the Crown Password Encoders plugin.
My code is super simple just an ASP.net based text based issues search with the results wired to a grid view code as below:
JiraSoapServiceService service = new JiraSoapServiceService();
string token = service.login("integration", "test");
if (!string.IsNullOrEmpty(txtSearch.Text))
{
RemoteIssue[] issues = service.getIssuesFromTextSearchWithLimit(token, txtSearch.Text, 0, 100);
gvwIssues.DataSource = issues;
gvwIssues.DataBind();
}
I get the error on the call to service.login, it worked fine before I imported the live data. And yes i did add the integration user again after i had imported the data.
Anyone any ideas on this.
No idea what was causing this but rebooting the PC that the dev Jira was installed on fixed it.
EDIT
I have had this a few times now. It tends to happen when you do something like restart the SQL server, restore the jira database via SQL server, restore jira via the inbuilt XML file import method or similar. You don't have to restart the machine just the jira windows service. No idea if this is a problem with other DBs or server types.
I've got problems with special characters being inserted and displayed properly. It's an ASP.NET 3.5 application (C#) and the database is MySql.
If I try to insert a special character into the database using a console app I get an error back saying 'Incorrect string value: '\x81 etc'. If I try to display them it displays for example ü instead of ü.
It works on my local machine, and the only difference I can find (and most likely the root of the problem) is that collation_server is set to latin1_swedish_ci on my machine, utf8_general_ci on the dev server, and character_set_server is latin1 on my machine, utf8 on the dev server. (Everything else is set to utf8 on both machines.)
I thought utf8 was the best thing to use, so now I'm not sure if I should change the dev server to be the same as my local machine which works, i.e. use latin, or change my local machine to be the same as the dev server and look for another solution to the problem? Any thoughts?
Thanks,
Annelie
Set your connection string as follows:
"server=<server>;port=<port>;database=<db>;uid=<uid>;pwd=<password>;charset=utf8;"
Example:
"server=localhost;port=3306;database=mydb;uid=root;pwd=root;charset=utf8;"
(add charset=utf8 to it)
I'm working on a simple windows service which reads csv data and transfer the data to ms-sql server. Csv contains non-unicode chars(Ç窺İıÖöÜüĞğ). ODBC doesn't transfer the right endocing. I try the copy data without ODBC, with text reading, encoding doesn't right too. But text encoding is right when I debug the service. Csv read code:
string[] csvLines = File.ReadAllLines(csvFile, Encoding.GetEncoding(new System.Globalization.CultureInfo("tr-TR").TextInfo.ANSICodePage));
Service is running as LocalSystem and I can't change "System Locale" for non-Unicode programs, I have to use system locale as United States(English)
The file reading code looks fine and I believe the problem is in how you call the ODBC function. Looks like the encoding information is lost somewhere in between the calls.
Can you post a code snippet showing your ODBC calls? Thanks
From memory, creating a new culture in the way you're doing it will use system defaults (i.e. "en-US" in your case).
So, rather than creating a new CultureInfo use the pre-cached one:
CultureInfo.GetCultureInfo("tr-TR")
It works when you debug because the code is running as you, not LocalSystem, and I assume your locale is Turkish.
Edit: Oops, should have been GetCultureInfo instead of GetCulture.
This works on my machine in a console app:
Console.WriteLine("en-US: {0}",
CultureInfo.GetCultureInfo("en-US").TextInfo.ANSICodePage);
Console.WriteLine("tr-TR: {0}",
CultureInfo.GetCultureInfo("tr-TR").TextInfo.ANSICodePage);
Outputs 1252 and 1254.