I'm working on a simple windows service which reads csv data and transfer the data to ms-sql server. Csv contains non-unicode chars(Ç窺İıÖöÜüĞğ). ODBC doesn't transfer the right endocing. I try the copy data without ODBC, with text reading, encoding doesn't right too. But text encoding is right when I debug the service. Csv read code:
string[] csvLines = File.ReadAllLines(csvFile, Encoding.GetEncoding(new System.Globalization.CultureInfo("tr-TR").TextInfo.ANSICodePage));
Service is running as LocalSystem and I can't change "System Locale" for non-Unicode programs, I have to use system locale as United States(English)
The file reading code looks fine and I believe the problem is in how you call the ODBC function. Looks like the encoding information is lost somewhere in between the calls.
Can you post a code snippet showing your ODBC calls? Thanks
From memory, creating a new culture in the way you're doing it will use system defaults (i.e. "en-US" in your case).
So, rather than creating a new CultureInfo use the pre-cached one:
CultureInfo.GetCultureInfo("tr-TR")
It works when you debug because the code is running as you, not LocalSystem, and I assume your locale is Turkish.
Edit: Oops, should have been GetCultureInfo instead of GetCulture.
This works on my machine in a console app:
Console.WriteLine("en-US: {0}",
CultureInfo.GetCultureInfo("en-US").TextInfo.ANSICodePage);
Console.WriteLine("tr-TR: {0}",
CultureInfo.GetCultureInfo("tr-TR").TextInfo.ANSICodePage);
Outputs 1252 and 1254.
Related
I'm developing a web app and trying to store some input data from a post request using asp.net core 3.1.
at first I should say that I'm using non-English language for inputs.
In local state everything goes fine, however when I publish it and try to store data in MySQL database, values are unexpected(in form of some question marks). the first thing came to my mind was maybe I used an inappropriate charset and encoding, so I change the encoding to the closest thing that I have in my local. didn't work.
I even analyzed the HTTP request and there were no problems.
then I tried to insert directly in phpMyAdmin with insert method. worked fine.
local encoding: utf8mb4_0900_ai_ci
remote encoding: utf8mb4_unicode_ci
any help would be appreciated.
The connection parameters determine the character set of the bytes in the client. To correctly store and retrieve data you must know the charset and announce it when connecting.
While transferring data between the client and server, MySQL will covert from that charset to the charset defined for the columns you are storing into / fetching from.
It sounds like you established the charset correctly for one client and incorrectly for the other.
If you can provide the hex of the characters while in the client, I can further explain.
utf8mb4_0900_ai_ci and utf8mb4_unicode_ci are different Collations for the same character set (encoding) utf8mb4. The characters will go between client and server without any transcoding. However, comparisons may cause trouble due to the different collations.
I developed a .net program to retrieve data through an api and import the data into postgresql database with utf8 encode hosted on AWS RDS.
The problem is that I developed the code on a windows machine with gb2312(active page code: 936) but I deployed the program on a windows server with utf8.en_us(active page code:437) on AWS. The application running on my machine can correctly store Chinese characters into database and displays well. But it turns out to import Gibberish data into database when running on server.
I tried to do a conversion directly on string within the code, like this:
private string Utf8y(string a)
{
return Encoding.UTF8.GetString(Encoding.Default.GetBytes(a));
}
But it's in vain.
Any idea or solution on this?
Thank you!
Finally find out a solution:
Go to control panel -> language, time, location blabla, change your non-unicode encoding to Chinese(Simplified).
Should work with other languages too.
I've tried searching but had no luck as I'm not sure I'm using the correct terminology.
I'm trying to figure out how to ask a user to input their server name on the first application run, store that and insert it into filepath/connection string.
Pseudocode for initial run:
I see this is the first time you ran this application. Please input your fileserver name.
user inputs: fileserver123x
Write Fileserver123x to text file.
Initialize database connection
string fileServername = read text file;
connection = new OleDbConnection(#"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=\'"+fileServerName+#"'\share\folder\Database.accdb;Persist Security Info=False;");
rest of code
I'm looking to see if there is a better/more professional way to store the user file server information other than via text file and read it each time the DB connection is initialized.
Any thoughts?
Your concept looks fine, I'd suggest a few improvements:
Store the file server name in the Windows registry instead of a text file. This is where well-behaved Windows programs store their configuration data.
Store the complete path to the database instead of the file server name. That way, your customers don't have to use a fixed share/folder name.
Make a configuration window where this configuration data can be modified. This can also serve as the window you show on your first run.
Don't read the configuration data each time a connection is opened. Read it once when your application starts and store it in a global variable.
(Note: In general, global variables are a code smell, but storing global configuration data is usually considered a legitimate use case. If you want a more advanced solution that simplifies unit testing, look into dependency injection for your configuration data.)
I have to read the first couple of lines of a csv file client side to present the user with a preview before uploading a large file.
I'm using c# with ASP.Net (VS2010)
Does anyone know how this can be done? Also that it works in all browsers!?!
(It's not possible to use activeX or so, we do not want our clients to install something!)
A sample code would be great!
Thanks in advance!
Firefox 3.6+ and Chrome (at least version 6, possibly older versions) support the File API which lets you read local files from a file input.
Here's a quick sample:
function handleFile(file) {
var reader = new FileReader();
reader.onload = function(e) {
// Here's where you would parse the first few lines of the CSV file
console.log(e.target.result);
};
reader.readAsText(file);
}
in your html, you would have something like this:
<input type="file" onchange="handleFile(this.files[0])" />
Of course, in real life you should make it degrade gracefully.
The file object has name and type properties that you could use to verify that it's a CSV file if you wanted to be more strict.
There is a W3C proposal for a File API in HTML5. I ran a few browsers against this test, and found that Firefox 3.6, and Chrome 6 support File Reader API and Local Storage. Safari 5 (on Windows) supported Local Storage but not File Reader API. Sorry I couldn't test IE9 as corporate policy has IE6 (?!) nailed in place.
You can't read a local file with vanilla JavaScript.
You could use activeX as a progressive enhancement to those using IE (even though offering an enhanced experience to IE users goes against everything I stand for!).
If you can use Flash, look at Read local file in Flash.
I've got problems with special characters being inserted and displayed properly. It's an ASP.NET 3.5 application (C#) and the database is MySql.
If I try to insert a special character into the database using a console app I get an error back saying 'Incorrect string value: '\x81 etc'. If I try to display them it displays for example ü instead of ü.
It works on my local machine, and the only difference I can find (and most likely the root of the problem) is that collation_server is set to latin1_swedish_ci on my machine, utf8_general_ci on the dev server, and character_set_server is latin1 on my machine, utf8 on the dev server. (Everything else is set to utf8 on both machines.)
I thought utf8 was the best thing to use, so now I'm not sure if I should change the dev server to be the same as my local machine which works, i.e. use latin, or change my local machine to be the same as the dev server and look for another solution to the problem? Any thoughts?
Thanks,
Annelie
Set your connection string as follows:
"server=<server>;port=<port>;database=<db>;uid=<uid>;pwd=<password>;charset=utf8;"
Example:
"server=localhost;port=3306;database=mydb;uid=root;pwd=root;charset=utf8;"
(add charset=utf8 to it)