I have an existing database with existing data that I can't change it's structure or values.
In that database there is a nvarchar column that contains values in the twilight unicode zone starting with F800, upward.
When I select those values in SQL or use SQL function, unicode - I get the proper values.
When I select the same values in .Net - I get an error value - all the values in that twilight zone become 65533.
I need those values - how can I presuade .Net to give me those values - something like chaninging the connection encoding to a custom one - or ucs-2 etc...
Here is a sample code that demonstraits the problem:
c.CommandText = "select NCHAR(55297)";
using (var r = c.ExecuteReader())
{
r.Read();
var result = r[0]; //expected 55297 but got 65533
}
55297 is D801 which isn't defined? you probably want f801 which is 63489? But it appears as if that one isn't defined either. Which characters do you want?
If I try doing a "select NCHAR(55297)" in SQL Server Management studio, I get back the diamond question mark, but if I do "select NCHAR(63489)" I get back a dot of some sort:
If what you want is the character values, you can ask for them directly:
select Unicode(NCHAR(63489))
This returns 63489 (as an integer)
If you want them as a byte array, you can ask for that:
select CONVERT(varbinary(MAX), FieldThatIsAnNvarchar) from ThatTable
After much investigations I failed to find any way around this. I couldn't find any two way conversion that would work here.
It seems that some unicode values are intended for some strange unicode scenario that isn't supported by .Net, but is partially supported in a way that breaks what we need here.
Related
I have an app doing searches with multiple fields for different criteria.
Search Bar
My data models seem to be updating accordingly when I debug, but what I pass as SQL parameters isn't correct I think. A SQL trace showed me this is whats being sent to the DB.
exec nu_sp_Codetable_GetWSub #address=N'1234 r%', #wo=N'%', #sub=N'%', #daysback=1096, #marketareaid=99, #po=N'%', #plan=N'%', #super=N'%', #builder=N'%'
My question has to do with those "N" in front of my actual parameter values (we are using modulus for default because reasons). I don't know where it came from, if it came from my code. Here is the code that submits this search to the DB.
UpdateMainGrid(String.Concat(Address, "%"), "%", "%", SelectedDaysBack, SelectedMarketAreaItem.marketareaid, "%", "%", "%", "%")
I ran this as the simplest form of the search and you can see where the code (for this particular search) has those default modulus hard coded for each parameter. In this example, only the Address variable is actually a set value.
Couple notes, the update method shown above passes the literal strings into the classes we use to connect to SQL, which is why I haven't provided the code for that. Debugging shows that the data doesn't change during these steps, it's just different in the SQL trace. Also, the other queries I've looked up in the trace DO NOT include the 'N'. I don't know what difference that makes, if any.
N is used in SQL to denote a unicode string.
It stands for National language character set.
Much like you can do this in C#:
decimal d = 0.3m;
The extra m on the end doesn't change the actual value, it is just a more explicit representation.
By default, parameters are sent to sql as an nvarchar. This is done to allow it to accept the broadest possible values for a parameter, and handle the conversion to type based on the column in the server. That N simply defines the string as Nvarcharrather than varchar.
There is DB2 database.
There is C#.NET application, which uses IBM.Data.DB2 driver to connect to the database (IBMDB2).
There is a parametrized query (DB2Command object, initialized with):
"SELECT $coid_ref FROM db.$ext WHERE $coid = #coid"
It's needed to substitute #coid with hexadecimal literal. For example, it should be executed like this:
"SELECT $coid_ref FROM db.$ext WHERE $coid = x'AA23909F'"
But, when I try to add parameter via command.Parameters.Add("#coid", "AA23909F") driver tries to add it as string, which leads to error. How can I solve this?
You are passing in a regular string.
You need to use a hexadecimal literal value.
From what I could find...
command.Parameters.Add("#coid", "\xAA\x23\x90\x9F")
What DB2 platform are you working with? If an EBCDIC platform (IBM i or z/OS) you might have a problem with your string being translated...
But this seems to be a really strange need. Is the column really defined as a 4 byte binary string? (is so it should look like CHAR(4) FOR BIT DATA assuming you're not using unicode.)
I am getting the error:
"Input string was not in a correct format."
Note: If I change line 182 to an actual number in quotes (ie; "3" or "875"), and comment out line 171, this code works perfectly fine. However, "{7}", in line 174 is a field that is supposed to auto-increment, but wont. So I am trying to get a "number" in line 171, that will use the number of rows, + 1, to do the auto-=increment.
Any takers on this one? :-)
171 string rowCount = string.Format("SELECT COUNT(*) FROM Log WHERE Location is NULL");
173 string sql = string.Format("insert into Log values " +
174 "('{0}','{1}',{2},{3},'{4}',#{5}#,'{6}','{7}')",
175 comboBox1.Text,
176 comboBox2.Text,
177 float.Parse(textBox1.Text),
178 float.Parse(comboBox3.Text),
179 textBox3.Text,
180 textBox2.Text,
181 addRemove,
182 int.Parse(rowCount)
183 );
Stop using that code immediately and use parameterized SQL instead. Otherwise you're vulnerable to SQL injection attacks, as well as potentially having data type conversion issues.
Next, think about what you've actually got in rowCount. It isn't a string representing an integer - it's some SQL. Trying to parse that with int.Parse isn't going to work, is it?
You'd need to execute the query first - or use a subquery within your insert statement. To be honest, if it's meant to be an auto-incrementing field, I would just concentrate on getting that working rather than fudging round it with code which is going to be vulnerable to race conditions.
int.Parse(rowCount) converts string to number, e.g. "100500" to 100500. But your string contains "SELECT COUNT(*) FROM Log WHERE Location is NULL" and this is not a number.
string.Format is not going to execute your SQL commands. So int.Parse sees exactly "SELECT COUNT(*) FROM Log WHERE Location is NULL", which of course is not a decimal representation of a number.
Virtually all databases have native support for auto-incrementing columns. You should not be trying to use an int column and increment it yourself. There are all sorts of race conditions, performance issues, etc. to make an incrementing column really robust, and the database designers have already taken care of all of that for you.
You are probably looking for an answer that will fix your problems with this particular posting. The answers that have been posted will help you do that.
You should examine other approaches.
Use a command object and use parameters (suggested by #JonSkeet)
Do some research on how auto increment columns work. This varies by database vendor. It appears that you may be using Microsoft Access. For MS Sql Server the auto-increment column is an identity column and in Oracle, the mechanism is a bit different again, using sequences. Basically, you do not supply values for auto-increment columns, you let the database engine handle that for you. (also mentioned by a previous poster)
I would also suggest that you assign the values of your text boxes to variables and do some validation of the data before putting into your insert statements or parameters. Try to program defensively.
i would like a clear answer of why can't i process a query like this one with OleDb
"select top 0 * FROM [APPELLATION$] WHERE (((([libel_app] LIKE '%g') OR ([region] LIKE '%g')) OR ([couleur] LIKE '%g')))"
I can with SQL but when i try to fetch some data from an Excel file it throws an error that tells me that i use some restricted word or i have some symbols that are restricted. I read the list of restricted words and symbols and did not find any of them in my query. My program is giving me SQL request, i would not like to end up looking if the TOP N number is whether or not is zero, but if i have no solution...
Thanks in advance.
Your collegues should remove hard-coded flavor-specific SQL from their code in the first place especially if it is to be used on different types of connections.
See the accepted answer on Is there an ANSI SQL alternative to the MYSQL LIMIT keyword? for some of the flavors. Excel is not listed there, I bet it uses the same syntax as MsAccess but I'm not suprised it acts completely different upon such a weird query. As Paddy already mentioned that query does not make any sense, and should be avoided.
Fixing this by matching "SELECT TOP 0" is ugly and will make maintaining software pure hell (been there :( )
The N in TOP N must be a numeric value of 1 (Banker's rounded) or greater.
e.g.
SELECT TOP 0.5 * FROM MyTable; -- rounds to zero = error
SELECT TOP 0.6 * FROM MyTable; -- rounds to 1 = no problem
Is it possible to get the information why/how given row returned by FTS query was matched (or which substring caused row to match)?
For example, consider simpliest table with id and text columns, with FTS index on the later one.
SELECT * FROM Example
WHERE CONTAINS(text, 'FORMSOF(INFLECTIONAL, jump)');
This examplary query could return, say row {1, 'Jumping Jack'}.
Now, is it possible to somehow get information that this very row was matched because of 'Jumping' word? It doesn't even have to be exact information, more of a which substring caused row to match.
Why I'm asking - I got C# app that builds up those queries basing on user input (keywords to search for), and I need the very basic information why/how row was matched back, to use further in C# code.
If it's not possible, any alternatives?
EDIT in regards of Mike Burton's and LesterDove's replies:
The above example was trivial for obvious reasons and your solutions are ok having that in mind, however FTS queries might return results where regex or simple string matching (eg. LIKE) won't cut in. Consider:
Search for bind returns bound (past form).
Search for extraordinary returns amazing (synonym).
Both valid matches.
I've been looking for solutions to this problem and found this: NHunspell. However, I already got FTS & valid results using SQL Server, duplicating similar mechanism (building extra indexes, storing additional words/thezaurus files etc) doesn't look good.
Lester's answer however gave me some ideas that perhaps I could indeed split the original string to temporary table, and run the original FTS query on this split result. As it might work for my case (where DB is fairly small and queries are not very complex), in general case this approach might be out of question.
1/ Use a SPLIT function (many variations can be Googled) on your original substring, which will dump the individual substrings into a temp table of some sort, with one row per substring snippet.
2/ EDIT: You need to use CROSS APPLY to join to a table valued function:
SELECT * FROM Example E CROSS APPLY Split(E.text, ' ') AS S
WHERE CONTAINS(E.text, 'FORMSOF(INFLECTIONAL, jump)') AND S.String LIKE '%jump%';
*NOTE: You need to forage for your own user-defined Split function. I used this one and applied the first commenter's edit to allow for the space character as a delimiter.
So, E is your Example table. You're still FT searching on the text field for the word 'jump'. And now you're "joining" to a table comprised of the individual substring values of your text field. Finally, you're matching that against the word 'jump' by using LIKE or Instr.
One simple post-processing method would be to generate an equivalent Regular Expression for each WHERE clause article and use it to discover after the fact how the found data matches the specified pattern.
You can get SQL to tell you how it interpreted your query, including how it transformed your input.
SELECT occurrence, special_term, display_term, expansion_type, source_term
FROM sys.dm_fts_parser('FORMSOF(INFLECTIONAL, bind)', 1033, 0, 0)
returns
occurrence special_term display_term expansion_type source_term
1 Exact Match binds 2 bind
1 Exact Match binding 2 bind
1 Exact Match bound 2 bind
1 Exact Match bind 0 bind
This isn't precisely what you asked for, but it's a start. You could search your results for anything in the display_term column and probably figure out why it matched.