Develop Reference

DB2/C# Fun with queries and strings - Problems building query

I've been tasked with what so far seems not possible but I was hoping that someone better at sql queries then me could figure this out if it's possible. I'm having problems querying data from a table. The source of the problem is I'm forced to query on names an date of birth. The date of birth is working and is out of the scope of my issue. My goal is to query using common values in both the source (an excel report) and the destination (the database) and thats the last name.
Fields: Name1, Name2
Table: Participant
Name2 in the database contains last name but if the person uses a middle name or a suffix it also contains those values. The source (report) for the most part contains only last names but sometimes it also has a small number of middle names also mixed into the last name. My goal is to strip out the middle name and suffix from the database query and also from the report last name string.
From the Database:
I need to strip the middle name which is to the left of the last name in the name2 field. They are separated by a space. I will also need to strip the suffix if it exists after the last name.
From the Report:
I need to strip out the middle name which would be to the left of the last name separated by a space. This would be done in c#
Please let me know if I can provide any more info to help with an answer.
My first guess with the query part is to do a wildcard search in which I would obtain the last name from the report and query the table using a (like '%%'). I think this will obtain the record I'm looking for in a search but I'm not sure how well it will work.

I have a similar table with an... integrated 'last name'.
I ended up writing a UDF that did the best it could, but there are still situations I haven't coded for which pop up from time to time. I wrote my UDF in ILE RPG. Unlike most databases, DB2 for i allows me to write in a HLL and simply register it as as UDF. I mention this because it's possible that the developers on the IBM side have already written the code to split the name parts - all they need to do now is make a UDF. Then you could
select getLast(combinedName) from ...

As John Clifford suggests, you could grab the last name using split like so (pseudocode):
// If the string contains a space, split it
string surname = Name2;
int spacePos = surname.IndexOf(" ");
if (spacePos > 0)
{
string[] words = surname.Split(' ');
surname = words[1];
}
Or you could find the space, and then get a substring:
int spaceTruckin = Name2.IndexOf(" ");
string surname = Name2.Substring(spaceTruckin);

C# implement raven db full text search by the part of word

I have a grid and I need to support full text search. I need to support search not only by start with and end with, but I need to support search by the part of word. For example if I have "MyWord", I need that search will found by the part of "wor". If I try to use string.contains() I get the following error:
Contains is not supported, doing a substring match over a text field is a very slow operation, and is not allowed using the Linq API.
The recommended method is to use full text search (mark the field as Analyzed and use the Search() method to query it.
If I build raven db index and mark field as Analyzed, contains is not working. It works with StartWith() and EndWith(), but not with contains. Using .Search() I'm getting the same results. Another option is to use lucene syntax:
.Where("Name:*partOfWord*")
and it works fine, but I don't want to combine linq with lucene syntax and I want to solve it using raven db indexes.
Have you any ideas how to implement full text search for raven db using indexes?

You want to be using an NGram analyzer, as described here. It's an analyzer you can add to your RavenDB server by dropping its DLL in the Analyzers folder.
You really don't want to do any *substr Lucene queries ("ending with" clauses, that is), because the performance is terrible. The inconsistency in coding style is a lesser problem.

I use this query to search for persons full names by just typing a part of the name. It is recommended to set a minimum length of search string.
.Search(x => x.Name, "word to search" + "*", escapeQueryOptions: EscapeQueryOptions.AllowPostfixWildcard)

Regex in Linq (EntityFramework), String processing in database

I have a column in my table which contains values as
"FilterA:123,234,34;FilterB:12,23;FilterC:;FilterD:45;"
Filters are separated by ';' and the values of each filter are separated by ','. There is a ':' in between a Filter's name and it's values.
Now, can I do anything which could fetch out the values part only? Like "123,234,34" for "FilterA". Or can I give it a number like "234" to search in the value part of "FilterA" and/or "54" in the value part of "FilterB"? I know it is possible with using regex, i guess, but I have no idea how.

You can't use regular expressions in Linq to Entities queries, because they cannot be translated into SQL. You even can't use String.Split to split your filters by ;. You have two options here:
Change database table structure. E.g. create table Foo_Filter which will link your entities to filters. And then create table Filters which will contain filters data.
Execute query in memory and use Linq to Objects. This option will be slow, because you have to fetch all data from database to memory

If your underlying database provider is SQL Server then you could make use of SqlMethods.Like to filter the database result set down to a manageable subset of data that can then be analysed locally with RegEx

+1 for #lazyberezovsky, you can't do regex in Linq to Entities because it can't translate that into SQL. You could pull all records back into memory then do Linq to Objects on it (do a .ToList() to a variable then a second linq query for this regex) but that means you'll load every db record into memory to process this query. Same recomendations: change the DB structure such that you don't need to do this.

I guess if you wanted everything to execute on the database server you'd need a SQL function that could parse these filters, perhaps returning a multi-column table. Unfortunately I am not on a machine where I could provide any sample code.

How to properly escape SQL when using BindingSource's Filter property

I've inherited a C# application that lists data from a database table via a DataGridView. There are also a few text fields used to filter that data. This is done like so:
String listFilter = string.Format("City = '{0}'", this.toolStripSearch.Text);
this.customersBindingSource.Filter = listFilter;
toolStripSearch is a text field for searching by city name. The problem is that there is no SQL escaping. Adding a quote to the field crashes the app.
What is the proper way of escaping SQL here?

I think the 'correct' way for BindingSource.Filter is to simply escape the single quotes using '' as horrible as that sounds. E.g, something like.
String listFilter = string.Format("City = '{0}'", this.toolStripSearch.Text.Replace("'", "''"));
this.customersBindingSource.Filter = listFilter;
The Filter property uses the same syntax as DataColumn expressions when BindingSource is bound to data from a database.
http://msdn.microsoft.com/en-us/library/system.windows.forms.bindingsource.filter.aspx
You can find documentation on the required syntax here:
http://msdn.microsoft.com/en-us/library/system.data.datacolumn.expression.aspx
From that link:
String values should be enclosed
within single quotation marks (and
each single quotation character in a
string value has to be escaped by
prepending it with another single
quotation character).

SQL Server FTS: possible to get information how/why rows were matched?

Is it possible to get the information why/how given row returned by FTS query was matched (or which substring caused row to match)?
For example, consider simpliest table with id and text columns, with FTS index on the later one.
SELECT * FROM Example
WHERE CONTAINS(text, 'FORMSOF(INFLECTIONAL, jump)');
This examplary query could return, say row {1, 'Jumping Jack'}.
Now, is it possible to somehow get information that this very row was matched because of 'Jumping' word? It doesn't even have to be exact information, more of a which substring caused row to match.
Why I'm asking - I got C# app that builds up those queries basing on user input (keywords to search for), and I need the very basic information why/how row was matched back, to use further in C# code.
If it's not possible, any alternatives?
EDIT in regards of Mike Burton's and LesterDove's replies:
The above example was trivial for obvious reasons and your solutions are ok having that in mind, however FTS queries might return results where regex or simple string matching (eg. LIKE) won't cut in. Consider:
Search for bind returns bound (past form).
Search for extraordinary returns amazing (synonym).
Both valid matches.
I've been looking for solutions to this problem and found this: NHunspell. However, I already got FTS & valid results using SQL Server, duplicating similar mechanism (building extra indexes, storing additional words/thezaurus files etc) doesn't look good.
Lester's answer however gave me some ideas that perhaps I could indeed split the original string to temporary table, and run the original FTS query on this split result. As it might work for my case (where DB is fairly small and queries are not very complex), in general case this approach might be out of question.

1/ Use a SPLIT function (many variations can be Googled) on your original substring, which will dump the individual substrings into a temp table of some sort, with one row per substring snippet.
2/ EDIT: You need to use CROSS APPLY to join to a table valued function:
SELECT * FROM Example E CROSS APPLY Split(E.text, ' ') AS S
WHERE CONTAINS(E.text, 'FORMSOF(INFLECTIONAL, jump)') AND S.String LIKE '%jump%';
*NOTE: You need to forage for your own user-defined Split function. I used this one and applied the first commenter's edit to allow for the space character as a delimiter.
So, E is your Example table. You're still FT searching on the text field for the word 'jump'. And now you're "joining" to a table comprised of the individual substring values of your text field. Finally, you're matching that against the word 'jump' by using LIKE or Instr.

One simple post-processing method would be to generate an equivalent Regular Expression for each WHERE clause article and use it to discover after the fact how the found data matches the specified pattern.

You can get SQL to tell you how it interpreted your query, including how it transformed your input.
SELECT occurrence, special_term, display_term, expansion_type, source_term
FROM sys.dm_fts_parser('FORMSOF(INFLECTIONAL, bind)', 1033, 0, 0)
returns
occurrence special_term display_term expansion_type source_term
1 Exact Match binds 2 bind
1 Exact Match binding 2 bind
1 Exact Match bound 2 bind
1 Exact Match bind 0 bind
This isn't precisely what you asked for, but it's a start. You could search your results for anything in the display_term column and probably figure out why it matched.