Develop Reference

Escaping SQL in C#/sqlite or: how can I use parameters here?

I realize that handcrafting SQL statements is evil. I also realize that the spontaneous answer of everyone seeing this post will be "Use parameters". I have found several occurences of similar questions around here, but I can't find out how they apply to my specific problem.
I use System.Data.SQLite with C#. I want to dynamically create a VIEW, say listview_asdf which I then can access inside a library with simple SELECT * FROM listview_asdf statements. That library uses loads of automatically generated statements for accessing my view.
Problem is, at one point I need to create a view like this:
CREATE VIEW listview_asdf AS SELECT * FROM tbl_asdf WHERE id IN (1,2,3)
not knowing how many and which parameters I have in the IN() list. Actually, the list will be different each time I use it (I delete the views in their Dispose() methods). Creating the view does - as far as I understand the relevant google result - not allow to use parameters.
So, I need to take an arbitrary-length array of parameters (easy) and write the CREATE VIEW statement with an IN (...) clause. Here, I need to insert the parameter values. This is no problem in case of integers, as shown above, but for strings or other types (System.DateTime comes to mind) I need some kind of escaping.
Else, I need some way of getting around the necessity of escaping. I don't know of any solution to either except the ominous
sql = sql.Replace("'","''");
which, as User Aur Saraf points out here, is a sure way of losing my job (which I do like).
Any ideas for a way out?

Linq to DataTable without enumerating fields

i´m trying to query a DataTable object without specifying the fields, like this :
var linqdata = from ItemA in ItemData.AsEnumerable()
select ItemA
but the returning type is
System.Data.EnumerableRowCollection<System.Data.DataRow>
and I need the following returning type
System.Data.EnumerableRowCollection<<object,object>>
(like the standard anonymous type)
Any idea?
Thanks

If I understand you correctly, you'd like to get a collection of objects that you don't need to define in your code but that are usable in a strongly typed fashion. Sadly, no you can't.
An anonymous type seems like some kind of variant or dynamic object, but it is in fact a strongly typed class that is defined at compile time. .NET defines the type for you automatically behind the scenes. In order for .net to be able to do this, it has to have some clue from the code with which to infer the type definition. It has to have something like:
from ItemA in ItemData.AsEnumerable()
select ItemA.Item("Name"), ItemA.Item("Email")
so it knows what members to define. There's no way to get around it, the information has to logically be there for the anonymous type to be defined.
Depending on why exactly your are trying to do this, there are some options.
If you want intellisense while still encapsulating your data access, you can return xml instead of a datatable from your encapsulated data access class. (You can convert data tables to xml very easily. You'll want to use the new System.Xml.Linq classes like the XElement. They're great!) Then you can use VS2008's ability to create an xsd schema from xml. Then use/import that schema at the top of your code page, and you have intellisense.
If you have to have an object an with properties for your data, but don't want to define a class/structure for them, you'll love the new dynamic objects coming in C#4.0/VB10. You have object properties based on what the sql returns, but you won't have intellisense. There is also a performance cost to this, but (a) that might not matter for your situation and (b) it actually is not so bad in some situations.
If you're just trying to avoid making a lot of classes, consider defining structs/structures on the same code file, beneath your class definition. When you add more columns to your result set, it's easy to adjust a struct with more public fields.
In short you can have any two of the following three: (a) dynamic, (b) strontly-typed objects, (3) intellisense. But not all three.

There is one way to accomplish what you want, but it required knowledge of dynamic linq. You would build the query during run-time and then use it. I am no expert and have never really played around with it, but here is a link to Scott Guthrie's blog about it - Dynamic Linq. Hope that helps.
Wade

Common problem for me in C#, is my solution good, stupid, reasonable? (Advanced Beginner)

Ok, understand that I come from Cold Fusion so I tend to think of things in a CF sort of way, and C# and CF are as different as can be in general approach.
So the problem is: I want to pull a "table" (thats how I think of it) of data from a SQL database via LINQ and then I want to do some computations on it in memory. This "table" contains 6 or 7 values of a couple different types.
Right now, my solution is that I do the LINQ query using a Generic List of a custom Type. So my example is the RelevanceTable. I pull some data out that I want to do some evaluation of the data, which first start with .Contains. It appears that .Contains wants to act on the whole list or nothing. So I can use it if I have List<string>, but if I have List<ReferenceTableEntry> where ReferenceTableEntry is my custom type, I would need to override the IEquatable and tell the compiler what exactly "Equals" means.
While this doesn't seem unreasonable, it does seem like a long way to go for a simple problem so I have this sneaking suspicion that my approach is flawed from the get go.
If I want to use LINQ and .Contains, is overriding the Interface the only way? It seems like if there way just a way to say which field to operate on. Is there another collection type besides LIST that maybe has this ability. I have started using List a lot for this and while I have looked and looked, a see some other but not necessarily superior approaches.
I'm not looking for some fine point of performance or compactness or readability, just wondering if I am using a Phillips head screwdriver in a Hex screw. If my approach is a "decent" one, but not the best of course I'd like to know a better, but just knowing that its in the ballpark would give me little "Yeah! I'm not stupid!" and I would finish at least what I am doing completely before switch to another method.
Hope I explained that well enough. Thanks for you help.

What exactly is it you want to do with the table? It isn't clear. However, the standard LINQ (-to-Objects) methods will be available on any typed collection (including List<T>), allowing any range of Where, First, Any, All, etc.
So: what is you are trying to do? If you had the table, what value(s) do you want?
As a guess (based on the Contains stuff) - do you just want:
bool x= table.Any(x=>x.Foo == foo); // or someObj.Foo
?

There are overloads for some of the methods in the List class that takes a delegate (optionally in the form of a lambda expression), that you can use to specify what field to look for.
For example, to look for the item where the Id property is 42:
ReferenceTableEntry found = theList.Find(r => r.Id == 42);
The found variable will have a reference to the first item that matches, or null if no item matched.
There are also some LINQ extensions that takes a delegate or an expression. This will do the same as the Find method:
ReferenceTableEntry found = theList.FirstOrDefault(r => r.Id == 42);

Ok, so if I'm reading this correctly you want to use the contains method. When using this with collections of objects (such as ReferenceTableEntry) you need to be careful because what you're saying is you're checking to see if the collection contains an object that IS the same as the object you're comparing against.
If you use the .Find() or .FindAll() method you can specify the criteria that you want to match on using an anonymous method.
So for example if you want to find all ReferenceTableEntry records in your list that have an Id greater than 1 you could do something like this
List<ReferenceTableEntry> listToSearch = //populate list here
var matches = listToSearch.FindAll(x => x.Id > 1);
matches will be a list of ReferenceTableEntry records that have an ID greater than 1.
having said all that, it's not completely clear that this is what you're trying to do.

Here is the LINQ query involved that creates the object I am talking about, and the problem line is:
.Where (searchWord => queryTerms.Contains(searchWord.Word))
List<queryTerm> queryTerms = MakeQueryTermList();
public static List<RelevanceTableEntry> CreateRelevanceTable(List<queryTerm> queryTerms)
{
SearchDataContext myContext = new SearchDataContext();
var productRelevance = (from pwords in myContext.SearchWordOccuranceProducts
where (myContext.SearchUniqueWords
.Where (searchWord => queryTerms.Contains(searchWord.Word))
.Select (searchWord => searchWord.Id)).Contains(pwords.WordId)
orderby pwords.WordId
select new {pwords.WordId, pwords.Weight, pwords.Position, pwords.ProductId});
}
This query returns a list of WordId's that match the submitted search string (when it was List and it was just the word, that works fine, because as an answerer mentioned before, they were the same type of objects). My custom type here is queryTerms, a List that contains WordId, ProductId, Position, and Weight. From there I go about calculating the relevance by doing various operations on the created object. Sum "Weight" by product, use position matches to bump up Weights, etc. My point for keeping this separate was that the rules for doing those operations will change, but the basic factors involved will not. I would have even rather it be MORE separate (I'm still learning, I don't want to get fancy) but the rules for local and interpreted LINQ queries seems to trip me up when I do.
Since CF has supported queries of queries forever, that's how I tend to lean. Pull the data you need from the db, then do your operations (which includes queries with Aggregate functions) on the in-memory table.
I hope that makes it more clear.

Get query terms from Lucene query for highlighting

My Lucene queries will usually exist of a bunch of AND combined fields.
Is it possible to get the queried fields out of the Query object again?

Did you mean extracting the terms or the field names? Since you already know you're handling a BooleanQuery, to extract the fields you can simply iterate the BooleanClause array returned by BooleanQuery.getClauses(), rewrite each clause to its base query (Query.rewrite) and apply recursively until you have a TermQuery on your hands.
If you did mean term extraction, I'm not sure about Lucene.NET, but in Java Lucene you can use org.apache.lucene.search.highlight.QueryTermExtractor; you pass a (rewritten) query to one of its getTerms overloads and get an array of WeightedTerms.
As far as I remember, the downsides to using this technique are:
Since it internally uses a term set it won't handle multiple instances of the same token, e.g. "dream within a dream"
It only supports base query types (TermQuery, BooleanQuery and any other query type which supports Query.extractTerms). I believe we've used it internally for SpanNearQuery and SpanNearOrderedQuery instances, but I may be wrong on this.
Either way I hope this is enough to get you started.

Dynamic "WHERE" like queries on memory objects

What would be the best approach to allow users to define a WHERE-like constraints on objects which are defined like this:
Collection<object[]> data
Collection<string> columnNames
where object[] is a single row.
I was thinking about dynamically creating a strong-typed wrapper and just using Dynamic LINQ but maybe there is a simpler solution?
DataSet's are not really an option since the collections are rather huge (40,000+ records) and I don't want to create DataTable and populate it every time I run a query.

What kind of queries do you need to run? If it's just equality, that's relatively easy:
public static IEnumerable<object[]> WhereEqual(
this IEnumerable<object[]> source,
Collection<string> columnNames,
string column,
object value)
{
int columnIndex = columnNames.IndexOf(column);
if (columnIndex == -1)
{
throw new ArgumentException();
}
return source.Where(row => Object.Equals(row[columnIndex], value);
}
If you need something more complicated, please give us an example of what you'd like to be able to write.

If I get your point : you'd like to support users writting the where clause externally - I mean users are real users and not developers so you seek solution for the uicontrol, code where condition bridge. I just though this because you mentioned dlinq.
So if I'm correct what you want to do is really :
give the user the ability to use column names
give the ability to describe a bool function (which will serve as where criteria)
compose the query dynamically and run
For this task let me propose : Rules from the System.Workflow.Activities.Rules namespace. For rules there're several designers available not to mention the ones shipped with Visual Studio (for the web that's another question, but there're several ones for that too).I'd start with Rules without workflow then examine examples from msdn. It's a very flexible and customizable engine.
One other thing: LINQ has connection to this problem as a function returning IQueryable can defer query execution, you can previously define a query and in another part of the code one can extend the returned queryable based on the user's condition (which then can be sticked with extension methods).

When just using object, LINQ isn't really going to help you very much... is it worth the pain? And Dynamic LINQ is certainly overkill. What is the expected way of using this? I can think of a few ways of adding basic Where operations.... but I'm not sure how helpful it would be.

How about embedding something like IronPython in your project? We use that to allow users to define their own expressions (filters and otherwise) inside a sandbox.

I'm thinking about something like this:
((col1 = "abc") or (col2 = "xyz")) and (col3 = "123")
Ultimately it would be nice to have support for LIKE operator with % wildcard.

Thank you all guys - I've finally found it. It's called NQuery and it's available from CodePlex. In its documentation there is even an example which contains a binding to my very structure - list of column names + list of object[]. Plus fully functional SQL query engine.
Just perfect.