I am using Microsoft's Dynamic Linq (System.Linq.Dynamic) library to generate some queries at run time. This has worked great for me, but for one specific scenario.
Simplified scenario - I am trying to query all claims which have some specific tags that the user has selected and whose Balance is greater than some number.
static void Main(string[] args)
{
var claims = new List<Claim>();
claims.Add(new Claim { Balance = 100, Tags = new List<string> { "Blah", "Blah Blah" } });
claims.Add(new Claim { Balance = 500, Tags = new List<string> { "Dummy Tag", "Dummy tag 1" } });
// tags to be searched for
var tags = new List<string> { "New", "Blah" };
var parameters = new List<object>();
parameters.Add(tags);
var query = claims.AsQueryable().Where("Tags.Any(#0.Contains(outerIt)) AND Balance > 100", parameters.ToArray());
}
public class Claim
{
public decimal? Balance { get; set; }
public List<string> Tags { get; set; }
}
This query throws an error:
An unhandled exception of type 'System.Linq.Dynamic.ParseException' occurred in System.Linq.Dynamic.dll
Additional information: No property or field 'Balance' exists in type 'String'
Dynamic linq parser seems to try to find the Balance property on the Tag and not on the Claim object.
I have tried to play around with outerIt, innerIt, It keywords in Dynamic Linq but none of it seems to work.
Changing the sequence works, but that's not an option for me, since in the real application the filters, operators and patterns will be dynamic (configured by end user).
Boxing the conditions in brackets (), also doesn't help.
Workaround - create a simple contains condition for every Tag selected e.g. Tags.Contains("New") OR Tags.Contains("Blah") etc.. But in the real application it results in a really complex / bad query for each condition and kills the performance.
I might be missing something or this could be a bug in the library.
I would really appreciate if someone could help me with this.
Found a/the bug in ParseAggregate... The pushing of it→outerIt and back doesn't work if there are multiple levels. The code supposes that the it and outerIt won't be changed by a third party before being reset (technically the code isn't reentrant). You can try with other variants of System.Linq.Dynamic (there are like two or three variants out of there). Probably some variants have already fixed it.
Or you can take the code from the linked site and recompile it inside your code (in the end the "original" System.Linq.Dynamic is a single cs file) and you can patch it like this:
Expression ParseAggregate(Expression instance, Type elementType, string methodName, int errorPos)
{
// Change starts here
var originalIt = it;
var originalOuterIt = outerIt;
// Change ends here
outerIt = it;
ParameterExpression innerIt = Expression.Parameter(elementType, elementType.Name);
it = innerIt;
Expression[] args = ParseArgumentList();
// Change starts here
it = originalIt;
outerIt = originalOuterIt;
// Change ends here
MethodBase signature;
if (FindMethod(typeof(IEnumerableSignatures), methodName, false, args, out signature) != 1)
I've already opened an Issue with the suggested bug fix in the github of the project.
This seems to be working correctly in my version: System.Linq.Dynamic.Core
See the test here:
https://github.com/StefH/System.Linq.Dynamic.Core/blob/master/test/System.Linq.Dynamic.Core.Tests/ComplexTests.cs#L19
Related
I realize this is answered in the documentation (basically, "use the dot syntax"), but I'm still missing something. I'm using the .NET SDK and need to be able to select just a few attributes from a scan, one of which is a boolean inside a Map attribute. Note that I'm not trying to filter the results. I want all of the items, but I only want some of the attributes returned to me.
var config = new ScanOperationConfig
{
AttributesToGet = new List<string> { "foo.bar" },
Select = SelectValues.SpecificAttributes
};
var search = Table.Scan(config);
var documents = await search.GetRemainingAsync();
This code gets me the items I expect, but it's missing the "foo.bar" attribute. I know I can select the entire foo object, but I'm trying to minimize the amount of data handed back. I don't want the other attributes inside the foo object.
The relevant attribute of the item has the following JSON format:
{
"foo": {
"bar": true
}
}
I checked spelling, case sensitivity, etc. to no avail. Any idea what's wrong?
Instead of using Table.Scan, use the AmazonDynamoDBClient and you get more options.
The Client's ScanAsync method takes a ScanRequest which has a ProjectionExpression string. This is not present on the ScanOperationConfig class and that was the source of confusion.
Use the ProjectionExpression like so:
var scanRequest = new ScanRequest(Table.TableName)
{
ProjectionExpression = "foo.bar"
};
According to the documentation on ProjectionExpression:
ProjectionExpression replaces the legacy AttributesToGet parameter.
I didn't realize AttributesToGet was legacy until finally looking at the client for a totally unrelated problem and happened to find my answer to this problem.
I'm working with TimeCode enumeration and I'd like to explore the one that relates to 3 (which is the next-to-last, for some weird reason in the table).
QueryScheduleRequest request = new QueryScheduleRequest
{
ResourceId = user.Id,
Start = DateTime.UtcNow,
End = DateTime.UtcNow.AddDays(3.5),
TimeCodes = new[] { TimeCode.Available, TimeCode.Unavailable,TimeCode.Filter }
};
I've googled my donkey off but I can't find any examples on how to specify the exact conditions for the filter (there's nothing sounding right amongst the properties of QueryScheduleRequest initializer.
Anybody with better google-fu or experience with filtering by time codes?
I am using linq2db and while it works well enough for most CRUD operations I have encountered many expressions that it just cannot translate into SQL.
It has gotten to the point where unless I know in advance exactly what kinds of expressions will be involved and have successfully invoked them before, I am worried that any benefit derived from linq2db will be outweighed by the cost of trying to find and then remove (or move away from the server side) the offending expressions.
If I knew how to tell linq2db how to parse an Expression<Func<T,out T>> or whatnot into SQL whenever on an ad-hoc, as-it-is-needed basis, then I would be much more confident and I could do many things using this tool.
Take, for instance, String.Split(char separator), the method that takes a string and a char to return a string[] of each substring between the separator.
Suppose my table Equipment has a nullable varchar field Usages that contains lists of different equipment usages separated by commas.
I need to implement IList<string> GetUsages(string tenantCode, string needle = null) that will give provide a list of usages for a given tenant code and optional search string.
My query would then be something like:
var listOfListOfStringUsages =
from et in MyConnection.GetTable<EquipmentTenant>()
join e in MyConnection.GetTable<Equipment>() on et.EquipmentId = e.EquipmentId
where (et.TenantCode == tenantCode)
where (e.Usages != null)
select e.Usages.Split(','); // cannot convert to sql here
var flattenedListOfStringUsages =
listOfListOfStringUsages.SelectMany(strsToAdd => strsToAdd)
.Select(str => str.Trim())
.Distinct();
var list = flattenedListOfStringUsages.ToList();
However, it would actually bomb out at runtime on the line indicated by comment.
I totally get that linq2db's creators cannot possibly be expected to ship with every combination of string method and major database package.
At the same time I feel as though could totally tell it how to handle this if I could just see an example of doing just that (someone implementing a custom expression).
So my question is: how do I instruct linq2db on how to parse an Expression that it cannot parse out of the box?
A few years ago I wrote something like this:
public class StoredFunctionAccessorAttribute : LinqToDB.Sql.FunctionAttribute
{
public StoredFunctionAccessorAttribute()
{
base.ServerSideOnly = true;
}
// don't call these properties, they are made private because user of the attribute must not change them
// call base.* if you ever need to access them
private new bool ServerSideOnly { get; set; }
private new int[] ArgIndices { get; set; }
private new string Name { get; set; }
private new bool PreferServerSide { get; set; }
public override ISqlExpression GetExpression(System.Reflection.MemberInfo member, params ISqlExpression[] args)
{
if (args == null)
throw new ArgumentNullException("args");
if (args.Length == 0)
{
throw new ArgumentException(
"The args array must have at least one member (that is a stored function name).");
}
if (!(args[0] is SqlValue))
throw new ArgumentException("First element of the 'args' argument must be of SqlValue type.");
return new SqlFunction(
member.GetMemberType(),
((SqlValue)args[0]).Value.ToString(),
args.Skip(1).ToArray());
}
}
public static class Sql
{
private const string _serverSideOnlyErrorMsg = "The 'StoredFunction' is server side only function.";
[StoredFunctionAccessor]
public static TResult StoredFunction<TResult>(string functionName)
{
throw new InvalidOperationException(_serverSideOnlyErrorMsg);
}
[StoredFunctionAccessor]
public static TResult StoredFunction<TParameter, TResult>(string functionName, TParameter parameter)
{
throw new InvalidOperationException(_serverSideOnlyErrorMsg);
}
}
...
[Test]
public void Test()
{
using (var db = new TestDb())
{
var q = db.Customers.Select(c => Sql.StoredFunction<string, int>("Len", c.Name));
var l = q.ToList();
}
}
(and of course you can write your wrappers around Sql.StoredFunction() methods to get rid of specifying function name as a string every time)
Generated sql (for the test in the code above):
SELECT
Len([t1].[Name]) as [c1]
FROM
[dbo].[Customer] [t1]
PS. We use linq2db extensively in our projects and completely satisfied with it. But yes, there is a learning curve (as with almost everything serious we learn) and one needs to spend some time learning and playing with the library in order to feel comfortable with it and see all the benefits it can give.
Try listOfListOfStringUsages.AsEnumerable(). It will enforce executing SelectMany on client side.
UPDATE:
I used the following code to reproduce the issue:
var q =
from t in db.Table
where t.StringField != null
select t.StringField.Split(' ');
var q1 = q
//.AsEnumerable()
.SelectMany(s => s)
.Select(s => s.Trim())
.Distinct()
.ToList();
It's not working. But if I uncomment .AsEnumerable(), it works just fine.
I am very new to C#, and got thrown into it by the company that I work for, so I apologize for my extreme lack of knowledge on all of this. The site uses a dropdown list so the user can select a language to view the site in. This is the code that I started with, which worked perfectly.
protected void CreateLanguageList()
{
// Get the available cultures.
string[] cultures = Telerik.Localization.Configuration.ConfigHelper.Handler.Cultures.Split(',');
foreach (string availableCulture in cultures)
{
var culture = new CultureInfo(availableCulture.Trim());
var item = new ListItem(culture.NativeName, culture.ToString());
LanguageSelector.Items.Add(item);
if (culture.ToString() == currentCultureString)
{
item.Selected = true;
}
}
}
Now that we are migrating to an updated CMS, Telerik.Localization is no longer supported. After contacting their support, they said that I could access the language files by using this code and accessing the name of the culture using the DisplayName() property:
var allCultures = AppSettings.CurrentSettings.AllLanguages.ToList();
Due to my very limited lack of knowledge on C# so far (I am trying to teach myself), I didn't want to change the code too much if I didn't have to, so I found where I could convert a list to a string, which i figured would allow me to leave the rest of the code alone, only having to change the one call to the language files. However, I am receiving a CS0030 error when I try and run this code:
protected void CreateLanguageList()
{
// Get the available cultures.
var allCultures = AppSettings.CurrentSettings.AllLanguages.ToList();
string combinedCultures = string.Join(",", allCultures);
foreach (string availableCulture in combinedCultures)
{
var culture = new CultureInfo(availableCulture.Trim());
var item = new ListItem(culture.NativeName, culture.ToString());
LanguageSelector.Items.Add(item);
if (culture.ToString() == currentCultureString)
{
item.Selected = true;
}
}
}
Any help in explaining what I'm doing wrong and how I could go about fixing it would be monumentally appreciated.
Your problem is that combinedCultures is defined as a string, but then you're trying to iterate over it with a type string in the foreach. When you iterate over a string you get chars. You should be able to just get rid of the combinedCultures variable altogether and foreach using allCultures.
The problem here is that foreach is only working on availableCulture because the type string is implicitly an array of char.
I'm not exactly sure what you're doing, but it looks like you're turning an array into a string when you just need to pass the array in directly.
Is there a class in the .NET framework which, if given a sequence/enumerable of key value pairs (or anything else I am not rigid on the format of this) can create a query string like this:
?foo=bar&gar=har&print=1
I could do this trivial task myself but I thought I'd ask to save myself from re-inventing the wheel. Why do all those string gymnastics when a single line of code can do it?
You can use System.Web.HttpUtility.ParseQueryString to create an empty System.Web.HttpValueCollection, and use it like a NameValueCollection.
Example:
var query = System.Web.HttpUtility.ParseQueryString(string.Empty);
query ["foo"] = "bar";
query ["gar"] = "har";
query ["print"] = "1";
var queryString = query.ToString(); // queryString is 'foo=bar&gar=har&print=1'
There's nothing built in to the .NET framework, as far as I know, though there are a lot of almosts.
System.Web.HttpRequest.QueryString is a pre-parsed NameValueCollection, not something that can output a querystring. System.NetHttpWebRequest expects you to pass a pre-formed URI, and System.UriBuilder has a Query property, but again, expects a pre-formed string for the entire query string.
However, running a quick search for "querystringbuilder" shows a couple of implementations for this out in the web that could serve. One such is this one by Brad Vincent, which gives you a simple fluent interface:
//take an existing string and replace the 'id' value if it exists (which it does)
//output : "?id=5678&user=tony"
strQuery = new QueryString("id=1234&user=tony").Add("id", "5678", true).ToString();
And, though not exactly very elegant, I found a method in RestSharp, as suggested by #sasfrog in the comment to my question. Here's the method.
From RestSharp-master\RestSharp\Http.cs
private string EncodeParameters()
{
var querystring = new StringBuilder();
foreach (var p in Parameters)
{
if (querystring.Length > 0)
querystring.Append("&");
querystring.AppendFormat("{0}={1}", p.Name.UrlEncode(), p.Value.UrlEncode());
}
return querystring.ToString();
}
And again, not very elegant and not really what I would have been expecting, but hey, it gets the job done and saves me some typing.
I was really looking for something like Xi Huan's answer (marked the correct answer) but this works as well.
There's nothing built into the .NET Framework that preserves the order of the query parameters, AFAIK. The following helper does that, skips null values, and converts values to invariant strings. When combined with a KeyValueList, it makes building URIs pretty easy.
public static string ToUrlQuery(IEnumerable<KeyValuePair<string, object>> pairs)
{
var q = new StringBuilder();
foreach (var pair in pairs)
if (pair.Value != null) {
if (q.Length > 0) q.Append('&');
q.Append(pair.Key).Append('=').Append(WebUtility.UrlEncode(Convert.ToString(pair.Value, CultureInfo.InvariantCulture)));
}
return q.ToString();
}