I have this function which calculate an checksumAggregate on a set of rows that should themselves be checksum-med. Unfortunately if the tables don't have a unique INT column so the checksumAggregate isn't enough. I have to use Checksum to be able to use other columns.
I am using System.Linq.Dynamic to allow the function to be generic.
Where the code below references x.ColumnName.ToString() is what I need to replace with something like "ColumnName" like I do for the OrderBy and Where clause. (I would pass the string in another parameter of the function)
Since it uses SQLFunctions I am now stuck at how to do this. It's a shame there is no "*" for this function.
I really hope this makes sense. Any pointers in the right direction will be appreciated.
public int calculateChecksum<T>(IQueryable<T> query, string SortColumn, DateTime? TimeStamp, Int64 Offset = 0)
{
if (TimeStamp == null) {TimeStamp = new DateTime(1800,1,1);}
string filterExp = SortColumn + " = #0";
query = query.Where<T>(filterExp, TimeStamp);
query = query.OrderBy<T>(SortColumn);
query = query.Take(varTakeCount);
int outChecksum = SqlFunctions.ChecksumAggregate(query.Select(x => SqlFunctions.Checksum( x.ColumnName.ToString() )));
return outChecksum;
}
EDIT:
With a few tweaks I am able to get the ChecksumAggregate to work on a single INT column. Close, but not ideal.
string filterType = typeof(T).FullName.Split('.').LastOrDefault();
var p = Expression.Parameter(typeof(T), filterType);
var e = System.Linq.Dynamic.DynamicExpression.ParseLambda(new[] { p }, null, "tablename.columnname");
int outChecksum = SqlFunctions.ChecksumAggregate(query.Select((Expression<Func<T, int>>)e));
Also I tried fooling around with stuff like this:
var checksumMethod = typeof (SqlFunctions).GetMethod("Checksum", new[] {typeof (string), typeof (string)});
var checksumExpression = Expression.Call(checksumMethod,e1,e1);
var e2=Expression.Lambda<Func<Int32>>(checksumExpression);
int outChecksum = SqlFunctions.ChecksumAggregate(query.Select(Expression<Func<T, bool>>)e2));
But can't quite make it passed runtime. I get a Nullable'1 Int32 can't be cast into an Int32. Sounds like it should.
EDIT 2:
Although this doesn't answer my original question I found a hack around it.
public int calculateChecksum<T, TContext>(DbContext context, IQueryable<T> query, string sortColumn, DateTime? TimeStamp, int Offset = 1)
{
if (TimeStamp == null) {TimeStamp = new DateTime(1800,1,1);}
string filterExp = sortColumn + " = #0";
query = query.Where<T>(filterExp, TimeStamp);
query = query.OrderBy<T>(sortColumn);
query = query.Take(Offset);
string sql = query.ToString();
sql = string.Format("select checksum_agg(BC) from ( select binary_checksum(*) as BC from ({0}) b ) c",sql);
int result = context.Database.SqlQuery<int>(sql).FirstOrDefault();
return result;
}
I need a beer!
Related
Currently I'm trying keeping a persistent where clause and then appending additional filter by appending if each textbox is present or not. The problem is this is giving me a good command to work with but I still receive an error when trying to use that command. If there is any simplification or guidance it would be much appreciated!
if (string.IsNullOrEmpty(make) && (string.IsNullOrEmpty(model)) && (string.IsNullOrEmpty(color)) && (string.IsNullOrEmpty(min)) && (string.IsNullOrEmpty(max)) && (string.IsNullOrEmpty(miles)))
{
SqlCommand updateDataGridViewCmd = new SqlCommand("select m.make, m.model, car.price, color.color, car.mileage, carlot.lotid, car.pic from car join makemodel as m ON m.mmid = car.mmid join color ON car.colorid = color.colorid join carlot ON carlot.carid = car.carid; ", sqlCon);
dt.Load(updateDataGridViewCmd.ExecuteReader());
dataGridView1.DataSource = dt;
}
else
{
StringBuilder sqlCommandText = new StringBuilder();
sqlCommandText.Append("select m.make, m.model, car.price, color.color, car.mileage, carlot.lotid, car.pic from car join makemodel as m ON m.mmid = car.mmid join color ON car.colorid = color.colorid join carlot ON carlot.carid = car.carid where");
string CommandText = sqlCommandText.ToString();
SqlCommand updateDataGridViewCmd = new SqlCommand(CommandText, sqlCon);
updateDataGridViewCmd.Parameters.AddWithValue("#make", make);
updateDataGridViewCmd.Parameters.AddWithValue("#model", model);
updateDataGridViewCmd.Parameters.AddWithValue("#min", min);
updateDataGridViewCmd.Parameters.AddWithValue("#max", max);
updateDataGridViewCmd.Parameters.AddWithValue("#mileage", miles);
updateDataGridViewCmd.Parameters.AddWithValue("#color", color);
if (!string.IsNullOrEmpty(make))
{
sqlCommandText.Append(" m.make = #make");
CommandText = sqlCommandText.ToString();
}
if (!string.IsNullOrEmpty(model))
{
sqlCommandText.Append(" OR m.model = #model");
CommandText = sqlCommandText.ToString();
}
if (!string.IsNullOrEmpty(min))
{
sqlCommandText.Append(" car.price between #min");
CommandText = sqlCommandText.ToString();
if (!string.IsNullOrEmpty(max))
{
sqlCommandText.Append(" AND #max");
CommandText = sqlCommandText.ToString();
}
else
{
sqlCommandText.Append(",");
CommandText = sqlCommandText.ToString();
}
}
if (!string.IsNullOrEmpty(color))
{
sqlCommandText.Append(" color.color = #color,");
CommandText = sqlCommandText.ToString();
}
if (!string.IsNullOrEmpty(miles))
{
sqlCommandText.Append(" car.price <= #mileage");
CommandText = sqlCommandText.ToString();
}
sqlCommandText.Append(";");
CommandText = sqlCommandText.ToString();
dt.Load(updateDataGridViewCmd.ExecuteReader());
dataGridView1.DataSource = dt;
}
}
}
ERROR:
You may see an error because you are not joining your condition strings either with and or or.
If ORM is not an option here (which may be practical when composiong queries) you can end the base query with where 1=1 and then chain the other conditions with and x=... or or x=...
Also you can set CommandText = sqlCommandText.ToString(); only once, after all filters have been applied
Consider this being a pseudo-code. I wrote it from my head in this editor. Basically, instead of dealing with endless conditions, create an object that will group your conditions and then group the output of instances of these objects. It should build a perfect filter for you.
public class WhereToken
{
private List<SqlParameter> _localColl;
// privates declared here
privat bool _between;
public WhereToken (string col, object[] values, SqlDbType t, SqlParameterCollection paramColl)
{
// assign privates here
}
public WhereToken (string col, object value1, object value2, SqlDbType t, SqlParameterCollection paramColl)
{
// assign privates here
_between = true;
_values = new object[]{value1, value2};
}
public string Write()
{
if (values.Length = 0)
return null;
_localColl = new List<SqlParameter>();
var b = new StringBuilder();
b.Append("(");
for (int i = 0; l < _values.Length; i++)
{
var pName = string.Concat("#", col, i);
var p = new SqlParameter(pName, values[i], _sqlType);
b.Append(_col);
if (_between)
b.Append(" BETWEEN ");
else
b.Append("=");
b.Append(pName);
if (i < values.Length - 1)
{
if (_between)
b.Append(" AND ");
else
b.Append(" OR ");
}
}
b.Append(")");
foreach(var pp in _localColl)
_paramColl.Parameters.Add(pp);
return b.ToString();
}
}
// In code
var tokens = new List<WhereToken>();
var wt1 = new WhereToken("make", new string[]{txt.Make.Text}, SqlDbType.NVarchar, sqlCommand.ParameterCollection);
tokens.Add(w1);
// in real life, you need to check if both values present, etc
var wt2 = new WhereToken("year", Convert.ToInt32(txt.YearMin.Text), Convert.ToInt32(txt.YearMax.Text), SqlDbType.Int, sqlCommand.ParameterCollection);
tokens.Add(w2);
// more items here . . . .
var builder = new StringBuilder(yourMainSQLSelect);
builder.Append(" WHERE 1=1"); // 1=1 what if the tokens don't generate
for (int i = 0; i < tokens.Length; i++)
{
string token = tokens[i].Write();
if (token == null) continue;
builder.Append(" AND ");
builder.Append(token);
}
sqlCommand.CommandText = builder.ToString();
This code ^^ should build you something like
... where 1=1 and make=#make0 and (year BETWEEN #year0 AND #year1)
Of course, you can improve built-in logic on how to parse and output from WhereToken. But this is the concept for you - parse small tokens then join them. Don't try build logic for all input fields
It seems to me, that you want to expand your WHERE with predicates if some input string are null or empty.
Let's ignore empty string, so I won't have to write: is null or empty string
SELECT ... FROM TABLECUSTOMERS WHERE <predicate1> OR <predicate2> OR ...
The predicates that you add depend on which input strings are null. It seems to me that most of the times, if the input string is null, you want to omit the predicate. Something like this:
if (inputStringName != null)
{
strBuilder.Append("OR CustomerName = #Name")
}
This is the same as:
OR (#Name != null AND #Name == CustomerName)
And if you want to use AND instead of OR:
AND (#Name == null OR #Name == CustomerName)
So my advice would be: make one SQL command, that contains all possible variables, and change the command such that the predicate also testing whether the parameter equals null
SELECT ... FROM car
JOIN ...
WHERE ( NOT #Make = NULL AND #Make = m.Make)
OR ( NOT #Model = NULL AND #Model = m.Model)
OR ( NOT #Mileage = NULL AND #Mileage > car.Mileage)
So just before you use the parameter you check whether the parameter is null or not. Depending on whether you want to append with AND or OR use something like:
OR (NOT #Value = NULL AND #Value = car.MyValue)
AND (#Value = NULL OR #Value = car.MyValue)
My SQL is a bit rusty, I use entity framework all the time, so I am a bit uncertain about the NOT =, maybe this should be !=, and maybe there should be more parentheses, but I guess you get the gist.
To prevent to have to check for Empty strings: make empty strings null, before you AddWithValue
What is the correct syntax for varname to make this query work?
I can get it to work with a single variable like
string varname = "TOTAL_NORWAY"
However, if I want to have a few variables in there, I get an empty array returned:
string varname = "'TOTAL_NORWAY', 'TOTAL_SWEDEN'";
return await _Context.theDataModel.FromSqlRaw(#"
select data
from data_table
where Variable in ({0})
", varname).ToListAsync();
Remember that you can combine FromSqlRaw with Linq:
string varnames = new [] { "TOTAL_NORWAY", "TOTAL_SWEDEN" };
var query = _Context.theDataModel.FromSqlRaw(#"
select data
from data_table");
query = query.Where(x => varnames.Contains(x.Variable));
// Add more where clauses as needed
return await query.ToListAsync();
ErikEJ's post was very helpful. The solution is not so trivial for someone who doesn't dabble in EF Core regularly.
I also had an extra where clause to consider, and this was done like so for anyone else wondering.
var items = new int[] { 1, 2, 3 };
var parameters = new string[items.Length];
var sqlParameters = new List<SqlParameter>();
for (var i = 0; i < items.Length; i++)
{
parameters[i] = string.Format("#p{0}", i);
sqlParameters.Add(new SqlParameter(parameters[i], items[i]));
}
sqlParameters.Add(new SqlParameter("#userid", "userXYZ123"));
var rawCommand = string.Format("SELECT * from dbo.Shippers WHERE ShipperId IN ({0}) and userid = {1}", string.Join(", ", parameters), "#userid");
var shipperList = db.Set<ShipperSummary>()
.FromSqlRaw(rawCommand, sqlParameters.ToArray())
.ToList();
I want to create a dynamic query with LINQ-to-SQL and everything works except for one thing: group by. I looked at the other questions but haven't found a reason so far why it does not work.
I use using System.Linq.Dynamic; with the following query (that works):
int numberOfRankedItems = 3;
string minMaxChoice = "max";
string orderBy = "";
if (minMaxChoice == "max")
{
orderBy = "queryGroup.Sum(row => row.POSITIONVALUE) descending";
} else if (minMaxChoice == "min")
{
orderBy = "queryGroup.Sum(row => row.POSITIONVALUE) ascending";
}
string minMaxFeatureColumnName = "BRANDNAME";
string selectMinMaxFeature = "new { minMaxFeature = queryGroup.Key." + minMaxFeatureColumnName + ", sumPosition = queryGroup.Sum(row => row.POSITIONVALUE)}";
var query = (from tableRow in Context.xxx
where /* some filters */
group tableRow by tableRow.BRANDNAME into queryGroup
orderby orderBy
select selectMinMaxFeature).Take(numberOfRankedItems).ToList();
The use of variables for orderby and select works fine, but for group by not no matter what I try to replace with a variable. The query actually works if I e.g. use a variable instead of tableRow.BRANDNAME but the query returns the wrong result. The goal is to be able to set the column for the grouping dynamically.
Ideas?
Thanks
Edit: there seem to be multiple issues also with the other variables. So I generalize the question a bit: how can I generalize the following query in terms of
The column to group by
The column to order by + ASC or DESC
In the select statement the columnname of the first statement (BRANDNAME)
Here is the query:
var query = (from tableRow in ctx.xxx
where /* filter */
group tableRow by new { tableRow.BRANDNAME } into queryGroup
orderby queryGroup.Sum(row => row.POSITIONVALUE) descending
select new { minMaxFeature = queryGroup.Key.BRANDNAME, sumPosition = queryGroup.Sum(row => row.POSITIONVALUE) }).Take(numberOfRankedItems).ToList();
Would be great without expression trees ;)
I have now elaborated a solution to the question:
The solution uses as before using System.Linq.Dynamic; allows now to set certain variables. In the end it returns a dictionary.
// set variables
int numberOfRankedItems;
if (queryData.NumberOfRankedItems != null)
{
numberOfRankedItems = (int) queryData.NumberOfRankedItems;
} else
{
numberOfRankedItems = 0;
}
string minMaxChoice = queryData.MinMaxChoice;
string minMaxFeatureColumnName = queryData.MinMaxFeatureColumnName;
string orderByPrompt = "";
if (minMaxChoice == "max")
{
orderByPrompt = "it.Sum(POSITIONVALUE) descending";
} else if (minMaxChoice == "min")
{
orderByPrompt = "it.Sum(POSITIONVALUE) ascending";
}
string groupByPrompt = queryData.MinMaxFeatureColumnName;
// execute query
IQueryable query = (from tableRow in Context.xxx
where /* some filters */
select tableRow).
GroupBy(groupByPrompt, "it").
OrderBy(orderByPrompt).
Select("new (it.Key as minMaxFeature, it.Sum(POSITIONVALUE) as sumPosition)").
Take(numberOfRankedItems);
// create response dictionary
Dictionary<int?, Dictionary<string, int?>> response = new Dictionary<int?, Dictionary<string, int?>>();
int count = 1;
foreach (dynamic item in query)
{
string minMaxFeatureReturn = item.GetType().GetProperty("minMaxFeature").GetValue(item);
int? sumPositionReturn = item.GetType().GetProperty("sumPosition").GetValue(item);
response[count] = new Dictionary<string, int?>() { { minMaxFeatureReturn, sumPositionReturn } };
count++;
}
return response;
I'm trying to have a method return a count statement.
public int getSettingsCount(string UserId, string Setting)
{
int LastSetting;
//var user = new SqlDataLayer();
using (var db = new SQLite.SQLiteConnection(this.DBPath))
{
{
List<int> _setting = db.Query<int>("SELECT COUNT(*) FROM QTabSettings WHERE UserId = 1058 AND Setting = 'ServerDropdown' GROUP BY UserId;");
LastSetting = Convert.ToInt32(_setting.SingleOrDefault());
}
return LastSetting;
}
}
When I execute the query it returns the correct value (6). However I am getting the value (0) from my above query.
How can I get the method to return the count as an int?
You are using LIMIT 1 and still using List<int> that's strange and unnecessary. Also since it's count(*) there is no need of LIMIT 1 since the result would be a scalar data. Should change it to
int _setting = db.Query<int>("SELECT COUNT(*) FROM QTabSettings WHERE UserId = 1058 AND Setting = 'ServerDropdown';");
LastSetting = _setting;
If I understand your question properly I think this is what you want:
LastSetting = _setting.FirstOrDefault();
or:
LastSetting = _setting[0];
Having something similar to:
SELECT (SELECT COUNT(*) from Table1),(SELECT COUNT(*) from Table2 )
How do I write it in linq? Or is it simple not possible?
Limitations:
Can only hit the database one time:
var result = new {
Sum1 = db.Table1.Count(),
Sum2 = db.Table2.Count()
}); // is not valid.....
I do not want to use something similar to (using a "helping" table):
var result = (from t3 in db.Table3
select new {
Sum1 = db.Table1.Count(),
Sum2 = db.Table2.Count()
}).firstOrDefault();
//In order to get only the first row
//but it will not return nothing if the table 3 has no entries......
Not using db.Database.ExecuteSqlCommand
I cannot see a solution which solves all your limitations. This is one of the caveats with using an ORM-mapper, you are not in control of the generated SQL.
In this case, if it is utterly unacceptable for you to send more than one query to the database, the harsh truth is that you will have to write the query yourself.
Update
I got curious and created an extension method that can do this! Of course it constructs its own SQL command, and it just works for Linq2SQL. Also massive disclaimer: It's fairly dirty code, if I have some time I'll fix it up in the weekend :)
public static TOut CountMany<TContext, TOut>(this TContext db, Expression<Func<TContext, TOut>> tableSelector)
where TContext: DataContext
{
var newExpression = (NewExpression) tableSelector.Body;
var tables =
newExpression.Arguments.OfType<MethodCallExpression>()
.SelectMany(mce => mce.Arguments.OfType<MemberExpression>())
.ToList();
var command = new string[tables.Count];
for(var i = 0; i < tables.Count; i++)
{
var table = tables[i];
var tableType = ((PropertyInfo) table.Member).PropertyType.GetGenericArguments()[0];
var tableName = tableType.GetCustomAttribute<TableAttribute>().Name;
command[i] = string.Format("(SELECT COUNT(*) FROM {0}) AS T{1}", tableName, i);
}
var dbCommand = db.Connection.CreateCommand();
dbCommand.CommandText = string.Format("SELECT {0}", String.Join(",", command));
db.Connection.Open();
IDataRecord result;
try
{
result = dbCommand.ExecuteReader().OfType<IDataRecord>().First();
}
finally
{
db.Connection.Close();
}
var results = new object[tables.Count];
for (var i = 0; i < tables.Count; i++)
results[i] = result.GetInt32(i);
var ctor = typeof(TOut).GetConstructor(Enumerable.Repeat(typeof(int), tables.Count).ToArray());
return (TOut) ctor.Invoke(results);
}
the code is called like this:
var counts = dbContext.CountMany(db => new
{
table1Count = db.Table1.Count(),
table2Count = db.Table2.Count()
//etc.
});