I have the following raw query being executed through Entity Framework via a SqlQuery call on the DbSet:
public IEnumerable<TEntity> GetComplicatedData<TEntity>()
{
return database
.Set<TEntity>()
.SqlQuery("SELECT * FROM <Complicated Query Here>");
}
...
var count = GetComplicatedData<Item>()
.Where(f => f.OwnerID == 5)
.Count();
This works, but is very slow due to the fact that SqlQuery executes immediately without the Where getting applied.
Is there any way to call SqlQuery in such a way that the Where gets applied server-side?
Essentially, I want Entity Framework to generate a store query like:
SELECT
<Columns>
FROM
(
SELECT * FROM <Complicated Query Here>
) a
WHERE a.OwnerID = 5
Or, is there a way to translate my where expression into a query that I can append manually (a.k.a, without manually writing a SQL query, the where clause is not always that simple)?
This cannot be done with LINQ methods, because "raw" queries are of type DbRawSqlQuery<TElement>, which lacks support required for "composing" dynamic queries with Where.
If you are interested in counting items, you could work around the issue by taking a condition:
public int CountComplicatedData<TEntity>(Func<TEntity,bool> condition) {
return database
.Set<TEntity>()
.SqlQuery("SELECT * FROM <Complicated Query Here>")
.Count(condition);
}
Related
I use ormlite with servicestack and I have got this problem.
I have saved a list of string in a column of my db so I want to do a select sql like this:
Select top 1 *
From MyTable
Where MyVariable In (MyListSavedInDb)
var orders = db.Select<Order>(o => Sql.In(o.Ldv, o.Waybills));
Where o.Ldv is a string and o.Waybills is a list of string saved on db
Any solutions ?
You can't query a blobbed field with server-side SQL, best you can do is a fuzzy string index search like:
var q = db.From<Order>();
q.Where($"CHARINDEX({q.Column<Order>(x=>x.Ldv)},{q.Column<Order>(x=>x.Waybills)}) > 0")
.Take(1);
var order = db.Single(q);
But essentially you shouldn't be blobbing any fields that you want to perform server-side SQL queries on.
A more typed and robust approach would be to perform the final query on a blobbed collection on the client after you've filtered the resultset, e.g:
var orders = db.Select(q);
var order = orders.FirstOrDefault(x => x.Waybills.Contains(term));
But as this query is done on the client you'll want to ensure it's being done on a limited filtered resultset.
Since EF Core does not currently execute group by queries in the database, I'm using the following query to get the job done (InvoiceQuery is a DbQuery type):
long merchantId = 177;
var invTokens = new List<string> {"qasas", "efsac"};
string inClause = string.Join(",", invTokens);
string query = $#"SELECT i.Token, i.Balance, COALESCE(SUM(i.Amount), 0) AS ProcessingAmount
FROM inv.Invoices i
WHERE i.Token IN ({inClause})
AND i.MerchantId = {merchantId}
AND i.Status = {(int)InvoiceStatus.Scheduled}
GROUP BY i.Token, i.Balance";
return await dbContext.InvoicesQuery.FromSql(query).ToListAsync();
The above code runs perfect but I don't like it because it gives me a warning of a 'Possible SQL injection vulnerability'.. I know I can use a #pragma to suppress this warning (that's what I'm currently using).
I have tried passing the query and provide the params to the FromSql method, and that works but the parameter that goes into the IN clause is not getting correctly mapped..
Tried using $"'{string.Join("', '", invTokens)}'" but it didn't work.
Any help would be appreciated
Since EF Core 2.0, you can use a FormattableString, where you can use string interpolation for queries.
String interpolation in FromSql and ExecuteSqlCommand
FormattableString Class
In EF Core 2.0 we added special support for interpolated strings to our two primary APIs that accept raw SQL strings: FromSql and ExecuteSqlCommand. This new support allows C# string interpolation to be used in a 'safe' manner.
The example from the docs:
var city = "London";
var contactTitle = "Sales Representative";
using (var context = CreateContext())
{
context.Set<Customer>()
.FromSql($#"
SELECT *
FROM ""Customers""
WHERE ""City"" = {city} AND
""ContactTitle"" = {contactTitle}")
.ToArray();
}
This will produce:
#p0='London' (Size = 4000)
#p1='Sales Representative' (Size = 4000)
SELECT *
FROM ""Customers""
WHERE ""City"" = #p0
AND ""ContactTitle"" = #p1
I'm working on a small ASP.NET Core project for tagging images using Entity Framework Core on a Sqlite database, mainly just for learning. There are two tables (and POCOs), Tags and Images, where multiple tags are related to each image. I'm trying to get a count of all Images that have tags associated with them.
In plain SQL I'd write SELECT COUNT(DISTINCT ImageId) FROM Tags to get the count, and in LINQ I came up with _context.Tags.Select(t => t.Image).Distinct().Count(). But that LINQ query appears to cause EF-Core to join the two tables, return all of the rows, and then do the Distinct and Count in code.
I tried to do _context.Tags.FromSql("SELECT COUNT(DISTINCT ImageId) FROM Tags"), but because that query only returns the count the call fails because EF can't map the result to a Tag. I also tried to use _context.Database.FromSql<int>, but wasn't able to find any real documentation on it and there doesn't seem to be IntelliSense for it.
What I have done for now is what's detailed in the "ADO.NET" section of this blog post from Eric Anderson:
int count;
using (var connection = _context.Database.GetDbConnection())
{
connection.Open();
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT COUNT(DISTINCT ImageId) FROM Tags";
string result = command.ExecuteScalar().ToString();
int.TryParse(result, out count);
}
}
But is that the best way to go about getting the count efficiently?
Edit: Here's the query that EF is putting in the Debug output:
SELECT "t"."TagId", "t"."Content", "t"."ImageId", "t.Image"."ImageId", "t.Image"."FileName", "t.Image"."Path", "t.Image"."Url"
FROM "Tags" AS "t"
LEFT JOIN "Images" AS "t.Image" ON "t"."ImageId" = "t.Image"."ImageId"
ORDER BY "t"."ImageId"
As of now, you can't define an ad-hoc result.
Good news is that it's currently on the backlog: https://github.com/aspnet/EntityFramework/issues/1862
In the meantime, here's an extension method that would work:
public static int IntFromSQL(this ApplicationDbContext context, string sql )
{
int count;
using (var connection = context.Database.GetDbConnection())
{
connection.Open();
using (var command = connection.CreateCommand())
{
command.CommandText = sql;
string result = command.ExecuteScalar().ToString();
int.TryParse(result, out count);
}
}
return count;
}
Usage:
int result = _context.IntFromSQL("SELECT COUNT(DISTINCT ImageId) FROM Tags");
Your original line of code should have done exactly what you wanted. It would also be recommended over inline SQL.
_context.Tags.Select(t => t.Image).Distinct().Count()
Are you sure that this called the database for the two tables and then queried them in memory? If you observed this behaviour while debugging then it's possible that your inspection caused the IQueryable to enumerate which would call the database using a different query than it would have otherwise.
A way to check the actual query, without breaking into the running code, is by using the MyLoggerProvider from the Entity Framework Core documentation.
https://docs.efproject.net/en/latest/miscellaneous/logging.html?highlight=logging
Once the logger is registered in the code then any SQL query ran against the server will be displayed in the console window and/or in the file c:\temp\log.txt.
The following log message was generated when using a Distinct() and a Count() on the database tables of the website example.
SELECT COUNT(*)
FROM (
SELECT DISTINCT [a.Blog].[BlogId], [a.Blog].[Url]
FROM [Posts] AS [a]
INNER JOIN [Blogs] AS [a.Blog] ON [a].[BlogId] = [a.Blog].[BlogId]
) AS [t]Closing connection to database '***' on server 'tcp:**************'.
Finally, since you do not need any of the properties on the t.Image then it seems that you should be using a Where() rather than a Select().
I am trying to perform a dapper query like this:
string query = "select * from MyTable where someNumber in #Nums;";
...
connection.Query<ReturnObj>(query, new {Nums = nums})
And I am getting a MySql syntax error if nums is empty. It looks like Dapper changes the query to look like this: WHERE 1 = 0) so I am guessing it the left ( is missing, which is causing the syntax error. Yes, I realize I could just check if the collection is empty before executing the query, but I would rather not if I don't have to.
This is a bug in Dapper where it creates a SQL statement that is invalid for MySQL Server 5.6 (and earlier).
Workarounds:
Upgrade to MySQL Server 5.7 (which accepts the SQL Dapper generates and returns the expected results)
As you said, check if the collection is empty before executing the query
A variant of checking if the collection is empty (that can be useful if you have a complex query, NOT IN, etc.):
var numsSql = nums.Any() ? "#Nums" : "(select null)";
var query = $"select * from MyTable where someNumber in {numsSql};";
conn.Query(query, new { Nums });
Up to now (with NHibernate) I've used entity mapping and not really got involved with creating raw sql queries - but somethings come up where I need to do exactly that.
The problem I have is I want to automatically map the columns aliases of my query to a Dto object.
This works, but i have to specify the column alias' in the order of the query.
SQL
string sql = "select mycol1 as ColumnOne from mytable";
NHibernate Query
var query = session.CreateSQLQuery(sql)
.AddScalar("ColumnOne", NHibernateUtil.Int32)
.SetResultTransformer(
NHibernate.Transform.Transformers.AliasToBean<MyDtoObject>()
);
MyDtoObject
public class MyDtoObject
{
public int ColumnOne {get;set;}
}
But is there a way to making NHibernate automate the mapping between the columns in the query and the Dto without creating a mapping class?
I've seen some examples of using aliases in the query e.g.
string sql = "select mycol1 as {ColumnOne} as ColumnOne from mytable"; /// ???
But cannot get this to work as the alias {ColumnOne} appear not to be replaced before being sent to the db as a sql statement.
Any idea?
TIA
Sam
Maybe System.Linq.Dymanic will help:
use System.Linq.Dynamic;
string ColumnAlisName = "ColumnOne";
var query = mytable.Select(String.Format("new (mycol1 as {0})",ColumnAlisName));