With Entity Framework Core removing dbData.Database.SqlQuery<SomeModel> I can't find a solution to build a raw SQL Query for my full-text search query that will return the tables data and also the rank.
The only method I've seen to build a raw SQL query in Entity Framework Core is via dbData.Product.FromSql("SQL SCRIPT"); which isn't useful as I have no DbSet that will map the rank I return in the query.
Any Ideas???
If you're using EF Core 3.0 or newer
You need to use keyless entity types, previously known as query types:
This feature was added in EF Core 2.1 under the name of query types.
In EF Core 3.0 the concept was renamed to keyless entity types. The
[Keyless] Data Annotation became available in EFCore 5.0.
To use them you need to first mark your class SomeModel with [Keyless] data annotation or through fluent configuration with .HasNoKey() method call like below:
public DbSet<SomeModel> SomeModels { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<SomeModel>().HasNoKey();
}
After that configuration, you can use one of the methods explained here to execute your SQL query. For example you can use this one:
var result = context.SomeModels.FromSqlRaw("SQL SCRIPT").ToList();
var result = await context.SomeModels.FromSql("SQL_SCRIPT").ToListAsync();
If you're using EF Core 2.1
If you're using EF Core 2.1 Release Candidate 1 available since 7 may 2018, you can take advantage of the proposed new feature which is query types:
In addition to entity types, an EF Core model can contain query types,
which can be used to carry out database queries against data that
isn't mapped to entity types.
When to use query type?
Serving as the return type for ad hoc FromSql() queries.
Mapping to database views.
Mapping to tables that do not have a primary key defined.
Mapping to queries defined in the model.
So you no longer need to do all the hacks or workarounds proposed as answers to your question. Just follow these steps:
First you defined a new property of type DbQuery<T> where T is the type of the class that will carry the column values of your SQL query. So in your DbContext you'll have this:
public DbQuery<SomeModel> SomeModels { get; set; }
Secondly use FromSql method like you do with DbSet<T>:
var result = context.SomeModels.FromSql("SQL_SCRIPT").ToList();
var result = await context.SomeModels.FromSql("SQL_SCRIPT").ToListAsync();
Also note that DbContexts are partial classes, so you can create one or more separate files to organize your 'raw SQL DbQuery' definitions as best suits you.
Building on the other answers I've written this helper that accomplishes the task, including example usage:
public static class Helper
{
public static List<T> RawSqlQuery<T>(string query, Func<DbDataReader, T> map)
{
using (var context = new DbContext())
{
using (var command = context.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
context.Database.OpenConnection();
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
while (result.Read())
{
entities.Add(map(result));
}
return entities;
}
}
}
}
Usage:
public class TopUser
{
public string Name { get; set; }
public int Count { get; set; }
}
var result = Helper.RawSqlQuery(
"SELECT TOP 10 Name, COUNT(*) FROM Users U"
+ " INNER JOIN Signups S ON U.UserId = S.UserId"
+ " GROUP BY U.Name ORDER BY COUNT(*) DESC",
x => new TopUser { Name = (string)x[0], Count = (int)x[1] });
result.ForEach(x => Console.WriteLine($"{x.Name,-25}{x.Count}"));
I plan to get rid of it as soon as built-in support is added. According to a statement by Arthur Vickers from the EF Core team it is a high priority for post 2.0. The issue is being tracked here.
In EF Core you no longer can execute "free" raw sql. You are required to define a POCO class and a DbSet for that class.
In your case you will need to define Rank:
var ranks = DbContext.Ranks
.FromSql("SQL_SCRIPT OR STORED_PROCEDURE #p0,#p1,...etc", parameters)
.AsNoTracking().ToList();
As it will be surely readonly it will be useful to include the .AsNoTracking() call.
EDIT - Breaking change in EF Core 3.0:
DbQuery() is now obsolete, instead DbSet() should be used (again). If you have a keyless entity, i.e. it don't require primary key, you can use HasNoKey() method:
ModelBuilder.Entity<SomeModel>().HasNoKey()
More information can be found here
For now, until there is something new from EFCore I would used a command
and map it manually
using (var command = this.DbContext.Database.GetDbConnection().CreateCommand())
{
command.CommandText = "SELECT ... WHERE ...> #p1)";
command.CommandType = CommandType.Text;
var parameter = new SqlParameter("#p1",...);
command.Parameters.Add(parameter);
this.DbContext.Database.OpenConnection();
using (var result = command.ExecuteReader())
{
while (result.Read())
{
.... // Map to your entity
}
}
}
Try to SqlParameter to avoid Sql Injection.
dbData.Product.FromSql("SQL SCRIPT");
FromSql doesn't work with full query. Example if you want to include a WHERE clause it will be ignored.
Some Links:
Executing Raw SQL Queries using Entity Framework Core
Raw SQL Queries
You can execute raw sql in EF Core - Add this class to your project.
This will allow you to execute raw SQL and get the raw results without having to define a POCO and a DBSet.
See https://github.com/aspnet/EntityFramework/issues/1862#issuecomment-220787464 for original example.
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Internal;
using Microsoft.EntityFrameworkCore.Storage;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.EntityFrameworkCore
{
public static class RDFacadeExtensions
{
public static RelationalDataReader ExecuteSqlQuery(this DatabaseFacade databaseFacade, string sql, params object[] parameters)
{
var concurrencyDetector = databaseFacade.GetService<IConcurrencyDetector>();
using (concurrencyDetector.EnterCriticalSection())
{
var rawSqlCommand = databaseFacade
.GetService<IRawSqlCommandBuilder>()
.Build(sql, parameters);
return rawSqlCommand
.RelationalCommand
.ExecuteReader(
databaseFacade.GetService<IRelationalConnection>(),
parameterValues: rawSqlCommand.ParameterValues);
}
}
public static async Task<RelationalDataReader> ExecuteSqlQueryAsync(this DatabaseFacade databaseFacade,
string sql,
CancellationToken cancellationToken = default(CancellationToken),
params object[] parameters)
{
var concurrencyDetector = databaseFacade.GetService<IConcurrencyDetector>();
using (concurrencyDetector.EnterCriticalSection())
{
var rawSqlCommand = databaseFacade
.GetService<IRawSqlCommandBuilder>()
.Build(sql, parameters);
return await rawSqlCommand
.RelationalCommand
.ExecuteReaderAsync(
databaseFacade.GetService<IRelationalConnection>(),
parameterValues: rawSqlCommand.ParameterValues,
cancellationToken: cancellationToken);
}
}
}
}
Here's an example of how to use it:
// Execute a query.
using(var dr = await db.Database.ExecuteSqlQueryAsync("SELECT ID, Credits, LoginDate FROM SamplePlayer WHERE " +
"Name IN ('Electro', 'Nitro')"))
{
// Output rows.
var reader = dr.DbDataReader;
while (reader.Read())
{
Console.Write("{0}\t{1}\t{2} \n", reader[0], reader[1], reader[2]);
}
}
You can use this:
public static class SqlQueryExtensions
{
public static IList<T> SqlQuery<T>(this DbContext db, string sql, params object[] parameters) where T : class
{
using (var db2 = new ContextForQueryType<T>(db.Database.GetDbConnection()))
{
// share the current database transaction, if one exists
var transaction = db.Database.CurrentTransaction;
if (transaction != null)
db2.Database.UseTransaction(transaction.GetDbTransaction());
return db2.Set<T>().FromSqlRaw(sql, parameters).ToList();
}
}
private class ContextForQueryType<T> : DbContext where T : class
{
private readonly DbConnection connection;
public ContextForQueryType(DbConnection connection)
{
this.connection = connection;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(connection, options => options.EnableRetryOnFailure());
base.OnConfiguring(optionsBuilder);
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<T>().HasNoKey();
base.OnModelCreating(modelBuilder);
}
}
}
And the usage:
using (var db = new Db())
{
var results = db.SqlQuery<ArbitraryType>("select 1 id, 'joe' name");
//or with an anonymous type like this
var results2 = db.SqlQuery(() => new { id =1, name=""},"select 1 id, 'joe' name");
}
try this: (create extension method)
public static List<T> ExecuteQuery<T>(this dbContext db, string query) where T : class, new()
{
using (var command = db.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
db.Database.OpenConnection();
using (var reader = command.ExecuteReader())
{
var lst = new List<T>();
var lstColumns = new T().GetType().GetProperties(BindingFlags.DeclaredOnly | BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic).ToList();
while (reader.Read())
{
var newObject = new T();
for (var i = 0; i < reader.FieldCount; i++)
{
var name = reader.GetName(i);
PropertyInfo prop = lstColumns.FirstOrDefault(a => a.Name.ToLower().Equals(name.ToLower()));
if (prop == null)
{
continue;
}
var val = reader.IsDBNull(i) ? null : reader[i];
prop.SetValue(newObject, val, null);
}
lst.Add(newObject);
}
return lst;
}
}
}
Usage:
var db = new dbContext();
string query = #"select ID , Name from People where ... ";
var lst = db.ExecuteQuery<PeopleView>(query);
my model: (not in DbSet):
public class PeopleView
{
public int ID { get; set; }
public string Name { get; set; }
}
tested in .netCore 2.2 and 3.0.
Note: this solution has the slow performance
Add Nuget package - Microsoft.EntityFrameworkCore.Relational
using Microsoft.EntityFrameworkCore;
...
await YourContext.Database.ExecuteSqlCommandAsync("... #p0, #p1", param1, param2 ..)
This will return the row numbers as an int
See - https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.relationaldatabasefacadeextensions.executesqlcommand?view=efcore-3.0
In Core 2.1 you can do something like this:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Query<Ranks>();
}
and then define you SQL Procedure, like:
public async Task<List<Ranks>> GetRanks(string value1, Nullable<decimal> value2)
{
SqlParameter value1Input = new SqlParameter("#Param1", value1?? (object)DBNull.Value);
SqlParameter value2Input = new SqlParameter("#Param2", value2?? (object)DBNull.Value);
List<Ranks> getRanks = await this.Query<Ranks>().FromSql("STORED_PROCEDURE #Param1, #Param2", value1Input, value2Input).ToListAsync();
return getRanks;
}
This way Ranks model will not be created in your DB.
Now in your controller/action you can call:
List<Ranks> gettingRanks = _DbContext.GetRanks(value1,value2).Result.ToListAsync();
This way you can call Raw SQL Procedures.
I used Dapper to bypass this constraint of Entity framework Core.
IDbConnection.Query
is working with either sql query or stored procedure with multiple parameters.
By the way it's a bit faster (see benchmark tests )
Dapper is easy to learn. It took 15 minutes to write and run stored procedure with parameters. Anyway you may use both EF and Dapper. Below is an example:
public class PodborsByParametersService
{
string _connectionString = null;
public PodborsByParametersService(string connStr)
{
this._connectionString = connStr;
}
public IList<TyreSearchResult> GetTyres(TyresPodborView pb,bool isPartner,string partnerId ,int pointId)
{
string sqltext "spGetTyresPartnerToClient";
var p = new DynamicParameters();
p.Add("#PartnerID", partnerId);
p.Add("#PartnerPointID", pointId);
using (IDbConnection db = new SqlConnection(_connectionString))
{
return db.Query<TyreSearchResult>(sqltext, p,null,true,null,CommandType.StoredProcedure).ToList();
}
}
}
I found the package EntityFrameworkCore.RawSQLExtensions on github. To use it, add the nuget package.
<PackageReference Include="EntityFrameworkCore.RawSQLExtensions" Version="1.2.0" />
The library is not documented but below is my using of it with .NET 6 + EF Core 6 + Npgsql 6
public class DbResult
{
public string Name { get; set; }
public int Age { get; set; }
}
using EntityFrameworkCore.RawSQLExtensions.Extensions;
var results = await context.Database
.SqlQuery<DbResult>(
#"select name, age from ""users"" where age > #Age",
new NpgsqlParameter("#Age", 15))
.ToListAsync();
Not directly targeting the OP's scenario, but since I have been struggling with this, I'd like to drop these ex. methods that make it easier to execute raw SQL with the DbContext:
public static class DbContextCommandExtensions
{
public static async Task<int> ExecuteNonQueryAsync(this DbContext context, string rawSql,
params object[] parameters)
{
var conn = context.Database.GetDbConnection();
using (var command = conn.CreateCommand())
{
command.CommandText = rawSql;
if (parameters != null)
foreach (var p in parameters)
command.Parameters.Add(p);
await conn.OpenAsync();
return await command.ExecuteNonQueryAsync();
}
}
public static async Task<T> ExecuteScalarAsync<T>(this DbContext context, string rawSql,
params object[] parameters)
{
var conn = context.Database.GetDbConnection();
using (var command = conn.CreateCommand())
{
command.CommandText = rawSql;
if (parameters != null)
foreach (var p in parameters)
command.Parameters.Add(p);
await conn.OpenAsync();
return (T)await command.ExecuteScalarAsync();
}
}
}
My case used stored procedure instead of raw SQL
Created a class
Public class School
{
[Key]
public Guid SchoolId { get; set; }
public string Name { get; set; }
public string Branch { get; set; }
public int NumberOfStudents { get; set; }
}
Added below on my DbContext class
public DbSet<School> SP_Schools { get; set; }
To execute the stored procedure:
var MySchools = _db.SP_Schools.FromSqlRaw("GetSchools #schoolId, #page, #size ",
new SqlParameter("schoolId", schoolId),
new SqlParameter("page", page),
new SqlParameter("size", size)))
.IgnoreQueryFilters();
I updated extension method from #AminRostami to return IAsyncEnumerable (so LINQ filtering can be applied) and it's mapping Model Column name of records returned from DB to models (Tested with EF Core 5):
Extension itself:
public static class QueryHelper
{
private static string GetColumnName(this MemberInfo info)
{
List<ColumnAttribute> list = info.GetCustomAttributes<ColumnAttribute>().ToList();
return list.Count > 0 ? list.Single().Name : info.Name;
}
/// <summary>
/// Executes raw query with parameters and maps returned values to column property names of Model provided.
/// Not all properties are required to be present in model (if not present - null)
/// </summary>
public static async IAsyncEnumerable<T> ExecuteQuery<T>(
[NotNull] this DbContext db,
[NotNull] string query,
[NotNull] params SqlParameter[] parameters)
where T : class, new()
{
await using DbCommand command = db.Database.GetDbConnection().CreateCommand();
command.CommandText = query;
command.CommandType = CommandType.Text;
if (parameters != null)
{
foreach (SqlParameter parameter in parameters)
{
command.Parameters.Add(parameter);
}
}
await db.Database.OpenConnectionAsync();
await using DbDataReader reader = await command.ExecuteReaderAsync();
List<PropertyInfo> lstColumns = new T().GetType()
.GetProperties(BindingFlags.DeclaredOnly | BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic).ToList();
while (await reader.ReadAsync())
{
T newObject = new();
for (int i = 0; i < reader.FieldCount; i++)
{
string name = reader.GetName(i);
PropertyInfo prop = lstColumns.FirstOrDefault(a => a.GetColumnName().Equals(name));
if (prop == null)
{
continue;
}
object val = await reader.IsDBNullAsync(i) ? null : reader[i];
prop.SetValue(newObject, val, null);
}
yield return newObject;
}
}
}
Model used (note that Column names are different than actual property names):
public class School
{
[Key] [Column("SCHOOL_ID")] public int SchoolId { get; set; }
[Column("CLOSE_DATE", TypeName = "datetime")]
public DateTime? CloseDate { get; set; }
[Column("SCHOOL_ACTIVE")] public bool? SchoolActive { get; set; }
}
Actual usage:
public async Task<School> ActivateSchool(int schoolId)
{
// note that we're intentionally not returning "SCHOOL_ACTIVE" with select statement
// this might be because of certain IF condition where we return some other data
return await _context.ExecuteQuery<School>(
"UPDATE SCHOOL SET SCHOOL_ACTIVE = 1 WHERE SCHOOL_ID = #SchoolId; SELECT SCHOOL_ID, CLOSE_DATE FROM SCHOOL",
new SqlParameter("#SchoolId", schoolId)
).SingleAsync();
}
Done this for Entity Framework Core 5, need to install
Microsoft.EntityFrameworkCore.Relational
The helper extension methods
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Storage;
using System;
using System.Collections.Generic;
using System.Data;
using System.Data.Common;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
public static class EfHelper
{
public static DbTransaction GetDbTransaction(this IDbContextTransaction source)
{
return (source as IInfrastructure<DbTransaction>).Instance;
}
private class PropertyMapp
{
public string Name { get; set; }
public Type Type { get; set; }
public bool IsSame(PropertyMapp mapp)
{
if (mapp == null)
{
return false;
}
bool same = mapp.Name == Name && mapp.Type == Type;
return same;
}
}
public static IEnumerable<T> FromSqlQuery<T>(this DbContext context, string query, params object[] parameters) where T : new()
{
const BindingFlags flags = BindingFlags.Public | BindingFlags.Instance | BindingFlags.NonPublic;
List<PropertyMapp> entityFields = (from PropertyInfo aProp in typeof(T).GetProperties(flags)
select new PropertyMapp
{
Name = aProp.Name,
Type = Nullable.GetUnderlyingType(aProp.PropertyType) ?? aProp.PropertyType
}).ToList();
List<PropertyMapp> dbDataReaderFields = new List<PropertyMapp>();
List<PropertyMapp> commonFields = null;
using (var command = context.Database.GetDbConnection().CreateCommand())
{
if (command.Connection.State != ConnectionState.Open)
{
command.Connection.Open();
}
var currentTransaction = context.Database.CurrentTransaction;
if (currentTransaction != null)
{
command.Transaction = currentTransaction.GetDbTransaction();
}
command.CommandText = query;
if (parameters.Any())
{
command.Parameters.AddRange(parameters);
}
using (var result = command.ExecuteReader())
{
while (result.Read())
{
if (commonFields == null)
{
for (int i = 0; i < result.FieldCount; i++)
{
dbDataReaderFields.Add(new PropertyMapp { Name = result.GetName(i), Type = result.GetFieldType(i) });
}
commonFields = entityFields.Where(x => dbDataReaderFields.Any(d => d.IsSame(x))).Select(x => x).ToList();
}
var entity = new T();
foreach (var aField in commonFields)
{
PropertyInfo propertyInfos = entity.GetType().GetProperty(aField.Name);
var value = (result[aField.Name] == DBNull.Value) ? null : result[aField.Name]; //if field is nullable
propertyInfos.SetValue(entity, value, null);
}
yield return entity;
}
}
}
}
/*
* https://entityframeworkcore.com/knowledge-base/35631903/raw-sql-query-without-dbset---entity-framework-core
*/
public static IEnumerable<T> FromSqlQuery<T>(this DbContext context, string query, Func<DbDataReader, T> map, params object[] parameters)
{
using (var command = context.Database.GetDbConnection().CreateCommand())
{
if (command.Connection.State != ConnectionState.Open)
{
command.Connection.Open();
}
var currentTransaction = context.Database.CurrentTransaction;
if (currentTransaction != null)
{
command.Transaction = currentTransaction.GetDbTransaction();
}
command.CommandText = query;
if (parameters.Any())
{
command.Parameters.AddRange(parameters);
}
using (var result = command.ExecuteReader())
{
while (result.Read())
{
yield return map(result);
}
}
}
}
}
Model
public class UserModel
{
public string Name { get; set; }
public string Email { get; set; }
public bool? IsDeleted { get; set; }
}
Manual mapping
List<UserModel> usersInDb = Db.FromSqlQuery
(
"SELECT Name, Email FROM Users WHERE Name=#paramName",
x => new UserModel
{
Name = (string)x[0],
Email = (string)x[1]
},
new SqlParameter("#paramName", user.Name)
)
.ToList();
usersInDb = Db.FromSqlQuery
(
"SELECT Name, Email FROM Users WHERE Name=#paramName",
x => new UserModel
{
Name = x["Name"] is DBNull ? "" : (string)x["Name"],
Email = x["Email"] is DBNull ? "" : (string)x["Email"]
},
new SqlParameter("#paramName", user.Name)
)
.ToList();
Auto mapping using reflection
List<UserModel> usersInDb = Db.FromSqlQuery<UserModel>
(
"SELECT Name, Email, IsDeleted FROM Users WHERE Name=#paramName",
new SqlParameter("#paramName", user.Name)
)
.ToList();
This solution leans heavily on the solution from #pius. I wanted to add the option to support query parameters to help mitigate SQL injection and I also wanted to make it an extension off of the DbContext DatabaseFacade for Entity Framework Core to make it a little more integrated.
First create a new class with the extension:
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using System;
using System.Collections.Generic;
using System.Data;
using System.Data.Common;
using System.Linq;
using System.Threading.Tasks;
namespace EF.Extend
{
public static class ExecuteSqlExt
{
/// <summary>
/// Execute raw SQL query with query parameters
/// </summary>
/// <typeparam name="T">the return type</typeparam>
/// <param name="db">the database context database, usually _context.Database</param>
/// <param name="query">the query string</param>
/// <param name="map">the map to map the result to the object of type T</param>
/// <param name="queryParameters">the collection of query parameters, if any</param>
/// <returns></returns>
public static List<T> ExecuteSqlRawExt<T, P>(this DatabaseFacade db, string query, Func<DbDataReader, T> map, IEnumerable<P> queryParameters = null)
{
using (var command = db.GetDbConnection().CreateCommand())
{
if((queryParameters?.Any() ?? false))
command.Parameters.AddRange(queryParameters.ToArray());
command.CommandText = query;
command.CommandType = CommandType.Text;
db.OpenConnection();
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
while (result.Read())
{
entities.Add(map(result));
}
return entities;
}
}
}
}
}
Note in the above that "T" is the type for the return and "P" is the type of your query parameters which will vary based on if you are using MySql, Sql, so on.
Next we will show an example. I'm using the MySql EF Core capability, so we'll see how we can use the generic extension above with this more specific MySql implementation:
//add your using statement for the extension at the top of your Controller
//with all your other using statements
using EF.Extend;
//then your your Controller looks something like this
namespace Car.Api.Controllers
{
//Define a quick Car class for the custom return type
//you would want to put this in it's own class file probably
public class Car
{
public string Make { get; set; }
public string Model { get; set; }
public string DisplayTitle { get; set; }
}
[ApiController]
public class CarController : ControllerBase
{
private readonly ILogger<CarController> _logger;
//this would be your Entity Framework Core context
private readonly CarContext _context;
public CarController(ILogger<CarController> logger, CarContext context)
{
_logger = logger;
_context = context;
}
//... more stuff here ...
/// <summary>
/// Get car example
/// </summary>
[HttpGet]
public IEnumerable<Car> Get()
{
//instantiate three query parameters to pass with the query
//note the MySqlParameter type is because I'm using MySql
MySqlParameter p1 = new MySqlParameter
{
ParameterName = "id1",
Value = "25"
};
MySqlParameter p2 = new MySqlParameter
{
ParameterName = "id2",
Value = "26"
};
MySqlParameter p3 = new MySqlParameter
{
ParameterName = "id3",
Value = "27"
};
//add the 3 query parameters to an IEnumerable compatible list object
List<MySqlParameter> queryParameters = new List<MySqlParameter>() { p1, p2, p3 };
//note the extension is now easily accessed off the _context.Database object
//also note for ExecuteSqlRawExt<Car, MySqlParameter>
//Car is my return type "T"
//MySqlParameter is the specific DbParameter type MySqlParameter type "P"
List<Car> result = _context.Database.ExecuteSqlRawExt<Car, MySqlParameter>(
"SELECT Car.Make, Car.Model, CONCAT_WS('', Car.Make, ' ', Car.Model) As DisplayTitle FROM Car WHERE Car.Id IN(#id1, #id2, #id3)",
x => new Car { Make = (string)x[0], Model = (string)x[1], DisplayTitle = (string)x[2] },
queryParameters);
return result;
}
}
}
The query would return rows like:
"Ford", "Explorer", "Ford Explorer"
"Tesla", "Model X", "Tesla Model X"
The display title is not defined as a database column, so it wouldn't be part of the EF Car model by default. I like this approach as one of many possible solutions. The other answers on this page reference other ways to address this issue with the [NotMapped] decorator, which depending on your use case could be the more appropriate approach.
Note the code in this example is obviously more verbose than it needs to be, but I thought it made the example clearer.
Actually you can create a generic repository and do something like this
public class GenericRepository<TEntity> : IGenericRepository<TEntity> where TEntity : BaseEntity
{
private readonly DataContext context;
private readonly DbSet<TEntity> dbSet;
public GenericRepository(DataContext context)
{
this.context = context;
this.dbSet = context.Set<TEntity>();
}
public IEnumerable<TEntity> ExecuteCommandQuery(string command)
=> dbSet.FromSqlRaw(command);
}
For Querying Data: Without existing Entity
string query = "SELECT r.Name as roleName, ur.roleId, u.Id as userId FROM dbo.AspNetUserRoles AS ur INNER JOIN dbo.AspNetUsers AS u ON ur.UserId = u.Id INNER JOIN dbo.AspNetRoles AS r ON ur.RoleId = r.Id ";
ICollection<object> usersWithRoles = new List<object>();
using (var command = _identityDBContext.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
await _identityDBContext.Database.OpenConnectionAsync();
using (var reader = await command.ExecuteReaderAsync())
{
while (await reader.ReadAsync())
{
usersWithRoles.Add(new {
roleName = reader.GetFieldValueAsync<string>(0).Result,
roleId = reader.GetFieldValueAsync<string>(1).Result,
userId = reader.GetFieldValueAsync<string>(2).Result
});
}
}
}
Detailed:
[HttpGet]
[Route("GetAllUsersWithRoles")]
public async Task<IActionResult> GetAllUsersWithRoles()
{
string query = "SELECT r.Name as roleName, ur.roleId, u.Id as userId FROM dbo.AspNetUserRoles AS ur INNER JOIN dbo.AspNetUsers AS u ON ur.UserId = u.Id INNER JOIN dbo.AspNetRoles AS r ON ur.RoleId = r.Id ";
try
{
ICollection<object> usersWithRoles = new List<object>();
using (var command = _identityDBContext.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
await _identityDBContext.Database.OpenConnectionAsync();
using (var reader = await command.ExecuteReaderAsync())
{
while (await reader.ReadAsync())
{
usersWithRoles.Add(new {
roleName = reader.GetFieldValueAsync<string>(0).Result,
roleId = reader.GetFieldValueAsync<string>(1).Result,
userId = reader.GetFieldValueAsync<string>(2).Result
});
}
}
}
return StatusCode(200, usersWithRoles); // Get all users
}
catch (Exception e)
{
return StatusCode(500, e);
}
}
RESULT looks like this:
[
{
"roleName": "admin",
"roleId": "7c9cb1be-e987-4ec1-ae4d-e4c9790f57d8",
"userId": "12eadc86-6311-4d5e-8be8-df30799df265"
},
{
"roleName": "user",
"roleId": "a0d5ef46-b1e6-4a53-91ce-9ff5959f1ed8",
"userId": "12eadc86-6311-4d5e-8be8-df30799df265"
},
{
"roleName": "user",
"roleId": "a0d5ef46-b1e6-4a53-91ce-9ff5959f1ed8",
"userId": "3e7cd970-8c52-4dd1-847c-f824671ea15d"
}
]
You can also use QueryFirst. Like Dapper, this is totally outside EF. Unlike Dapper (or EF), you don't need to maintain the POCO, you edit your sql SQL in a real environment, and it's continually revalidated against the DB. Disclaimer: I'm the author of QueryFirst.
I've came to this question because we have over 100 instances of entity-less usages of SqlQuery in Entity Framework 6 and so going the Microsoft suggested way(s) simply cannot not easily work in our case.
In addition, we had to maintain a single EF (Entity Framework 6) / EFC (Entity Framework Core 5) code base for several months, while migrating from EF to EFC. The code base is fairly large and it was simply impossible to migrate "overnight".
The answer below is based on great answers above and it is just a small extension to make them work for a few more edge cases.
First, for each EF based project we created an EFC based project (e.g. MyProject.csproj ==> MyProject_EFC.csproj) and inside all such EFC projects we defined a constant EFCORE. If you are doing a quick one-time migration from EF to EFC, then you don't need that and you can just keep what's inside #if EFCORE ... #else and remove what's inside #else ... #endif below.
Here is the main interop extension class.
using System;
using System.Collections.Generic;
using System.Threading;
#if EFCORE
using System.ComponentModel.DataAnnotations.Schema;
using System.Data;
using System.Data.Common;
using System.Linq;
using System.Reflection;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.ChangeTracking;
using Microsoft.EntityFrameworkCore.Storage;
using Database = Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade;
using MoreLinq.Extensions;
#else
using System.Data.Entity;
using System.Data.Entity.Infrastructure;
#endif
namespace YourNameSpace.EntityFrameworkCore
{
/// <summary>
/// Collection of extension methods to simplify migration from EF to EFC.
/// </summary>
public static class EntityFrameworkCoreInterop
{
/// <summary>
/// https://stackoverflow.com/questions/6637679/reflection-get-attribute-name-and-value-on-property
/// </summary>
public static TAttribute? TryGetAttribute<TAttribute>(this PropertyInfo prop) where TAttribute : Attribute =>
prop.GetCustomAttributes(true).TryGetAttribute<TAttribute>();
public static TAttribute? TryGetAttribute<TAttribute>(this Type t) where TAttribute : Attribute =>
t.GetCustomAttributes(true).TryGetAttribute<TAttribute>();
public static TAttribute? TryGetAttribute<TAttribute>(this IEnumerable<object> attrs) where TAttribute : Attribute
{
foreach (object attr in attrs)
{
switch (attr)
{
case TAttribute t:
{
return t;
}
}
}
return null;
}
/// <summary>
/// Returns true if the source string matches *any* of the passed-in strings (case insensitive)
/// </summary>
public static bool EqualsNoCase(this string? s, params string?[]? targets)
{
if (s == null && (targets == null || targets.Length == 0))
{
return true;
}
if (targets == null)
{
return false;
}
return targets.Any(t => string.Equals(s, t, StringComparison.OrdinalIgnoreCase));
}
#if EFCORE
public class EntityException : Exception
{
public EntityException(string message) : base(message)
{
}
}
public static TEntity GetEntity<TEntity>(this EntityEntry<TEntity> entityEntry)
where TEntity : class => entityEntry.Entity;
#region SqlQuery Interop
/// <summary>
/// kk:20210727 - This is a little bit ugly but given that this interop method is used just once,
/// it is not worth spending more time on it.
/// </summary>
public static List<T> ToList<T>(this IOrderedAsyncEnumerable<T> e) =>
Task.Run(() => e.ToListAsync().AsTask()).GetAwaiter().GetResult();
private static string GetColumnName(this MemberInfo info) =>
info.GetCustomAttributes().TryGetAttribute<ColumnAttribute>()?.Name ?? info.Name;
/// <summary>
/// See: https://stackoverflow.com/questions/35631903/raw-sql-query-without-dbset-entity-framework-core
/// Executes raw query with parameters and maps returned values to column property names of Model provided.
/// Not all properties are required to be present in the model. If not present then they will be set to nulls.
/// </summary>
private static async IAsyncEnumerable<T> ExecuteQuery<T>(this Database database, string query, params object[] parameters)
{
await using DbCommand command = database.GetDbConnection().CreateCommand();
command.CommandText = query;
command.CommandType = CommandType.Text;
if (database.CurrentTransaction != null)
{
command.Transaction = database.CurrentTransaction.GetDbTransaction();
}
foreach (var parameter in parameters)
{
// They are supposed to be of SqlParameter type but are passed as objects.
command.Parameters.Add(parameter);
}
await database.OpenConnectionAsync();
await using DbDataReader reader = await command.ExecuteReaderAsync();
var t = typeof(T);
// TODO kk:20210825 - I do know that the code below works as we use it in some other place where it does work.
// However, I am not 100% sure that R# proposed version does. Check and refactor when time permits.
//
// ReSharper disable once CheckForReferenceEqualityInstead.1
if (t.IsGenericType && t.GetGenericTypeDefinition().Equals(typeof(Nullable<>)))
{
t = Nullable.GetUnderlyingType(t)!;
}
var lstColumns = t
.GetProperties(BindingFlags.DeclaredOnly | BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic)
.ToList();
while (await reader.ReadAsync())
{
if (t.IsPrimitive || t == typeof(string) || t == typeof(DateTime) || t == typeof(Guid) || t == typeof(decimal))
{
var val = await reader.IsDBNullAsync(0) ? null : reader[0];
yield return (T) val!;
}
else
{
var newObject = Activator.CreateInstance<T>();
for (var i = 0; i < reader.FieldCount; i++)
{
var name = reader.GetName(i);
var val = await reader.IsDBNullAsync(i) ? null : reader[i];
var prop = lstColumns.FirstOrDefault(a => a.GetColumnName().EqualsNoCase(name));
if (prop == null)
{
continue;
}
prop.SetValue(newObject, val, null);
}
yield return newObject;
}
}
}
#endregion
public static DbRawSqlQuery<TElement> SqlQuery<TElement>(this Database database, string sql, params object[] parameters) =>
new(database, sql, parameters);
public class DbRawSqlQuery<TElement> : IAsyncEnumerable<TElement>
{
private readonly IAsyncEnumerable<TElement> _elements;
internal DbRawSqlQuery(Database database, string sql, params object[] parameters) =>
_elements = ExecuteQuery<TElement>(database, sql, parameters);
public IAsyncEnumerator<TElement> GetAsyncEnumerator(CancellationToken cancellationToken = new ()) =>
_elements.GetAsyncEnumerator(cancellationToken);
public async Task<TElement> SingleAsync() => await _elements.SingleAsync();
public TElement Single() => Task.Run(SingleAsync).GetAwaiter().GetResult();
public async Task<TElement> FirstAsync() => await _elements.FirstAsync();
public TElement First() => Task.Run(FirstAsync).GetAwaiter().GetResult();
public async Task<TElement?> SingleOrDefaultAsync() => await _elements.SingleOrDefaultAsync();
public async Task<int> CountAsync() => await _elements.CountAsync();
public async Task<List<TElement>> ToListAsync() => await _elements.ToListAsync();
public List<TElement> ToList() => Task.Run(ToListAsync).GetAwaiter().GetResult();
}
#endif
}
}
and the usages are indistinguishable from the former EF usages:
public async Task<List<int>> GetMyResults()
{
using var ctx = GetMyDbContext();
const string sql = "select 1 as Result";
return await ctx.GetDatabase().SqlQuery<int>(sql).ToListAsync();
}
where GetMyDbContext is a method to get your database context and GetDatabase is an one-liner interop that returns ((DbContext)context).Database for a given IMyDbContext : DbContext. This is to simplify simultaneous EF / EFC operations.
This works for primitive types (the example is above), entities, local classes (but not anonymous ones). Column renaming is supported via GetColumnName, but, ... it was already done above.
With Entity Framework 6 you can execute something like below
Create Modal Class as
Public class User
{
public int Id { get; set; }
public string fname { get; set; }
public string lname { get; set; }
public string username { get; set; }
}
Execute Raw DQL SQl command as below:
var userList = datacontext.Database.SqlQuery<User>(#"SELECT u.Id ,fname , lname ,username FROM dbo.Users").ToList<User>();
I've managed to get something up and running today as small sandbox/POC project, but have seemed to bump my head on one issue...
Question:
Is there a way to get dapper to map to SQL column names with spaces in them.
I have something to this effect as my result set.
For example:
SELECT 001 AS [Col 1],
901 AS [Col 2],
00454345345345435349 AS [Col 3],
03453453453454353458 AS [Col 4]
FROM [Some Schema].[Some Table]
And my class would look like this
public class ClassA
{
public string Col1 { get; set; }
public string Col2 { get; set; }
///... etc
}
My implementation looks like this at the moment
public Tuple<IList<TClass>, IList<TClass2>> QueryMultiple<TClass, TClass2>(object parameters)
{
List<TClass> output1;
List<TClass2> output2;
using (var data = this.Connection.QueryMultiple(this.GlobalParameter.RpcProcedureName, parameters, CommandType.StoredProcedure))
{
output1 = data.Read<TClass>().ToList();
output2 = data.Read<TClass2>().ToList();
}
var result = new Tuple<IList<TClass>, IList<TClass2>>(output1, output2);
return result;
}
Note: The SQL cant be modified in any way.
Currently I'm going through the dapper code, and my only foreseeable solution is to add some code to "persuade" the column comparison, but not having much luck so far.
I've seen on StackOverflow that there are things like dapper extensions, but I'm hoping I can get this done without adding an extention, if not. I'll take whatever is quickest to implement.
There's a nuget package Dapper.FluentMap that allows you to add column name mappings (including spaces). It's similar to EntityFramework.
// Entity class.
public class Customer
{
public string Name { get; set; }
}
// Mapper class.
public class CustomerMapper : EntityMap<Customer>
{
public CustomerMapper()
{
Map(p => p.Name).ToColumn("Customer Name");
}
}
// Initialise like so -
FluentMapper.Initialize(a => a.AddMap(new CustomerMapper()));
see https://github.com/henkmollema/Dapper-FluentMap for more.
One option here would be to go via the dynamic / non-generic API, and then fetch the values out via the IDictionary<string,object> API per row, but that might be a bit tedious.
As an alternative, you can create a custom mapper, and tell dapper about it; for example:
SqlMapper.SetTypeMap(typeof(ClassA), new RemoveSpacesMap());
with:
class RemoveSpacesMap : Dapper.SqlMapper.ITypeMap
{
System.Reflection.ConstructorInfo SqlMapper.ITypeMap.FindConstructor(string[] names, Type[] types)
{
return null;
}
SqlMapper.IMemberMap SqlMapper.ITypeMap.GetConstructorParameter(System.Reflection.ConstructorInfo constructor, string columnName)
{
return null;
}
SqlMapper.IMemberMap SqlMapper.ITypeMap.GetMember(string columnName)
{
var prop = typeof(ClassA).GetProperty(columnName.Replace(" ", ""));
return prop == null ? null : new PropertyMemberMap(columnName, prop);
}
class PropertyMemberMap : Dapper.SqlMapper.IMemberMap
{
private string columnName;
private PropertyInfo property;
public PropertyMemberMap(string columnName, PropertyInfo property)
{
this.columnName = columnName;
this.property = property;
}
string SqlMapper.IMemberMap.ColumnName
{
get { throw new NotImplementedException(); }
}
System.Reflection.FieldInfo SqlMapper.IMemberMap.Field
{
get { return null; }
}
Type SqlMapper.IMemberMap.MemberType
{
get { return property.PropertyType; }
}
System.Reflection.ParameterInfo SqlMapper.IMemberMap.Parameter
{
get { return null; }
}
System.Reflection.PropertyInfo SqlMapper.IMemberMap.Property
{
get { return property; }
}
}
}
I had a similar problem when trying to get mapped results from a call to the system sp_spaceused procedure. Marc's code didn't quite work for me as it complained about not being able to find a default constructor. I also made my version generic so it could theoretically be re-used. This may not be the fastest performing piece of code, but it works for me and in our situation these calls are made infrequently.
class TitleCaseMap<T> : SqlMapper.ITypeMap where T: new()
{
ConstructorInfo SqlMapper.ITypeMap.FindConstructor(string[] names, Type[] types)
{
return typeof(T).GetConstructor(Type.EmptyTypes);
}
SqlMapper.IMemberMap SqlMapper.ITypeMap.GetConstructorParameter(ConstructorInfo constructor, string columnName)
{
return null;
}
SqlMapper.IMemberMap SqlMapper.ITypeMap.GetMember(string columnName)
{
string reformattedColumnName = string.Empty;
foreach (string word in columnName.Replace("_", " ").Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries))
{
reformattedColumnName += char.ToUpper(word[0]) + word.Substring(1).ToLower();
}
var prop = typeof(T).GetProperty(reformattedColumnName);
return prop == null ? null : new PropertyMemberMap(prop);
}
class PropertyMemberMap : SqlMapper.IMemberMap
{
private readonly PropertyInfo _property;
public PropertyMemberMap(PropertyInfo property)
{
_property = property;
}
string SqlMapper.IMemberMap.ColumnName
{
get { throw new NotImplementedException(); }
}
FieldInfo SqlMapper.IMemberMap.Field
{
get { return null; }
}
Type SqlMapper.IMemberMap.MemberType
{
get { return _property.PropertyType; }
}
ParameterInfo SqlMapper.IMemberMap.Parameter
{
get { return null; }
}
PropertyInfo SqlMapper.IMemberMap.Property
{
get { return _property; }
}
}
}
I know this is an old question nevertheless i faced the same problem in my last project, so i just created an own mapper using attributes.
I defined an attribute class called ColumnNameAttribute.cs
using System;
namespace DapperHelper.Attributes
{
[System.AttributeUsage(AttributeTargets.All, Inherited = true, AllowMultiple = true)]
sealed class ColumNameAttribute : Attribute
{
private string _columName;
public string ColumnName
{
get { return _columName; }
set { _columName = value; }
}
public ColumNameAttribute(string columnName)
{
_columName = columnName;
}
}
}
After defining the attribute, i implemeted a dynamic mapper that uses the Query method from Dapper but works as the Query<T>:
using Dapper;
using DapperHelper.Attributes;
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Reflection;
using System.Web.Routing;
namespace DapperHelper.Tools
{
public class DynamicMapper<T> :IDisposable where T : class, new()
{
private readonly Dictionary<string, PropertyInfo> _propertiesMap;
public DynamicMapper()
{
_propertiesMap = new Dictionary<string, PropertyInfo>();
PropertyInfo[] propertyInfos = typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance);
foreach (PropertyInfo propertyInfo in propertyInfos)
{
if (propertyInfo.GetCustomAttribute(typeof(ColumNameAttribute)) is ColumNameAttribute columNameAttribute)
{
_propertiesMap.Add(columNameAttribute.ColumnName, propertyInfo);
}
else
{
_propertiesMap.Add(propertyInfo.Name, propertyInfo);
}
}
}
public List<T> QueryDynamic(IDbConnection dbConnection, string sqlQuery)
{
List<dynamic> results = dbConnection.Query(sqlQuery).ToList();
List<T> output = new List<T>();
foreach (dynamic dynObj in results)
{
output.Add(AssignPropertyValues(dynObj));
}
return output;
}
private T AssignPropertyValues(dynamic dynamicObject)
{
T output = new T();
RouteValueDictionary dynamicObjProps = new RouteValueDictionary(dynamicObject);
foreach (var propName in dynamicObjProps.Keys)
{
if (_propertiesMap.TryGetValue(propName, out PropertyInfo propertyMapped)
&& dynamicObjProps.TryGetValue(propName, out object value))
{
propertyMapped.SetValue(output, value);
}
}
return output;
}
public void Dispose()
{
_propertiesMap.Clear();
}
}
}
To use it, you have to refer to your Model class and define the attribute:
using DapperHelper.Attributes;
namespace Testing
{
public class Sample
{
public int SomeColumnData { get; set; }
[ColumnName("Your Column Name")]
public string SpecialColumn{ get; set; }
}
}
and then you can implement something like this:
DynamicMapper<Sample> mapper = new DynamicMapper<Sample>();
List<Sample> samples = mapper.QueryDynamic(connection, "SELECT * FROM Samples");
I hope it can help someone looking for an alternative.