Develop Reference

FromSqlInterpolated and in clause

sonar is given me a hard time because the following code:
var listaChaves = chaves.ToList();
var parametros = new string[chaves.Count];
var parametrosSql = new List<NpgsqlParameter>();
for (int i = 0; i < listaChaves.Count; i++)
{
parametros[i] = string.Format("#param_{0}", i);
parametrosSql.Add(new NpgsqlParameter(parametros[i], listaChaves[i]));
}
var comandoSql = string
.Format("SELECT distinct on(chave_identificacao) chave_identificacao, data from ultimos_acessos where chave_identificacao in({0}) order by chave_identificacao, data desc",
string.Join(", ", parametros));
var ultimosAcessos = await Entidade.FromSqlRaw(comandoSql, parametrosSql.ToArray())
.Select(a => new ProjecaoListagemUltimoAcesso(a.Data, a.ChaveIdentificacao))
.ToListAsync();
it thinks a sql injection can happen because the string interpolation. So i tried to change to "FromSqlInterpolated" method, as follow:
var listaChaves = chaves.ToList();
var ultimosAcessos = await Entidade.FromSqlInterpolated(#$"SELECT distinct on(chave_identificacao) chave_identificacao, data from ultimos_acessos where chave_identificacao in({string.Join(", ", listaChaves)})) ) order by chave_identificacao, data desc")
.Select(a => new ProjecaoListagemUltimoAcesso(a.Data, a.ChaveIdentificacao))
.ToListAsync();
But it just dont work, anyone can help me on how can I create a secure sql from a interpolated string using a "in" clause?

Something like this should work. Build an OR clause in the same loop as your parameters and stuff it into your SQL:
string ORclause = "";
for (int i = 0; i < listaChaves.Count; i++)
{
ORclause += $" chave_identificacao = #param_{i} OR ";
parametros[i] = string.Format("#param_{0}", i);
parametrosSql.Add(new NpgsqlParameter(parametros[i], listaChaves[i]));
}
//remove last "OR"
ORclause = ORclause.Substring(0, s.Length - 4);
#$"SELECT distinct on(chave_identificacao) chave_identificacao, data
from ultimos_acessos
where {ORclause}
order by chave_identificacao, data desc";
If you have a lot of listaChaves then consider using StringBuilder instead of +=

C# entity framework fromsqlraw query with includes (in) instead of where clause

What is the correct syntax for varname to make this query work?
I can get it to work with a single variable like
string varname = "TOTAL_NORWAY"
However, if I want to have a few variables in there, I get an empty array returned:
string varname = "'TOTAL_NORWAY', 'TOTAL_SWEDEN'";
return await _Context.theDataModel.FromSqlRaw(#"
select data
from data_table
where Variable in ({0})
", varname).ToListAsync();

Remember that you can combine FromSqlRaw with Linq:
string varnames = new [] { "TOTAL_NORWAY", "TOTAL_SWEDEN" };
var query = _Context.theDataModel.FromSqlRaw(#"
select data
from data_table");
query = query.Where(x => varnames.Contains(x.Variable));
// Add more where clauses as needed
return await query.ToListAsync();

ErikEJ's post was very helpful. The solution is not so trivial for someone who doesn't dabble in EF Core regularly.
I also had an extra where clause to consider, and this was done like so for anyone else wondering.
var items = new int[] { 1, 2, 3 };
var parameters = new string[items.Length];
var sqlParameters = new List<SqlParameter>();
for (var i = 0; i < items.Length; i++)
{
parameters[i] = string.Format("#p{0}", i);
sqlParameters.Add(new SqlParameter(parameters[i], items[i]));
}
sqlParameters.Add(new SqlParameter("#userid", "userXYZ123"));
var rawCommand = string.Format("SELECT * from dbo.Shippers WHERE ShipperId IN ({0}) and userid = {1}", string.Join(", ", parameters), "#userid");
var shipperList = db.Set<ShipperSummary>()
.FromSqlRaw(rawCommand, sqlParameters.ToArray())
.ToList();

LINQ Dynamic Query with group by and string variable

I want to create a dynamic query with LINQ-to-SQL and everything works except for one thing: group by. I looked at the other questions but haven't found a reason so far why it does not work.
I use using System.Linq.Dynamic; with the following query (that works):
int numberOfRankedItems = 3;
string minMaxChoice = "max";
string orderBy = "";
if (minMaxChoice == "max")
{
orderBy = "queryGroup.Sum(row => row.POSITIONVALUE) descending";
} else if (minMaxChoice == "min")
{
orderBy = "queryGroup.Sum(row => row.POSITIONVALUE) ascending";
}
string minMaxFeatureColumnName = "BRANDNAME";
string selectMinMaxFeature = "new { minMaxFeature = queryGroup.Key." + minMaxFeatureColumnName + ", sumPosition = queryGroup.Sum(row => row.POSITIONVALUE)}";
var query = (from tableRow in Context.xxx
where /* some filters */
group tableRow by tableRow.BRANDNAME into queryGroup
orderby orderBy
select selectMinMaxFeature).Take(numberOfRankedItems).ToList();
The use of variables for orderby and select works fine, but for group by not no matter what I try to replace with a variable. The query actually works if I e.g. use a variable instead of tableRow.BRANDNAME but the query returns the wrong result. The goal is to be able to set the column for the grouping dynamically.
Ideas?
Thanks
Edit: there seem to be multiple issues also with the other variables. So I generalize the question a bit: how can I generalize the following query in terms of
The column to group by
The column to order by + ASC or DESC
In the select statement the columnname of the first statement (BRANDNAME)
Here is the query:
var query = (from tableRow in ctx.xxx
where /* filter */
group tableRow by new { tableRow.BRANDNAME } into queryGroup
orderby queryGroup.Sum(row => row.POSITIONVALUE) descending
select new { minMaxFeature = queryGroup.Key.BRANDNAME, sumPosition = queryGroup.Sum(row => row.POSITIONVALUE) }).Take(numberOfRankedItems).ToList();
Would be great without expression trees ;)

I have now elaborated a solution to the question:
The solution uses as before using System.Linq.Dynamic; allows now to set certain variables. In the end it returns a dictionary.
// set variables
int numberOfRankedItems;
if (queryData.NumberOfRankedItems != null)
{
numberOfRankedItems = (int) queryData.NumberOfRankedItems;
} else
{
numberOfRankedItems = 0;
}
string minMaxChoice = queryData.MinMaxChoice;
string minMaxFeatureColumnName = queryData.MinMaxFeatureColumnName;
string orderByPrompt = "";
if (minMaxChoice == "max")
{
orderByPrompt = "it.Sum(POSITIONVALUE) descending";
} else if (minMaxChoice == "min")
{
orderByPrompt = "it.Sum(POSITIONVALUE) ascending";
}
string groupByPrompt = queryData.MinMaxFeatureColumnName;
// execute query
IQueryable query = (from tableRow in Context.xxx
where /* some filters */
select tableRow).
GroupBy(groupByPrompt, "it").
OrderBy(orderByPrompt).
Select("new (it.Key as minMaxFeature, it.Sum(POSITIONVALUE) as sumPosition)").
Take(numberOfRankedItems);
// create response dictionary
Dictionary<int?, Dictionary<string, int?>> response = new Dictionary<int?, Dictionary<string, int?>>();
int count = 1;
foreach (dynamic item in query)
{
string minMaxFeatureReturn = item.GetType().GetProperty("minMaxFeature").GetValue(item);
int? sumPositionReturn = item.GetType().GetProperty("sumPosition").GetValue(item);
response[count] = new Dictionary<string, int?>() { { minMaxFeatureReturn, sumPositionReturn } };
count++;
}
return response;

Working with multiple resultset in .net core

While retrieving the results using stored procedure how can I retrieve and store multiple result set in view model in .net core
For e.g. from stored procedure I am returning records for below two queries
Select * LMS_Survey
Select * from LMS_SurveyQuestion
Select * from LMS_SurveyQuestionOptionChoice
and below is view model for two tables
public class LMS_SurveyTraineeViewModel
{
public LMS_SurveyDetailsViewModel SurveyDetailsViewModel { get; set; }
public LMS_SurveyQuestionsViewModel SurveyQuestionsViewModel { get; set; }
public LMS_SurveyQuestionOptionChoiceViewModel SurveyQuestionOptionChoiceViewModel { get; set; }
}
This is how I am executing the stored procedure
public List<LMS_SurveyTraineeViewModel> GetTraineeSurvey(int surveyID)
{
try
{
List<LMS_SurveyTraineeViewModel> modelList = new List<LMS_SurveyTraineeViewModel>();
modelList = dbcontext.Set<LMS_SurveyTraineeViewModel>().FromSql("LMSSP_GetTraineeSurvey #surveyID = {0},#LanguageID = {1}", surveyID, AppTenant.SelectedLanguageID).ToList();
return modelList;
}
catch (Exception ex)
{
throw ex;
}
}
How can stored the multiple result set using stored procedure in view model ?

Currently, EF Core doesn't not support this. There is an open issue to address this.
https://github.com/aspnet/EntityFramework/issues/8127
Update 12th Sep 2018: This is still not a priority for EF Core even for release 3.0; so best use Dapper or plain ADO.NET when you have multiple results scenario
Update 25th Jun 2020: still on the backlog for EF Core even for release 5.0; so best use Dapper or plain ADO.NET when you have multiple results scenario
Update 7th Feb 2021: still on the backlog for EF Core
Update 8th Aug 2022: still on the backlog for EF Core, looks like its not a high priority use-case. Recommend to follow alternatives like using straight ADO.NET or Dapr or the below workaround for this
In the interim an alternative solution can be achieved via extension method(s)
public static async Task<IList<IList>> MultiResultSetsFromSql(this DbContext dbContext, ICollection<Type> resultSetMappingTypes, string sql, params object[] parameters)
{
var resultSets = new List<IList>();
var connection = dbContext.Database.GetDbConnection();
var parameterGenerator = dbContext.GetService<IParameterNameGeneratorFactory>()
.Create();
var commandBuilder = dbContext.GetService<IRelationalCommandBuilderFactory>()
.Create();
foreach (var parameter in parameters)
{
var generatedName = parameterGenerator.GenerateNext();
if (parameter is DbParameter dbParameter)
commandBuilder.AddRawParameter(generatedName, dbParameter);
else
commandBuilder.AddParameter(generatedName, generatedName);
}
using var command = connection.CreateCommand();
command.CommandType = CommandType.Text;
command.CommandText = sql;
command.Connection = connection;
for (var i = 0; i < commandBuilder.Parameters.Count; i++)
{
var relationalParameter = commandBuilder.Parameters[i];
relationalParameter.AddDbParameter(command, parameters[i]);
}
var materializerSource = dbContext.GetService<IEntityMaterializerSource>();
if (connection.State == ConnectionState.Closed)
await connection.OpenAsync();
using var reader = await command.ExecuteReaderAsync();
foreach (var pair in resultSetMappingTypes.Select((x, i) => (Index: i, Type: x)))
{
var i = pair.Index;
var resultSetMappingType = pair.Type;
if (i > 0 && !(await reader.NextResultAsync()))
throw new InvalidOperationException(string.Format("No result set at index {0}, unable to map to {1}.", i, resultSetMappingType));
var type = resultSetMappingType;
var entityType = dbContext.GetService<IModel>()
.FindEntityType(type);
if (entityType == null)
throw new InvalidOperationException(string.Format("Unable to find a an entity type (or query type) matching '{0}'", type));
var relationalTypeMappingSource = dbContext.GetService<IRelationalTypeMappingSource>();
var columns = Enumerable.Range(0, reader.FieldCount)
.Select(x => new
{
Index = x,
Name = reader.GetName(x)
})
.ToList();
var relationalValueBufferFactoryFactory = dbContext.GetService<IRelationalValueBufferFactoryFactory>();
int discriminatorIdx = -1;
var discriminatorProperty = entityType.GetDiscriminatorProperty();
var entityTypes = entityType.GetDerivedTypesInclusive();
var instanceTypeMapping = entityTypes.Select(et => new
{
EntityType = et,
Properties = et.GetProperties()
.Select(x =>
{
var column = columns.FirstOrDefault(y => string.Equals(y.Name,
x.GetColumnName() ?? x.Name, StringComparison.OrdinalIgnoreCase)) ?? throw new InvalidOperationException(string.Format("Unable to find a column mapping property '{0}'.", x.Name));
if (x == discriminatorProperty)
discriminatorIdx = column.Index;
return new TypeMaterializationInfo(x.PropertyInfo.PropertyType, x, relationalTypeMappingSource, column.Index);
})
.ToArray()
})
.Select(x => new
{
EntityType = x.EntityType,
Properties = x.Properties,
ValueBufferFactory = relationalValueBufferFactoryFactory.Create(x.Properties)
})
.ToDictionary(e => e.EntityType.GetDiscriminatorValue() ?? e.EntityType, e => e)
;
var resultSetValues = (IList)Activator.CreateInstance(typeof(List<>).MakeGenericType(type));
while (await reader.ReadAsync())
{
var instanceInfo = discriminatorIdx < 0 ? instanceTypeMapping[entityType] : instanceTypeMapping[reader[discriminatorIdx]];
var valueBuffer = instanceInfo.ValueBufferFactory.Create(reader);
var materializationAction = materializerSource.GetMaterializer(instanceInfo.EntityType);
resultSetValues.Add(materializationAction(new MaterializationContext(valueBuffer, dbContext)));
}
resultSets.Add(resultSetValues);
}
return resultSets;
}
And the extension typed methods
public static async Task<(IReadOnlyCollection<T1> FirstResultSet, IReadOnlyCollection<T2> SecondResultSet)> MultiResultSetsFromSql<T1, T2>(this DbContext dbContext, string sql, params object[] parameters)
{
var resultSetMappingTypes = new[]
{
typeof(T1), typeof(T2)
};
var resultSets = await MultiResultSetsFromSql(dbContext, resultSetMappingTypes, sql, parameters);
return ((IReadOnlyCollection<T1>)resultSets[0], (IReadOnlyCollection<T2>)resultSets[1]);
}
public static async Task<(IReadOnlyCollection<T1> FirstResultSet, IReadOnlyCollection<T2> SecondResultSet, IReadOnlyCollection<T3> ThirdResultSet)> MultiResultSetsFromSql<T1, T2, T3>(this DbContext dbContext, string sql, params object[] parameters)
{
var resultSetMappingTypes = new[]
{
typeof(T1), typeof(T2), typeof(T3)
};
var resultSets = await MultiResultSetsFromSql(dbContext, resultSetMappingTypes, sql, parameters);
return ((IReadOnlyCollection<T1>)resultSets[0], (IReadOnlyCollection<T2>)resultSets[1], (IReadOnlyCollection<T3>)resultSets[2]);
}

Currently, EF Core doesn't not support this. see this example for retrieve multiple result sets.
https://github.com/nilendrat/EfCoreMultipleResults/

It works with this tiny change on EF core 5 according to Ricky G answer
change
command.CommandType = CommandType.Text;
to
command.CommandType = CommandType.StoredProcedure;
and as sql parameter value for this extension method type your stored procedure name "dbo.testproc"
example of usage:
var t1 = await _context.MultiResultSetsFromSql(new [] {typeof(proctestprocResult) },"dbo.testproc", sqlParameters);
works for me

Querying Azure Table Storage - compare using a static array of values

I need to modify and existing Azure Table Storage query, assuming i is an integer query retrieves latest report:
string rowCompare = String.Format(CommonDefs.inverseTimeStampRowKeyFormat, DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
var result = (from er in this.serviceContext.EntityReportsTable
where er.PartitionKey.Equals(i.ToString(), StringComparison.OrdinalIgnoreCase) && er.RowKey.CompareTo(rowCompare) > 0
select er).Take(1)).FirstOrDefault();
I need to modify it to retrieve latest reports for several known entities, replacing single integer i with array of integers - like int[]{1, 6, 10}.
Apart from running existing query sequentially for the each parameter in array, is there a way to do it in one query? Like IN clause in Sql?

You can use the lastest version of the Azure Storage Client Library this is the complete pseudo code for your task:
var rowCompare = String.Format("{0}", DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
var items = new []{"1", "6", "10"};
var filters =
items.Select(key => TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, key)).ToArray();
var combine =
filters.Length > 0
? filters[0]
: null;
for (var k = 0; k < filters.Length; k++)
combine = TableQuery.CombineFilters(combine, TableOperators.Or, filters[k]);
var final = TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.GreaterThan, rowCompare);
if (!string.IsNullOrEmpty(combine))
final = TableQuery.CombineFilters(final, TableOperators.And, combine);
var query = new TableQuery<EntityReport>().Where(final);
var client = CloudStorageAccount.DevelopmentStorageAccount.CreateCloudTableClient();
var table = client.GetTableReference("EntityReports");
var result = table.ExecuteQuery(query);

Azure Table Storage does not support IN clause like SQL. However instead of doing a query sequentially, you could fire queries in parallel and compare the result. For example look at the pseudo code below:
List<Task<T>> tasks = new List<Task<T>>();
foreach (var i in integerArray)
{
tasks.Add(Task.Factory.StartNew<T>(() => {
string rowCompare = String.Format(CommonDefs.inverseTimeStampRowKeyFormat, DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
var result = (from er in this.serviceContext.EntityReportsTable
where er.PartitionKey.Equals(i.ToString(), StringComparison.OrdinalIgnoreCase) && er.RowKey.CompareTo(rowCompare) > 0
select er).Take(1)).FirstOrDefault();
}));
}
Task.WaitAll(tasks.ToArray());
foreach (var task in tasks)
{
var queryResult = task.Result;
//Work on the query result
}