I need to modify and existing Azure Table Storage query, assuming i is an integer query retrieves latest report:
string rowCompare = String.Format(CommonDefs.inverseTimeStampRowKeyFormat, DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
var result = (from er in this.serviceContext.EntityReportsTable
where er.PartitionKey.Equals(i.ToString(), StringComparison.OrdinalIgnoreCase) && er.RowKey.CompareTo(rowCompare) > 0
select er).Take(1)).FirstOrDefault();
I need to modify it to retrieve latest reports for several known entities, replacing single integer i with array of integers - like int[]{1, 6, 10}.
Apart from running existing query sequentially for the each parameter in array, is there a way to do it in one query? Like IN clause in Sql?
You can use the lastest version of the Azure Storage Client Library this is the complete pseudo code for your task:
var rowCompare = String.Format("{0}", DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
var items = new []{"1", "6", "10"};
var filters =
items.Select(key => TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, key)).ToArray();
var combine =
filters.Length > 0
? filters[0]
: null;
for (var k = 0; k < filters.Length; k++)
combine = TableQuery.CombineFilters(combine, TableOperators.Or, filters[k]);
var final = TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.GreaterThan, rowCompare);
if (!string.IsNullOrEmpty(combine))
final = TableQuery.CombineFilters(final, TableOperators.And, combine);
var query = new TableQuery<EntityReport>().Where(final);
var client = CloudStorageAccount.DevelopmentStorageAccount.CreateCloudTableClient();
var table = client.GetTableReference("EntityReports");
var result = table.ExecuteQuery(query);
Azure Table Storage does not support IN clause like SQL. However instead of doing a query sequentially, you could fire queries in parallel and compare the result. For example look at the pseudo code below:
List<Task<T>> tasks = new List<Task<T>>();
foreach (var i in integerArray)
{
tasks.Add(Task.Factory.StartNew<T>(() => {
string rowCompare = String.Format(CommonDefs.inverseTimeStampRowKeyFormat, DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks);
var result = (from er in this.serviceContext.EntityReportsTable
where er.PartitionKey.Equals(i.ToString(), StringComparison.OrdinalIgnoreCase) && er.RowKey.CompareTo(rowCompare) > 0
select er).Take(1)).FirstOrDefault();
}));
}
Task.WaitAll(tasks.ToArray());
foreach (var task in tasks)
{
var queryResult = task.Result;
//Work on the query result
}
Related
sonar is given me a hard time because the following code:
var listaChaves = chaves.ToList();
var parametros = new string[chaves.Count];
var parametrosSql = new List<NpgsqlParameter>();
for (int i = 0; i < listaChaves.Count; i++)
{
parametros[i] = string.Format("#param_{0}", i);
parametrosSql.Add(new NpgsqlParameter(parametros[i], listaChaves[i]));
}
var comandoSql = string
.Format("SELECT distinct on(chave_identificacao) chave_identificacao, data from ultimos_acessos where chave_identificacao in({0}) order by chave_identificacao, data desc",
string.Join(", ", parametros));
var ultimosAcessos = await Entidade.FromSqlRaw(comandoSql, parametrosSql.ToArray())
.Select(a => new ProjecaoListagemUltimoAcesso(a.Data, a.ChaveIdentificacao))
.ToListAsync();
it thinks a sql injection can happen because the string interpolation. So i tried to change to "FromSqlInterpolated" method, as follow:
var listaChaves = chaves.ToList();
var ultimosAcessos = await Entidade.FromSqlInterpolated(#$"SELECT distinct on(chave_identificacao) chave_identificacao, data from ultimos_acessos where chave_identificacao in({string.Join(", ", listaChaves)})) ) order by chave_identificacao, data desc")
.Select(a => new ProjecaoListagemUltimoAcesso(a.Data, a.ChaveIdentificacao))
.ToListAsync();
But it just dont work, anyone can help me on how can I create a secure sql from a interpolated string using a "in" clause?
Something like this should work. Build an OR clause in the same loop as your parameters and stuff it into your SQL:
string ORclause = "";
for (int i = 0; i < listaChaves.Count; i++)
{
ORclause += $" chave_identificacao = #param_{i} OR ";
parametros[i] = string.Format("#param_{0}", i);
parametrosSql.Add(new NpgsqlParameter(parametros[i], listaChaves[i]));
}
//remove last "OR"
ORclause = ORclause.Substring(0, s.Length - 4);
#$"SELECT distinct on(chave_identificacao) chave_identificacao, data
from ultimos_acessos
where {ORclause}
order by chave_identificacao, data desc";
If you have a lot of listaChaves then consider using StringBuilder instead of +=
I'm using Microsoft Graph client and want to retrieve users based on a list of objectIds. So far I've managed to do it like this:
// Create filterstring based on objectId string list.
var filterString = string.Join(" or ", objectIds.Where(x => !string.IsNullOrEmpty(x)).Select(objectId => $"id eq '{objectId}'"));
// Get users by filter
var users = await _graphServiceClient.Users.Request()
.Select(x => new { x.UserPrincipalName, x.Id })
.Filter(filterString)
.GetAsync(ct).ConfigureAwait(false);
But I've hit this error here:
Too many child clauses specified in search filter expression containing 'OR' operators: 22. Max allowed: 15.
Is there another way to only get a portion of users? Or do I need to "chunk" the list up in 15 each?
You should probably split your query and send a BATCH request to the Graph API.
This will send only 1 request to the server, but allow you to query for more data at once.
https://learn.microsoft.com/en-us/graph/sdks/batch-requests?tabs=csharp
This could look something like this: (untested code)
var objectIds = new string[0];
var batchRequestContent = new BatchRequestContent();
var requestList = new List<string>();
for (var i = 0; i < objectIds.Count(); i += 15)
{
var batchObjectIds = objectIds.Where(x => !string.IsNullOrEmpty(x)).Skip(i).Take(15);
var filterString = string.Join(" or ", batchObjectIds.Select(objectId => $"id eq '{objectId}'"));
var request = _graphServiceClient.Users.Request()
.Select(x => new { x.UserPrincipalName, x.Id })
.Filter(filterString);
var requestId = batchRequestContent.AddBatchRequestStep(request);
requestList.Add(requestId);
}
var batchResponse = await _graphServiceClient.Batch.Request().PostAsync(batchRequestContent);
var allUsers = new List<User>();
foreach (var it in requestList)
{
var users = await batchResponse.GetResponseByIdAsync<GraphServiceUsersCollectionResponse>(it);
allUsers.AddRange(users.Value.CurrentPage);
}
What is the correct syntax for varname to make this query work?
I can get it to work with a single variable like
string varname = "TOTAL_NORWAY"
However, if I want to have a few variables in there, I get an empty array returned:
string varname = "'TOTAL_NORWAY', 'TOTAL_SWEDEN'";
return await _Context.theDataModel.FromSqlRaw(#"
select data
from data_table
where Variable in ({0})
", varname).ToListAsync();
Remember that you can combine FromSqlRaw with Linq:
string varnames = new [] { "TOTAL_NORWAY", "TOTAL_SWEDEN" };
var query = _Context.theDataModel.FromSqlRaw(#"
select data
from data_table");
query = query.Where(x => varnames.Contains(x.Variable));
// Add more where clauses as needed
return await query.ToListAsync();
ErikEJ's post was very helpful. The solution is not so trivial for someone who doesn't dabble in EF Core regularly.
I also had an extra where clause to consider, and this was done like so for anyone else wondering.
var items = new int[] { 1, 2, 3 };
var parameters = new string[items.Length];
var sqlParameters = new List<SqlParameter>();
for (var i = 0; i < items.Length; i++)
{
parameters[i] = string.Format("#p{0}", i);
sqlParameters.Add(new SqlParameter(parameters[i], items[i]));
}
sqlParameters.Add(new SqlParameter("#userid", "userXYZ123"));
var rawCommand = string.Format("SELECT * from dbo.Shippers WHERE ShipperId IN ({0}) and userid = {1}", string.Join(", ", parameters), "#userid");
var shipperList = db.Set<ShipperSummary>()
.FromSqlRaw(rawCommand, sqlParameters.ToArray())
.ToList();
i'm trying to execute a linq off a DataTable but having problems with Like statement, and need some assistance.
var query = from results in referenceDt.AsEnumerable()
where results.IsNull("ClosedTime") &&
**????**
select new
{
Cluster = results.Field<string>("FaultCode"),
DC = results.Field<Int32>("FaultCode"),
Region = results.Field<string>("FabricName")
};
Here is what my sql query looks like:
SELECT FaultCode, COUNT(FaultCode) AS Count, FabricName
FROM RmaSummary
WHERE ClosedTime IS null AND FaultCode LIKE '60%'
) GROUP BY FaultCode, FabricName
ORDER BY FabricName
Try this:
var query = from results in referenceDt.AsEnumerable()
where results.Field<DateTime?>("ClosedTime") == null &&
results.Field<int>("FaultCode").ToString().StartsWith("60")
select results;
var count = results.Count();
Use StartsWith() method available for strings,
var query = from results in referenceDt.AsEnumerable()
where results.IsNull("ClosedTime") &&
results.Field<Int32>("FaultCode").ToString().StartsWith("60")
select new
{
Cluster = results.Field<string>("FaultCode"),
DC = results.Field<Int32>("FaultCode"),
Region = results.Field<string>("FabricName")
};
Having something similar to:
SELECT (SELECT COUNT(*) from Table1),(SELECT COUNT(*) from Table2 )
How do I write it in linq? Or is it simple not possible?
Limitations:
Can only hit the database one time:
var result = new {
Sum1 = db.Table1.Count(),
Sum2 = db.Table2.Count()
}); // is not valid.....
I do not want to use something similar to (using a "helping" table):
var result = (from t3 in db.Table3
select new {
Sum1 = db.Table1.Count(),
Sum2 = db.Table2.Count()
}).firstOrDefault();
//In order to get only the first row
//but it will not return nothing if the table 3 has no entries......
Not using db.Database.ExecuteSqlCommand
I cannot see a solution which solves all your limitations. This is one of the caveats with using an ORM-mapper, you are not in control of the generated SQL.
In this case, if it is utterly unacceptable for you to send more than one query to the database, the harsh truth is that you will have to write the query yourself.
Update
I got curious and created an extension method that can do this! Of course it constructs its own SQL command, and it just works for Linq2SQL. Also massive disclaimer: It's fairly dirty code, if I have some time I'll fix it up in the weekend :)
public static TOut CountMany<TContext, TOut>(this TContext db, Expression<Func<TContext, TOut>> tableSelector)
where TContext: DataContext
{
var newExpression = (NewExpression) tableSelector.Body;
var tables =
newExpression.Arguments.OfType<MethodCallExpression>()
.SelectMany(mce => mce.Arguments.OfType<MemberExpression>())
.ToList();
var command = new string[tables.Count];
for(var i = 0; i < tables.Count; i++)
{
var table = tables[i];
var tableType = ((PropertyInfo) table.Member).PropertyType.GetGenericArguments()[0];
var tableName = tableType.GetCustomAttribute<TableAttribute>().Name;
command[i] = string.Format("(SELECT COUNT(*) FROM {0}) AS T{1}", tableName, i);
}
var dbCommand = db.Connection.CreateCommand();
dbCommand.CommandText = string.Format("SELECT {0}", String.Join(",", command));
db.Connection.Open();
IDataRecord result;
try
{
result = dbCommand.ExecuteReader().OfType<IDataRecord>().First();
}
finally
{
db.Connection.Close();
}
var results = new object[tables.Count];
for (var i = 0; i < tables.Count; i++)
results[i] = result.GetInt32(i);
var ctor = typeof(TOut).GetConstructor(Enumerable.Repeat(typeof(int), tables.Count).ToArray());
return (TOut) ctor.Invoke(results);
}
the code is called like this:
var counts = dbContext.CountMany(db => new
{
table1Count = db.Table1.Count(),
table2Count = db.Table2.Count()
//etc.
});