I'm using Entity Framework and I want to perform a bulk update. It is way too inefficient to load each row, update those rows, and then save them back to the database.
So I'd prefer to use DbContext.Database.ExecuteSqlCommand(). But how can I use this method to update all those rows with an ID contained in my list of IDs?
Here's what I have so far.
IEnumerable<int> Ids;
DbContext.Database.ExecuteSqlCommand("UPDATE Messages SET Viewed = 1 WHERE Id IN (#list)", Ids);
I realize I could manually build a string with the correct query, but I'd prefer to pass my parameters as is generally recommended.
Is there any way to make that happen?
You can still build the parameters and include them in the parameterized query.
The query would look something like this when generated
UPDATE Messages SET Viewed = 1 WHERE Id IN (#p0, #p1, #p2, ..., #pn)
So given
IEnumerable<int> Ids;
Then
var parameters = Ids.Select((id, index) => new SqlParameter(string.Format("#p{0}", index), id));
var parameterNames = string.Join(", ", parameters.Select(p => p.ParameterName));
var query = string.Format("UPDATE Messages SET Viewed = 1 WHERE Id IN ({0})", parameterNames);
int affected = DbContext.Database.ExecuteSqlCommand(query, parameters.ToArray());
Instead of generating query string with exact values, you can generate query string with as many parameters as you have.
So you'll get smth like:
DbContext.Database.ExecuteSqlCommand("UPDATE Messages SET Viewed = 1 WHERE Id IN (#p0,#p1,#p2,...,#pN)", Ids);
by smth like this:
var paramsDef = string.Concat(Ids.Select(x=>$"{(Ids.IndexOf(x) > 0 ? "," : "")}p{Ids.IndexOf(x)}"));
DbContext.Database.ExecuteSqlCommand($"UPDATE Messages SET Viewed = 1 WHERE Id IN {paramsDef}", Ids);
Some links I found people doing similar with SqlCommand:
http://www.svenbit.com/2014/08/using-sqlparameter-with-sqls-in-clause-in-csharp/
http://nodogmablog.bryanhogan.net/2016/01/parameterize-sql-where-in-clause-c/
Related
I have a list of items with different ids which represent a SQL table's PK values.
Is there any way to build an efficient and safe statement?
Since now I've always prepared a string representing the statement and build it as I traversed the list via a foreach loop.
Here's an example of what I'm doing:
string update = "UPDATE table SET column = 0 WHERE";
foreach (Line l in list)
{
update += " id = " + l.Id + " OR";
}
// To remove last OR
update.Remove(update.Length - 3);
MySqlHelper.ExecuteNonQuery("myConnectionString", update);
Which feels very unsafe and looks very ugly.
Is there a better way for this?
So yeah, in SQL you've got the 'IN' keyword which allows you to specify a set of values.
This should accomplish what you would like (syntax might be iffy, but the idea is there)
var ids = string.Join(',', list.Select(x => x.Id))
string update = $"UPDATE table SET column = 0 WHERE id IN ({ids})";
MySqlHelper.ExecuteNonQuery("myConnectionString", update);
However, the way you're performing your SQL can be considered dangerous (you should be fine as this just looks like ids from a DB, who knows, better to be safe than sorry). Here you're passing parameters straight into your query string, which is a potential risk to SQL injection which is very dangerous. There are ways around this, and using the inbuilt .NET 'SqlCommand' object
https://www.w3schools.com/sql/sql_injection.asp
https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlcommand?view=dotnet-plat-ext-6.0
It would be more efficient to use IN operator:
string update = "UPDATE table SET column = 0 WHERE id IN (";
foreach (Line l in list)
{
update += l.Id + ",";
}
// To remove last comma
update.Remove(update.Length - 1);
// To insert closing bracket
update += ")";
If using .NET Core Framework, see the following library which creates parameters for a WHERE IN. The library is a port from VB.NET which I wrote in Framework 4.7 years ago. Clone the repository, get SqlCoreUtilityLibrary project for creating statements.
Setup.
public void UpdateExample()
{
var identifiers = new List<int>() { 1, 3,20, 2, 45 };
var (actual, exposed) = DataOperations.UpdateExample(
"UPDATE table SET column = 0 WHERE id IN", identifiers);
Console.WriteLine(actual);
Console.WriteLine(exposed);
}
Just enough code to create the parameterizing SQL statement. Note ActualCommandText method is included for development, not for production as it reveals actual values for parameters.
public static (string actual, string exposed) UpdateExample(string commandText, List<int> identifiers)
{
using var cn = new SqlConnection() { ConnectionString = GetSqlConnection() };
using var cmd = new SqlCommand() { Connection = cn };
cmd.CommandText = SqlWhereInParamBuilder.BuildInClause(commandText + " ({0})", "p", identifiers);
cmd.AddParamsToCommand("p", identifiers);
return (cmd.CommandText, cmd.ActualCommandText());
}
For a real app all code would be done in the method above rather than returning the two strings.
Results
UPDATE table SET column = 0 WHERE id IN (#p0,#p1,#p2,#p3,#p4)
UPDATE table SET column = 0 WHERE id IN (1,3,20,2,45)
I want to create simple database in runtime, fill it with data from internal resource and then read each record through loop. Previously I used LiteDb for that but I couldn't squeeze time anymore so
I choosed SQLite.
I think there are few things to improve I am not aware of.
Database creation process:
First step is to create table
using var create = transaction.Connection.CreateCommand();
create.CommandText = "CREATE TABLE tableName (Id TEXT PRIMARY KEY, Value TEXT) WITHOUT ROWID";
create.ExecuteNonQuery();
Next insert command is defined
var insert = transaction.Connection.CreateCommand();
insert.CommandText = "INSERT OR IGNORE INTO tableName VALUES (#Id, #Record)";
var idParam = insert.CreateParameter();
var valueParam = insert.CreateParameter();
idParam.ParameterName = "#" + IdColumn;
valueParam.ParameterName = "#" + ValueColumn;
insert.Parameters.Add(idParam);
insert.Parameters.Add(valueParam);
Through loop each value is inserted
idParameter.Value = key;
valueParameter.Value = value.ValueAsText;
insert.Parameters["#Id"] = idParameter;
insert.Parameters["#Value"] = valueParameter;
insert.ExecuteNonQuery();
Transaction commit transaction.Commit();
Create index
using var index = transaction.Connection.CreateCommand();
index.CommandText = "CREATE UNIQUE INDEX idx_tableName ON tableName(Id);";
index.ExecuteNonQuery();
And after that i perform milion selects (to retrieve single value):
using var command = _connection.CreateCommand();
command.CommandText = "SELECT Value FROM tableName WHERE Id = #id;";
var param = command.CreateParameter();
param.ParameterName = "#id";
param.Value = id;
command.Parameters.Add(param);
return command.ExecuteReader(CommandBehavior.SingleResult).ToString();
For all select's one connection is shared and never closed. Insert is quite fast (less then minute) but select's are very troublesome here. Is there a way to improve them?
Table is quite big (around ~2 milions records) and Value contains quite heavy serialized objects.
System.Data.SQLite provider is used and connection string contains this additional options: Version=3;Journal Mode=Off;Synchronous=off;
If you go for performance, you need to consider this: each independent SELECT command is a roundtrip to the DB with some extra costs. It's similar to a N+1 select problem in case of parent-child relations.
The best thing you can do is to get a LIST of items (values):
SELECT Value FROM tableName WHERE Id IN (1, 2, 3, 4, ...);
Here's a link on how to code that: https://www.mikesdotnetting.com/article/116/parameterized-in-clauses-with-ado-net-and-linq
You could have the select command not recreated for every Id but created once and only executed for every Id. From your code it seems every select is CreateCommand/CreateParameters and so on. See this for example: https://learn.microsoft.com/en-us/dotnet/api/system.data.idbcommand.prepare?view=net-5.0 - you run .Prepare() once and then only execute (they don't need to be NonQuery)
you could then try to see if you can be faster with ExecuteScalar and not having reader created for one data result, like so: https://learn.microsoft.com/en-us/dotnet/api/system.data.idbcommand.executescalar?view=net-5.0
If scalar will not prove to be faster then you could try to use .SingleRow instead of .SingleResult in your ExecuteReader for possible performance optimisations. According to this: https://learn.microsoft.com/en-us/dotnet/api/system.data.commandbehavior?view=net-5.0 it might work. I doubt that but if first two don't help, why not try it too.
I use ormlite with servicestack and I have got this problem.
I have saved a list of string in a column of my db so I want to do a select sql like this:
Select top 1 *
From MyTable
Where MyVariable In (MyListSavedInDb)
var orders = db.Select<Order>(o => Sql.In(o.Ldv, o.Waybills));
Where o.Ldv is a string and o.Waybills is a list of string saved on db
Any solutions ?
You can't query a blobbed field with server-side SQL, best you can do is a fuzzy string index search like:
var q = db.From<Order>();
q.Where($"CHARINDEX({q.Column<Order>(x=>x.Ldv)},{q.Column<Order>(x=>x.Waybills)}) > 0")
.Take(1);
var order = db.Single(q);
But essentially you shouldn't be blobbing any fields that you want to perform server-side SQL queries on.
A more typed and robust approach would be to perform the final query on a blobbed collection on the client after you've filtered the resultset, e.g:
var orders = db.Select(q);
var order = orders.FirstOrDefault(x => x.Waybills.Contains(term));
But as this query is done on the client you'll want to ensure it's being done on a limited filtered resultset.
I want to get list of specific columns returned by stored procedure.In the result set two column names will be common and one column name will be dynamic.
ALTER PROC DBO.GETLANGUAGETRANSLATION(#LANGCODE VARCHAR(10))
AS
BEGIN
DECLARE #QUERY NVARCHAR(255)
SET #QUERY= N'SELECT RESOURCENAME,ENText,'+UPPER(#LANGCODE)+' INTO #TEMP FROM DBO.LANGUAGETRANSLATION SELECT * FROM #TEMP';
EXEC(#QUERY);
END
RESOURCENAME and ENText column will be same in output each time but based in input parameter third column name will vary.
I am writing below code in C# using Entity Framework Core LINQ
public async Task<IEnumerable<LanguageTranslation>> GetAsync(string langCode)
{
var context = new LQMSDbContext(AppConstants.DB_CONNECTION_STRING_KEY);
try
{
string query = "GetLanguageTranslation '" + langCode + "'";
var result = context.LanguageTranslation.FromSql(query).ToList();
var result1 = context.Database.ExecuteSqlCommand("GETLANGUAGETRANSLATION #p0", parameters: langCode );
}
catch(Exception ex)
{
string a = ex.Message;
}
return await _dbSet
.Where(x => x.Equals(langCode))
.ToListAsync();
}
I am trying to call stored procedure GETLANGUAGETRANSLATION using two different approaches. But both are failing with below error
The required column 'ESText' was not present in the results of a
'FromSql' operation
Where ESText refers to the column which I am not returning from stored procedure but present in table.
I want to store only few columns in result set in c# and not all.
Can any body help me in this?
NOTE : It works fine with Select * from LanguageTranslation query
Have a look at how to execute raw queries.
https://learn.microsoft.com/en-us/ef/core/querying/raw-sql
You can write the query to store the results temporarily in a temp table then select from that table.
FromSql takes more parameters for your injection, you will want to use those extra parameters to place yours into the query. Don't string concatenate.
Another note: when you use ToList or ToArray or Count you will execute the query on the server.
Modifying an answer from this question slightly, suppose I run this code:
public int SaveOrUpdate(MyEntity entity)
{
var sql = #"MERGE INTO MyEntity
USING
(
SELECT #id as Id
#myField AS MyField
) AS entity
ON MyEntity.Id = entity.Id
WHEN MATCHED THEN
UPDATE
SET Id = #id
MyField = #myField
WHEN NOT MATCHED THEN
INSERT (Id, MyField)
VALUES (#Id, #myField);"
object[] parameters = {
new SqlParameter("#id", entity.Id),
new SqlParameter("#myField", entity.myField)
};
return context.Database.ExecuteSqlCommand(sql, parameters);
}
This will actually run and it returns an int. What does the int mean? The documentation just says
The result returned by the database after executing the command.
I did a couple tests and it looks like it's 1 if it modified a row, and 0 if nothing changed. Is the return value the number of rows modified?
For most databases, that means the number of rows affected by the command. I theory though, god forbid that such a thing exists, the database vendor is free to return whatever and you would then need to look in the documentation for the particular database for what the number means.