How do I cache the result of such a query:
Session.Query<Entity>().Count(e=> e.Property == someConstant)
I cannot place Cacheable() after it, and if I place it before the Count(), it would fetch the entire result set, wouldnt it?
Adding Cacheable before the count will cause the aggregate count results to be cached. This can be seen by the SQL generated from my example below.
Domain and Mapping Code
public class Entity
{
public virtual long id { get; set; }
public virtual string name { get; set; }
}
public class EntityMap : ClassMap<Entity>
{
public EntityMap()
{
Id(x => x.id).GeneratedBy.Identity();
Map(x => x.name);
Cache.ReadOnly();
}
}
The test code itself.
using (var session = NHibernateHelper.OpenSession())
using (var tx = session.BeginTransaction())
{
session.Save(new Entity() { name = "Smith" });
session.Save(new Entity() { name = "Smithers" });
session.Save(new Entity() { name = "Smithery" });
session.Save(new Entity() { name = "Smith" });
tx.Commit();
}
String name_constant = "Smith";
using (var session = NHibernateHelper.OpenSession())
using (var tx = session.BeginTransaction())
{
var result = session.Query<Entity>().Cacheable().Count(e => e.name == name_constant);
}
using (var session = NHibernateHelper.OpenSession())
using (var tx = session.BeginTransaction())
{
var result = session.Query<Entity>().Cacheable().Count(e => e.name == name_constant);
}
The SQL generated from the above code can be seen below. As you can see, there are four INSERT statements, one for each session.Save. Whereas there is only one SELECT despite the two queries, each performing under a separate session. This is because the result of the count has been cached by NHibernate.
NHibernate: INSERT INTO [Entity] (name) VALUES (#p0); select SCOPE_IDENTITY();
#p0 = 'Smith' [Type: String (4000)]
NHibernate: INSERT INTO [Entity] (name) VALUES (#p0); select SCOPE_IDENTITY();
#p0 = 'Smithers' [Type: String (4000)]
NHibernate: INSERT INTO [Entity] (name) VALUES (#p0); select SCOPE_IDENTITY();
#p0 = 'Smithery' [Type: String (4000)]
NHibernate: INSERT INTO [Entity] (name) VALUES (#p0); select SCOPE_IDENTITY();
#p0 = 'Smith' [Type: String (4000)]
NHibernate: select cast(count(*) as INT) as col_0_0_ from [Entity] entity0_
where entity0_.name=#p0;
#p0 = 'Smith' [Type: String (4000)]
The possible scenarios which will cause NHibernate to ignore Cacheable and go back to the DB are when:
The second level cache is not enabled in the session factory configuration.
The entity has not been marked as cacheable.
The only other scenario I know of that will cause NHibernate to perform two SELECT queries is when the entity has been evicted from the cache, either explicitly using sessionFactory.Evict or by the cached entity becoming expired between the two calls.
Related
In my .Net application, EntityFramework 6 is used and I am writing a logic in C# to update one of the column values of the ORDERTABLE to Yes or No.
In ORDERTABLE, for a Single TranID there will be one or more OrderNumber. I need to update the last OrderNumber (i.e., MAX(OrderNumber)) value of a TranID to 'NO' and for remaining OrderNumber value will be 'YES'.
Below SQL query is giving me the expected result but I am not sure of converting this into LINQ to Entities code logic.
UPDATE SC
SET ORDERVALUE = CASE WHEN SC1.ORDERNUMBER IS NULL THEN 'YES' ELSE 'NO' END
FROM ORDERTABLE SC
LEFT JOIN (SELECT MAX(ORDERNUMBER) ORDERNUMBER, SID FROM ORDERTABLE WHERE STATUS ='A' GROUP BY SID) SC1
ON SC.ORDERNUMBER = SC1.ORDERNUMBER AND SC.SID = SC1.SID
WHERE SC.STATUS ='A' AND SC.SID IN (SELECT ID FROM ORDERMASTER(NOLOCK) WHERE MID = variablename)
Select Query:
SELECT
ORDERVALUE = CASE WHEN SC1.ORDERNUMBER IS NULL THEN 'YES' ELSE 'NO' END,
SC.*
FROM ORDERTABLE SC
LEFT JOIN (SELECT MAX(ORDERNUMBER) ORDERNUMBER, SID FROM ORDERTABLE WHERE STATUS ='A' GROUP BY SID) SC1
ON SC.ORDERNUMBER = SC1.ORDERNUMBER AND SC.SID = SC1.SID
WHERE SC.STATUS ='A' AND SC.SID IN (SELECT ID FROM ORDERMASTER(NOLOCK) WHERE MID = variablename)
In C#, LINQ to Entities code should looks like somewhat similar to below first format (LINQ Method) but not like the second one(LINQ query).
//1. LINQ Method
using (var context = new ProductDBEntities())
{
dbContextTransaction = context.Database.BeginTransaction();
ORDERDETAILS od = context.ORDERDETAILS.Single(G => G.ID == 1);
od.OrderNumber = OrdNumber;
od.LastModifiedBy = createdBy;
od.LastModifiedTS = DateTime.UtcNow;
context.SaveChanges();
}
//2. LINQ Query
using (var context = new ProductDBEntities())
{
var query = from st in context.ORDERTABLE
where ...
select st;
var ORDERTABLE = query.FirstOrDefault<ORDERTABLE >();
}
I've created a database trigger on my table that updates a field in the table after an insert. When doing an insert using EF I'll get the ID and the number. On the database I've created this code:
create table Things (
ID int primary key identity not null,
Number nvarchar(20)
);
create trigger UpdateThingsNumberTrigger on Things
after insert
as
begin
declare #month nvarchar(2);
select #month = cast(month(getdate()) as nvarchar(2));
declare #code nvarchar(15);
select #code = cast(year(getdate()) as nvarchar(4)) +
'.' +
replicate('0', 2 - len(#month)) +
#month +
'.';
declare #max nvarchar(20);
select #max = t.ID
from Things t
where ID like #code + '%';
with CTE_UPD as
(
select
replicate('0',
4 -
len(cast(coalesce(cast(right(#max, 4) as int), 0) + row_number() over (order by ins.ID) as nvarchar(4)))) +
cast(coalesce(cast(right(#max, 4) as int), 0) + row_number() over (order by ins.ID) as nvarchar(4)) as NextNo,
ID
from Things ins
)
update Things
set Number = #code + NextNo
from Things t inner join CTE_UPD ins on ins.ID = t.ID;
end
Note: For the logical flaw inside the trigger, I'll refer to Create an incremental number with year and month without updating the entire table using a trigger on Database Administrators SE.
This part of my code works fine, ignoring the logical flaw inside the trigger… The problem I'll try to solve in this question, is when I insert a thing in my table from Entity Framework (database first). There is my code and the output:
using (Database db = new Database())
{
Thing thing = new Thing(); // --> just an empty constructor.
db.Entry(thing).State = EntityState.Added;
await db.SaveChangesAsync();
Console.WriteLine($"ID = {thing.ID}");
Console.WriteLine($"Number = {thing.Number}");
}
// Output:
// ID = 1
// Number =
In the background EF is doing this code on the server when calling SaveChangesAsync():
INSERT [dbo].[Things]([Number])
VALUES (NULL)
SELECT [ID]
FROM [dbo].[Things]
WHERE ##ROWCOUNT > 0 AND [ID] = scope_identity()
Now can EF update the ID in the C# object. But how could I get the number without using code below before closing the using block?
Thing recentlyInsertedThing = await db.Things.FindAsync(thing.ID);
I've found it to get the ID and the Number without writing a 2nd select statement. This is my code:
using (Database db = new Database())
{
Thing thing = new Thing();
string sql = #"insert into Things()
values ();
select ID, Number
from Things
where ##rowcount > 0 and ID = scope_identity();";
KeyMapper recentlyRecevedKeys = await db
.Database
.SqlQuery<KeyMapper>(sql)
.FirstAsync();
thing.ID = recentlyRecevedKeys.ID;
thing.Number = recentlyRecevedKeys.Number;
}
// Nested class
private class KeyMapper
{
public int ID { get; set; }
public string Number { get; set; }
}
I have to import a hundreds records to database from Excel.
Each record has to be verified:
Against duplicate
Has to has foreign key in another table
I’m wondering how should I do this with the highest performance. I know that I shouldn’t use db.SaveChanges(); after each record so after verification - I’m adding each record to temporary list (var recordsToAdd), and I’m saving that list after all.
Please check my code below, is this good approach to do this?
using (var db = new DbEntities())
{
var recordsToAdd = new List<User>();
for (var row = 2; row <= lastRow; row++)
{
var newRecord = new User
{
Id = Int32.Parse(worksheet.Cells[idColumn + row].Value.ToNullSafeString()),
FirstName = worksheet.Cells[firstNameColumn + row].Value.ToNullSafeString(),
LastName = worksheet.Cells[lastNameColumn + row].Value.ToNullSafeString(),
SerialNumber = worksheet.Cells[serialNumber + row].Value.ToNullSafeString()
};
bool exists = db.User.Any(u => u.Id == newRecord.Id) || recordsToAdd.Any(u => u.Id == newRecord.Id);
if (!exists)
{
bool isSerialNumberExist = db.SerialNumbers.Any(u => u.SerialNumber == newRecord.SerialNumber);
if (isSerialNumberExist)
{
recordsToAdd.Add(newRecord);
}
else
{
resultMessages.Add(string.Format("SerialNumber doesn't exist"));
}
}
else
{
resultMessages.Add(string.Format("Record already exist"));
}
}
db.User.AddRange(recordsToAdd);
db.SaveChanges();
}
First of all let's separate the code into two parts. First part is creating a list of valid User records to be inserted. Second part is inserting those records to the database (last two lines of your code).
Assuming you are using EntityFramework as your ORM, second part may be optimized by bulk inserting the records. It has many existing solutions that can be easily found. (example)
There are some suggestions concerning the first part.
Load user ids in a HashSet or Dictionary. These data structures are optimized for searching. var userDbIds = new HashSet<int>(db.User.Select(x => x.Id));. You will quickly check if id exists without making a request to DB.
Do the same for serialNumber. var serialNumbers = new HashSet<string>(db.SerialNumber.Select(x => x.SerialNumber)); assuming that type of SerialNumber property is string.
Change the type of your recordToAdd variable to be Dictionary<int, User> for the same reason.
In the check would look like this:
bool exists = userDbIds.Contains(newRecord.Id) || recordsToAdd.ContainsKey(newRecord.Id);
if (!exists)
{
bool isSerialNumberExist = serialNumbers.Contains(newRecord.SerialNumber);
if (isSerialNumberExist)
{
recordsToAdd[newRecord.Id] = newRecord;
}
else
{
resultMessages.Add(string.Format("SerialNumber doesn't exist"));
}
}
else
{
resultMessages.Add(string.Format("Record already exist"));
}
One way to improve the performance is to minimize the db calls and linear searches by using a fast lookup data structures for performing the verification - HashSet<string> for Id and Dictionary<string, bool> for SerialNumber:
using (var db = new DbEntities())
{
var recordsToAdd = new List<User>();
var userIdSet = new HashSet<string>();
var serialNumberExistsInfo = new Dictionary<string, bool>();
for (var row = 2; row <= lastRow; row++)
{
var newRecord = new User
{
Id = Int32.Parse(worksheet.Cells[idColumn + row].Value.ToNullSafeString()),
FirstName = worksheet.Cells[firstNameColumn + row].Value.ToNullSafeString(),
LastName = worksheet.Cells[lastNameColumn + row].Value.ToNullSafeString(),
SerialNumber = worksheet.Cells[serialNumber + row].Value.ToNullSafeString()
};
bool exists = !userIdSet.Add(newRecord.Id) || db.User.Any(u => u.Id == newRecord.Id);
if (!exists)
{
bool isSerialNumberExist;
if (!serialNumberExistsInfo.TryGetValue(newRecord.SerialNumber, out isSerialNumberExist))
serialNumberExistsInfo.Add(newRecord.SerialNumber, isSerialNumberExist =
db.SerialNumbers.Any(u => u.SerialNumber == newRecord.SerialNumber));
if (isSerialNumberExist)
{
recordsToAdd.Add(newRecord);
}
else
{
resultMessages.Add(string.Format("SerialNumber doesn't exist"));
}
}
else
{
resultMessages.Add(string.Format("Record already exist"));
}
}
db.User.AddRange(recordsToAdd);
db.SaveChanges();
}
It would be most efficient to use a Table-Valued Parameter instead of LINQ. That way you can handle this in a set-based approach that is a single connection, single stored procedure execution, and single transaction. The basic setup is shown in the example code I provided in the following answer (here on S.O.):
How can I insert 10 million records in the shortest time possible?
The stored procedure can handle both validations:
don't insert duplicate records
make sure that SerialNumber exists
The User-Defined Table Type (UDTT) would be something like:
CREATE TYPE dbo.UserList AS TABLE
(
Id INT NOT NULL,
FirstName NVARCHAR(50) NOT NULL,
LastName NVARCHAR(50) NULL,
SerialNumber VARCHAR(50) NOT NULL
);
-- Uncomment the following if you get a permissions error:
-- GRANT EXECUTE ON TYPE::[dbo].[UserList] TO [ImportUser];
GO
The stored procedure (executed via SqlCommand.ExecuteNonQuery) would look something like:
CREATE PROCEDURE dbo.ImportUsers
(
#NewUserList dbo.UserList READONLY
)
AS
SET NOCOUNT ON;
INSERT INTO dbo.User (Id, FirstName, LastName, SerialNumber)
SELECT tmp.Id, tmp.FirstName, tmp.LastName, tmp.SerialNumber
FROM #NewUserList tmp
WHERE NOT EXISTS (SELECT *
FROM dbo.User usr
WHERE usr.Id = tmp.[Id])
AND EXISTS (SELECT *
FROM dbo.SerialNumbers sn
WHERE sn.SerialNumber = tmp.[SerialNumber]);
The stored procedure above simply ignores the invalid records. If you need notification of the "errors", you can use the following definition (executed via SqlCommand.ExecuteReader):
CREATE PROCEDURE dbo.ImportUsers
(
#NewUserList dbo.UserList READONLY
)
AS
SET NOCOUNT ON;
CREATE TABLE #TempUsers
(
Id INT NOT NULL,
FirstName NVARCHAR(50) NOT NULL,
LastName NVARCHAR(50) NULL,
SerialNumber VARCHAR(50) NOT NULL,
UserExists BIT NOT NULL DEFAULT (0),
InvalidSerialNumber BIT NOT NULL DEFAULT (0)
);
INSERT INTO #TempUsers (Id, FirstName, LastName, SerialNumber)
SELECT tmp.Id, tmp.FirstName, tmp.LastName, tmp.SerialNumber
FROM #NewUserList tmp;
-- Mark existing records
UPDATE tmp
SET tmp.UserExists = 1
FROM #TempUsers tmp
WHERE EXISTS (SELECT *
FROM dbo.User usr
WHERE usr.Id = tmp.[Id]);
-- Mark invalid SerialNumber records
UPDATE tmp
SET tmp.InvalidSerialNumber = 1
FROM #TempUsers tmp
WHERE tmp.UserExists = 0 -- no need to check already invalid records
AND NOT EXISTS (SELECT *
FROM dbo.SerialNumbers sn
WHERE sn.SerialNumber = tmp.[SerialNumber]);
-- Insert remaining valid records
INSERT INTO dbo.User (Id, FirstName, LastName, SerialNumber)
SELECT tmp.Id, tmp.FirstName, tmp.LastName, tmp.SerialNumber
FROM #TempUsers tmp
WHERE tmp.UserExists = 0
AND tmp.InvalidSerialNumber = 0;
-- return temp table to caller as it contains validation info
SELECT tmp.Id, tmp.FirstName, tmp.LastName, tmp.SerialNumber,
tmp.UserExists, tmp.InvalidSerialNumber
FROM #TempUsers tmp
-- optionally only return records that had a validation error
-- WHERE tmp.UserExists = 1
-- OR tmp.InvalidSerialNumber = 1;
When this version of the stored procedure completes, cycle through SqlDataReader.Read() to get the validation info.
I'm using Entity Framework to get the total row count for a table. I simply want the row count, no where clause or anything like that. The following query works, but is slow. It took about 7 seconds to return the count of 4475.
My guess here is that it's iterating through the entire table, just like how IEnumerable.Count() extension method works.
Is there a way I can get the total row count "quickly"? is there a better way?
public int GetLogCount()
{
using (var context = new my_db_entities(connection_string))
{
return context.Logs.Count();
}
}
You can even fire Raw SQL query using entity framework as below:
var sql = "SELECT COUNT(*) FROM dbo.Logs";
var total = context.Database.SqlQuery<int>(sql).Single();
That is the way to get your row count using Entity Framework. You will probably see faster performance on the second+ queries as there is an initialization cost the first time that you run it. (And it should be generating a Select Count() query here, not iterating through each row).
If you are interested in a faster way to get the raw row count in a table, then you might want to try using a mini ORM like Dapper or OrmLite.
You should also make sure that your table is properly defined (at the very least, that it has a Primary Key), as failure to do this can also affect the time to count rows in the table.
If you have access to do so, it would be much quicker to query the sys tables to pull this information.
E.g.
public Int64 GetLogCount()
{
var tableNameParam = new SqlParameter("TableName", "Logs");
var schemaNameParam = new SqlParameter("SchemaName", "dbo");
using (var context = new my_db_entities(connection_string))
{
var query = #"
SELECT ISNULL([RowCount],0)
FROM (
SELECT SchemaName,
TableName,
Sum(I.rowcnt) [RowCount]
FROM sysindexes I
JOIN sysobjects O (nolock) ON I.id = o.id AND o.type = 'U'
JOIN (
SELECT so.object_id,
ss.name as SchemaName,
so.name as TableName
FROM sys.objects SO (nolock)
JOIN sys.schemas SS (nolock) ON ss.schema_id = so.schema_id
) SN
ON SN.object_id = o.id
WHERE I.indid IN ( 0, 1 )
AND TableName = #TableName AND SchemaName = #SchemaName
GROUP BY
SchemaName, TableName
) A
";
return context.ExecuteStoreQuery<Int64>(query, tableNameParam, schemaNameParam).First();
}
}
I am particularly confused by the following test case:
public void TestMapping()
{
var autoPersistenceModel = AutoMap.AssemblyOf<RepositoryEntity>().Where(
x => x.Namespace.EndsWith("Descriptors"));
var configuration = Fluently.Configure()
.Database(SQLiteConfiguration.Standard.ShowSql().InMemory)
.Mappings(x => x.AutoMappings.Add(autoPersistenceModel))
.ExposeConfiguration(x => new NHibernate.Tool.hbm2ddl.SchemaExport(x).Create(true, false));
var sessionFactory = configuration.BuildSessionFactory();
using (var session = sessionFactory.OpenSession())
{
new PersistenceSpecification<IndicatorUnitDescriptor>(session)
.CheckProperty(x => x.Name, "Name1")
.CheckProperty(x => x.Timestamp, new DateTime(2000, 10, 10))
.VerifyTheMappings();
}
}
As you can see, I'm experimenting with automappings, but unfortunately the following test case raises the following SQLite exception (the first contains the actual queries done):
drop table if exists "IndicatorUnitDescriptor"
drop table if exists "StockUnitDescriptor"
create table "IndicatorUnitDescriptor" (
Id integer,
Name TEXT,
Timestamp DATETIME,
primary key (Id)
)
create table "StockUnitDescriptor" (
Id integer,
Name TEXT,
Timestamp DATETIME,
primary key (Id)
)
NHibernate: INSERT INTO "IndicatorUnitDescriptor" (Name, Timestamp) VALUES (#p0, #p1); select last_insert_rowid();#p0 = 'Name1' [Type: String (0)], #p1 = 10.10.2000 0:00:00 [Type: DateTime (0)]
System.Data.SQLite.SQLiteException: SQLite error
no such table: IndicatorUnitDescriptor
And I can't understand why does it happen this way - the SQL commands seem to be working appropriately and the corresponding table should be created by the create table query.
I assume something is wrong in my code (I probably missed something). Could you help me?
I think your using two sessions. One during database creation and in your test proper. Try setting up like this.
Configuration cfg = Fluently.Configure()
.Database(SQLiteConfiguration.Standard.InMemory())
.Mappings(m => m.HbmMappings.AddFromAssembly(_mappingsAssembly))
.BuildConfiguration();
var session = cfg.BuildSessionFactory().OpenSession();
new SchemaExport(cfg).Execute(false, true, false, session.Connection, null);