Why I lose performance if I use LINQ on MongoDB? - c#

This is the situation. I have a Domain object Product like this...
[DataContract]
public class Product : IStorableEntity
{
[DataMember]
public String Id { get; set; }
[DataMember]
public String RemoteId { get; set; }
[DataMember]
public String LanguageId { get; set; }
[DataMember]
public DateTime? CreationDate { get; set; }
[DataMember]
public DateTime? LastUpdate { get; set; }
ETC..ETC...
}
into my repository layer I have the following method.
public IEnumerable<TElement> Read()
{
var mongoCollection = _mongoDatabase.GetCollection<TElement>(_partitionName);
return mongoCollection.AsQueryable<TElement>();
}
With this method I want to expose via LINQ my repository layer without exporting information about technology.
In this way I can do this:
var _repository = new MyRepositoryFactory<Product>();
var result = _repository.Read().Where(p=>p.RemoteId == "1")
this query it takes 1 or 2 milliseconds.
instead...
var _repository = new MyRepositoryFactory<Product>();
var result = _repository.Read().Where(p=>p.RemoteId == "29000")
it takes 2800 milliseconds!!!
I have correctly created a unique index with the command
db.products.ensureIndex({"RemoteId":1, unique:true})
NB: Yes, I have also rebuilt the indexes with .reIndex() command
Here the strange thing...
Avoiding LINQ and modifying the repository method in...
public IEnumerable<TElement> Read(string remoteId)
{
var mongoCollection = _mongoDatabase.GetCollection<TElement>(_partitionName);
var query = Query<TElement>.EQ(p => p.RemoteId, remoteId);
return mongoCollection.Find(query);
}
if then I invoke the method whit the same id before..
var _repository = new MyMongoRepository<Product>();
var result = _repository.Read("29000")
it takes 1 or 2 milliseconds. WHY??
Why with the first approach do I have a performance degradation as the id increases instead with the second is not it?
Ps. Erm... really sorry for my english

As WiredPrainie stated in comments you should use IQueryable instead of IEnumerable otherwise the whole collection will be retrieved.
Read this guide carefully
http://docs.mongodb.org/ecosystem/tutorial/use-linq-queries-with-csharp-driver/

Related

EF LINQ query with Contains returns not all matched records

I have a .NET Core 3.1 application with Entity Framework talking to PostgreSQL database.
I use Npgsql library 3.1.0, code-first pattern and LINQ to make queries.
So, I have a table Meetings with object like this:
public class Meeting
{
[Key]
public string Id { get; set; }
public string CreatorId { get; set; }
public List<string> Members { get; set; }
}
My query is:
var userId = "...";
using var db = new DatabaseContext();
var meetings = db.Meetings.Where(m => m.CreatorId == userId || m.Members.Contains(userId));
And it returns all records that matche first criteria: m.CreatorId == userId, but no records for the second: m.Members.Contains(userId).
This also doesn't work:
var meetings = db.Meetings.Where(m => m.Members.Contains(userId));
Returns zero records. But there are definitely matching records, because this:
var meetings = db.Meetings.ToList().Where(m => m.Members.Contains(userId));
Returns several records as expected.
Why does it happen? How can I use Contains in query like this?
Ok, I think I've figured it out myself.
According to the documentation ...Contains() query should transform to WHERE 3 = ANY(c."SomeArray") SQL operator, but there is an annotaion below:
Note that operation translation on List<> is limited at this time, but will be improved in the future. It's recommended to use an array for now.
I changed my model to:
public class Meeting
{
[Key]
public string Id { get; set; }
public string CreatorId { get; set; }
public string[] Members { get; set; }
}
and now it works.

Implementing all read api with a intercepting filter using Servicestack.Ormlite

To elaborate what I try to achieve with servicestack.ormlite. Imagine that a franchise business has some branches, each branch has system and local database, all of these database are replicating each other. In the system, each model is with a property called store_id like below.
public class UserEntity : EntityBase
{
[PrimaryKey, AutoIncrement]
public int id { get; set; }
public string user_id { get; set; }
public string name { get; set; }
public string email { get; set; }
public string password { get; set; }
public int role { get; set; }
}
public class EntityBase
{
public int store_id {get;set;}
public bool is_delete {get;set;}
}
We have 40+ entity and repos, is there any way to have all servicestack.ormlite read api filtered by store_id in one action instead of coding repo by repo ? I've a abstract repobase from which all repos are derived. And some repos needs to read all data across different store_id.
any help is much appreciated !!
This question is still unclear on what answer it wants, the screenshot says it doesn't know which API to use to filter by store_id but your screenshot includes 2 different examples of filtering by store_id?
db.Where<T>(new { store_id = _store_id });
db.Where<T>("store_id", _store_id);
Both of which should work. Although I'd recommend using the Typed version when possible, you can also use nameof() instead of magic strings:
db.Where<T>(nameof(EntityBase.store_id), _store_id);
Maybe you're after different examples of doing the same thing inside a generic repo?
You can also query using a typed SqlExpression<T>:
var q = db.From<T>().Where(x => (x as EntityBase).store_id == _store_id);
var all = db.Select(q);
Or if you want to combine it with an additional typed expression:
var q = db.From<T>().Where(x => (x as EntityBase).store_id == _store_id);
var filtered = db.Select(q.And(expr));
Since you're already using generic constraints, you can also add a constraint that the entity must be a EntityBase as well, e.g:
class RepoBase<T> where T : EntityBase, new() { ... }
That way you can query without casting, e.g:
var q = db.From<T>().Where(x => x.store_id == _store_id);
var all = db.Select(q);
and
var q = db.From<T>().Where(x => x.store_id == _store_id);
var filtered = db.Select(q.And(expr));

Access count of related records of an entity Entity Framework

I have two models:
public class HouseType
{
public int Id { get; set; }
public string TypeName { get; set; }
public virtual IEnumerable<HouseModel> HouseModels { get; set; }
}
and
public class HouseModel
{
public int Id { get; set; }
public string ModelName { get; set; }
[DisplayFormat(DataFormatString = "{0:n2}")]
public double StandardPrice { get; set; }
[ForeignKey("HouseType")]
public int HouseTypeID { get; set; }
public virtual HouseType HouseType { get; set; }
public virtual IEnumerable<HouseUnit> HouseUnits { get; set; }
}
I am returning a JSON result, so as expected I cannot manipulate it in a view, because the display is handled by a javascript file that I made.
I am trying to retrieve the number of HouseModel that is contained by HouseType. I have tried:
db.HouseTypes.Select(h => new
{
HouseCount = h.HouseModels.Count()
}).ToList();
But Entity Framework complains about it. How can I access the count of related records inside an entity? Any help will be much appreciated. Thanks.
Use
public virtual ICollection<HouseUnit> HouseUnits { get; set; }
instead of
public virtual IEnumerable<HouseUnit> HouseUnits { get; set; }
Hope this helps.
Simply speaking, the trouble is that EF is trying to execute the .Select() statement on the db server but, of course, the db server does not know how to create a new object.
You first need to bring back the counts then create your objects so something like this should work better:
var listOfCounts = db.HouseTypes
.Select(h => h.HouseModels.Count())
.ToList()
.Select(c => new
{
HouseCount = c
})
.ToList();
in this example when the first .ToList() is executed the db needs only return a set of numbers (the counts of HouseModels in each HouseType) then we have a List<int> in local memory from which we can create our objects with the second Select statement.
As an aside...
It wasn't part of your original question but maybe you'd want to consider a dictionary rather than a list so you have some means of identifying which count of HouseModels belonged to each HouseType? in which case we could do something like:
Dictionary<int,string> houseModelCounts = db.HouseTypes
.ToDictionary(h => h.Id, h => h.HouseModels.Count());
which would give a dictionary keyed with the HouseType Id with values for the count of HouseModels in each type. I don't know your context though so maybe unnecessary for you?

Linq Query where related entity contains value from array

I have a Model called JobReport which looks like this (simplified)
public class JobReport
{
public JobReport()
{
WorkOrders = new List<WorkOrder>();
}
public int JobID { get; set; }
public List<WorkOrder> WorkOrders{ get; set; }
}
public class WorkOrder
{
public WorkOrder()
{
Total = 0;
}
public string Trade { get; set; }
public int WorkOrderID { get; set; }
public decimal? Total { get; set; }
}
I'd like to run a Linq query which gets me all the Jobs that have WorkOrders that have a trade which is in a passed array.
var trades = new string[] { }
I've tried something like the following which doesn't work, as it tries to get me a list of workorders, when I actually need the underlying jobs.
The problem appears to be because I'm calling Select
var jobsDB = db.Jobs.Include(x=>x.WorkOrders).ToList();
var jobs = (from p in jobsDB
select new JobReport()
{
JobID = p.JobID,
WorkOrders = p.WorkOrders.ToList()
}
jobs = jobs
.Select(x => x.WorkOrders
.Where(y => trades.Contains(y.Trade)));
This will work:
jobs = jobs
.Where(x => x.WorkOrders.Any(y => trades.Contains(y.Trade)));
The way I usually tackle these problems is that I look at what the outcome must be (a list of jobs) - that means we need to put the Where first, and we must look for a condition for a job to be included. It's a bit like constructing a SQL query - in fact you can use the SQL like query syntax for most LINQ tasks, if you prefer.

C# - The ObjectContext instance has been disposed and can no longer be used for operations that require a connection

I'm sorry for the repetition. I'm heading with a same problem but still could not handle it. I'm using Angularjs framework and Asp.net mvc.
My person class:
public partial class Person
{
public int PersonID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string Telephone { get; set; }
public string Adress { get; set; }
public int UserID { get; set; }
public virtual User User { get; set; }
}
My User class:
public partial class User
{
public User()
{
this.People = new HashSet<Person>();
}
public int UserID { get; set; }
public string Username { get; set; }
public string Password { get; set; }
public string FullName { get; set; }
public string Email { get; set; }
public string Gender { get; set; }
public virtual ICollection<Person> People { get; set; }
}
My js code:
$http.get('/Data/GetPeople', { params: { UserID: "1" } }).success(function (data) {
$scope.model = data;
});
I'm trying to get records from my database:
public ActionResult GetPeople(int UserID)
{
using (PersonEntities dc = new PersonEntities())
{
var model = new PersonIndexVM();
model.People = dc.People.Where(b => b.UserID == UserID).ToList();
//model.People = dc.People.Include("User").ToList();
return Json(model, JsonRequestBehavior.AllowGet);
}
}
As I see with debugging, I'm getting the right objects from database in GetPeople method. But after that I'm getting this error:
'The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.'
I tried to Eagerly load: model.People = dc.People.Include("User").Where(b => b.UserID == UserID).ToList(); Still getting the same error.
It would be a pleasure if you help me.
The problem is solved. I get help from my friend. Its about unclosed connection. Its my fault. I didn't mention it.
In PersonIndexVM(), I created People = new List<Person>(); Person class is created by entity framework. It is related with database. When I create a model that is an object of PersonIndexVM() in GetPeople() method and return this model object as a json, model object try to reach User class informations after the connection closed. And I'm getting this error. To solve this problem:
Closing the lazy loading to prevent reaching User information. dc.Configuration.LazyLoadingEnabled = false;
Creating another class not related with database and return its object as Json.
You can strongly type your inclusion, which will give you hints as to whether or not your object structure is correctly related. This solves any "magic strings" issues, such as your table being named Users instead of User inside of your EF context, after including DbExtension.
using System.Data.Entity.DbExtension;
model.People = dc.People.Include(c => c.User).ToList();
However, if you are using ObjectContext instead of DbContext, you may be stuck with magic strings. That said, EF will "pluralize" most names, so your table is most likely named "Users". Check to see if the table exists as dc.Users and, if it does, change your magic string to match.
model.People = dc.People.Include("Users").ToList();
It's look like the problem is when you using keyword Using.
Look at this How to solve the error The ObjectContext
In my case, i was passsing all model 'Users' to a column, and it doesn't mapped correctly, so i just pass 'Users.Name' and it fixed it.!!
var data = db.ApplicationTranceLogs
.Include(q=>q.Users).Include(q => q.LookupItems)
.Select(q => new { Id = q.Id, FormatDate = q.Date.ToString("yyyy/MM/dd"), Users = ***q.Users***, ProcessType = q.ProcessType, CoreProcessId = q.CoreProcessId, Data = q.Data })
.ToList();
--
var data = db.ApplicationTranceLogs
.Include(q=>q.Users).Include(q => q.LookupItems)
.Select(q => new { Id = q.Id, FormatDate = q.Date.ToString("yyyy/MM/dd"), Users = ***q.Users.Name***, ProcessType = q.ProcessType, CoreProcessId = q.CoreProcessId, Data = q.Data })
.ToList();
In my case I had a bug in the front end, which was causing the same function in the backend to be triggered 3 times, causing a threading issue.
Maybe look into this as a possibility too.

Categories

Resources