Develop Reference

C# MongoDB, LINQ unable to match by Id

I'm using MongoDB.Driver (2.12.4) while trying to stay away from Bson documents. Apart from the Mongo collection methods I use LINQ for all of my queries.
I have a baseclass like this:
public abstract class Entity
{
public Guid Id { get; protected set; }
}
And a subclass like this:
public class User : Entity
{
public string Name { get; set; }
public string Email { get; set; }
}
public User(string name = "", string email = "")
{
Id = Guid.NewGuid();
Name = name;
Email = email;
}
And my database class is initialized like this:
class Database
{
const string MongoConnection = "mongodb+srv://user:password#cluster0.mongodb.net/";
public static MongoClient Client { get; private set; }
public static IMongoDatabase Directory { get; private set; }
public static IMongoCollection<User> Users { get; private set; }
public Database()
{
BsonSerializer.RegisterSerializer(new GuidSerializer(GuidRepresentation.Standard));
Client = new MongoClient(MongoConnection);
Directory = Client.GetDatabase("DB");
Users = Directory.GetCollection<User>("users");
}
}
What has been working so far:
Database.Users.InsertOne((new User("name", "email#domain.com")); //cloud.mongodb.com shows {_id : UUID('a12937ba-7a7c-4c5d-a8ff-4049e9878ca3')}
var document = Database.Users.AsQueryable().Where(x => x.Email == "email#domain.com").FirstOrDefault();
Database.Users.AsQueryable().ToList().Where(x => x.Id == document.Id).FirstOrDefault()
But none of the following is working:
Database.Users.Find(x => x.Id == document.Id).FirstOrDefault(); //returns null
Database.Users.AsQueryable().Any(x => x.Id == document.Id); //returns false
Database.Users.ReplaceOne(x => x.Id == document.Id, document); //returns MatchedCount = 0
So it seems that as soon as I use .ToList then everything works as expected. But if I use .AsQueryable, .Find, .Any or .ReplaceOne directly on the collection then it's not able to perform a match between Ids.
I am not able to find a similar question that does not use [BsonId] or [BsonElement] and does not make use of filters and builders. If I understand correctly the MongoDB.Driver is supposed to do the mapping automatically, and for every property it works except the Id. What am I missing?
PS: I am not using Entity Framework

you need to add this on start of the app:
BsonDefaults.GuidRepresentation = GuidRepresentation.Standard;
this makes the query possible with the Guid Id.
it does have a comment of "obsolete" on the latest driver but i guess that's a bug.

EF LINQ query with Contains returns not all matched records

I have a .NET Core 3.1 application with Entity Framework talking to PostgreSQL database.
I use Npgsql library 3.1.0, code-first pattern and LINQ to make queries.
So, I have a table Meetings with object like this:
public class Meeting
{
[Key]
public string Id { get; set; }
public string CreatorId { get; set; }
public List<string> Members { get; set; }
}
My query is:
var userId = "...";
using var db = new DatabaseContext();
var meetings = db.Meetings.Where(m => m.CreatorId == userId || m.Members.Contains(userId));
And it returns all records that matche first criteria: m.CreatorId == userId, but no records for the second: m.Members.Contains(userId).
This also doesn't work:
var meetings = db.Meetings.Where(m => m.Members.Contains(userId));
Returns zero records. But there are definitely matching records, because this:
var meetings = db.Meetings.ToList().Where(m => m.Members.Contains(userId));
Returns several records as expected.
Why does it happen? How can I use Contains in query like this?

Ok, I think I've figured it out myself.
According to the documentation ...Contains() query should transform to WHERE 3 = ANY(c."SomeArray") SQL operator, but there is an annotaion below:
Note that operation translation on List<> is limited at this time, but will be improved in the future. It's recommended to use an array for now.
I changed my model to:
public class Meeting
{
[Key]
public string Id { get; set; }
public string CreatorId { get; set; }
public string[] Members { get; set; }
}
and now it works.

Linq to Entities - is it possible to achieve this in single query to DB?

I have multiple entity objects which I would like to secure by using custom Access Lists (ACL) stored in a SQL Server database. All my "securables" implement an ISecurable interface:
public interface ISecurable
{
int ACLId { get; }
ACL ACL { get; }
}
AccessList (ACL) entity looks like this:
public class ACL
{
public int Id { get; set; }
public virtual ICollection<ACE> ACEntries { get; set; }
}
... and each ACL has multiple ACE entries:
public class ACE
{
public int Id { get; set; }
public int ACLId { get; set; }
public virtual ACL ACL { get; set; }
public int PrincipalId { get; set; }
public virtual Principal Principal { get; set; }
public int AllowedActions { get; set; } // flags
}
Actions can be granted both to Users and to Workgroups.
Principal is an abstract class, and both User and Workgroup inherits from it:
public abstract class Principal
{
public int Id { get; set; }
public string DisplayName { get; set; }
public virtual ICollection<ACE> ACEntries { get; set; }
}
public class User : Principal
{
public string Email { get; set; }
public virtual ICollection<Workgroup> Workgroups { get; set; }
}
public class Workgroup : Principal
{
public virtual ICollection<User> Users { get; set; }
}
User and Workgroup are obviously in many-to-many relation.
My DbContext is looks like this:
public class MyDbContext : DbContext
{
public DbSet<User> Users { get; set; }
public DbSet<Workgroup> Workgroups { get; set; }
public DbSet<ACL> ACLs { get; set; }
public DbSet<ACE> ACEntries { get; set; }
public DbSet<SecurableClass> SecurableClassItems { get; set; }
}
Finally, my question : I would like to write extension method to filter out all my ISecurable classes by ACL, based on user and required action. And I would like to have single query to the DB:
public static IQueryable<ISecurable> FilterByACL(this IQueryable<ISecurable> securables, User user, int requiredAction)
{
var userId = user.Id;
return securables.Where(s =>
s.ACL.ACEntries.Any(e =>
(e.PrincipalId == userId || user.Workgroups.Select(w => w.Id).Contains(userId)) &&
(e.AllowedActions & requiredAction) == requiredAction));
}
This does not work, due to error:
Unable to create a constant value of type 'Test.Entities.Workgroup'. Only primitive types or enumeration types are supported in this context
even though I select only IDs from Workgroup:
user.Workgroups.Select(w => w.Id)
Is there any way to handle this scenario ? Sorry for long question, but existing simplified code will probably best explain my intention.
UPDATE
After #vittore suggestion, my extension method might be something like this:
public static IQueryable<ISecurable> FilterByACL2(this IQueryable<ISecurable> securables, User user, int requiredAction, MyDbContext db)
{
var userId = user.Id;
var workgroups = db.Entry(user).Collection(u => u.Workgroups).Query();
return securables.Where(s =>
s.ACL.ACEntries.Any(e =>
(e.PrincipalId == userId || workgroups.Select(w => w.Id).Contains(e.PrincipalId)) &&
(e.AllowedActions & requiredAction) == requiredAction));
}
.. but I'll have to reference 'MyDbContext' as additional parameter ?
On the other side, if I write SQL function (TVF) which will return all allowable ACLIDs based on 'UserId' and 'RequiredAction':
CREATE function [dbo].[AllowableACLs]
(
#UserId int,
#RequiredAction int
)
returns table
as
return
select
ace.ACLId
from
dbo.ACEntries ace
where
(#UserId = ace.PrincipalId or #UserId in (select wu.User_Id from dbo.WorkgroupUsers wu where ace.PrincipalId = wu.Workgroup_Id))
and
((ace.AllowedActions & #RequiredAction) = #RequiredAction)
.. theoretically I could just make 'inner join' or 'exist (select 1 from dbo.AllowableACLs(#UserId, #RequiredAction)' from my ISecurable Entity.
Is there any way to get this working from code-first ?

You need to move "user.Workgroups.Select..." out of expression. EF tries to translate that expression to sql and fails because "user" is local variable. So, this should help to fix the error:
public static IQueryable<ISecurable> FilterByACL(this IQueryable<ISecurable> securables,
User user, int requiredAction)
{
var userId = user.Id;
var groupIds = user.Workgroups.Select(w => w.Id).ToArray();
return securables.Where(s =>
s.ACL.ACEntries.Any(e =>
(e.PrincipalId == userId || groupIds.Contains(userId)) &&
(e.AllowedActions & requiredAction) == requiredAction));
}
Note that to ensure this is only one query you need to load user's Workgroups property when retrieving user from database. Otherwise, if you have lazy loading enabled, this will first load Workgroups for given user, so 2 queries.

There are number of options you have in order to get that information in one trip.
the most obvious one would be to use join and linq syntax:
Something close to:
var accessbleSecurables = from s in securables
from u in ctx.Users
from g in u.Workgroups
from e in s.ACL.entries
where u.id == userId ...
select s
so it will use user and her groups from db while querying securables
convert your workgroups to the query, using CreateSourceQuery
This way you will let EF know that it needs to do your contains on db level.
UPDATE: CreateSourceQuery is old name from EF3 - EF4 days. Right now you can use DbCollectionEntry.Query() method, like that:
var wgQuery = ctx.Entry(user).Collection(u=>u.Workgroups).Query();
Then you'll be able to use that in your query, but you need reference to EF context in order to do that.
Create store procedure that will do the same sql query
UPDATE 2: One of the unique PROs of creating store procedure is ability to map multiple return result sets for it (MARS). This way you might for instance query all workgroups AND all securables for a user in one trip to db.

user.Workgroups is a local variable consisting of a sequence of c# objects. There is no translation of these objects into SQL. And EF requires this translation because the variable is used within an expression.
But it is simple to avoid the exception: first create a list of the primitive ID values:
var workGroupIds = user.Workgroups.Select(w => w.Id).ToArray();
and in the query:
e.PrincipalId == userId || workGroupIds.Contains(userId)
After your comments it seems that you can also do...
return securables.Where(s =>
s.ACL.ACEntries.Any(e =>
(e.PrincipalId == userId
|| e.Principal.Workgroups.Select(w => w.Id).Contains(userId))
&& (e.AllowedActions & requiredAction) == requiredAction));
...which would do everything in one query.

How can I extend the EF Repository Model to do more complex queries?

I have the following:
public partial class Subject
{
public Subject()
{
this.Contents = new List<Content>();
}
public int SubjectId { get; set; }
public string Name { get; set; }
public virtual ICollection<Content> Contents { get; set; }
}
public partial class Content
{
public int ContentId { get; set; }
public int ContentTypeId { get; set; }
public string Title { get; set; }
public string Text { get; set; }
public int SubjectId { get; set; }
public virtual Subject Subject { get; set; }
}
In my SQL Server database I have an index on the Content table of SubectId and ContentTypeId
My classes are working find with a standard repository that has such methods such as GetAll() and GetId(id) however using the repository model is there a way I can do more complex queries. In this case I would somehow want todo a query for a particular SujectId and a contentTypeId. What I want to avoid is having a query that gets every content record and then filters out what I need. I'd like some way to send a real query of exactly what I need to SQL Server.
Currently my generic repository has the following:
public virtual T GetById(int id)
{
return DbSet.Find(id);
}
Could I do what I need by implementing creating a ContentRepository and having something like the following:
public IQuerable<Content> GetAllBySubjectId(int id)
{
return DbSet.Where(c => c.SubjectId == id);
}
If so then how could I use the GetAllBySubjectId and add in the check for where ContentId == "01" for example?

You could add to your repository a method like this:
public IQueryable<T> Find(Expression<Func<T, bool>> predicate)
{
return DbSet.Where<T>(predicate);
}
Then write sth like:
repository.Find(c => c.SubjectId == myId);

If you use Entity Framework with LINQ, it will try to generate and send optimized queries to the database, for example, if you do something like:
var contents =
from c in Context.Contents // or directly the DbSet of Contents
where c.ContentTypeId == 2
select new { c.Title, c.ContentId };
It should generate a query along the lines of the following (you can use a SQL Profiler):
select
c.Title as Title,
c.ContentId as ContentId
from Contents c
where
c.ContentTypeId == 2
There are some considerations to think of, but most of the time EF will generate good performance queries.
To know more about that, I recommend the following URL: http://www.sql-server-performance.com/2012/entity-framework-performance-optimization/

OrderBy embedded documents

I'm trying to order my results by the value in an embedded document.
Consider a model such as:
public class Car
{
public Guid ID { get; set; }
public string Name { get; set; }
public IEnumerable<Passenger> Passengers { get; set; }
}
public class Passenger
{
public Guid ID { get; set; }
public virtual string Name { get; set; }
public virtual int Age { get; set; }
}
I'm trying to query my Car collection, and order by Passenger.Age
My query looks something like:
var results = (from car in _db.GetCollection<Car>("car").AsEnumerable()
from passenger in car.Passengers
where car.Name == "Ford"
orderby passenger.Age).ToList();
With this, I get the following exception:
The SelectMany query operator is not supported.
This is understandably a limitation of the C# mongo driver.
Is there a workaround?
Failing that, how could I order them after my .ToList() ?

You can probably re-write this with AsQueryable() to get an IEnumerable collection from ToList() back, from which you can then further query with any LINQ you want to use, not just the operations directly supported by the MongoCollection:
var passengers = _db.GetCollection<Car>("car").AsQueryable().ToList()
.Where(car => car.Name == "Ford")
.SelectMany(ford => ford.Passengers)
.OrderBy(p => p.Age);
Here's where you can find the directly supported LINQ operations for the MongoDb C# driver.