Develop Reference

How to use database sharding with EF Core and C#"

I'm currently in the process of converting my 6 years old C# application to .NET Core v3 and EF Core (and also using Blazor).
Most of it is working except for the Sharding part.
Our application creates a new database for each client. We use more or less this code for it: https://learn.microsoft.com/en-us/azure/sql-database/sql-database-elastic-scale-use-entity-framework-applications-visual-studio
I'm now trying to convert it to EF Core, but get stuck at this part:
// C'tor to deploy schema and migrations to a new shard
protected internal TenantContext(string connectionString)
: base(SetInitializerForConnection(connectionString))
{
}
// Only static methods are allowed in calls into base class c'tors
private static string SetInitializerForConnection(string connnectionString)
{
// We want existence checks so that the schema can get deployed
Database.SetInitializer<TenantContext<T>>(new CreateDatabaseIfNotExists<TenantContext<T>>());
return connnectionString;
}
// C'tor for data dependent routing. This call will open a validated connection routed to the proper
// shard by the shard map manager. Note that the base class c'tor call will fail for an open connection
// if migrations need to be done and SQL credentials are used. This is the reason for the
// separation of c'tors into the DDR case (this c'tor) and the internal c'tor for new shards.
public TenantContext(ShardMap shardMap, T shardingKey, string connectionStr)
: base(CreateDDRConnection(shardMap, shardingKey, connectionStr), true /* contextOwnsConnection */)
{
}
// Only static methods are allowed in calls into base class c'tors
private static DbConnection CreateDDRConnection(ShardMap shardMap, T shardingKey, string connectionStr)
{
// No initialization
Database.SetInitializer<TenantContext<T>>(null);
// Ask shard map to broker a validated connection for the given key
var conn = shardMap.OpenConnectionForKey<T>(shardingKey, connectionStr, ConnectionOptions.Validate);
return conn;
}
The above code doesn't compile because the Database object doesn't exist in this way in EF Core.
I assume I can simplify it using TenantContext.Database.EnsureCreated(); somewhere. But I can't figure out how to modify the methods, which to remove, which to change (and how).
Of course, I've been searching for an example using sharding and EF Core but couldn't find it.
Does anybody here has done this before in EF Core and is willing the share?
I'm specifically looking for what to put in startup.cs and how to create a new sharding/database when I create a new client.

In EF.Core just resolve the shard in OnConfiguring. EG
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
var con = GetTenantConnection(this.tenantName);
optionsBuilder.UseSqlServer(con,o => o.UseRelationalNulls());
base.OnConfiguring(optionsBuilder);
}
Note that if you have a service or factory that returns open DbConnections, then you'll need to Close()/Dispose() them in the DbContext.Dispose(). If you get a connection string or a closed connection then DbContext will take care of closing the connection.
ASP.NET Core best-practices probably call for injecting an ITenantConfiguration service or somesuch in your DbContext. But the pattern is the same. Just save the injected service instance to a DbContext field and use it in OnConfiguring.

With the app that I'm working on, the desired shard is not discoverable until request time (for example, knowing what user is making the request, and then routing that user to their database). This meant that the OnConfiguring solution proposed above was not viable.
I worked around this by using IDbContextFactory<TContext>, and defining an extension on top of it, which sets the connection string based on whatever you want. I believe the database connection is created lazily in EF, and you are able to set the connection string up until the EF first needs to actually connect to the database.
In my case, it looked something like this:
var dbContext = _dbContextFactory.CreateDbContext();
var connectionString = $"DataSource={_sqlliteDirectory}/tenant_{tenant.TenantId}.db";
dbContext.Database.SetConnectionString(connectionString);
The downside is that it breaks the database abstraction (this code knows that my database is a local sqllite instance). An abstraction was not necessary in this layer of my app, but it is something very solvable if it's required.

Switch database using routes

Here's my scenario:
I Have a single app, but I need to switch the database connection by route.
Example:
switch(route)
{
case(URL/A):
{
USE DATABASE 1
}
case(URL/B):
{
USE DATABASE 2
}
DEFAULT:
USE DATABASE DEFAULT
}
Is it possible?

Since you're using ASP.NET MVC, your routes depends on your controllers. Then you can imagine having ControllerA using DatabaseA and ControllerB using DatabaseB.
To use multiple database connections, you need a connection string for each one of them.
I would use the following pieces of code to inject instances of DbContextOptionsBuilder inside of Startup.ConfigureServices()
var ContextAOptionsBuilder = new DbContextOptionsBuilder<ContextA>();
var ContextBOptionsBuilder = new DbContextOptionsBuilder<ContextB>();
Then you can configure your builders this way (depending on your parameters)
ContextAOptionsBuilder.UseSqlServer(Configuration.GetConnectionString("ContextAConnectionString"), builder =>
{
builder.EnableRetryOnFailure(5, TimeSpan.FromSeconds(30), null);
});
ContextAOptionsBuilder.EnableSensitiveDataLogging();
Then you can inject them as singletons this way :
services.AddSingleton(typeof(DbContextOptionsBuilder<ContextA>),ContextAOptionsBuilder);
You can use a BaseController, whose constructor parameters can access to services this way :
public BaseController(IConfiguration configuration, IMemoryCache memoryCache,
IHttpContextAccessor contextAccessor,
DbContextOptionsBuilder<ContextA> ContextAOptionsBuilder,
DbContextOptionsBuilder<ContextB> ContextBOptionsBuilder){}
Of course, ControllerA and ControllerB being heir classes of BaseController, you can access desired builder quite simply.
public ControllerA(IConfiguration configuration,
IMemoryCache cache,
IHttpContextAccessor contextAccessor,
DbContextOptionsBuilder<ContextA> ContextAOptionsBuilder,
DbContextOptionsBuilder<ContextB> ContextBOptionsBuilder)
:base(configuration, cache, contextAccessor, ContextAOptionsBuilder,ContextBOptionsBuilder)
{
//Create your DbContext using the builder
}
This way you can use one, the other, or both database to build your context
A simpler way would have been injecting your configuration file and building your context from it's content but ppumkin's comment suggested it's a bad idea to do this at a controller level.
This solution is working for me in an ASP.NET Core MVC application, I am still learning the framework but maybe my answer gave you precisions about multiple DbContexts.

You can create 3 connection string also 3 data access Classes. First of your class uses for example DropCreateDatabaseIfModelChanges others use CreateDatabaseIfNotExists. When you call first class your database creates when you need others there will no need recreate it.

Register your context (as scoped, per request) and use factory method for dynamically creating context with specified connection string based on current route (which should be available from HttpContext or something similar). If the databases schemas are same and just data is different this should work easily. I can't provide a snippet for you because it's mostly depends on what DI framework you have.

IdentityServer 4, Create Panel to CRUD Clients

Currently I Have configured Identityserver4 as separated project + My WebAPI and store in DB Credentials in IdentityServer.
Now i have problem how to make CRUD(In my frontend API) to IdentityServer(I want from my API add Clients to IdentityServer)
How to make property?

From IdentityServer4.EntityFramework and IdentityServer4.EntityFramework.Storage, you have access to IConfigurationDbContext (once you've added the required services in ConfigureServices using e.g. AddConfigurationStore). Because this is registered as part of the Dependency Injection system, you can take a dependency on it in one of your controllers. e.g.:
public class ClientsController : ControllerBase
{
private readonly IConfigurationDbContext _configurationDbContext;
public ClientsController(IConfigurationDbContext configurationDbContext)
{
_configurationDbContext = configurationDbContext;
}
// ...
}
IConfigurationDbContext is an abstraction of a standard DbContext, with the following DbSet<T> properties:
Clients
IdentityResources
ApiResources
It also includes both SaveChanges and SaveChangesAsync - Everything one might expect from a DbContext. Because of all of this, you can CRUD each of these entities just like any other Entity Framework Core driven database.
One final thing to note is that there are both Models (in IdentityServer4.Storage) and Entities (in IdentityServer4.EntityFramework.Storage). There are also a few extension methods for mapping between these (e.g. ClientMappers.ToEntity).
Given all of this, you can create a Model inside of your controller (or perhaps somewhere much better encapsulated than directly there). Here's a basic example for creating a new Client:
var clientModel = new Client
{
ClientId = "",
ClientName = "",
// ...
};
_configurationDbContext.Clients.Add(clientModel.ToEntity());
await _configurationDbContext.SaveChangesAsync();
The Client class here comes from IdentityServer4.Models and is then converted to an Entity using a ToEntity extension method I hinted at above. Working with a Model and converting to an Entity is simpler than trying to manipulate an Entity directly - If you're interested, you can see the mapping that takes place here.
This works in the same way for ApiResources, IdentityResources, etc. Use the source code links I've provided if you want to find out more about those specifically, but the information I've provided here should have you covered.
In order to use IdentityServer4 and IdentityServer4.EntityFramework in your API project, you can just add the two references to your API project. After that, you can configure the DI in the same way (using AddIdentityServer in ConfigureServices), but you don't need to add the middleware (using UseIdentityServer in Configure). You can even just use AddIdentityServer().AddConfigurationStore(...) to set up the relevant services, as you don't need a signing key, etc.

One way you can do this is by bootstrapping the ID4 Quickstart (tutorial located here):
http://docs.identityserver.io/en/release/quickstarts/3_interactive_login.html
Other option is to use their quickstart seeds located here to speed this up:
https://github.com/IdentityServer/IdentityServer4.Samples
Now if you want to implement restfull login there are constraints around it (i wanted to find out as well) check out this question:
IdentityServer 4 Restfull Login/Logout

Autofac, best practices on runtime adding tenants?

I am using the multitenant container and each tenant has its own database + connectionstring registered in a InstancePerLifeTime scope. The tenant is identified using a subdomain which is mapped in a "master database" with a generated database name.
Now I have two use cases:
Use Case A: Creating new Tenants:
Someone fills in a registration form with the companyname, submits, and after submission we generate a new database and that tenant should be able to access the application under companyname.domain.com
However we want to do that without restarting the application which impacts all current tenants.
Let's say I want to add a new tenant, runtime. What is the best way to register this without restarting the application?
At first I thought about registering the container, inject it in my MVC Controller, and add the new registration runtime but after reading some questions this appears to be bad practice.
I could also get the DependencyResolver from within the Controller and access the container from there. Are there better practices available?
Use Case B: Register on demand
Assuming we have a big amount of tenants and want to prevent registering them all at once on application startup. We could register these in the multitenantcontainer on the first request when the subdomain can be matched to an existing account.
This might be premature optimization though, since basically we don't have lots of tenants yet.
But again, this would result in runtime registrations.
Container:
var tenantIdentificationStrategy= new TenantIdentificationStrategy();
var multitenantContainer = new MultitenantContainer(tenantIdentificationStrategy, builder.Build());
var tenants = new[]
{
"companyA.domain",
"localhost"
};
foreach (var id in tenants)
{
var databaseName = $"tenant-{id}";
multitenantContainer.ConfigureTenant(id, b =>
{
// Init RavenDB
b.Register(context => new RavenDocumentSessionFactory(databaseName))
.InstancePerTenant()
.AsSelf();
// Session per request
b.Register(context => context.Resolve<RavenDocumentSessionFactory>()
.FindOrCreate(context.Resolve<IDocumentStore>()))
.As<IDocumentSession>()
.InstancePerLifetimeScope()
.OnRelease(x =>
{
x.SaveChanges();
x.Dispose();
});
});
}

Your best bet is to hold a static reference to the application container somewhere and register your tenants from there. This is pretty common practice and, since your tenant registration code is going to have to "know" what a MultitenantContainer is anyway, it's not going to change your assembly references or spread the "knowledge" of the container around more than it would otherwise have to be.
Create the multitenant container at app startup.
Register the tenants you already know about.
Store the container in a static property somewhere that is globally accessible.
Reference the static property when you need to register a tenant.

is there another way of changing Database Instance in Autofac

I have an application that use multiple Database.
i found out i can change that by using the connection builder. like so :
var configNameEf = "ProjectConnection";
var cs = System.Configuration.ConfigurationManager.ConnectionStrings[configNameEf].ConnectionString;
var sqlcnxstringbuilder = new SqlConnectionStringBuilder(cs);
sqlcnxstringbuilder.InitialCatalog = _Database;
but then i need to change the autofac Lifescope of UnitOfWork so that it will now redirect the request to the good Database instance.
what i found out after quite a while is that i can do it like this from a DelegatedHandler :
HttpConfiguration config = GlobalConfiguration.Configuration;
DependencyConfig.Register(config, sqlcnxstringbuilder.ToString());
request.Properties["MS_DependencyScope"] = config.DependencyResolver.GetRequestLifetimeScope();
The question is, is there any other way to do that, that change the MS_DependencyScope parametter of the request. This solution work but i think it is kind of shady.
here is the registry in DependencyConfig:
public static void Register(HttpConfiguration config, String bdContext = null)
{
var builder = new ContainerBuilder();
builder.RegisterApiControllers(Assembly.GetExecutingAssembly());
builder.Register(_ => new ProjectContext(bdContext)).As<ProjectContext>().InstancePerApiRequest();
builder.RegisterType<UnitOfWork>().As<IUnitOfWork>().InstancePerApiRequest();
// Register IMappingEngine
builder.Register(_ => Mapper.Engine).As<IMappingEngine>().SingleInstance();
config.DependencyResolver = new AutofacWebApiDependencyResolver(builder.Build());
config.DependencyResolver.BeginScope();
}

The way the question is described and the way the answer to my comment sounds, you have the following situation:
The application uses per-request lifetime units of work. I see this from your registrations.
Only one database is used in the application at a given point in time. That is, each request doesn't have to determine a different database; they all use the same one until the connection string changes. This is seen in the way the database is retrieved from using a fixed application setting.
The connection string in configuration may change, at which point the database used needs to change.
Assuming I have understood the question correctly...
If the app setting is in web.config (as it appears), then changing the string in web.config will actually restart the application. This question talks about that in more detail:
How to prevent an ASP.NET application restarting when the web.config is modified?
If that's the case, you don't have any work to do - just register the database as a singleton and when the web.config changes, the app restarts, re-runs the app startup logic, gets the new database, and magic happens.
If the app setting is not in web.config then you should probably create a project context factory class.
The factory would serve as the encapsulation for the logic of reading configuration and building the connection to the database. It'll also serve as the place to cache the connection for the times when the setting hasn't changed.
The interface would look something like this:
public interface IProjectContextFactory
{
ProjectContext GetContext();
}
A simple implementation (without locking, error handling, logging, and all the good stuff you should put in) might be:
public class ProjectContextFactory : IProjectContextFactory
{
private ProjectContext _currentContext = null;
private string _currentConnectionString = null;
private const string ConnectionKey = "ProjectConnection";
public ProjectContext GetContext()
{
// Seriously, don't forget the locking, etc. in here
// to make this thread-safe! I'm omitting it for simplicity.
var cs = ConfigurationManager.ConnectionStrings[ConnectionKey].ConnectionString;
if(this._currentConnectionString != cs)
{
this._currentConnectionString = cs;
var builder = new SqlConnectionStringBuilder(cs);
builder.InitialCatalog = _Database;
this._currentContext = new ProjectContext(builder.ToString());
}
return this._currentContext;
}
}
OK, now you have a factory that caches the built project context and only changes it if the configuration changes. (If you're not caching the ProjectContext and are, instead, caching the database connection string or something else, the principle still holds - you need a class that manages the caching and checking of the configuration so the change can happen as needed.)
Now that you have a cache/factory, you can use that in your Autofac registrations rather than a raw connection string.
builder.RegisterType<ProjectContextFactory>()
.As<IProjectContextFactory>()
.SingleInstance();
builder.Register(c => c.Resolve<IProjectContextFactory>().GetContext())
.As<ProjectContext>()
.InstancePerRequest();
The ProjectContext will now change on a per request basis when the configured connection string changes.
Aside: I see odd stuff going on with the request lifetime scope. I see in your registration that you're creating your own request lifetime scope. With this method you shouldn't have to do that. If, however, you find that you still need to (or want to), you need to make sure both the originally-created lifetime scope and the one you created are disposed. Lifetime scopes do not get automatically disposed and do hang onto object references so they can handle disposal. There is a high probability that if you're not handling this properly then you have a subtle memory leak. The Autofac Web API integration will take care of creation and disposal of the request lifetime for you, but if you change out the request lifetime, odd things are going to happen.