I am evaluating JOliver's EventStore library. In particular, I am trying to use RavenDB as the persistence engine for EventStore. EventStore comes with a plugin for this. NOTE: The database is empty with no indexes (other than the default ones).
In wiring up the store, I used the following:
var store = Wireup.Init()
.UsingRavenPersistence("EventStore", new DocumentObjectSerializer())
.Build();
However, when I run my program, I get an exception that indicates the "RavenCommitByRevisionRange" index could not be found.
In digging around the EventStore code, I think the problem happens to be that the RavenPersistenceEngine is not being initialized. The initialization code installs the needed indexes in the RavenDB server.
On the SQL server side of things, I noticed that the wiring code in the example project shows a call to an extension method called: "InitializeStorageEngine". This extension method is tied to the class "PersistenceWireup". However, the extension method I'm using to wire in RavenDB persistence returns the class "Wireup". So I wrapped part of my wireup code in a new PersistenceWireup instance, and was able to call ".InitializeStorageEngine()" like so:
var store = new PersistenceWireup(
Wireup.Init()
.UsingRavenPersistence("EventStore", new DocumentObjectSerializer()))
.InitializeStorageEngine()
.Build();
This works great! The RavenDB database now contained the necessary indexes.
So... My questions: Shouldn't ".UsingRavenPersistence(...)" return an instance of "PersistenceWireup" rather than simply "Wireup"? Or is there a better way to wire up RavenDB persistence in EventStore?
Hmmm. Well, I think that the reason raven doesn't implement PersistenceWireup is b/c that interface exposes ISerializer specific methods which the document store implementations don't use. We probably need to move the initialize to Wireup. I'll look into this.
However, I see that you're missing the dispatcher that will dispatch the event after storage. Was this intentional? If you're not planning on publishing from another enpoint, the add the Synchronous / Asynchronous dispatcher. The dispatcher is currently calling initialize.
Wireup.Init()
.UsingRavenPersistence("EventStore", new DocumentObjectSerializer())
.UsingSynchronousDispatcher()
.PublishTo(new DelegateMessagePublisher(c => PublishMessages(c))
.Build();
private IStoreEvents GetInitializedEventStore(IDispatchCommits bus)
{
return Wireup.Init()
.UsingRavenPersistence(BootStrapper.RavenDbEventStoreConnectionStringName)
.UsingAsynchronousDispatchScheduler(bus)
.Build();
}
Related
Perhaps I'm just using the wrong terms while searching, but I haven't found any solid guidance around how to do what I'm seeking to do.
All the guidance around DI registration follows something like the following:
builder.Services.AddSingleton<MyService>(() => new MyService("connectionString"));
But this seems too simple for me to use over here in the real world. I don't store my various credentials in my applications, but rather put them somewhere else like Azure Key Vault or authenticate using a managed identity that itself retrieves connection strings and keys.
This introduces the need then to access the credentials/connection string first, which increasingly is exposed only as an asynchronous operation and introduces the problem I regularly face: namely, asynchronous registration isn't a thing.
I could register a service that itself retrieves and exposes the credential in an async method, but now every downstream service is going to need to know about that method in order to utilize it - I can't simply abstract it away in a DI registration.
I could just use .Result or Wait(), but there's plenty of solid guidance that suggests this shouldn't be done for deadlocking reasons. Because this code may be intended for a library that's consumed by an app with a UI, that's a no-go.
So the question is: When I'm unable to synchronously provide my credentials, how do I register my services?
Real-world example
For example, let's say I've got a web app that needs to access Cosmos DB, but via a managed identity, following the instructions here. I need to store some information about the Cosmos DB instance which means a dependency on IConfiguration and I'd like to use a singleton HttpClient to retrieve the necessary keys.
I want to put this into a separate service responsible for setting up the Cosmos DB client so that downstream usages can just inject the CosmosClient, so my class looks like:
public class CosmosKeyService
{
private readonly MyCosmosOptions _cosmosOptions;
private readonly HttpClient _http;
public CosmosKeyService(IOptions<MyCosmosOptions> options, HttpClient http)
{
_cosmosOptions = options.Value;
_http = http;
}
private async Task<string> GetCosmosKey()
{
//Follow instructions at https://learn.microsoft.com/en-us/azure/cosmos-db/managed-identity-based-authentication#programmatically-access-the-azure-cosmos-db-keys
//...
var keys = await result.Content.ReadFromJsonAsync<CosmosKeys>();
return keys.PrimaryMasterKey;
}
public async Task<CosmosClient> GetCosmosClient()
{
var key = await GetCosmosKey();
return new CosmosClient(_cosmosOptions.CosmosDbEndpoint, key);
}
}
To support the DI used in this class, my registration then looks like:
builder.Services.Configure<MyCosmosOptions>(builder.Configuration.GetSection("cosmosdb"));
builder.Services.AddSingleton<HttpClient>();
And of course I'm going to need to register this service:
builder.Services.AddSingleton<CosmosKeyService>();
But now I'd also like to register the CosmosClient as created by the method in that service and this is where I start getting confused about the best way forward.
I can't retrieve an instance of the CosmosKeyService from the builder because I haven't yet built it, and after I do, I can't then register new services.
I can't use async methods in the registration itself or I could easily do something like:
builder.Services.AddSingleton<CosmosClient>(async services => {
var keyService = services.GetService<CosmosKeyService>();
return await keyService.GetCosmosClient();
});
...and downstream services could simply inject CosmosClient in their various constructors.
Again, any downstream consumer can just inject a CosmosKeyService, but now they're all going to have to "remember" to call the initialization method first so they can retrieve the CosmosClient and utilize it. I'd rather that be handled in registration so that 1) this initialization is hidden and centrally located and 2) the CosmosClient is truly a singleton and not just an artifact of every utilization.
I could create another intermediate service that injects this Key resolver service and retrieve the keys, but it too will need to have this async method that retrieves the keys since I can't just hide that initialization in a registration somewhere (for lack of async support).
For example, I could make another service:
public class CosmosBuilder
{
private readonly CosmosKeyService _keySvc;
public CosmosBuilder(CosmosKeyService keySvc)
{
_keySvc = keySvc;
}
public async Task<CosmosClient> GetCosmosClient()
{
return async _keySvc.GetCosmosClient();
}
}
But this ultimately still requires a downstream service to inject this service and call that initialization method and if that's necessary, I might as well just stick with injecting the CosmosKeyService and call the method there.
What I'd ideally like to see is some way to hide any async initialization in the registration so that downstream consumers can simply inject CosmosClient and it works, but it's beyond me how that's intended to happen. Can anyone shed some light on this?
Edit to address comment:
I don't want to comment on a 4-year old answer, but the issue I assert with the accepted answer boils down to this part:
Move [initialization] into the Composition Root. At that point, you can create an initialize those classes before registering them in the container and feed those initialized classes into the container as part of registrations.
That's all well and good except:
I only get to "build" my container a single time. I can't build it, then utilize the registrations to accomplish the initialization, then append still more registrations to it for later use.
In my example above, I explicitly utilize elements registered in DI by ASP.NET Core itself (namely IConfiguration), so there's simply no way to even access these except via DI (which, per #1, precludes me from being able to initialize and later supplement my registrations with more implementations).
If we set up a Saga and immediately Publish(context => ...) then a message successfully hits the bus.
If however, we have something like
Initially(
When(SomeCommand)
.Then(context => { context.Instance.SomeField = 5 })
.TransitionTo(SomeState)
.Then(context => this.RaiseEvent(context.Instance, SomeEvent)));
During(SomeState,
When(SomeEvent)
// ConsumeContext is not available here
.Publish(context => new SomeEventClass
{
Foo = context.Instance.SomeField
})
.Finalize());
The machine also never transitions to the Final state presumably because of the exception locating a ConsumeContext.
We have seen some references to passing a ConsumeContext as a parameter in Publish() however it's unclear as to which context this needs (Intellisense just makes reference to context1, context2, context3, etc).
Is there a way to use Publish() after RaiseEvent() has already been called? If not, is there a way to publish an event using some other mechanism?
MassTransit version is 5.1.5 and Automatonymous is 4.1.2
EDIT Based on Chris Patterson's answer here we have tried adding the below outside of any Initially or During
WhenEnter(NewState, state => state.Publish(context =>
new EventClass { Foo = context.Instance.Foo }
)
.Finalize();
However it still doesn't publish anything and the state never transitions to Final. If we add a Then it also never hits this code block. There don't seem to be any exceptions occurring. We also tried using When(SomeState.Enter) and it also doesn't work.
Side question as maybe this will help with my understanding of why Publish and RaiseEvent don't seem to play nicely together - why does Publish need the ConsumeContext? Is it to locate the bus and bindings?
You should be able to add
When(SomeState.Enter)
to your Initially section, and it will use the existing context. By creating a new EventContext with RaiseEvent, you aren't retaining the ConsumeContext.
You could also add WhenEnter(SomeState) outside of any Initially or During block.
The solution to this turned out to be using this:
.Then(context => context.Raise(SomeEvent))
instead of this:
.Then(context => this.RaiseEvent(context.Instance, SomeEvent))
It kinda makes sense now - the latter (as mentioned by Chris Patterson) creates a new EventContext while the former uses the given context. We didn't know that Raise was a method available on the passed in context.
I am new to NHibernate and even newer to MOQ (or other similar frameworks). After searching online day and night (google + stackoverflow + others), I am turning here for help.
The scenario is (should be) simple. I am trying to unit test a call on a C# WCF service that uses NHibernate as the ORM layer. The method, after doing some initial work, finds a database to connect to, and then calls on the SessionProvider (a manager of session factories) to return a nhibernate session for a sharded DB. I am then trying to use ISession.Get<>() to retrieve an object from the database aand then do some work. The problem is that the GUID (the key for the entry that I am looking up in the db) is generated at the begining of the call and I have no way of knowing what it might be beforehand outside the scope of the WCF call. Hence, I cannot use sqllite or other techniques to pre-populate the necessary data to control the test. What I was hoping for was that I can somehow mock (inject a fake layer to?) the call to Session.Get to return an invalid object which should cause the WCF call to throw.
Here's the test code snippet:
var testRequest = ... (request DTO)
var dummyBadObject = ... (entity in DB)
var mock = new Mock<ISession>(MockBehavior.Strict);
mock.Setup(m => m.Get<SampleObject>(It.IsAny<Guid>())).Returns(dummyBadObject);
var exception = Assert.Throws<FaultException>(() => applicationService.SomeMethod(testRequest));
Assert.AreEqual(exception.Code.ToString(), SystemErrorFault.Code.ToString());
When I run this test, instead of interacting with the mock ISession object, the app service code calls the Get on the actual ISession object from the session factory, connects to the database and gets the right object. Seems like I am missing something very basic about mocks or injection. Any help will be appreciated.
Thanks,
Shawn
Based on our comments, the problem is that mocks are completely different from how you thought of them.
They don't magically intercept creations of classes derived from an interface. They are just dynamic implementations of it.
Creating a Mock<ISession> is not much different from creating a class that implements ISession. You still have to inject it in the services that depend on it.
You'll probably have to review your whole stack, as the capability to do this depends on a good decoupled design.
Suggested read: Inversion of control
I re-designed the components in my application to have a ServiceContext object which in turns holds all the other (what used to be static) components that the application uses. In this case, this would be the session provider (or the ISessionFactory cache), and similarly a WCF channel factory cache. The difference is that the ServiceContext provides methods to override the default instances of the different components allowing me to replace them with mock ones for testing and restoring the original ones when testing is done. This has allowed me to create a test where I mock all the way from the session cache to the ISession.Get/Save/Load etc.
var mockDatabaseSessionFactory = new Mock<DatabaseSessionManager>(MockBehavior.Strict);
var mockSession = new Mock<ISession>(MockBehavior.Strict);
var mockTransaction = new Mock<ITransaction>(MockBehavior.Strict);
mockDatabaseSessionFactory.Setup(x => x.GetIndividualMapDbSession()).Returns(mockSession.Object);
mockDatabaseSessionFactory.Setup(x => x.GetIndividualDbSession(It.IsAny<UInt32>())).Returns(mockSession.Object);
mockDatabaseSessionFactory.Setup(x => x.Dispose());
mockSession.Setup(x => x.BeginTransaction()).Returns(mockTransaction.Object);
mockSession.Setup(x => x.Dispose());
mockTransaction.Setup(x => x.Commit());
mockTransaction.Setup(x => x.Dispose());
// Setups to allow for the map insertion/deletion to pass
mockSession.Setup(x => x.Get<IndividualMap>(It.IsAny<string>())).Returns((IndividualMap)null);
mockSession.Setup(x => x.Load<IndividualMap>(It.IsAny<string>())).Returns((IndividualMap)null);
mockSession.Setup(x => x.Save(It.IsAny<IndividualMap>())).Returns(new object());
mockSession.Setup(x => x.Delete(It.IsAny<IndividualMap>()));
// Our test condition for this test: throw on attempt to save individual
mockSession.Setup(x => x.Save(It.IsAny<Individual>()))
.Throws(new FaultException(ForcedTestFault.Reason, ForcedTestFault.Code));
// Test it - but be sure to back up the previous database session factory
var originalDbSessionFactory = ServiceContext.DatabaseSessionManager;
ServiceContext.OverrideDatabaseSessionManager(mockDatabaseSessionFactory.Object);
try
{
var exception = Assert.Throws<FaultException>(() => applicationService.AddIndividual(addIndividualRequest));
Assert.IsTrue(ForcedTestFault.Code.Name.Equals(exception.Code.Name));
}
catch (Exception)
{
// Restore the original database session factory before rethrowing
ServiceContext.OverrideDatabaseSessionManager(originalDbSessionFactory);
throw;
}
ServiceContext.OverrideDatabaseSessionManager(originalDbSessionFactory);
ServiceContext.CommunicationManager.CloseChannel(applicationService);
Luckily the code design wasn't too bad o_O :) so i re-factored this bit easily and now code coverage is at 100! Thanks Diego for nudging me in the right direction.
I created an application that provides several services. Each service provides a specific processing capabilities, except one service (that is the main service) that returns true or false to the clients which request if the specified processing capabilities is available or not.
Now I would modify the application, leaving the main service unchanged and adding the support for the installation of plugin with new processing capabilities: each plugin should add new processing capabilities without the need of implement a new service, but after installing the new plugin, a new service should be avaible. In this way, a plugin should not handle the communication layer. In other words, I would like to separate the layer of the communication and processing, in order to simplify the creation of new plugins.
Is it possible?
I could create two services: the main service and the service for processing.
The first service may be used by clients to know if a certain feature is present on the server (for example, clients may ask the server if it has installed the plugin that provides the functionality for solving differential equations).
The second service could be used to send a generic task and to receive a general result, for example:
Result executeTask(Task task);
where Result and Task are abstract classes...
For example, if I develop a plugin to solve the differential equations, I first create the classes for transferring data:
public class DifferentialEquationTask : Task
// This class contains the data of the differential equation to be solved.
...
public class DifferentialEquationResult : Result
// This class contains the the result.
...
Therefore, the client should instantiate a new object DifferentialEquationTask and pass it to the method of the second service:
DifferentialEquationTask myTask = new DifferentialEquationTask(...);
...
Result result = executeTask(myTask); // called by basic application
// The second service receives myTask as a Task object.
// This Task object also contains the destination plugin, so myTask is send
// to the correct plugin, which converts it to DifferentialEquationTask
...
myResult = result as DifferentialEquationResult;
// received by the client
Moreover, each plugin should have a version for the application server and a version for the client application.
An alternative would be to include the service in the plugin itself: in this way, a new plugin should implement a new functionality and expose it via an additional service.
In summary, I thought the following two alternatives:
a main service to ask the server if it has a plugin or not, and a second service to deliver tasks at the correct plugin;
a main service to ask if the server has a plugin or not, and various additional services (an additional service for each plugin installed).
In order to choose the best approach, I could use the following requirements:
Which of the two alternatives may provide better performance?
What advantages would be obtained using a new service for each plugin than using a single service that delivers tasks at the correct plugin?
Which of the two alternatives simplifies the development of a new plugin?
Being a novice, I was wondering if there was a better approach...
Thanks a lot!
It seems like the main service could maintain a dictionary of plugins, indexed by name. Then for a client to see if the server provides a particular service, all the main service has to do is look up the name in the dictionary. And to process, the service just has to call a method on the object that's in the value portion of the dictionary entry. An example:
You have three abstract classes: Service, ServiceResult, and ServiceTask. The contents of ServiceTask and ServiceResult aren't really important for this discussion. Service must have a parameterless constructor and a method called Process that takes a ServiceTask as its sole parameter. So your differential equation solver would look like:
public class DiffeqSolver : Service
{
public DiffeqSolver()
{
// do any required initialization here
}
public ServiceResult Process(ServiceTask task)
{
DiffeqTask dtask = task as DiffeqTask;
if (dtask == null)
{
// Error. User didn't pass a DiffeqTask.
// Somehow communicate error back to client.
}
// Here, solve the diff eq and return the result.
}
}
The main service is somehow notified of existing plugins. It maintains a dictionary:
Dictionary<string, Service> Services = new Dictionary<string, Service>();
I assume you have some idea how you're going to load the plugins. What you want, in effect, is for the dictionary to contain:
Key = "DiffeqSolver", Value = new DiffeqSolver();
Key = "ServiceType1", Value = new ServiceType1();
etc., etc.
You can then have two methods for the main service: ServiceIsSupported and Process:
bool ServiceIsSupported(string serviceName)
{
return Services.ContainsKey(serviceName);
}
ServiceResult Process(string serviceName, ServiceTask task)
{
Service srv;
if (Services.TryGetValue(serviceName, out srv))
{
return srv.Process(task);
}
else
{
// The service isn't supported.
// Return a failure result
return FailedServiceResult;
}
}
I've simplified that to some extent. In particular, I'm using a Dictionary, which is not thread safe. You'd want to use a ConcurrentDictionary, or use locks to synchronize access to your dictionary.
The more difficult part, I think, will be loading the plugins. But there are many available examples of creating a plugin architecture. I think you can find what you need.
I am trying to create a sample app using OData and Silverlight, using (what else?) the Netflix service. I've already succeeded in creating the app using WPF, but am struggling to port my service class to an async model.
My existing service class (simplified) looks like this:
public IEnumerable<Title> BlockingSearch(TitleSearchParam param)
{
var catalog = new NetflixCatalog(new Uri("http://odata.netflix.com/Catalog/"));
return catalog.Titles.Where(t =>
t.Instant.AvailableFrom > param.InstantStartDate && t.Instant.AvailableFrom < param.InstantEndDate &&
(string.IsNullOrEmpty(param.TitleName) || t.Name.Contains(param.TitleName))).ToList();
}
All of the examples of consuming OData asynchronously employ some kind of Respository Pattern and/or require an instantiated collection to be passed in. I would like to model the method signature for the Silverlight/Async call to look something like this (with the service class itself being stateless):
public void AsyncSearch(TitleSearchParam param, Action<IEnumerable<Title>> completedCallback, Action<MyErrorClass> errorCallback, object callBackOwner)
{
}
I think I could so something along the lines of what MS outlines in Calling Synchronous Methods Asynchronously, but I was hoping there was a more elegant solution that I was missing.
As you already know, you can always run the call on a different thread than the UI thread, so there will be no blocking per se. That seems like a decent enough solution. Then you can (in fact, must) use dispatchers to handle results coming from callbacks if you intend to use them with UI.