Develop Reference

Sharing the same scope in Masstransit mediator produces error: The ConsumeContext was already set

I'm trying to use MassTransit mediator on my Asp.Net web API. Currently I'm using Masstransit 7.3.0
I wanted for consumers to share the same scope. So I did everything step-by-step as in documentation.
Here I'm setting up the mediator in startup and adding the consumers, requestclients:
services.AddHttpContextAccessor();
services.AddMediator(cfg =>
{
cfg.ConfigureMediator((regContext, options) =>
{
options.UseHttpContextScopeFilter(regContext);
});
cfg.AddConsumer<BillingFile.Create>();
cfg.AddRequestClient<BillingFile.Command.CreateCommand>();
cfg.AddConsumer<Payment.CreateBasePayment>();
cfg.AddRequestClient<Payment.Command.CreateBasePaymentCommand>();
});
I'm using HttpContextScopeFilter directly from the documentation on the website.
When I try to send a message from one consumer to the other, an exception is thrown:
The ConsumeContext was already set
Here is my Create consumer:
public class Create : IConsumer<CreateCommand>
{
private readonly IMediator mediator;
public Create(IMediator mediator)
{
this.mediator = mediator;
}
public async Task Consume(ConsumeContext<CreateCommand> context)
{
var request = context.Message.Request;
/* Create base payment <--- here be the exception */
var basePaymentHandler = mediator.CreateRequestClient<CreateBasePaymentCommand>(context);
var basePaymentResponse = await basePaymentHandler.GetResponse<CreateBasePaymentResponse>(new CreateBasePaymentCommand { Request = Request, CorrelationId = Guid.NewGuid()}, cancellationToken: context.CancellationToken);
....
Is this not allowed by Masstransit mediator?

MassTransit won't let you dispatch multiple messages in the same container scope. It results in the very exception you are experiencing. You're also using an obsolete/unsupported version of MassTransit.

How to authenticate a user when consuming MassTransit messages in Asp.Net Core Web API?

I have several Asp.Net Core Web APIs that use Bearer authentication and IdentityServer4.AccessTokenValidation middleware to introspect tokens, authenticate the user and create claims. This works fine for HTTP requests.
I am in the process of configuring these APIs to also be MassTransit endpoints (for both Publishing and Consuming messages) using RabbitMQ as transport. I followed the instructions here for adding MassTransit to the API and for setting up message consumers. A typical workflow will be something like:
HTTP Request to API > Publish message on MassTransit > RabbitMQ > Message consumed in another API
What I'm struggling to understand is how I can create a ClaimsPrincipal when consuming messages off the bus so that I know which user to perform actions on behalf of? Where it's not an HTTP request there is no AuthenticationHandler being invoked.
What I've tried so far:
I thought I'd approach this by passing a token (and/or individual claim values) in message headers. The publish part seemed easily enough as MassTransit allows adding any number of custom headers when publishing messages using MassTransit.PublishContextExecuteExtensions.Publish. This allowed me to get messages onto the transport with information identifying a user and this info can be viewed in a consumer by manually viewing the headers e.g.
public class SomeEventConsumer : IConsumer<SomeEventData>
{
public async Task Consume(ConsumeContext<SomeEventData> context)
{
var token = context.Headers["token"];
}
}
At this point I could take the token and call the Introspection endpoint in Identity Server manually but then I'd need to:
Do this in every consumer every time and then ...
... pass that information down to logic classes etc manually instead of making use of IHttpContextAccessor.HttpContext.User.Claims or by wrapping the claims and using Dependency Injection.
To address point 1 I created a new custom middleware ...
public class AuthenticationFilter<T> : IFilter<ConsumeContext<T>> where T : class
{
public void Probe(ProbeContext context)
{
var scope = context.CreateFilterScope("authenticationFilter");
}
public async Task Send(ConsumeContext<T> context, IPipe<ConsumeContext<T>> next)
{
var token = context.Headers.Where(x => x.Key == "token").Select(x => x.Value.ToString()).Single();
// TODO: Call token introspection
await next.Send(context);
}
}
public class AuthenticationFilterSpecification<T> : IPipeSpecification<ConsumeContext<T>> where T : class
{
public void Apply(IPipeBuilder<ConsumeContext<T>> builder)
{
var filter = new AuthenticationFilter<T>();
builder.AddFilter(filter);
}
public IEnumerable<ValidationResult> Validate()
{
return Enumerable.Empty<ValidationResult>();
}
}
public class AuthenticationFilterConfigurationObserver : ConfigurationObserver, IMessageConfigurationObserver
{
public AuthenticationFilterConfigurationObserver(IConsumePipeConfigurator receiveEndpointConfigurator) : base(receiveEndpointConfigurator)
{
Connect(this);
}
public void MessageConfigured<TMessage>(IConsumePipeConfigurator configurator)
where TMessage : class
{
var specification = new AuthenticationFilterSpecification<TMessage>();
configurator.AddPipeSpecification(specification);
}
}
public static class AuthenticationExtensions
{
public static void UseAuthenticationFilter(this IConsumePipeConfigurator configurator)
{
if (configurator == null)
{
throw new ArgumentNullException(nameof(configurator));
}
_ = new AuthenticationFilterConfigurationObserver(configurator);
}
}
... and then added that into the pipeline ...
IBusControl CreateBus(IServiceProvider serviceProvider)
{
return Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host("rabbitmq://localhost");
cfg.UseAuthenticationFilter();
// etc ...
});
}
And this is where I'm stuck. I don't know how to authenticate the user for the scope of the request. Where it's not an HTTP request I'm not sure what best practice is here. Any suggestions or pointers would be gratefully received. Thanks...

I've just been watching a Kevin Dockx course on Pluralsight that covers this scenario on Azure Service Bus, but the same principal would apply to Mass Transit or any other asynchronous communication between services using a message bus. Here's a link to the section: Securing Microservices in ASP.NET Core
Kevin's technique is to include the access token (JWT) as a property on the bus message and to then validate this in the consumer using IdentityModel.
To summarise:
In the Producer:
Get the Access Token from the request (e.g. HttpContext.GetUserAccessTokenAsync()).
Set this as a property in the message before sending.
In the Consumer:
Use IdentityModel to get the IdP Discovery Document
Extract the public signing keys from the discovery response (these must be converted to RsaSecurityKey)
Call JwtSecurityTokenHandler.ValidateToken() to validate the JWT from the message. This returns a ClaimsPrincipal if successful.
If you're concerned about Access Token expiration, you can make use of the datetime that the message was enqueued as part of the token validation logic in the consumer.
Here's how the validator works (simplified):
var discoveryDocumentResponse = await httpClient.GetDiscoveryDocumentAsync("https://my.authority.com");
var issuerSigningKeys = new List<SecurityKey>();
foreach (var webKey in discoveryDocumentResponse.KeySet.Keys)
{
var e = Base64Url.Decode(webKey.E);
var n = Base64Url.Decode(webKey.N);
var key = new RsaSecurityKey(new RSAParameters
{ Exponent = e, Modulus = n })
{
KeyId = webKey.Kid
};
issuerSigningKeys.Add(key);
}
var tokenValidationParameters = new TokenValidationParameters()
{
ValidAudience = "my-api-audience",
ValidIssuer = "https://my.authority.com",
IssuerSigningKeys = issuerSigningKeys
};
var claimsPrincipal = new JwtSecurityTokenHandler().ValidateToken(tokenToValidate,
tokenValidationParameters, out var rawValidatedToken);
return claimsPrincipal;

Communication between Topshelf service (acting as TCP server) and selfhosted OWIN WebAPI

I have a Topshelf windows service that acts as a TCP server. Inside this service, I also have a self-hosted (OWIN) WebAPI.
My goal is to somehow allow the WebAPI to communicate with the TCP server that's contained and running in the same service. Naturally I could simply use something like a "trigger" file or a shared DB that could be polled frequently, though I'd like to know of any more optimal/native ways to achieve this.
To get a better idea of the project, think of a single page application consuming my API and making certain calls with arbitrary string parameters. This data should then be passed to clients (C++ console apps using winsock) that are connected to the running TCP server.
The following Container is instantiated and passed to the Topshelf HostConfigurator
class ContainerService
{
private APIService _apiService;
private EngineService _engineService;
protected IDisposable WebAppHolder { get; set; }
public bool Start(HostControl hostControl)
{
var host = hostControl;
_apiService = new APIService();
_engineService = new EngineService();
// Initialize API service
if (WebAppHolder == null)
{
WebAppHolder = _apiService.Initialize();
}
// Initialize Engine service
_engineService.Initialize();
return true;
}
public bool Stop(HostControl hostControl)
{
// Stop API service
if (WebAppHolder != null)
{
WebAppHolder.Dispose();
WebAppHolder = null;
}
// Stop Engine service
_engineService.Stop();
return true;
}
}
Standard Topshelf stuff in program entry point (as mentioned above):
HostFactory.Run(hostConfigurator =>
{
hostConfigurator.Service<ContainerService>(containerService =>
{
containerService.WhenStarted((service, control) => service.Start(control));
containerService.WhenStopped((service, control) => service.Stop(control));
});
hostConfigurator.RunAsLocalSystem();
hostConfigurator.SetServiceName("Educe Service Host");
hostConfigurator.SetDisplayName("Communication Service");
hostConfigurator.SetDescription("Responsible for API and Engine services");
});
TCP Server:
public void Initialize()
{
_serverListener = new TcpListener(new IPEndPoint(hostAddress, (int)port));
_serverListener.Start();
_threadDoBeginAcceptTcpClient = new Thread(() => DoBeginAcceptTcpClient(_serverListener));
_threadDoBeginAcceptTcpClient.Start();
}
...
public void DoBeginAcceptTcpClient(TcpListener listener)
{
while(!_breakThread)
{
// Set the event to nonsignaled state.
TcpClientConnected.Reset();
// Start to listen for connections from a client.
Console.WriteLine("Waiting for a connection...");
// Accept the connection.
listener.BeginAcceptTcpClient(DoAcceptTcpClientCallback, listener);
// Wait until a connection is made and processed before continuing.
TcpClientConnected.WaitOne();
}
}
// Process the client connection.
public void DoAcceptTcpClientCallback(IAsyncResult ar)
{
// Get the listener that handles the client request.
TcpListener listener = (TcpListener)ar.AsyncState;
// End the operation and display the received data on the console.
Console.WriteLine("Client connection completed");
Clients.Add(listener.EndAcceptTcpClient(ar));
// Signal the calling thread to continue.
TcpClientConnected.Set();
}
WebAPI Controller:
public class ValuesController : ApiController
{
// GET api/values/5
public string Get(int id)
{
return $"Foo: {id}";
}
}
As mentioned earlier, what I seek is "communication" between the WebAPI and the windows service. How can I pass the "id" parameter from the WebAPI call to the _engineService object in my windows service? Perhaps something similar to WPF's MVVM Light Messenger? The idea is that it would then be parsed and sent to the appropriate TcpClient that is stored in the Clients List.
Any advice on how to achieve this will be appreciated. Please feel free to ask for clarification/more code.

Did you find any answer to your issue yet ?
I don't quite understand what you try to achieve looking for a communication between the two of them ? Do you want to somehow rely on TCP/IP to relay this id or in-memory ?
Potentially, you could consider a Mediator pattern and use this kind of library that seems quite useful in the case I understood : https://github.com/jbogard/MediatR
In a simpler approach, I would rely on events to achieve what you are trying to do, which is having a reactive communication from the HTTP request to the c++ users.
Did I understand you needs ? I am quite curious about the solution

I'm assuming you are trying to take an HTTP GET request's ID parameter and send it to TCP clients who are connected to the EngineService. If your EngineService is initialized before your ApiService, I think this is a question of how to get a handle to the one-and-only EngineService instance from within an ApiService's controller instances.
If I'm following you, you could make the EngineService a public static property of your ContainerService and reference it as ContainerService.EngineService from the controller (or anywhere in the app for that matter) or better register your EngineService as a singleton in a DI container an inject it into the ApiService.

Solution (calls to WebAPI trigger EngineService)
I now use RabbitMQ/EasyNetQ to achieve communication between the WebApi and the EngineService object containing my TCP clients.
I have incidentally split them into two separate Projects/Topshelf services now.
The following is the new "communication" component and it is instantiated in the EngineService constructor.
public class Communication
{
private readonly Logger _logger;
private readonly IBus _bus;
public delegate void ReceivedEventHandler(string data);
public event ReceivedEventHandler Received;
protected virtual void OnReceive(string data)
{
Received?.Invoke(data);
}
public Communication()
{
_logger = new Logger();
_bus = RabbitHutch.CreateBus("host=localhost", reg => reg.Register<IEasyNetQLogger>(log => _logger));
SubscribeAllQueues();
}
private void SubscribeAllQueues()
{
_bus.Receive<Message>("pipeline", message =>
{
OnReceive(message.Body);
});
}
public void SubscribeQueue(string queueName)
{
_bus.Receive<Message>(queueName, message =>
{
OnReceive(message.Body);
});
}
}
An event handler is then added.
This means that as soon as a message arrives to the bus, the data will be relayed to the event handler which will subsequently relay it to the first connected TCP client in the list.
public void Handler(string data)
{
//Console.WriteLine(data);
_clients[0].Client.Send(Encoding.UTF8.GetBytes(data));
}
...
_comPipe.Received += Handler;
And finally on the WebApi's controller:
public string Get(int id)
{
ServiceCom.SendMessage("ID: " + id);
return "value";
}
ServiceCom class. Allows sending a string message on the bus.
public static class ServiceCom
{
public static void SendMessage(string messageBody)
{
var messageBus = RabbitHutch.CreateBus("host=localhost");
messageBus.Send("pipeline", new Message { Body = messageBody });
}
}
Now that this is done, I am now looking to implement a way for the connected TCP clients to trigger updates/events in an additional SPA project that will act as a Portal / Client Management App.
My approach will probably make use of KnockOut.js and SignalR to achieve dynamic Views where TCP client events are displayed immediately and similarly actions on to WebAPI will trigger events in the TCP clients. I know it sounds like a bizarre combination of processes but it is all according to plan and working out as expected :)

How to host NServiceBus in ASP.NET Core

I recently upgraded from NServiceBus 5x to 6.0.0-beta0004 to be able to host a ASP.NET Core application (whose main function is to listen to NServiceBus messages). I'm having problems with the startup of the host as the endpoint doesn't seem to subscribe to the publisher.
I am using the pubsub example to fix the problem. Apart from the projects that are in there by default, I added one extra project, a fresh ASP.NET Core project (based on the full .NET framework). I tried to use the exact same NServiceBus configuration, but instead of using the app/web.config, I am using the following configuration:
public class ConfigurationSource : IConfigurationSource
{
public T GetConfiguration<T>() where T : class, new()
{
UnicastBusConfig config = new UnicastBusConfig()
{
MessageEndpointMappings = new MessageEndpointMappingCollection()
};
var endpointMapping = new MessageEndpointMapping
{
AssemblyName = "Shared",
Endpoint = "Samples.PubSub.MyPublisher"
};
config.MessageEndpointMappings.Add(endpointMapping);
return config as T;
}
}
The Startup class was extended with the following code:
public IConfigurationRoot Configuration
{
get;
}
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddApplicationInsightsTelemetry(Configuration);
services.AddMvc();
services.AddSingleton(this.GetBus());
}
private IEndpointInstance GetBus()
{
LogManager.Use<DefaultFactory> ().Level(NServiceBus.Logging.LogLevel.Info);
var endpointConfiguration = new EndpointConfiguration("Samples.PubSub.Subscriber3");
endpointConfiguration.UseSerialization<JsonSerializer>();
endpointConfiguration.DisableFeature<AutoSubscribe>();
endpointConfiguration.UsePersistence<InMemoryPersistence>();
endpointConfiguration.SendFailedMessagesTo("error");
endpointConfiguration.EnableInstallers();
// Skip web.config settings and use programmatic approach
endpointConfiguration.CustomConfigurationSource(new ConfigurationSource());
var endpointInstance = Endpoint.Start(endpointConfiguration).Result;
endpointInstance.Subscribe<IMyEvent>().Wait();
return endpointInstance;
}
The Message Handler is identical to the other Subscriber projects in the solution:
public class EventMessageHandler : IHandleMessages<IMyEvent>
{
static ILog log = LogManager.GetLogger<EventMessageHandler>();
public Task Handle(IMyEvent message, IMessageHandlerContext context)
{
log.Info($"Subscriber 2 received IEvent with Id {message.EventId}.");
log.Info($"Message time: {message.Time}.");
log.Info($"Message duration: {message.Duration}.");
return Task.FromResult(0);
}
}
If I run the sample, I notice Subscriber1 and Subscriber2 are subscribed perfectly and they receive messages if I execute some of the commands in the Publisher console application. However Subscriber3 doesn't appear to be doing anything. No exceptions are thrown and in this sample I don't seem to find any relevant log information that could lead to misconfiguration.
Has anyone tried a similar setup with ASP.NET Core & NServiceBus 6 and if so, what should I do next?
Update 04/04/2017
There are some updates that provide some interesting insights:
https://groups.google.com/forum/#!topic/particularsoftware/AVrA1E-VHtk
https://particular.net/blog/nservicebus-on-net-core-why-not

Injecting Singleton Instance of class into SignalR Hub using Autofac

I am creating an application where SignalR is used to broadcast real-time tweets to a map. I am using the C# Tweetinvi library (tweetinvi.codeplex.com) to handle all of the logic associated with connecting to the Twitter Streaming API.
The Twitter API specifies that only one streaming connection can be open to Twitter at any time. As I am using SignalR, there is a dependency between the Streaming connection and the Hub class. I know that the Hub class is transient, meaning that it is created each time a client requests it, so I need to ensure that the instance of my Twitter Stream class injected into the Hub class is a singleton, or at least IFilteredStream is only created once in the lifetime of the application. Here is the boilerplate code to connect to the API:
public class TweetStream
{
private IFilteredStream _stream;
public TweetStream()
{
var consumerKey = ConfigurationManager.AppSettings.Get("twitter:ConsumerKey");
var consumerSecret = ConfigurationManager.AppSettings.Get("twitter:ConsumerSecret");
var accessKey = ConfigurationManager.AppSettings.Get("twitter:AccessKey");
var accessToken = ConfigurationManager.AppSettings.Get("twitter:AccessToken");
TwitterCredentials.SetCredentials(accessKey, accessToken, consumerKey, consumerSecret);
_stream = Stream.CreateFilteredStream();
}
// Return singular instance of _stream to Hub class for usage.
public IFilteredStream Instance
{
get { return _stream; }
}
}
The IFilteredStream interface exposes a lambda method as below which allows for receiving Tweets in real-time, which I would like to be able to access from within my SignalR Hub class:
_stream.MatchingTweetReceived += (sender, args) => {
Clients.All.broadcast(args.Tweet);
};
The source for this method can be found here
I've tried to implement Autofac, and it seems that the connection to the Twitter API happens, however nothing more happens. I've tried to debug this, but am unsure how to debug such a scenario using dependency injection. My Hub class currently looks like this:
public class TwitterHub : Hub
{
private readonly ILifetimeScope _scope;
private readonly TweetStream _stream;
// Inject lifetime scope and resolve reference to TweetStream
public TwitterHub(ILifetimeScope scope)
{
_scope = scope.BeginLifetimeScope();
_stream = scope.Resolve<TweetStream>();
var i = _stream.Instance;
_stream.MatchingTweetReceived += (sender, args) => {
Clients.All.broadcast(args.Tweet);
};
i.StartStreamMatchingAllConditions();
}
}
And finally, my OWIN Startup class, where I register my dependencies and Hub with Autofac:
[assembly: OwinStartup(typeof(TwitterMap2015.App_Start.OwinStartup))]
namespace TwitterMap2015.App_Start
{
public class OwinStartup
{
public void Configuration(IAppBuilder app)
{
var builder = new ContainerBuilder();
// use hubconfig, not globalhost
var hubConfig = new HubConfiguration {EnableDetailedErrors = true};
builder.RegisterHubs(Assembly.GetExecutingAssembly()); // register all SignalR hubs
builder.Register(i => new TweetStream()).SingleInstance(); // is this the correct way of injecting a singleton instance of TweetStream?
var container = builder.Build();
hubConfig.Resolver = new AutofacDependencyResolver(container);
app.MapSignalR("/signalr", hubConfig);
}
}
}
Sorry if this question is a bit of a mess, I'm having a hard time of understand what kind of architecture I need to implement to get this working! Open to advice / recommendations on how this could be improved, or how it should be done!

IMO this cannot work because you are wiring your event to call over the context of a specific hub instance, regardless of any code related to Autofac (which might have issues too but I'm not a big expert on it). Your hub's constructor will be called each time a new connection happens or a method is called from a client, so:
you are subscribing that event potentially multiple times per client. I don't know the Twitter API you are using, but on this note the fact that you call i.StartStreamMatchingAllConditions() all these times seems wrong to me
each time you create a closure over the Clients member of that instance in your event handler, which is supposed to go away when the hub is destroyed (so probably you are leaking memory)
What you need to do, given that your are calling over Client.All, and therefore this is a pure broadcast independent on any specific caller, is:
initialize your Twitter connection in the constructor of your TwitterStream service
in that same place (maybe with some indirection, but probably not necessary) take an instance of the hub context of your TwitterHub
subscribe to the event and use the context you just retrieved to broadcast over it
Such constructor might look like this:
public service TwitterStream : ??? <- an interface here?
{
...
public TwitterStream (ILifetimeScope scope ??? <- IMO you don't need this...)
{
//Autofac/Twitter stuff
...
var context = GlobalHost.DependencyResolver.GetHubContext<TwitterHub>();
_stream.MatchingTweetReceived += (sender, args) => {
context.Clients.All.broadcast(args.Tweet);
};
//maybe more Autofac/Twitter stuff
...
}
...
}
TwitterHub must exist, but in case you just need it to do this kind of broadcast to all, with no special code needed to monitor connections or handle client-generated calls, it could well be empty and it's just fine that your actual hub-related code lives outside of it and uses a IHubContext to broadcast messages. Such a code would take care of handling all the existing connected clients each time a tweet arrives, so no need to track them.
Of course if you have more requirements for actually handling clients separarely, then things might need to be different, but your code does not make me think otherwise.