Develop Reference

Sharing the same scope in Masstransit mediator produces error: The ConsumeContext was already set

I'm trying to use MassTransit mediator on my Asp.Net web API. Currently I'm using Masstransit 7.3.0
I wanted for consumers to share the same scope. So I did everything step-by-step as in documentation.
Here I'm setting up the mediator in startup and adding the consumers, requestclients:
services.AddHttpContextAccessor();
services.AddMediator(cfg =>
{
cfg.ConfigureMediator((regContext, options) =>
{
options.UseHttpContextScopeFilter(regContext);
});
cfg.AddConsumer<BillingFile.Create>();
cfg.AddRequestClient<BillingFile.Command.CreateCommand>();
cfg.AddConsumer<Payment.CreateBasePayment>();
cfg.AddRequestClient<Payment.Command.CreateBasePaymentCommand>();
});
I'm using HttpContextScopeFilter directly from the documentation on the website.
When I try to send a message from one consumer to the other, an exception is thrown:
The ConsumeContext was already set
Here is my Create consumer:
public class Create : IConsumer<CreateCommand>
{
private readonly IMediator mediator;
public Create(IMediator mediator)
{
this.mediator = mediator;
}
public async Task Consume(ConsumeContext<CreateCommand> context)
{
var request = context.Message.Request;
/* Create base payment <--- here be the exception */
var basePaymentHandler = mediator.CreateRequestClient<CreateBasePaymentCommand>(context);
var basePaymentResponse = await basePaymentHandler.GetResponse<CreateBasePaymentResponse>(new CreateBasePaymentCommand { Request = Request, CorrelationId = Guid.NewGuid()}, cancellationToken: context.CancellationToken);
....
Is this not allowed by Masstransit mediator?

MassTransit won't let you dispatch multiple messages in the same container scope. It results in the very exception you are experiencing. You're also using an obsolete/unsupported version of MassTransit.

AWS.Logger.AspNetCore with .NET Core 3.1 Web API project - log message formatting and delivery questions

I am currently using AWS.Logger.AspNetCore v3.0.1 as my logging system inside a .Net Core Web API project.
I've got my logging set up simply in Program.cs, like so:
.ConfigureLogging(logging =>
{
logging.AddAWSProvider();
}
with the log stream and flags set in appsettings.json. So far, so good.
I was wondering if anyone could possibly help with the following:
I'd like to prepend text to all the messages logged - even before the log level, category, scope.
I know there is an overload of AddAWSProvider() which lets me supply my own custom message formatter which AWSLogger instance uses. I could add this to Program.cs for example:
[IHostBuilder].ConfigureLogging(logging =>
{
logging.AddAWSProvider(((level, state, exception) =>
{
// TODO: Add formatting code which returns a formatted log entry string here
}));
})
however this approach doesn't give me access to the "Scope" supplied to ILogger.BeginScope() which I need to see my connection ID, trace ID etc.
Any ideas on how I can intercept the full string (not just the message!) to be sent to AWS and then transform it? I don't have a lot of time left to spend on this, so switching to something like Log4Net, Serilog or creating a full LogProvider or ScopeProvider is not an option.
Even something that auto-prefixes a custom formatted TimeStamp would be satisfactory. (And yes, I know AWS CloudWatch logs display a timestamp, but not when you "View As Text" )
Are there any docs on the AWSLoggerCore daemon? (AWS.Logger.Core.AWSLoggerCore::StartMonitor() and Monitor() )
I've looked at the code and noticed it dispatches to AWS a max of 5 requests (log messages) per second. However, I don't know how regularly it checks for new messages, and whether we can control that to ensure messages are logged immediately. Nor do I know how often or under what circumstances it calls Flush(). Losing vital log messages could prevent us tracing future issues.
The API runs within an EC2 container.
Thanks in advance,
Scott

Maybe you can try to create a decorator for the ILogger instance:
namespace ConsoleSample
{
public class Program
{
public static void Main(string[] args)
{
var config = new AWS.Logger.AWSLoggerConfig("AspNetCore.ConsoleSample");
config.Region = "us-east-1";
LoggerFactory logFactory = new LoggerFactory();
logFactory.AddAWSProvider(config);
// use the Decorator instead
var logger = new LoggerDecorator<Program>(logFactory.CreateLogger<Program>());
logger.LogInformation("Check the AWS Console CloudWatch Logs console in us-east-1");
logger.LogInformation("to see messages in the log streams for the");
logger.LogInformation("log group AspNetCore.ConsoleSample");
}
}
}
namespace ConsoleSample
{
public class LoggerDecorator<T> : ILogger<T>
{
private readonly ILogger<T> _logger;
public LoggerDecorator(ILogger<T> logger)
{
_logger = logger;
}
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
string message = $"{DateTime.UtcNow} [{logLevel.ToString().ToUpper()}] {state}";
_logger.Log(logLevel, eventId, message, exception, formatter);
}
public bool IsEnabled(LogLevel logLevel)
{
return _logger.IsEnabled(logLevel);
}
public IDisposable BeginScope<TState>(TState state)
{
return _logger.BeginScope(state);
}
}
}

NServiceBus In MVC Core - Sending with Dependency Injection

I am currently attempting to send messages to an Azure ServiceBus queue via NServiceBus 7.1.9.
We are using
dotNet Core 2.0
NServiceBus 7.1.9
NServiceBus.MSDependencyInjection 0.1.4
NServiceBus.Persistence.AzureStorage 2.3.0
NServiceBus.Transport.AzureServiceBus 1.1.1
However, messages appear to send, but never arrive at the destination queue.
We are attempting to use the default Microsoft Dependency Injection, which again, appears to configure correctly, but doesn't send any messages.
In startup.cs we configure the service and add it to the DI container
private void ConfigureNServiceBus(IServiceCollection services)
{
// Sending Service
services.AddScoped<IServiceBusSender, ServiceBusSender>();
// Create endpoint
var endpointConfiguration = new EndpointConfiguration("LinkGreenEndpoint");
// Set Queue Name
var context = Configuration["ServiceBusContext"]?.ToLower() ?? "local";
endpointConfiguration.OverrideLocalAddress($"horticulture.nservicebusbackend.{context}");
// Use Azure ServiceBus Queue
var transport = endpointConfiguration.UseTransport<AzureServiceBusTransport>();
transport.ConnectionString(Configuration["ConnectionStrings:LinkGreenServiceBus"]);
// ConnectionStrings:LinkGreenServiceBus= "Endpoint=sb://xxx.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx"
endpointConfiguration.UsePersistence<AzureStoragePersistence>();
endpointConfiguration.UseContainer<ServicesBuilder>(customizations =>
{
customizations.ExistingServices(services);
});
var endpoint = NServiceBus.Endpoint.Start(endpointConfiguration).GetAwaiter().GetResult();
// Add to Dependency Injection Container
services.AddScoped(typeof(IEndpointInstance), x => endpoint);
}
To send a message we use ServiceBusSender
public class ServiceBusSender : IServiceBusSender
{
private readonly IEndpointInstance _endpointInstance;
public ServiceBusSender(IEndpointInstance endpointInstance)
{
_endpointInstance = endpointInstance;
}
public Task Send(ICommand message)
{
// Also tried _endpointInstance.SendLocal(); - gives "handler not found"
return _endpointInstance.Send(message);
}
}
And an example of a command that we send:
public class CloneSupplierItemCommandBase : ICommand
{
public int BuyerId { get; set; }
public IdAndQuantity[] CloneItems { get; set; }
}
We currently use NServiceBus v5.0.0 in .NET 4.5 with this ServiceBus endpoint successfully.

Found the issue and managed to get messages posted to a queue.
Not 100% sure on which of the following was the solution
Set the name of the queue on creation to the endpoint
var context = Configuration["ServiceBusContext"]?.ToLower() ?? "local";
var endpointConfiguration = new EndpointConfiguration($"horticulture.nservicebusbackend.{context}");
Add the endpoint name to the Send() command
return _endpointInstance.Send(_endpoint, message);

It appears your endpoint configuration in the ConfigureNServiceBus method does not have any routing defined. Without this, the endpoint will not know where to deliver command messages.
I suspect you got it to work because you added the destination endpoint name in the Send() command directly. This works, but it can become unmanageable when you have a lot of endpoints and/or you want to modify the routing at run-time.
The better approach is to configure routing during start up. See Command Routing on Particular's documentation site.

How to host NServiceBus in ASP.NET Core

I recently upgraded from NServiceBus 5x to 6.0.0-beta0004 to be able to host a ASP.NET Core application (whose main function is to listen to NServiceBus messages). I'm having problems with the startup of the host as the endpoint doesn't seem to subscribe to the publisher.
I am using the pubsub example to fix the problem. Apart from the projects that are in there by default, I added one extra project, a fresh ASP.NET Core project (based on the full .NET framework). I tried to use the exact same NServiceBus configuration, but instead of using the app/web.config, I am using the following configuration:
public class ConfigurationSource : IConfigurationSource
{
public T GetConfiguration<T>() where T : class, new()
{
UnicastBusConfig config = new UnicastBusConfig()
{
MessageEndpointMappings = new MessageEndpointMappingCollection()
};
var endpointMapping = new MessageEndpointMapping
{
AssemblyName = "Shared",
Endpoint = "Samples.PubSub.MyPublisher"
};
config.MessageEndpointMappings.Add(endpointMapping);
return config as T;
}
}
The Startup class was extended with the following code:
public IConfigurationRoot Configuration
{
get;
}
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddApplicationInsightsTelemetry(Configuration);
services.AddMvc();
services.AddSingleton(this.GetBus());
}
private IEndpointInstance GetBus()
{
LogManager.Use<DefaultFactory> ().Level(NServiceBus.Logging.LogLevel.Info);
var endpointConfiguration = new EndpointConfiguration("Samples.PubSub.Subscriber3");
endpointConfiguration.UseSerialization<JsonSerializer>();
endpointConfiguration.DisableFeature<AutoSubscribe>();
endpointConfiguration.UsePersistence<InMemoryPersistence>();
endpointConfiguration.SendFailedMessagesTo("error");
endpointConfiguration.EnableInstallers();
// Skip web.config settings and use programmatic approach
endpointConfiguration.CustomConfigurationSource(new ConfigurationSource());
var endpointInstance = Endpoint.Start(endpointConfiguration).Result;
endpointInstance.Subscribe<IMyEvent>().Wait();
return endpointInstance;
}
The Message Handler is identical to the other Subscriber projects in the solution:
public class EventMessageHandler : IHandleMessages<IMyEvent>
{
static ILog log = LogManager.GetLogger<EventMessageHandler>();
public Task Handle(IMyEvent message, IMessageHandlerContext context)
{
log.Info($"Subscriber 2 received IEvent with Id {message.EventId}.");
log.Info($"Message time: {message.Time}.");
log.Info($"Message duration: {message.Duration}.");
return Task.FromResult(0);
}
}
If I run the sample, I notice Subscriber1 and Subscriber2 are subscribed perfectly and they receive messages if I execute some of the commands in the Publisher console application. However Subscriber3 doesn't appear to be doing anything. No exceptions are thrown and in this sample I don't seem to find any relevant log information that could lead to misconfiguration.
Has anyone tried a similar setup with ASP.NET Core & NServiceBus 6 and if so, what should I do next?
Update 04/04/2017
There are some updates that provide some interesting insights:
https://groups.google.com/forum/#!topic/particularsoftware/AVrA1E-VHtk
https://particular.net/blog/nservicebus-on-net-core-why-not

Ninject scope when running Azure Webjob in parallel

I have a console application as my webjob to process notifications inside my application. The processes are triggered using a queue. The application interacts with a SQL Azure database using entity framework 6. The Process() method that's being called reads/write data to the database.
I'm getting several errors when the queue messages are processed. They never get to the poison queue since they are reprocessed successfully after 2-3 times. Mainly the errors are the following:
An unhandled exception of type 'System.StackOverflowException' occurred in mscorlib.dll
Error: System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.
The default batch size is 16, so the messages are processed in parallel.
My guess is that the Ninject setup for processing messages in parallel is wrong. Therefore, when they are processed at the same time, there are some errors and eventually they are processed successfully.
My question is: Does this setup look ok? Should I use InThreadScope() maybe since I don't know the parallel processing is also multi-threaded.
Here's the code for my application.
Program.cs
namespace NotificationsProcessor
{
public class Program
{
private static StandardKernel _kernel;
private static void Main(string[] args)
{
var module = new CustomModule();
var kernel = new StandardKernel(module);
_kernel = kernel;
var config =
new JobHostConfiguration(AzureStorageAccount.ConnectionString)
{
NameResolver = new QueueNameResolver()
};
var host = new JobHost(config);
//config.Queues.BatchSize = 1; //Process messages in parallel
host.RunAndBlock();
}
public static void ProcessNotification([QueueTrigger("%notificationsQueueKey%")] string item)
{
var n = _kernel.Get&ltINotifications>();
n.Process(item);
}
public static void ProcessPoison([QueueTrigger("%notificationsQueueKeyPoison%")] string item)
{
//Process poison message.
}
}
}
Here's the code for Ninject's CustomModule
namespace NotificationsProcessor.NinjectFiles
{
public class CustomModule : NinjectModule
{
public override void Load()
{
Bind&ltIDbContext>().To&ltDataContext>(); //EF datacontext
Bind&ltINotifications>().To&ltNotificationsService>();
Bind&ltIEmails>().To&ltEmailsService>();
Bind&ltISms>().ToSmsService>();
}
}
}
Code for process method.
public void ProcessMessage(string message)
{
try
{
var notificationQueueMessage = JsonConvert.DeserializeObject&ltNotificationQueueMessage>(message);
//Grab message and check if it has to be processed
var notification = _context.Set().Find(notificationQueueMessage.NotificationId);
if (notification != null)
{
if (notification.NotificationType == NotificationType.AppointmentReminder.ToString())
{
notificationSuccess = SendAppointmentReminderEmail(notification); //Code that sends email using the SendGrid Api
}
}
}
catch (Exception ex)
{
_logger.LogError(ex + Environment.NewLine + message, LogSources.EmailsService);
throw;
}
}
Update - Added Exception
The exception is being thrown at the Json serializer. Here's the stack trace:
Error: System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.
at System.String.CtorCharCount(Char c, Int32 count) at Newtonsoft.Json.JsonTextWriter.WriteIndent() at Newtonsoft.Json.JsonWriter.AutoCompleteClose(JsonContainerType type) at Newtonsoft.Json.JsonWriter.WriteEndObject() at Newtonsoft.Json.JsonWriter.WriteEnd(JsonContainerType type) at Newtonsoft.Json.JsonWriter.WriteEnd() at Newtonsoft.Json.JsonWriter.AutoCompleteAll() at Newtonsoft.Json.JsonTextWriter.Close() at Newtonsoft.Json.JsonWriter.System.IDisposable.Dispose() at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, Formatting formatting, JsonSerializerSettings settings) at Core.Services.Communications.EmailsService.SendAppointmentReminderEmail(Notificaciones email) in c:\ProjectsGreenLight\EAS\EAS\EAS\Core\Services\Communications\EmailsService.cs:line 489 at Core.Services.Communications.EmailsService.ProcessMessage(String message) in c:\ProjectsGreenLight\EAS\EAS\EAS\Core\Services\Communications\EmailsService.cs:line 124 at Core.Services.NotificacionesService.Process(String message) in c:\ProjectsGreenLight\EAS\EAS\EAS\Core\Services\NotificacionesService.cs:line 56

Since you're receiving OutOfMemoryExceptions and StackOverflowExceptions, i suggest that there may be a recursive or deeply nested method. It would be extremely helpful if you'd have a stacktrace to the exception, sadly that's not the case for StackOverflowExceptions. However, OutOfMemoryException has a stack trace, so you need to log this and see what it says / add it to your question. Also, as far as the StackoverflowException goes, you can also try this.
Scoping
You should not use .InThreadScope() in such a scenario. Why? usually the thread-pool is used. Thread's are reused. That means, that scoped objects live longer than for the processing of a single messages.
Currently you are using .InTransientScope() (that's the default if you don't specify anything else). Since you are using the IDbContext in only one place, that's ok. If you'd wanted to share the same instance across multiple objects, then you'd have to use a scope or pass along the instance manually.
Now what may be problematic in your case is that you may be creating a lot of new IDbContexts but not disposing of them after you've used them. Depending on other factors this may result in the garbage collector taking longer to clean-up memory. See Do i have to call dispose on DbContext.
However i grant that this won't fix your issue. It might just speed up your application.
Here's how you can do it anyway:
What you should do is: Have INotifications derive from IDisposable:
public interface INotifications : IDisposable
{
(...)
}
internal class Notifications : INotifications
{
private readonly IDbContext _context;
public Notifications(IDbContext context)
{
_context = context;
}
(...)
public void Dispose()
{
_context.Dispose();
}
}
and alter your calling code to dispose of INotifications:
public static void ProcessNotification([QueueTrigger("%notificationsQueueKey%")] string item)
{
using(var n = _kernel.Get<INotifications>())
{
n.Process(item);
}
}