Unit testing MassTransit consumers that make utilize asynchronous calls - c#

We are using MassTransit asynchronous messaging (on top of RabbitMQ) for our microservice architecture.
We ran into issues testing consumers that in turn make asynchronous calls.
The example below shows a simple MassTransit consumer that uses RestSharp to make an outbound call and utilized the ExecuteAsync asynchronous method.
public class VerifyPhoneNumberConsumer : Consumes<VerifyPhoneNumber>.Context
{
IRestClient _restClient;
RestRequest _request;
PhoneNumber _phoneNumber;
PhoneNumberVerificationResponse _responseData;
public VerifyPhoneNumberConsumer(IRestClient client)
{
_restClient = client;
}
public void Consume(IConsumeContext<VerifyPhoneNumber> context)
{
try
{
//we can do some standard message verification/validation here
_restClient.ExecuteAsync<PhoneNumberVerificationResponse>(_request, (response) =>
{
//here we might do some standard response verification
_responseData = response.Data;
_phoneNumber = new PhoneNumber()
{
Number = _responseData.PhoneNumber
};
context.Respond(new VerifyPhoneNumberSucceeded(context.Message)
{
PhoneNumber = _phoneNumber
});
});
}
catch (Exception exception)
{
context.Respond(new VerifyPhoneNumberFailed(context.Message)
{
PhoneNumber = context.Message.PhoneNumber,
Message = exception.Message
});
}
}
}
A sample unit test for this might look like the following:
[TestFixture]
public class VerifyPhoneNumberConsumerTests
{
private VerifyPhoneNumberConsumer _consumer;
private PhoneNumber _phoneNumber;
private RestResponse _response;
private VerifyPhoneNumber _command;
private AutoResetEvent _continuationEvent;
private const int CONTINUE_WAIT_TIME = 1000;
[SetUp]
public void Initialize()
{
_continuationEvent = new AutoResetEvent(false);
_mockRestClient = new Mock<IRestClient>();
_consumer = new VerifyPhoneNumberConsumer(_mockRestClient.Object);
_response = new RestResponse();
_response.Content = "Response Test Content";
_phoneNumber = new PhoneNumber()
{
Number = "123456789"
};
_command = new VerifyPhoneNumber(_phoneNumber);
}
[Test]
public void VerifyPhoneNumber_Succeeded()
{
var test = TestFactory.ForConsumer<VerifyPhoneNumberConsumer>().New(x =>
{
x.ConstructUsing(() => _consumer);
x.Send(_command, (scenario, context) => context.SendResponseTo(scenario.Bus));
});
_mockRestClient.Setup(
c =>
c.ExecuteAsync(Moq.It.IsAny<IRestRequest>(),
Moq.It
.IsAny<Action<IRestResponse<PhoneNumberVerificationResponse>, RestRequestAsyncHandle>>()))
.Callback<IRestRequest, Action<IRestResponse<PhoneNumberVerificationResponse>, RestRequestAsyncHandle>>((
request, callback) =>
{
var responseMock = new Mock<IRestResponse<PhoneNumberVerificationResponse>>();
responseMock.Setup(r => r.Data).Returns(GetSuccessfulVericationResponse());
callback(responseMock.Object, null);
_continuationEvent.Set();
});
test.Execute();
_continuationEvent.WaitOne(CONTINUE_WAIT_TIME);
Assert.IsTrue(test.Sent.Any<VerifyPhoneNumberSucceeded>());
}
private PhoneNumberVerificationResponse GetSuccessfulVericationResponse()
{
return new PhoneNumberVerificationResponse
{
PhoneNumber = _phoneNumber
};
}
}
Because of the invocation of the ExecuteAsync method in the consumer, this test method would fall through if we did not put something to block it until it was signaled (or timed out). In the sample above, we are using AutoResetEvent to signal from the callback to continue and run assertions.
THIS IS A TERRIBLE METHOD and we are exhausting all resources to try to find out alternatives. If its not obvious, this can potentially cause false failures and race conditions during testing. Not too mention potentially crippling automated testing times.
What alternatives do we have that are BETTER than what we currently have.
EDIT Here is a source that I originally used for how to mock RestSharp asynchronous calls.
How to test/mock RestSharp ExecuteAsync(...)

Honestly, the complexity of doing asynchronous methods is one of the key drivers of MassTransit 3. While it isn't ready yet, it makes asynchronous method invocation from consumers so much better.
What you're testing above, because you are calling ExecuteAsync() on your REST client, and not waiting for the response (using .Result, or .Wait) in the consumer, the HTTP call is continuing after the message consumer has returned. So that might be part of your problem.
In MT3, this consumer would be written as:
public async Task Consume(ConsumeContext<VerifyPhoneNumber> context)
{
try
{
var response = await _restClient
.ExecuteAsync<PhoneNumberVerificationResponse>(_request);
var phoneNumber = new PhoneNumber()
{
Number = response.PhoneNumber
};
await context.RespondAsync(new VerifyPhoneNumberSucceeded(context.Message)
{
PhoneNumber = _phoneNumber
});
}
catch (Exception exception)
{
context.Respond(new VerifyPhoneNumberFailed(context.Message)
{
PhoneNumber = context.Message.PhoneNumber,
Message = exception.Message
});
}
}

I was able to come up with the following solution which seems far more elegant and proper. Feel free to correct me if I am wrong in assuming this.
I modified the RestSharp execution in my consumer so my consumer looks like the following:
public class VerifyPhoneNumberConsumer : Consumes.Context
{
IRestClient _restClient;
RestRequest _request;
PhoneNumber _phoneNumber;
PhoneNumberVerificationResponse _responseData;
public VerifyPhoneNumberConsumer(IRestClient client)
{
_restClient = client;
}
public void Consume(IConsumeContext<VerifyPhoneNumber> context)
{
try
{
//we can do some standard message verification/validation here
var response = await _restClient.ExecuteGetTaskAsync<PhoneNumberVerificationResponse>(_request);
_responseData = response.Data;
_phoneNumber = new PhoneNumber()
{
Number = _responseData.PhoneNumber
};
}
catch (Exception exception)
{
context.Respond(new VerifyPhoneNumberFailed(context.Message)
{
PhoneNumber = context.Message.PhoneNumber,
Message = exception.Message
});
}
}
}
This utilizes the TPL async capabilities of RestSharp so that I don't have to do it myself.
Because of this, I am able to change my test code to the following:
[Test]
public void VerifyPhoneNumber_Succeeded()
{
var test = TestFactory.ForConsumer<VerifyPhoneNumberConsumer>().New(x =>
{
x.ConstructUsing(() => _consumer);
x.Send(_command, (scenario, context) => context.SendResponseTo(scenario.Bus));
});
var response = (IRestResponse<PhoneNumberVerificationResponse>)new RestResponse<PhoneNumberVerificationResponse>();
response.Data = GetSuccessfulVericationResponse();
var taskResponse = Task.FromResult(response);
Expect.MethodCall(
() => _client.ExecuteGetTaskAsync<PhoneNumberVerificationResponse>(Any<IRestRequest>.Value.AsInterface))
.Returns(taskResponse);
test.Execute();
Assert.IsTrue(test.Sent.Any<VerifyPhoneNumberSucceeded>());
}

Related

No customer received the message

I am making a notification microservice for users. It is impossible to use the SignalR built into MassTransit due to the built-in microservice architecture.
After accepting messages from the queue RabbitMq, it is called StartAsync:
public class MassTransitManager : IMassTransitManager
{
public MassTransitManager(Func<string, Task<string>> processDelegateAsync)
{
ProcessDelegateAsync = processDelegateAsync;
}
public Func<string, Task<string>> ProcessDelegateAsync { get; set; }
public async Task<string> StartAsync(string requestString)
{
string response;
try
{
response = await ProcessMessageAsync(requestString);
}
catch (Exception exception)
{
response = exception.ToString();
}
return response;
}
private async Task<string> ProcessMessageAsync(string requestString)
{
if (ProcessDelegateAsync == null)
throw new InvalidOperationException($"Not found delegate {nameof(ProcessDelegateAsync)}.");
var result = await ProcessDelegateAsync(requestString);
return result;
}
}
MassTransitManager is in Class Library. Delegate created in Startup:
services.AddSingleton<IMassTransitManager, MassTransitManager>(provider =>
{
return new MassTransitManager(async serializedMessage =>
{
var hubContext = provider.GetRequiredService<IHubContext<NoticeHub>>();
await hubContext.Clients.All.SendAsync("SendResponse", "Hi all.");
return "Hi man";
});
});
But no client receives this message. What can be the ways to solve this problem?

Masstransit Problem adding Payload to request/response in Unit Testing, alternativly using Pipes in Unit Tests

currently I have the problem that I want to write unit tests for Masstransit in .NET. My request/response consumer has some consumer filters, one of these filters are generating extra data as message payload and attaching this to the request message. In order to test my consumer in a unit test I would like to add the Payload.
Q1) Is it possible to add the payload to the request message
Q2) Alternativly, can I make a mocking filter and set it as consumer filter in the pipeline? (Which sets the payload)
This is my latest attempt:
public class ContactCommandConsumerTest
{
[Fact]
public async Task CreateContactOnUserRequestConsumer_RequestConsumer_IsAttached()
{
var harness = new InMemoryTestHarness { TestTimeout = TimeSpan.FromSeconds(5) };
[...]
var consumer = harness.Consumer<CreateContactOnUserRequestCommandConsumer>(() => new CreateContactOnUserRequestCommandConsumer(loggerConsumer, mapper,kontakteintragRep,machineTime));
var pipe = Pipe.New<PipeContext>(x => x.UseFilter(new MockFilter<PipeContext>()));
// harness.Consumer<CreateContactOnUserRequestCommandConsumer>();
await harness.Start();
try
{
harness.Bus.ConnectConsumePipe<CreateContactOnUserRequestCommandConsumer>(pipe);
var requestClient = await harness.ConnectRequestClient<CreateContactOnUserRequestCommand>();
var response = await requestClient.GetResponse<AcceptedResponse, FaultedResponse>(new
{
EntityInfo = "Vb48cc135-4593-4b96-bb29-2cf136b3d1ee",
});
Assert.True(consumer.Consumed.Select<CreateContactOnUserRequestCommand>().Any());
Assert.True(harness.Sent.Select<FaultedResponse>().Any());
}
finally
{
await harness.Stop();
}
}
}
internal class MockFilter<T> : IFilter<T> where T: class, PipeContext
{
public void Probe(ProbeContext context)
{
context.CreateFilterScope("mock");
}
public async Task Send(T context, IPipe<T> next)
{
context.GetOrAddPayload(() => new ContextUserPayload() { ContextUser = new Guid("dc6e091f-669e-45b3-9dd6-a36316f70527") });
await next.Send(context);
}
}
I tried to build a pipe and add it to "harness.bus.ConnectConsumerPipe". But the mock filter is never called ???
You use use the OnConfigureInMemoryBus event on the InMemoryTestHarness to add your filter to the bus endpoint.
Similar to:
harness.OnConfigureInMemoryBus += configurator =>
{
configurator.UseFilter(...);
}
To add a filter to the request, use:
using RequestHandle<TRequest> requestHandle = requestClient.Create(message, cancellationToken);
requestHandle.UseFilter(...);
return await requestHandle.GetResponse<TResponse>().ConfigureAwait(false);

Unable to catch InvalidOperationException in xUnit

I've been trying to make the following test pass for the last hour, but I can't seem to make it work:
[Fact]
public async void TestDetachedRecordsArentUpdatedWithoutIDs()
{
var options = new DbContextOptionsBuilder<ClientContext>()
.UseInMemoryDatabase(databaseName: "cant_update_detached_clients_with_bankinfos")
.Options;
int clientID, bankInfoID;
using(var context = new ClientContext(options))
{
var service = new ClientService(context);
var bankInfo = new BankInfo { RoutingNumber = "12345" };
var client = new Client { FirstName = "Javier", LastName = "Garcia", BankInfo = bankInfo };
await service.Save(client);
clientID = client.ID;
bankInfoID = client.BankInfo.ID;
}
using(var context = new ClientContext(options))
{
var service = new ClientService(context);
var bankInfo = new BankInfo { RoutingNumber = "Modified" };
var client = new Client { ID = clientID, FirstName = "Modified", BankInfo = bankInfo };
try
{
await service.Save(client);
}
catch (System.InvalidOperationException ex)
{
var expected = "The property 'ID' on entity type 'BankInfo' is part of a key and so cannot be modified or marked as modified.";
Assert.Contains(expected, ex.Message);
}
}
}
The issue I'm finding is that I can't seem to catch the exception. This is the result of the test every time I run it:
To give a little extra context, when ClientService::Save is called, it calls a method called HandleDisconnectedEntities, which blows up when the below line runs:
94: _context.Entry(existingClient.BankInfo).CurrentValues.SetValues(client.BankInfo);
I do understand the nature of the exception, but I don't understand why my test would not be able to catch it. Any insight is greatly appreciated!
The issue because of the async void of the test definition.
public async void TestDetachedRecordsArentUpdatedWithoutIDs() { ...
Async void methods have different error-handling semantics. When an exception is thrown out of an async Task or async Task<T> method, that exception is captured and placed on the Task object. With async void methods, there is no Task object, so any exceptions thrown out of an async void method will be raised directly on the SynchronizationContext that was active when the async void method started.
Exceptions from an Async Void Method Can’t Be Caught with Catch
For example
private async void ThrowExceptionAsync()
{
throw new InvalidOperationException();
}
public void AsyncVoidExceptions_CannotBeCaughtByCatch()
{
try
{
ThrowExceptionAsync();
}
catch (Exception)
{
// The exception is never caught here!
throw;
}
}
Change the test to use Task instead
public async Task TestDetachedRecordsArentUpdatedWithoutIDs() {
//...
}
Reference Async/Await - Best Practices in Asynchronous Programming

Create delay between two message reads of a Queue?

I am using Azure Queues to perform a bulk import.
I am using WebJobs to perform the process in the background.
The queue dequeues very frequently. How do I create a delay between 2 message
reads?
This is how I am adding a message to the Queue
public async Task<bool> Handle(CreateFileUploadCommand message)
{
var queueClient = _queueService.GetQueueClient(Constants.Queues.ImportQueue);
var brokeredMessage = new BrokeredMessage(JsonConvert.SerializeObject(new ProcessFileUploadMessage
{
TenantId = message.TenantId,
FileExtension = message.FileExtension,
FileName = message.Name,
DeviceId = message.DeviceId,
SessionId = message.SessionId,
UserId = message.UserId,
OutletId = message.OutletId,
CorrelationId = message.CorrelationId,
}))
{
ContentType = "application/json",
};
await queueClient.SendAsync(brokeredMessage);
return true;
}
And Below is the WebJobs Function.
public class Functions
{
private readonly IValueProvider _valueProvider;
public Functions(IValueProvider valueProvider)
{
_valueProvider = valueProvider;
}
public async Task ProcessQueueMessage([ServiceBusTrigger(Constants.Constants.Queues.ImportQueue)] BrokeredMessage message,
TextWriter logger)
{
var queueMessage = message.GetBody<string>();
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(_valueProvider.Get("ServiceBaseUri"));
var stringContent = new StringContent(queueMessage, Encoding.UTF8, "application/json");
var result = await client.PostAsync(RestfulUrls.ImportMenu.ProcessUrl, stringContent);
if (result.IsSuccessStatusCode)
{
await message.CompleteAsync();
}
else
{
await message.AbandonAsync();
}
}
}
}
As far as I know, azure webjobs sdk enable concurrent processing on a single instance(the default is 16).
If you run your webjobs, it will read 16 queue messages(peeklock and calls Complete on the message if the function finishes successfully, or calls Abandon) and create 16 processes to execute the trigger function at same time. So you feel the queue dequeues very frequently.
If you want to disable concurrent processing on a single instance.
I suggest you could set ServiceBusConfiguration's MessageOptions.MaxConcurrentCalls to 1.
More details, you could refer to below codes:
In the program.cs:
JobHostConfiguration config = new JobHostConfiguration();
ServiceBusConfiguration serviceBusConfig = new ServiceBusConfiguration();
serviceBusConfig.MessageOptions.MaxConcurrentCalls = 1;
config.UseServiceBus(serviceBusConfig);
JobHost host = new JobHost(config);
host.RunAndBlock();
If you want to create a delay between 2 message reads, I suggest you could create a custom ServiceBusConfiguration.MessagingProvider.
It contains CompleteProcessingMessageAsync method, this method completes processing of the specified message, after the job function has been invoked.
I suggest you could add thread.sleep method in CompleteProcessingMessageAsync to achieve delay read.
More detail, you could refer to below code sample:
CustomMessagingProvider.cs:
Notice: I override the CompleteProcessingMessageAsync method codes.
public class CustomMessagingProvider : MessagingProvider
{
private readonly ServiceBusConfiguration _config;
public CustomMessagingProvider(ServiceBusConfiguration config)
: base(config)
{
_config = config;
}
public override NamespaceManager CreateNamespaceManager(string connectionStringName = null)
{
// you could return your own NamespaceManager here, which would be used
// globally
return base.CreateNamespaceManager(connectionStringName);
}
public override MessagingFactory CreateMessagingFactory(string entityPath, string connectionStringName = null)
{
// you could return a customized (or new) MessagingFactory here per entity
return base.CreateMessagingFactory(entityPath, connectionStringName);
}
public override MessageProcessor CreateMessageProcessor(string entityPath)
{
// demonstrates how to plug in a custom MessageProcessor
// you could use the global MessageOptions, or use different
// options per entity
return new CustomMessageProcessor(_config.MessageOptions);
}
private class CustomMessageProcessor : MessageProcessor
{
public CustomMessageProcessor(OnMessageOptions messageOptions)
: base(messageOptions)
{
}
public override Task<bool> BeginProcessingMessageAsync(BrokeredMessage message, CancellationToken cancellationToken)
{
// intercept messages before the job function is invoked
return base.BeginProcessingMessageAsync(message, cancellationToken);
}
public override async Task CompleteProcessingMessageAsync(BrokeredMessage message, FunctionResult result, CancellationToken cancellationToken)
{
if (result.Succeeded)
{
if (!MessageOptions.AutoComplete)
{
// AutoComplete is true by default, but if set to false
// we need to complete the message
cancellationToken.ThrowIfCancellationRequested();
await message.CompleteAsync();
Console.WriteLine("Begin sleep");
//Sleep 5 seconds
Thread.Sleep(5000);
Console.WriteLine("Sleep 5 seconds");
}
}
else
{
cancellationToken.ThrowIfCancellationRequested();
await message.AbandonAsync();
}
}
}
}
Program.cs main method:
static void Main()
{
var config = new JobHostConfiguration();
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
}
var sbConfig = new ServiceBusConfiguration
{
MessageOptions = new OnMessageOptions
{
AutoComplete = false,
MaxConcurrentCalls = 1
}
};
sbConfig.MessagingProvider = new CustomMessagingProvider(sbConfig);
config.UseServiceBus(sbConfig);
var host = new JobHost(config);
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
Result:

MVC, ASP.NET Identity, EmailService and Async calls w/Angularjs

After reviewing and trying many of the suggestions surrounding the error message:
"An asynchronous module or handler completed while an asynchronous
operation was still pending."
I found myself in the situation where even though the call to the MVC accountController actually EXECUTED the desired code (an email was sent to the right place with the right content) and a Try/Catch in the controller method would not 'catch' the error, the AngularJS factory that was initiating the call would receive a server error "page".
Factory:(AngularJS)
InitiateResetRequest: function (email) {
var deferredObject = $q.defer();
$http.post(
'/Account/InitiateResetPassword', { email: email }
)
.success(function (data) {
deferredObject.resolve(data);
})
.error(function (data) {
//This is a stop-gap solution that needs to be fixed..!
if (data.indexOf("An asynchronous module or handler completed while an asynchronous operation was still pending.") > 0) {
deferredObject.resolve(true);
} else {
deferredObject.resolve(false);
}
});
return deferredObject.promise;
}
MVC Controller (C#):
[HttpPost]
[AllowAnonymous]
public async Task<int> InitiateResetPassword(string email)
{
try
{
_identityRepository = new IdentityRepository(UserManager);
string callbackUrl = Request.Url.AbsoluteUri.Replace(Request.Url.AbsolutePath, "/account/reset?id=");
await _identityRepository.InitiatePasswordReset(email, callbackUrl);
return 0;
}
catch(Exception ex)
{
Console.WriteLine(ex.ToString());
return 1;
}
}
Identity Repository/InitiatePasswordReset:
public async Task InitiatePasswordReset(string email, string callbackUrl)
{
try
{
var u = await _applicationUserManager.FindByEmailAsync(email);
string passwordResetToken = await GetResetToken(u);
callbackUrl = callbackUrl + HttpUtility.UrlEncode(passwordResetToken);
await _applicationUserManager.SendEmailAsync(u.Id, RESET_SUBJECT, string.Format(RESET_BODY, u.FirstName, u.LastName, callbackUrl));
}
catch(Exception ex)
{ //another vain attempt to catch the exception...
Console.WriteLine(ex.ToString());
throw ex;
}
}
The EmailService injected into the ASP.NET Identity "ApplicationUserManager"
public class EmailService : IIdentityMessageService
{
XYZMailer xyzMailer;
public EmailService()
{
xyzMailer = XYZMailer.getCMRMailer();
}
public async Task SendAsync(IdentityMessage message)
{
//original code as posted:
//await Task.FromResult(xyzMailer.SendMailAsync(message));
//solution from #sirrocco-
await xyzMailer.SendMailAsync(message);
}
}
and finally...the XYZMailer class
class XYZMailer
{
#region"Constants"
private const string SMTP_SERVER = "XYZEXCHANGE.XYZ.local";
private const string NO_REPLY = "noReply#XYZCorp.com";
private const string USER_NAME = "noreply";
private const string PASSWORD = "theMagicP#55word"; //NO, that is not really the password :)
private const int SMTP_PORT = 587;
private const SmtpDeliveryMethod SMTP_DELIVERY_METHOD = SmtpDeliveryMethod.Network;
#endregion//Constants
internal XYZMailer()
{
//default c'tor
}
private static XYZMailer _XYZMailer = null;
public static XYZMailer getXYZMailer()
{
if (_XYZMailer == null)
{
_XYZMailer = new XYZMailer();
}
return _XYZMailer;
}
public async Task<int> SendMailAsync(IdentityMessage message)
{
#if DEBUG
message.Body += "<br/><br/>DEBUG Send To: " + message.Destination;
message.Destination = "me#XYZCorp.com";
#endif
// Create the message:
var mail =
new MailMessage(NO_REPLY, message.Destination)
{
Subject = message.Subject,
Body = message.Body,
IsBodyHtml = true
};
// Configure the client:
using (SmtpClient client = new SmtpClient(SMTP_SERVER, SMTP_PORT)
{
DeliveryMethod = SMTP_DELIVERY_METHOD,
UseDefaultCredentials = false,
Credentials = new System.Net.NetworkCredential(USER_NAME, PASSWORD),
EnableSsl = true
})
{
// Send:
await client.SendMailAsync(mail);
}
return 0;
}
}
(note: originally the controller method was simply "public async Task InitiateResetPassword, I added the return type as an attempt to trap the error on the server. At runtime, return 0; does hit (breakpoint) the catch does not get hit and at the client")
At the moment I am simply filtering for the expected error message and telling javascript to treat it as a success. This solution has the benefit of 'actually working'... but it is not 'ideal'.
How do I prevent the error on the server?
or alternately,
How do I catch the error on the server?
You need to remove await Task.FromResult from EmailService because that makes it so the code executes synchronously instead of async.
As to why the the exception was still raised and bubbled up outside the try/catch - I suspect the Task.FromResult was the culprit here too - if you now raise an exception in SendAsync (just to test it) you should catch in the controller.

Categories

Resources