Develop Reference

Do not collect child opentelemetry trace when parent trace is not recorded

I'm in the middle of introducing distributed tracing in a microservice app. All my services have the tracing enabled and everything works fine. But...
My app runs on a K8S cluster, so that cluster makes a lot of call to health endpoint. A lot is a lot. for the moment when nothing happens on a setup, I get 50Gb of trace recorder per day.
trace lock like this: one trace for the /health call and one children trace for a database call (call to check if db is available)
So, i decided to not record Health trace this is easily done by
// in startup
tracerProviderBuilder.AddAspNetCoreInstrumentation(x => x.Filter = AspnetCoreOtelFilter.Filter)
//the filter
public static class AspnetCoreOtelFilter
{
public static bool Filter(HttpContext httpContext)
{
if (httpContext.Request.Path == "/health/liveness")
return false;
if (httpContext.Request.Path == "/health/readiness")
return false;
return true;
}
}
as a result, I don't see all the health trace in my APM, anymore, but I keep receiving the trace regarding the log in DB
I was expecting that ParentBasedSampler (https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry/Trace/ParentBasedSampler.cs) would do the magic, but it's not the case.
Do you have any idea.
Thx for your help
Here is my test class:
[TestMethod, Ignore("Didn't succeed to avoid child message for now")]
public async Task HealthWithSubTrace_NoTracing()
{
var exportedItems = new List<Activity>();
var factory = new WebApplicationFactory<Program>();
using (var client = factory.WithWebHostBuilder(builder =>
builder.ConfigureTestServices(serviceCollection => ConfigureInMemoryExporterForOpenTelemetry()).CreateClient())
{
using var response = await client.GetAsync((string?)"/health/details").ConfigureAwait(false);
response.EnsureSuccessStatusCode();
WaitForActivityExport(exportedItems, 1);
}
Assert.AreEqual(0, exportedItems.Count);
}
//OTEL configuration is this
tracerProviderBuilder
.SetResourceBuilder(ResourceBuilder.CreateDefault()
.AddService(serviceName))
.SetSampler(sp => new ParentBasedSampler(new AlwaysOnSampler())
.AddAspNetCoreInstrumentation(x => x.Filter = AspnetCoreOtelFilter.Filter)
.AddHttpClientInstrumentation()
.AddProcessor<K8STagsProcessor>()
.AddSource(OpenTelemetrySources.Npgsql)
.AddSource(Program.source.Name);
//with program defined as this
public class Program
{
public static readonly ActivitySource source = new("OpenTelemtry.Test.App");
public static void Main(string[] args)
{
var builder = WebApplication.CreateBuilder(args);
builder.Services.ConfigurePhoenixHealth()
.AddHealthChecks().AddCheck(HealthConstants.Self, () => HealthCheckResult.Healthy());
var app = builder.Build();
app.UsePhoenixHealth();
app.MapGet("/health/details", () =>
{
using var activity = source.StartActivity("health.detail");
return "all fine";
});
app.Run();
}
}

Unittesting method which uses EF.Functions

I have a method that should return all projects with a modified date older than 5 years.
var projekte = await this.db.Projects.Include(x => x.ProjectStatus)
.Where(x =>
x.ModifiedDate.HasValue
&& EF.Functions.DateDiffYear(
x.ModifiedDate.Value.AddYears(-5),
this.dateTimeProvider.Today) >= 5)
.ToListAsync(cancellationToken: cancellationToken);
and my unit test to cover this method:
[TestMethod]
public async Task Handle_WhenCalled_ThenReturnProjekteWithGLKenntnisnameOlderThan5Years()
{
var today = new DateTime(2022, 8, 1);
var projekt1 = new ProjektBuilder()
.WithProjektId(new Guid("5a38062d-1992-4110-a49f-04cdf1eb21f0"))
.WithModifiedDate(today.AddYears(-5))
.Build();
var projekt2 = new ProjektBuilder()
.WithProjektId(new Guid("b5deaec7-17dd-4f2e-83fe-1badd7deeadb"))
.WithModifiedDate(today.AddYears(-6))
.Build();
var projekt3 = new ProjektBuilder()
.WithProjektId(new Guid("a7ca47ec-e5b0-4268-8df1-da562af1acd7"))
.WithModifiedDate(today.AddYears(-5).AddDays(1))
.Build();
this.AddToInMemoryContext(new[] { projekt1, projekt2, projekt3 });
this.dateTimeProviderFake.Setup(x => x.Today).Returns(today);
var result = await this.testee.Handle(new GetProjekteToArchiveQuery(), CancellationToken.None);
result.Should().BeEquivalentTo(new[] { projekt1, projekt2 });
}
The InMemoryDb is setup like this:
protected void InitializeInMemoryContext()
{
var randomAuditDatabaseName = $"{nameof(IAuditingContext)}_{Guid.NewGuid()}";
var auditOptions = new DbContextOptionsBuilder<AuditDbContext>().UseInMemoryDatabase(randomAuditDatabaseName).Options;
var auditContext = new AuditDbContext(auditOptions);
var randomDatabaseName = $"{nameof(TContextInterface)}_{Guid.NewGuid()}";
var options = new DbContextOptionsBuilder<TContext>().UseInMemoryDatabase(randomDatabaseName, b => b.EnableNullChecks(false)).EnableSensitiveDataLogging().Options;
this.InMemoryContext = (TContext)Activator.CreateInstance(typeof(TContext), options);
}
protected int AddToInMemoryContext<TEntity>(ICollection<TEntity> entities)
{
foreach (var entity in entities)
{
this.InMemoryContext.Add(entity);
}
return this.InMemoryContext.SaveChangesAsync().Result;
}
When running the unit test I get the following error:
System.InvalidOperationException: The 'DateDiffYear' method is not supported because the query has switched to client-evaluation. This usually happens when the arguments to the method cannot be translated to server. Rewrite the query to avoid client evaluation of arguments so that method can be translated to server.
at Microsoft.EntityFrameworkCore.SqlServerDbFunctionsExtensions.DateDiffYear(DbFunctions _, DateTime startDate, DateTime endDate)
I guess it's because the DateDiffYear is specifically for the SQLServer but how can I unit test this method if this is the case?
Thanks in advance

For this case, the best approach for really testing your code is to have a docker image with an SQL Server instance. The downside of this approach is that your unit tests would have a dependency, so in your CI/CD pipelines you will also need an SQL Server instance.
When you do not depend on any specific DB function, you can use InMemory or SQLite in memory, otherwise you need to test against the real technology. This is also the recommended approach by Microsoft:
Testing EF Core Applications
Testing against your production database system

Controllers Not Mapping on WebApplicationFactory for TestClient .NET 6

So I'm upgrading a bunch of our APIs and migrating them to the new shiny top-level statement style of startup.
These projects all also have integration tests that rely upon a WebApplicationFactory to create a TestClient for them. For the most-part, retargetting that at Program rather than Startup has worked just fine.
However, on one of my APIs, I just get 404s whenever I try to call a controller and I can't for the life of me work out why.
If I add in a minimal app.MapGet("test", () => "test") I can hit that from the test, so I guess for some reason, the controllers are getting unmapped. That being said, I'm struggling to see much difference between this API and the other ones.
So, all of my APIs follow the same basic pattern in Program:
var builder = WebApplication.CreateBuilder();
BuildConfiguration();
ConfigureLogging();
ConfigureXRay();
ConfigureServices();
ConfigureHost();
var app = builder.Build();
ConfigureApp();
app.Run();
I haven't included everything for brevity, but in ConfigureServices() I believe this line should add and configure all of the controllers:
void ConfigureServices()
{
builder.Services.ConfigureHealthChecks();
builder.Services.ConfigureAppMetrics();
builder.Services.ConfigureFromAppSettingsConfig(builder.Configuration);
builder.Services.ConfigureIOCServices(builder.Configuration);
builder.Services.AddAutoMapper(AppDomain.CurrentDomain.GetAssemblies());
builder.Services.AddControllers(opt => opt.AddGlobalErrorHandling()); //<----- This one
builder.Services.AddSoapCore();
builder.Services.ConfigureSwagger();
}
Then in ConfigureApp() I've got this call which I believe should map the controllers:
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.UseSoapEndpoint<ICorporateActionsSoapService>("/investments/api/CorporateActions.asmx", new BasicHttpBinding());
});
So if I run the app directly, all of that seems to work. For my tests, I've got the following.
public class CustomWebApplicationFactory : WebApplicationFactory<Program>
{
public CustomWebApplicationFactory()
{
SetAppSettingsValues();
DisableMetricsBuiltFlag();
}
private void DisableMetricsBuiltFlag()
{
var metricsBuiltField = typeof(MetricsAspNetHostBuilderExtensions).GetField("_metricsBuilt",
BindingFlags.Static |
BindingFlags.NonPublic);
metricsBuiltField?.SetValue(null, false);
}
private void SetAppSettingsValues()
{
Environment.SetEnvironmentVariable("IntegrationTests", "true");
}
}
And then this is injected into my test classes as an IClassFixture<CustomWebApplicationFactory> by XUnit. I then replace a few of the services with mocks and create a test client.
public GlobalErrorHandlingIntegrationTest(CustomWebApplicationFactory factory)
{
_mockLogger = _fixture.Create<ILogger<CorporateActionsController>>();
_mockAuditService = _fixture.Create<ICorporateActionsEmailAuditService>();
_mockEmailService = _fixture.Create<ICorporateActionsEmailService>();
_mockEmailService.GetEmails().Returns(Task.Run(() => _fixture.Create<CorporateActionResponse>()));
_appFactory = factory
.WithWebHostBuilder
(
builder =>
{
builder.ConfigureTestServices
(
x =>
{
x.Add(new ServiceDescriptor(typeof(ILogger<CorporateActionsController>), _mockLogger));
x.Add(new ServiceDescriptor(typeof(ICorporateActionsEmailService), _mockEmailService));
x.Add(new ServiceDescriptor(typeof(ICorporateActionsEmailAuditService), _mockAuditService));
}
);
}
);
_testClient = _appFactory.CreateClient();
}
And then all of my tests using that _testClient just 404. Any idea what's going wrong here and why these controllers might be getting unmapped?
I've seen this question but the answer doesn't seem to apply to my case.

You have to pass the args to the WebApplication.CreateBuilder(args);

How to update HttpClient base address at runtime in ASP.net core

I created several microservices using the ASP.net core API
One of these microservices returns the exact address of the other microservices
How to update the address of any of these microservices without restarting if the address is changed
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.AddHttpClient("MainMicroservice", x =>
{
x.BaseAddress = new Uri("http://mainmicroservice.com");
});
services.AddHttpClient("Microservice1", x =>
{
x.BaseAddress = new Uri("http://microservice1.com");
});
services.AddHttpClient("Microservice2", x =>
{
x.BaseAddress = new Uri("http://microservice2.com");
});
services.AddHttpClient("Microservice3", x =>
{
x.BaseAddress = new Uri("http://microservice3.com");
});
}
}
public class Test
{
private readonly IHttpClientFactory _client;
public Test(IHttpClientFactory client)
{
_client = client;
}
public async Task<string> Test()
{
var repeat = false;
do
{
try
{
return await _client
.CreateClient("Microservice1")
.GetStringAsync("Test")
.ConfigureAwait(false);
}
catch (HttpRequestException e) when (e.StatusCode == HttpStatusCode.NotFound)
{
var newAddress = await _client
.CreateClient("MainMicroservice")
.GetStringAsync("Microservice1")
.ConfigureAwait(false);
//todo change address of microservice1
repeat = true;
}
} while (repeat);
}
}

If you are building a microservice-based solution sooner or later (rather sooner) you will encounter a situation when one service needs to talk to another. In order to do this caller must know the exact location of a target microservice in a network, they operate in.
You must somehow provide an IP address and port where the target microservice listens for requests. You can do it using configuration files or environment variables, but this approach has some drawbacks and limitations.
First is that you have to maintain and properly deploy
configuration files for all your environments: local development,
test, pre-production, and production. Forgetting to update any of
these configurations when adding a new service or moving an existing
one to a different node will result in errors discovered at runtime.
Second, the more important issue is that it works only in a static
environment, meaning you cannot dynamically add/remove nodes,
therefore you won’t be able to dynamically scale your system. The
ability to scale and deploy given microservice autonomously is one
of the key advantages of microservice-based architecture, and we do
not want to lose this ability.
Therefore we need to introduce service discovery. Service discovery is a mechanism that allows services to find each other's network location. There are many possible implementations of this pattern.
We have two types of service discovery: client-side and server-side which you can find a good NuGet package to handle in ASP.NET projects.

How do I write unit tests for ASP.NET Core controllers that actually use my Database Context?

There seems to be little information about how to write good unit tests for actual ASP.NET Core controller actions. Any guidance about how to make this work for real?

I've got a system that seems to be working pretty well right now, so I thought I'd share it and see if it doesn't help someone else out. There's a really useful article in the Entity Framework documentation that points the way. But here's how I incorporated it into an actual working application.
1. Create an ASP.NET Core Web App in your solution
There are tons of great articles out there to help you get started. The documentation for basic setup and scaffolding is very helpful. For this purpose, you'll want to create a web app with Individual User Accounts so that your ApplicationDbContext is setup to work with EntityFramework automatically.
1a. Scaffold a controller
Use the information included in the documentation to create a simple controller with basic CRUD actions.
2. Create a separate class library for your unit tests
In your solution, create a new .NET Core Library and reference your newly created web app. In my example, the model I'm using is called Company, and it uses the CompaniesController.
2a. Add the necessary packages to your test library
For this project, I use xUnit as my test runner, Moq for mocking objects, and FluentAssertions to make more meaningful assertions. Add those three libraries to your project using NuGet Package Manager and/or Console. You may need to search for them with the Show Prerelease checkbox selected.
You will also need a couple of packages to use EntityFramework's new Sqlite-InMemory database option. This is the secret sauce. Below are a list of the package names on NuGet:
Microsoft.Data.Sqlite
Microsoft.EntityFrameworkCore.InMemory [emphasis added]
Microsoft.EntityFrameworkCore.Sqlite [emphasis added]
3. Setup Your Test Fixture
Per the article I mentioned earlier, there is a simple, beautiful way to set up Sqlite to work as an in-memory, relational database which you can run your tests against.
You'll want to write your unit test methods so that each method has a new, clean copy of the database. The article above shows you how to do that on a one-off basis. Here's how I set up my fixture to be as DRY as possible.
3a. Synchronous Controller Actions
I've written the following method that allows me to write tests using the Arrange/Act/Assert model, with each stage acting as a parameter in my test. Below is the code for the method and the relevant class properties in the TestFixture that it references, and finally an example of what it looks like to call the code.
public class TestFixture {
public SqliteConnection ConnectionFactory() => new SqliteConnection("DataSource=:memory:");
public DbContextOptions<ApplicationDbContext> DbOptionsFactory(SqliteConnection connection) =>
new DbContextOptionsBuilder<ApplicationDbContext>()
.UseSqlite(connection)
.Options;
public Company CompanyFactory() => new Company {Name = Guid.NewGuid().ToString()};
public void RunWithDatabase(
Action<ApplicationDbContext> arrange,
Func<ApplicationDbContext, IActionResult> act,
Action<IActionResult> assert)
{
var connection = ConnectionFactory();
connection.Open();
try
{
var options = DbOptionsFactory(connection);
using (var context = new ApplicationDbContext(options))
{
context.Database.EnsureCreated();
// Arrange
arrange?.Invoke(context);
}
using (var context = new ApplicationDbContext(options))
{
// Act (and pass result into assert)
var result = act.Invoke(context);
// Assert
assert.Invoke(result);
}
}
finally
{
connection.Close();
}
}
...
}
Here's what it looks like to call the code to test the Create method on the CompaniesController (I use parameter names to help me keep my expressions straight, but you don't strictly need them):
[Fact]
public void Get_ReturnsAViewResult()
{
_fixture.RunWithDatabase(
arrange: null,
act: context => new CompaniesController(context, _logger).Create(),
assert: result => result.Should().BeOfType<ViewResult>()
);
}
My CompaniesController class requires a logger, that I mock up with Moq and store as a variable in my TestFixture.
3b. Asynchronous Controller Actions
Of course, many of the built-in ASP.NET Core actions are asynchronous. To use this structure with those, I've written the method below:
public class TestFixture {
...
public async Task RunWithDatabaseAsync(
Func<ApplicationDbContext, Task> arrange,
Func<ApplicationDbContext, Task<IActionResult>> act,
Action<IActionResult> assert)
{
var connection = ConnectionFactory();
await connection.OpenAsync();
try
{
var options = DbOptionsFactory(connection);
using (var context = new ApplicationDbContext(options))
{
await context.Database.EnsureCreatedAsync();
if (arrange != null) await arrange.Invoke(context);
}
using (var context = new ApplicationDbContext(options))
{
var result = await act.Invoke(context);
assert.Invoke(result);
}
}
finally
{
connection.Close();
}
}
}
It's almost exactly the same, just setup with async methods and awaiters. Below, an example of calling these methods:
[Fact]
public async Task Post_WhenViewModelDoesNotMatchId_ReturnsNotFound()
{
await _fixture.RunWithDatabaseAsync(
arrange: async context =>
{
context.Company.Add(CompanyFactory());
await context.SaveChangesAsync();
},
act: async context => await new CompaniesController(context, _logger).Edit(1, CompanyFactory()),
assert: result => result.Should().BeOfType<NotFoundResult>()
);
}
3c. Async Actions with Data
Of course, sometimes you'll have to pass data back-and-forth between the stages of testing. Here's a method I wrote that allows you to do that:
public class TestFixture {
...
public async Task RunWithDatabaseAsync(
Func<ApplicationDbContext, Task<dynamic>> arrange,
Func<ApplicationDbContext, dynamic, Task<IActionResult>> act,
Action<IActionResult, dynamic> assert)
{
var connection = ConnectionFactory();
await connection.OpenAsync();
try
{
object data;
var options = DbOptionsFactory(connection);
using (var context = new ApplicationDbContext(options))
{
await context.Database.EnsureCreatedAsync();
data = arrange != null
? await arrange?.Invoke(context)
: null;
}
using (var context = new ApplicationDbContext(options))
{
var result = await act.Invoke(context, data);
assert.Invoke(result, data);
}
}
finally
{
connection.Close();
}
}
}
And, of course, an example of how I use this code:
[Fact]
public async Task Post_WithInvalidModel_ReturnsModelErrors()
{
await _fixture.RunWithDatabaseAsync(
arrange: async context =>
{
var data = new
{
Key = "Name",
Message = "Name cannot be null",
Company = CompanyFactory()
};
context.Company.Add(data.Company);
await context.SaveChangesAsync();
return data;
},
act: async (context, data) =>
{
var ctrl = new CompaniesController(context, _logger);
ctrl.ModelState.AddModelError(data.Key, data.Message);
return await ctrl.Edit(1, data.Company);
},
assert: (result, data) => result.As<ViewResult>()
.ViewData.ModelState.Keys.Should().Contain((string) data.Key)
);
}
Conclusion
I really hope this helps somebody getting on their feet with C# and the awesome new stuff in ASP.NET Core. If you have any questions, criticisms, or suggestions, please let me know! I'm still new at this, too, so any constructive feedback is invaluable to me!