I need to store connection strings and other settings in my .NET Core 2.0 integration tests (so, it's a Test project, with xUnit). How can I do this safely considering those tests run locally and in VSTS? Ideally, I need those settings separate for local environment and running on VSTS.
You can add multiple configuration files, such as appsettings.json, appsettings.dev.json, appsettings.test.json, then set the dev value for related environment variable and set test value for related variable of build.
After that replace the value in appsettings.test.json file through Token Replace task
Simple steps:
Install Microsoft.Extensions.Configuration.Json package to your xUnit test project
Add appsettings.json, appsettings.dev.json and appsettings.test.json configuration files to project
appsettings.json:
{
"ConnectionStrings": {
"BloggingDatabase": "Server=(localdb)\\mssqllocaldb;Database=EFGetStarted.ConsoleApp.NewDb;Trusted_Connection=True;"
},
"OtherSettings": {
"UserName": "name1",
"Pass": "pass1"
}
}
appsettings.dev.json:
{
"ConnectionStrings": {
"BloggingDatabase": "Server=(localdb)\\mssqllocaldb;Database=dev;Trusted_Connection=True;"
},
"OtherSettings": {
"UserName": "devname1",
"Pass": "pass1"
}
}
appsettings.test.json:
{
"ConnectionStrings": {
"BloggingDatabase": "Server=(localdb)\\mssqllocaldb;Database=#{testDev}#;Trusted_Connection=True;"
},
"OtherSettings": {
"UserName": "#{testName}#",
"Pass": "pass1"
}
}
Set Copy to Output Directory property of these files to Copy if newer.
4: Simple test code
:
[Fact]
public void Test1()
{
var envVariable = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
Console.WriteLine($"env: {envVariable}");
var config = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.AddJsonFile($"appsettings.{envVariable}.json", optional: true)
.Build();
var conn = config.GetConnectionString("BloggingDatabase");
var otherSettings = config["OtherSettings:UserName"];
Console.WriteLine(conn);
Console.WriteLine(otherSettings);
}
Add ASPNETCORE_ENVIRONMENT (value: test), testDev and testName variables to build definition, you can click the lock icon to change variable type to secret.
Add Replace Tokens task before build task (Target files: **\appsettings.test.json)
Related
I'm attempting to port an existing Functions app from core3.1 v3 to net5.0 I but can't figure out how to get the IOptions configuration pattern to work.
The configuration in my local.settings.json is present in the configuration data, and I can get to it using GetEnvironmentVariable. Still, the following does not bind the values to the IOptions configuration like it used to.
.Services.AddOptions<GraphApiOptions>()
.Configure<IConfiguration>((settings, configuration) => configuration.GetSection("GraphApi").Bind(settings))
The values are in the local.settings.json just as they were before:
"GraphApi:AuthenticationEndPoint": "https://login.microsoftonline.com/",
"GraphApi:ClientId": "316f9726-0ec9-4ca5-8d04-f39966bebfe1",
"GraphApi:ClientSecret": "VJ7qbWF-ek_Amb_e747nXW-fMOX-~6b8Y6",
"GraphApi:EndPoint": "https://graph.microsoft.com/",
"GraphApi:TenantId": "NuLicense.onmicrosoft.com",
Is this still supported?
What am I missing?
I had the same issue, but turns out that the json was not correctly formatted.
Just for reference, here it is how I configured it:
var host = new HostBuilder()
.ConfigureFunctionsWorkerDefaults()
.ConfigureServices(s =>
{
s.AddOptions<ApplicationSettings>().Configure<IConfiguration>((settings, configuration) =>
{
configuration.GetSection(nameof(ApplicationSettings)).Bind(settings);
});
})
.Build();
And here is an example of local.setting.json:
{
"IsEncrypted": false,
"Values": {
"ApplicationSettings:test": "testtest",
"ApplicationSettings:testtest": "test"
}
}
I've looked around and I'm surprised no one else has asked this yet, but I am trying to use User Secrets with my Azure Function app (.NET 6, v4). The value is never populated with I run locally from Visual Studio.
Function1.cs
public static class Function1
{
[FunctionName("Function1")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
{
var secret = Environment.GetEnvironmentVariable("mysecret");
return new OkObjectResult($"Secret is: {secret}");
}
}
local.settings.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet"
}
}
secrets.json
{
"mysecret": "test1",
"Values:mysecret": "test2"
}
FunctionApp1.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<AzureFunctionsVersion>v4</AzureFunctionsVersion>
<UserSecretsId>24c7ab90-4094-49a9-94ef-c511ac530526</UserSecretsId>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" Version="5.0.0" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="4.1.1" />
</ItemGroup>
<ItemGroup>
<None Update="host.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Update="local.settings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
</ItemGroup>
</Project>
The endpoint returns: "Secret is: " and I would expect it to return either "Secret is: test1" or "Secret is: test2"
However, when I change my local.settings.json to this it works:
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"mysecret": "test3"
}
}
How can I get the user secrets to pass to local.settings.json correctly? I'm guessing they don't show up as environment variables.
secrets.json is file used by Secret Manager which is for Development environment only. To store your secrets you have several choices for example provide them as app settings or use Azure Key Vault.
Out of the box, my conclusion is that User Secrets do not work with Azure Functions using .Net6 or .Net7. I have tried but unable to get it to work even though when the function starts the console will display "Found xxxxx.csproj. Using for user secrets file configuration."
Here is a github issue that mentions the problem. Within that same issue is a step-by-step set of instructions to make the User Secrets load by reading the file into the IConfiguration pipeline.
I chose not to alter the startup pipeline and instead use Azure KeyVault binding in the local.settings.json. For example:
"SuperSecret": "#Microsoft.KeyVault(SecretUri=https://XXXXXXXX.vault.azure.net/secrets/SuperSecret/)"
Using Azure KeyVault allows the secret to be safe during development and requires no changes in production. Just make sure when you develop locally that your Default Credential has access to the Azure KeyVault and grant access to your function app once deployed.
TLDR:
Might not be the best thing to do but it can be solved like this. I have done this a few times as well when there are settings that I want to check in. However if you do have settings that can be checked in you might reconsider adding them in their own class unless they should be overridden during deployment.
In my opinion you should keep local.settings.json and not try to replace this with appsettings.json since some values are usually needed to make it run like these:
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated"
},
Install NuGet Microsoft.Extensions.Configuration.UserSecrets
Set local.settings.json Copy to Output Directory to Copy if newer.
Modify Program.cs to look like this for .NET 5 <. Tested up to .NET 7:
var host = new HostBuilder()
.ConfigureFunctionsWorkerDefaults()
.ConfigureAppConfiguration((hostContext, config) =>
{
if (hostContext.HostingEnvironment.IsDevelopment())
{
config.AddJsonFile("local.settings.json");
config.AddUserSecrets<Program>();
}
})
.ConfigureServices((hostContext, services) =>
{
var connectionString = hostContext.Configuration.GetConnectionString("DefaultConnection");
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(connectionString));
var configuration = hostContext.Configuration;
var settings = new Settings();
configuration.Bind(settings);
services.AddSingleton(settings);
}
)
.Build();
host.Run();
public class Settings
{
public ConnectionStrings ConnectionStrings { get; set; } = new ConnectionStrings();
}
public class ConnectionStrings
{
public string DefaultConnection { get; set; } = "";
}
The only thing really needed is below but I like to show more complete examples from the real world:
.ConfigureAppConfiguration((hostContext, config) =>
{
if (hostContext.HostingEnvironment.IsDevelopment())
{
config.AddJsonFile("local.settings.json");
config.AddUserSecrets<Program>();
}
})
You can now inject any Settings you have from the constructor and they will be overridden if a secret is present.
public class TimerTrigger
{
private readonly ILogger _logger;
private readonly ApplicationDbContext _dbContext;
private readonly Settings _settings;
public TimerTrigger(ILoggerFactory loggerFactory, ApplicationDbContext dbContext, Settings settings)
{
_logger = loggerFactory.CreateLogger<TimerTrigger>();
_dbContext = dbContext;
_settings=settings;
}
[Function("TimerTrigger")]
public void Run([TimerTrigger("0 */1 * * * *")] MyInfo myTimer)
{
if (myTimer is not null && myTimer.ScheduleStatus is not null)
{
_logger.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
_logger.LogInformation($"Next timer schedule at: {myTimer.ScheduleStatus.Next}");
var defaultConnection = _settings.ConnectionStrings.DefaultConnection;
Long answer:
ASP.NET Core web apps created with dotnet new or Visual Studio generates code similar to this for .NET 7:
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello World!");
app.Run();
WebApplication.CreateBuilder initializes a new instance of the WebApplicationBuilder class with preconfigured defaults. The initialized WebApplicationBuilder (builder) provides default configuration and calls AddUserSecrets when the EnvironmentName is Development:
https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-7.0&tabs=windows#register-the-user-secrets-configuration-source
The method AddUserSecrets is located in the namespace Microsoft.Extensions.Configuration from package Microsoft.Extensions.Configuration.UserSecrets
https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.configuration.usersecretsconfigurationextensions.addusersecrets?view=dotnet-plat-ext-7.0
https://www.nuget.org/packages/Microsoft.Extensions.Configuration.UserSecrets/
If we then look at how Environment variables works:
Environment variables are used to avoid storage of app secrets in code
or in local configuration files. Environment variables override
configuration values for all previously specified configuration
sources.
https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-7.0&tabs=windows#environment-variables
Given that local.settings.json is meant to be local and the default .gitignore created when you add a Azure Functions project explicitly ignores it and therefore work like secrets:
## Ignore Visual Studio temporary files, build results, and
## files generated by popular Visual Studio add-ons.
# Azure Functions localsettings file
local.settings.json
The app secrets aren't checked into source control.
https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-7.0&tabs=windows#secret-manager
However if you still want to use the Secret Manager tool and secret storage read my TLDR.
Example values:
local.settings.json:
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated",
"Movies:ServiceApiKey": "--SECRET--"
},
"ConnectionStrings": {
"DefaultConnection": "--SECRET--"
}
}
secrets.json:
{
"Movies": {
"ServiceApiKey": "12345"
}
}
Reading values:
var moviesApiKeyConfig = _config["Movies:ServiceApiKey"];
var moviesApiKeySettings = _settings.Movies.ServiceApiKey;
var moviesApiKeyEnvironmentVariable = Environment.GetEnvironmentVariable("Movies:ServiceApiKey");
As you can see IConfiguration from Microsoft.Extensions.Configuration picks up secrets as expected but Environment.GetEnvironmentVariable does not. Also as expected from documentation.
I am upgrading nuget-packages for a webjob-project rocking the 4.3.0 version of Microsoft.Azure.WebJobs.Extensions.ServiceBus, where the connection string can be set programmatically like this:
.ConfigureWebJobs(b =>
{
b.AddServiceBus(options =>
{
options.ConnectionString = connectionString;
});
})
Which works great. We store the connection string in Azure Key Vault instead of in the appsettings-file.
However, after upgrading to 5.5.1, there is no option to set the connection string.
Is there any other way to set this? On the trigger itself, you can only specify the name of the setting in appsettings.json.
https://github.com/Azure/azure-sdk-for-net/blob/Microsoft.Azure.WebJobs.Extensions.ServiceBus_5.5.1/sdk/servicebus/Microsoft.Azure.WebJobs.Extensions.ServiceBus/src/Config/ServiceBusOptions.cs
Managed Identity can be used to avoid secrets in the settings-file but this specific project has not that configured in Azure yet for service bus and I would prefer to keep the current solution until we can move to use managed identity for all connections at once.
Package: https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.ServiceBus
Documentation:
https://github.com/Azure/azure-sdk-for-net/blob/Microsoft.Azure.WebJobs.Extensions.ServiceBus_5.5.1/sdk/servicebus/Microsoft.Azure.WebJobs.Extensions.ServiceBus/README.md
A solution I thought might work was to add Key Vault to the configuration (system assigned managed identity is used here):
.ConfigureAppConfiguration((_, config) =>
{
var keyVaultUrl = "...";
config.AddAzureKeyVault(new Uri(keyVaultUrl), new DefaultAzureCredential());
})
And then specify the secret name in the attribute:
[ServiceBusTrigger(QueueName, Connection = "secret-name")] string messageJson
But that only gives me an error that I think it related to it not accessing the service bus correctly.
System.InvalidOperationException: Can't bind parameter 'messageReceiver' to type 'Microsoft.Azure.ServiceBus.Core.MessageReceiver'
Storing Service Bus Connection string , you can use Azure Key Vault Service .
add this line of code in local.settings.json
"Connectionstring": "#Microsoft.KeyVault(SecretUri=https://connectionmt.vault.azure.net/secrets/AzureFunctionKeyVaultConnections/abcd46389abdeujewfuew898)"
local.settings.json code.
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"ServiceBusConnectionString": "Endpoint=sb://xxxservicebusnamespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx",
"Connectionstring": "#Microsoft.KeyVault(SecretUri=https://connectionstring-mt.vault.azure.net/secrets/AzureFunctionKeyVaultConnections/xxx)"
}
}
we create function , where we print connection string in console.
[FunctionName("Function1")]
public void Run([ServiceBusTrigger("myqueue", Connection = "ServiceBusConnectionString")]string myQueueItem, ILogger log)
{
string connection = Environment.GetEnvironmentVariable("Connectionstring");
Console.WriteLine(connection);
Console.ReadKey();
}
in this code we print connection string value, which come from azure key vault.
I have created a Worker Service using C# / .Net Core 6 (Visual Studio 2022).
It writes to a log file as expected if run via Visual Studio or started directly from Windows Explorer / PowerShell. However, when installed as a Windows Service, it does not create or write to a log file.
This is my program.cs:
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Information()
.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
.MinimumLevel.Override("Microsoft.Hosting.Lifetime", LogEventLevel.Information)
.Enrich.FromLogContext()
.WriteTo.File("./logs/log-.txt", rollingInterval:RollingInterval.Day)
.CreateBootstrapLogger();
try
{
Log.Information("Starting the Service");
IHost host = Host.CreateDefaultBuilder(args)
.UseWindowsService()
.ConfigureAppConfiguration((hostContext, configBuilder) =>
{
configBuilder
//.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json")
.AddJsonFile($"appsettings.{hostContext.HostingEnvironment.EnvironmentName}.json", optional: true, reloadOnChange: true)
.AddJsonFile($"appsettings.{Environment.MachineName}.json", true, true)
.Build();
})
.UseSerilog((context, services, configuration) => configuration
.ReadFrom.Configuration(context.Configuration)
.ReadFrom.Services(services)
.Enrich.FromLogContext())
.ConfigureServices((hostContext, services) =>
{
...
})
.Build();
await host.RunAsync();
}
catch (Exception ex)
{
Log.Fatal(ex, "There was a problem starting the service");
}
finally
{
Log.Information("Service successfully stopped");
Log.CloseAndFlush();
}
I have this in appsettings.json:
"Serilog": {
"Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File", "Serilog.Sinks.Debug" ],
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Warning",
"System": "Warning"
}
},
"WriteTo": [
{ "Name": "Console" },
{
"Name": "File",
"Args": {
"path": ".\\logs\\log-.txt",
"rollingInterval": "Day"
}
}
],
"Enrich": [ "FromLogContext" ]
}
If I include the "SetBasePath" part, the service fails to start (although it still runs via Visual Studio).
I am publishing the service from Visual Studio with Target Framework = net6.0, Deployment Mode = Framework Dependent, Target runtime = win-x64.
Note: I have created a Logger in Program.cs because I wanted to log the start and stop / crash of the service.
As others have noted, services run with a working directory different than the process location. One old trick I like to use is to set the working directory to the process' binary directory first thing:
Environment.CurrentDirectory = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
In my case, I just changed Deployment mode to Self-Contained and choose the Produce single file.
Although it will create a large file but now compiler embed all dependent libraries into a single file.
I am writing an azure webjob which reads events from eventhub using NET Core 3.1.
I have a config file as below:
{
"JobHostConfig": {
"DashboardConnectionString": "",
"StorageConnectionString": "xx"
},
"EventHubConfig": {
"EventHubConnectionString": "xx",
"EventHubName": "xx",
"EventProcessorHostName": "xx",
"ConsumerGroupName": "xx",
"StorageConnectionString": "xx",
"StorageContainerName": "xx"
}
}
In the Main method, I call ConfigureServices method which looks something like:
var builder = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory() + $"\\..\\..\\..\\ConfigFiles")
.AddJsonFile($"applicationConfig.json", optional: true, reloadOnChange: true)
.AddJsonFile($"applicationConfig.{environment}.json", optional: true, reloadOnChange: true);
Configuration = builder.AddEnvironmentVariables()
.Build();
services.AddSingleton<IConfigurationRoot>(Configuration);
services.AddSingleton<IConfiguration>(Configuration);
services.AddMvcCore();
services.AddSingleton(GetInstance<EventHubConfig>());
services.AddSingleton(GetInstance<JobHostConfig>());
I confirmed that at runtime configs are getting populated in Configuration only like this: Configuration["EventHubConfig:EventHubName"]. But I also debugged that environment variables have not been set and its value is null.
So when I do:
ProcessEvent([EventHubTrigger("%EventHubName%", ConsumerGroup = "%ConsumerGroupName%", Connection = "%ConnectionString%")] EventData eventData) I get that %EventHubName% is not resolved.
Also, when I hard-code the values of these then I get: No event hub receiver named.
Can someone suggest what is wrong with my registration?
Furthermore, I replaced the values with string in EventHubTrigger, and I get Value cannot be null. Parameter name: receiverConnectionString
When using %% format, you should use Custom binding expressions.
For more details, please refer to this answer.
Please let me know if you still have more issues about it.
Right click the appsettings.json file -> click propertities -> set the "Copy to output directory" to "copy if newer"
and the code should be
public static void Trigger([EventHubTrigger("my eventhub name",Connection = "EventHubConnectionString")] EventData message, ILogger logger)
{
string data = Encoding.UTF8.GetString(message.Body);
Console.WriteLine(data+";;xxx");
}
Make your appsettings.json simpler like
{
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;EndpointSuffix=core.windows.net",
"EventHubConnectionString": "Endpoint=sb://xxxx"
}