Develop Reference

where to store database credentials in an ASP.NET core web api project

this is my first project in asp.net core and i don't seem to figure out where to store a postgres db credentials.
i've read that they need to be stored in appsettings.json but if i do so, do i need to add it to the .gitignore file and do i have the ability to add it when pushing to production or should i use a .env file.
PS: i still don't know where the project will be hosted so do all hosts support appsettings.json configurations ?

Answer to second question - appsettings.json is just a file, part of dotnet publish output and it should work on any host that supports uploading files.
This is somewhat pet issue of mine, so answer to first question will be longer. You definitely don't want your passwords to go to git, you are right on that. If you want to stay with official solution, you are supposed to use application secrets for local development, and appsettings.{Develoment,Staging,Production}.json on servers where you deploy the application. They stack, what is in appsettings.json will be overriden by anything you put in one of the .env files with same key. I myself do have several issues with this approach.
appsecrets only work in development environment (unless you rewrite your default Startup class). If you are doing both development and maintenance, you have to mentally switch between two approaches to one thing.
appsecrets have their own special utility which you have to learn, while appsettings.* family are files in a well known path inside a project that you can edit with anything (secrets are a file too, with the same syntax, but it lives somewhere deep in %APPDATA%, out of normal reach. You have to search for it).
appsettings.{env}.json - should they be in git or not? No offical answer on that that I know of. Most projects I dealt with had some kind of a problem with these two files. If it lives in git, you have the same problem as appsettings.json - you can't put sensitive info into them. And you have to exclude if from publishing, because your git version won't have correct passwords. I have seen a few horrible teamcity scripts trying ensure this during build/publish step, and then randomly fail when devs change things that should have no impact on this (like target dotnet version). If they are not in git and you DO need to update server version? You have to make sure to merge the changes to live version manually. Which is easy to forget.
Neither of those points is a dealbreaker, but they leave bad taste for me.
After some meditation on those issues, I introduced appsettings.local.json file as standard part of any of our projects. It's in .gitignore, it never leaves the machine.
return Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((hostBuilderContext, configurationBuilder) =>
{
ContentRootPath = hostBuilderContext.HostingEnvironment.ContentRootPath;
for (int pos = configurationBuilder.Sources.Count - 1; pos >= 0; --pos)
{
if (configurationBuilder.Sources[pos] is JsonConfigurationSource)
{
var source = new JsonConfigurationSource()
{
Path = Path.Join(ContentRootPath, "appsettings.local.json"),
Optional = true,
ReloadOnChange = true,
};
source.ResolveFileProvider();
configurationBuilder.Sources.Insert(pos + 1, source);
}
}
})
This belongs in Program.cs (old style, it needs slight modification if you use toplevel statements) and places it into the chain of "stock" configuration providers, just after all other configuration files. They continue to work as before, but if .local file is present, values from it override other .json files.
With this, appsettings.json serves just as a template, it's tracked by git and if you have some discipline, it also serves as a structured overview of everything that can be configured, with default values. If the project has different needs for staging/prod environments, corresponding .env files still work, and can be tracked by git. Passwords go to .local file, everywhere.

Unsure how to set C# Connection string depending on solution configuration

I'm trying to figure out how to use Solution Configuration Management in Visual Studio to set a MySQL Connection string depending on what build configuration but am fairly new to this.
I've found plenty of info on the XML for the App.config file for creating connection strings, but am unclear on how to tell the different configuration modes (excuse my lack of known terminology) to use different connection strings.
EG. I want independent environments for development and production, each use a different SQL database so as to not be developing on any production database.
What I have so far in app.config:
<connectionStrings>
<add name="Debug" connectionString="debguconnectionstring"/>
<add name="Production" connectionString="productionconnectionstring"/>
</connectionStrings>
It may simply be that my lack of terminology is hindering my Google-ing capability, so even if someone could point me to a guide or dupe thread that would be excellent.
Thanks!
EDIT:
Thanks all for your answers, sadly I can only mark one of them as the correct answer. +Rep for everyone, exactly what I was looking for. Thanks again!

This is the article which shows how to build such a config:
https://mitasoft.wordpress.com/2011/09/28/multipleappconfig/
Points to remember:
give same name to the connection string, only change the other details, so in code you'll be using same name but it will change depending on the Configuration (release vs debug)
PS: Sorry not enough reputation points to post as comment to question

You will need three config files, like this:
- app.config
- app.Debug.config
- app.Release.config
Also there is already a question for this here:
How to select different app.config for several build configurations

You're looking for config transforms, (msdn article: https://msdn.microsoft.com/en-us/library/vstudio/dd465318%28v=vs.100%29.aspx)
However this will only work for web project. Using this in other projects will require custom code or, as I usually do, external packages, e.g. Slow Cheetah (https://www.nuget.org/packages/SlowCheetah/) that will allow you to use transforms in any config file.
The syntax is quite simple. You define the same node on the additional configuration file and then set the attribute "xdt:Transform" with the desired mode and "xdt:Locator" with the attribute you want to match with the transform (e.g. giving it xdt:Locator="Match(name)" will transform the item with the same name attribute)

Managing passwords in continuous deployment

We are well into our deployment of continuous integration environment using TeamCity. As we work through the CI process and move toward continuous deployment, we have run into a problem with how we manage production passwords. For other changes in the config, we use the Web.Config transform. However, I don't really want to bake the production password in a build profile.
Before CI/CD, we would take the Web.config, use aspnet_regiis to decrypt the connection strings, change the password, then re-encrypt. Obviously, this is error prone and not at all in the spirit of CI/CD.
I've had several other thoughts that were basically all about using something in the deploy script to re-write and then encrypt the connection strings section of the file, but it seems like this must be a common problem and that there must be some generally accepted solution. But so far, I can't find it. Is there a "right way"?
Thanks!

One possible solution, available since TeamCity 7.0, is to use typed parameters. You can define a parameter in TeamCity of type password, and pass it somehow to your build script (either as environment variable or as your build script property).
TeamCity stores values of such parameters in its own configuration files and in database in scrambled form. If password appears in build log or on build parameters page, it will be replaced with ***.

Use config transformations. You can even build your own transformation that can handle encryption/decryption. The easiest way is encrypt the production strings in the release.web.config and use a transformation to handle replacing the connection strings.
http://msdn.microsoft.com/en-us/library/dd465318.aspx
http://sedodream.com/2010/09/09/ExtendingXMLWebconfigConfigTransformation.aspx
If this doesn't work for you, use a postbuild event to call aspnet_regiis. If you chose to extend the config transformation, you can do ANYTHING with it. The encryption keys could be on the moon as long as you can get to them.

Can't you define the connection strings IN iis. Do they change alot? I think i'd just set them in IIS for a site, and then not deploy them as part of the app, or atleast take preference of my IIS settings.

How to make the connection string of an application in C#.net,Computer independent?

As I transfer my application in C#.net from one computer to another, I have to change the connection string every time as the location of the Database file has changed.
How can I can I prevent this,so that I don't have to change the connection string again and again?

When location of the database changes, something has to change somewhere. Back in the ODBC days, you could define a system-wide connection and specify the just the name in the connection string. But if the server moves you would have to change the ODBC anyway.
I can think of a few solutions here. One is that if your database runs on the local machine, use the localhost instead of the machine name.
In case it is a file, create a network share and put it on that so that you use \\localhost\shareName\file.db.
If it is a server database and could be on other machines, use a DNS name by using a host file and assign a common name so that you could do that in different networks.

This one is an old one that comes up from time to time.
Visual Studio has the ability to support multiple build environments and will allow you to specify environment specific values (i.e. databse connection strings) so that you may test wherever you want.
Pulled this from the Gu's blog and full post may be seen here:
It turns out you can easily automate this configuration process within the Visual Studio build environment (and do so in a way that works both within the IDE, as well as with command-line/automated builds). Below are the high-level steps you take to do this. They work with both VS 2005 and VS 2008.
Use ASP.NET Web Application Projects (which have MSBuild based project files)
Open the VS Configuration Manager and create new "Dev", "QA", "Staging" build configurations for your project and solution
Add new "web.config.dev", "web.config.qa", and "web.config.staging" files in your project and customize them to contain the app's mode specific configuration settings
Add a new "pre-build event" command to your project file that can automatically copy over the web.config file in your project with the appropriate mode specific version each time you build the project (for example: if your solution was in the "Dev" configuration, it would copy the web.config.dev settings to the main web.config file).
In VS 2010 I believe most of the setup for this is done for you when you make your new application, but the principles are still the same. You can skip the pre-build event in favor of pulling the connection string in via code if you're comfortable with that way of doing things. There is an example of this on StackOverflow here on this answered question
The start of the answer (or at least how I'm more likely to implement this) is to store each connection string value in a separate connections.config file with a name unique to the environment. This will only contain the values from connectionStrings portion of the web.config file. I like doing this because it means adding a new environment doesn't mean a developer touching the main web.config file and you can also do some server-side trickery to remove the database connection strings from the site's webroot making it a bit more secure.
Method for removing the connectionStrings to another fole may be seen here: http://www.bigresource.com/Tracker/Track-ms_sql-cberGbNT/
I would also look into having a network accessible version of the database instead of making each developer have one in their environment. If you have to have disconnected work being done that makes sense, but from a management standpoint there should really only be one database for each stage of the project (Development, Staging, and Production).
Hope that's helpful, and feel free to ask follow-up questions on anything that doesn't make sense.

Use a relative path. For most scenarios, you can just use the built-in |DataDirectory|, which translates to the base directory you're running from (or App_Data in ASP.NET scenarios).
If you are using a client-server database, like SQL Express, then you should also set the server name to something like .\SQLEXPRESS - which will use the local instance.

what I usually do is having a configFile.txt that can be modified.
I can easily read it from the program and since it's path is fixed (relative path), I don't need to change the code anymore.
I don't know if this is a good or bad habit but it works fine.
Here it takes the first line :
public string pathConfig = "../../myProject/configFile.txt";
string yourDBPath = File.ReadLines(pathConfig).Skip(0).Take(1).Split(':')[3];

Understanding .Net Configuration Options

I'm really confused by the various configuration options for .Net configuration of dll's, ASP.net websites etc in .Net v2 - especially when considering the impact of a config file at the UI / end-user end of the chain.
So, for example, some of the applications I work with use settings which we access with:
string blah = AppLib.Properties.Settings.Default.TemplatePath;
Now, this option seems cool because the members are stongly typed, and I won't be able to type in a property name that doesn't exist in the Visual Studio 2005 IDE. We end up with lines like this in the App.Config of a command-line executable project:
<connectionStrings>
<add name="AppConnectionString" connectionString="XXXX" />
<add name="AppLib.Properties.Settings.AppConnectionString" connectionString="XXXX" />
</connectionStrings>
(If we don't have the second setting, someone releasing a debug dll to the live box could have built with the debug connection string embedded in it - eek)
We also have settings accessed like this:
string blah = System.Configuration.ConfigurationManager.AppSettings["TemplatePath_PDF"];
Now, these seem cool because we can access the setting from the dll code, or the exe / aspx code, and all we need in the Web or App.config is:
<appSettings>
<add key="TemplatePath_PDF" value="xxx"/>
</appSettings>
However, the value of course may not be set in the config files, or the string name may be mistyped, and so we have a different set of problems.
So... if my understanding is correct, the former methods give strong typing but bad sharing of values between the dll and other projects. The latter provides better sharing, but weaker typing.
I feel like I must be missing something. For the moment, I'm not even concerned with the application being able to write-back values to the configuration files, encryption or anything like that. Also, I had decided that the best way to store any non-connection strings was in the DB... and then the very next thing that I have to do is store phone numbers to text people in case of DB connection issues, so they must be stored outside the DB!

If you use the settings tab in VS 2005+, you can add strongly typed settings and get intellisense, such as in your first example.
string phoneNum = Properties.Settings.Default.EmergencyPhoneNumber;
This is physically stored in App.Config.
You could still use the config file's appSettings element, or even roll your own ConfigurationElementCollection, ConfigurationElement, and ConfigurationSection subclasses.
As to where to store your settings, database or config file, in the case of non-connection strings: It depends on your application architecture. If you've got an application server that is shared by all the clients, use the aforementioned method, in App.Config on the app server. Otherwise, you may have to use a database. Placing it in the App.Config on each client will cause versioning/deployment headaches.

Nij, our difference in thinking comes from our different perspectives. I'm thinking about developing enterprise apps that predominantly use WinForms clients. In this instance the business logic is contained on an application server. Each client would need to know the phone number to dial, but placing it in the App.config of each client poses a problem if that phone number changes. In that case it seems obvious to store application configuration information (or application wide settings) in a database and have each client read the settings from there.
The other, .NET way, (I make the distinction because we have, in the pre .NET days, stored application settings in DB tables) is to store application settings in the app.config file and access via way of the generated Settings class.
I digress. Your situation sounds different. If all different apps are on the same server, you could place the settings in a web.config at a higher level. However if they are not, you could also have a seperate "configuration service" that all three applications talk to get their shared settings. At least in this solution you're not replicating the code in three places, raising the potential of maintenance problems when adding settings. Sounds a bit over engineered though.
My personal preference is to use strong typed settings. I actually generate my own strongly typed settings class based on what it's my settings table in the database. That way I can have the best of both worlds. Intellisense to my settings and settings stored in the db (note: that's in the case where there's no app server).
I'm interested in learning other peoples strategies for this too :)

I think your confusion comes from the fact that it looks like your first example is a home-brewed library, not part of .NET.
The configurationmanager example is an example of built-in functionality.

I support Rob Grays answer, but wanted to add to it slightly. This may be overly obvious, but if you are using multiple clients, the app.config should store all settings that are installation specific and the database should store pretty much everything else.
Single client (or server) apps are somewhat different. Here it is more personal choice really. A noticable exception would be if the setting is the ID of a record in the database, in which case I would always store the setting in the database with a foreign key to ensure the reference doesn't get deleted.

Yes - I think I / we are in the headache situation Rob descibes - we have something like 5 or 6 different web-sites and applications across three independent servers that need to access the same DB. As things stand, each one has its own Web or App.config with the settings described setting and / or overriding settings in our main DB-access dll library.
Rob - when you say application server, I'm not sure what you mean? The nearest thing I can think is that we could at least share some settings between sites on the same machine by putting them in a web.config higher in the directory hierarchy... but this too is not something I've been able to investigate... having thought it more important to understand which of the strong or weak-typed routes is 'better'.