Currently we have set up our UI test automation in one environment(Dev). We are using C#, .Net, Visual Studio, specflow and MSTest
CONFIG
We read an app.config file for environment specific variables. We use Azure Pipeline for CI and run these tests on a nightly build.
<configuration>
<appSettings>
<add key="DevApp" value="https://dev.websitename.com />
</appSettings>
<connectionStrings>
<add name="DevDatabase" connectionString="http://dev.url/" />
</connectionStrings>
</configuration>
Now we want to run these tests on our UAT environment as well. The setup will be the same, and we want to run the same tests with the same data. The only difference is that for UAT we will point to different URL's and a different database.
for example
Dev env = https://dev.websitename.com
UAT env = https://uat.websitename.com
server name="DevDatabase" connectionString="http://dev.url/"
server name="UATDatabase" connectionString="http://uat.url/"
PASSWORDS
In terms of password, out application is an internal application and we use windows auth. So in our dev and uat environment we have the same password set up for all users. So in Dev = devpassword and UAT = uatpassword
For both dev and test we are using the same users with the password being the only difference.When testing we launch the browser using impersonation and launch the browser as 'run as' for that user
var service = ChromeDriverService.CreateDefaultService(driverpath)
if user is not null then we do this
var pwd = new SecureString()
service.StartDomain = Configurationhelper.Domain
service.StartupUserName = username
service.StartupPassword= = pwd
service.StartupLoadUserProfile = true
we store domain and password and other environmental variables in a separate config file as constants.
**Main issue: **
This wont work now so I think it could be best to store passwords as secrets in AZURE pipeline variables? if so, how would i change this code? for example
The server team, db team and devops team have taken care of server,db setup and urls etc
So for me its just configuring the test automation repo with my changes to configuration
What could be an elegant approach to do this?
AZURE PIPELINE
How could we run tests for both these environments in parallel? by parallel i mean having both run on a nightly run. Our Azure pipeline has 2 separate clients UAT and DEV pointing to the same artifact. The tasks and Variable are the same for both environments but with different values obviously
Currently they both would run in isolation
Solutions to this problem come down to how does the context (in your case the environment and all its associated connection strings and URLs) get to the tests where they will be consumed. In your question, you stated several orthogonal concerns:
using the same data
running in a different environment
running in parallel
Not mentioned is another concern
how to handle secrets (e.g. passwords in connection strings)
I'll explain one solution (a strategy really) that addresses these concerns, and why it appears to be a maintainable and extensible solution.
Using the same data
This can be very simple or very complex. The simple solution is to create a database of canonical and representative test data, and to then swap in that database to your environment. That can be done via your database's backup/restore or by creating the data programmatically. You would need to have a mechanism to restore or wipe the data whether the test(s) succeeds or fails.
Very rarely will using the environment's database "as is" lead to reliable tests; tests often modify state, and a database is the ultimate form of state; mutations of state will affect future tests.
It is with this last sentence that a full swap that occurs before/after each test is probably a) faster (occurring at a bulk/macro level with a quicker swap function), b) more maintainable (data can be reviewed/created ahead of time) and c) less brittle.
Running in a different environment
This, like the heart of your question discusses, is where you come down to whether to use multiple files or a single file. Using multiple files means you can take advantage of some of the built-in .NET configuration mechanisms that allow you to specify the environment. That means duplicating the files and changing the values to reflect the environment.
The other way, you mentioned, is storing all of this information into a single configuration file. If you do it this way, you need some sort of discriminator to disambiguate the entries, and your test needs to be able to pass in the environment name to some API to pull the value. I prefer this mechanism personally, because when you add a new feature/test you can add all the configuration in one place.
So the two mechanisms are roughly the same in terms of work, except the latter leads to a more compact editing session when adding new config, and from a readability/maintainability scenario, you have fewer places to look.
Secrets
If you follow the single-source of configuration approach, you simply extend that to your secrets, but you select the appropriate secret store (e.g. a secrets file or Azure Key Vault, or some such... again with an environment-based discriminator). Here's an example:
{
"DEV.Some.Key" : "http://devhost/some/path",
"UAT.Some.Key" : "https://uathost/some/other/path"
...
}
Using a discriminator means far less changes to your DevOps pipeline, which is, from a developer/editing experience, most likely slower and more cumbersome than editing a file or key vault.
Running in parallel
While you could rotate out the context and design your solution to run in parallel using the MSTest mechanisms, it would be more elegant to allocate this to your pipeline itself, and have enough resources to be able to run these pipelines in parallel by having enough build agents and so on.
Conclusion
It comes down to which parts of the solution should be addressed by which resources. The solution above allocates environmental selection and test execution into the pipeline itself. Granular values such as connection strings and secrets are allocated to a single source to reduce the friction that occurs when having to edit these values.
Following this strategy might better leverage your team's skills as well. Those with a DevOps mindset can most likely spin up new environments and parallelize more readily than a Developer mindset, who would be more aware of what data needs to be setup and how to craft the tests.
Related
I was wondering if it is possible to run multiple MassTransmit or RabbitMQ instances on the same server. Basically we have a .net app using MassTransmit on top of RabbitMQ. Unfortunately a lot of our clients run both live and test environments on the same server so in order to deploy to the real world we need a way of having either multiple instances or the ability to segregate messages between live and test.
A few ideas I've had
1) Do something like: https://lazareski.com/multiple-rabbitmq-instances-on-1-machine/
The problem here is it relies on a lot of config on clients sites.
2) I could include a header in all messages and each consumer checks for the present of the correct header before consuming the message (e.g. header has 'live' or 'test'.) Obviously this means all messages being received from all instances whether they are meant for them or not which is far from ideal.
Ideally I would like to be able to do something with minimal setup on a clients site, like a virtual sub instance or directory for each environment.
There are two ways to work around this issue.
The first way is the most obvious - you need to use virtual hosts.
From the documentation:
Virtual hosts provide logical grouping and separation of resources.
Separation of physical resources is not a goal of virtual hosts and
should be considered an implementation detail.
Create two virtual hosts in your RMQ instance, called test and prod and the only thing you would need to do on MassTransit side is to change the RMQ connection string:
Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host(new Uri("rabbitmq://localhost/test"), host =>
{
host.Username("username");
host.Password("password");
});
});
So you will use rabbitmq://localhost/prod for production. Naturally, those values should not be hard-coded but come from the configuration.
I believe that virtual hosts cover your needs entirely.
If you really need to run the test environment completely separated, you can just run it in a Docker container. This option will give you the ability to kill the whole thing and start from scratch when you need a clean environment. You can easily remap default ports to avoid conflicts with the production instance:
docker run -d --name test-rmq -p 5673:5672 -p 8080:15672 rabbitmq:3-management
If you run the command above, the new instance will be accessible via AMQP on localhost:5673 and the management plugin will be on http://localhost:8080
I'm working on an AWS serverless application, I'm new to AWS so please correct any misconceptions in this question. We have around 30 lambda functions doing basic CRUD operations on a SQL database hosted in RDS. We used EntityFramework Core to create the database. This means that, in order to use the database, the lambda functions each need access to the EF connection string. I didn't want to leave the connection string in memory (it contains a plaintext password) so I put it in an encrypted environment variable.
The only way I could work out how to do this was to add an encrypted environment variable through the lambda management console GUI to every lambda function individually. This was tedious but it worked. I've now changed the solution a bit and we require a different set of lambda functions, these need environment variables adding to them too and I don't want to do it manually again.
My question:
Is there an easy way to add encrypted environment variables (or something similar) that can be accessed by all of my lambda functions? Is there a method to add them in bulk? A higher-level variable that I can use?
I have tried to find information in the Amazon docs (here for example) but had little success.
Lambda is only concerned about getting the environment variables that it needs and it is not responsible for centrally managing environment variables.
What you would need to do is to handle your environment variables in your deployment process.
How do you currently deploy your 30 lambda functions?
If you use CodeBuild, it can access AWS EC2 Parameter Store where you can centrally store and manage your sensitive environment variables.
Other CI/CD tools also have their own way of centrally managing environment variables.
Update:
You can use the AWS EC2 Parameter Store programmatically. This means that you can retrieve your stored environment variables from inside your Lambda function during startup so you don't need to do this at deployment time.
Doing it on deployment time is still better if that is possible in your use case.
I would not call this as a solution rather a work around to address your problem.
You can configure your environment variables and store it onto S3 bucket.
And using S3 client within your each lambda , you can read specific variable you are interested in.
Hope this helps.
I would suggest to add ConnectionString at AWS console and to not use production connection string at config file to avoid those kinds of problems. Log into AWS console, go to lambda function and add connection string as an environment variable.
PS. In case you will struggle with syntax. If your json config file looks like this:
{
"Settings":
{
"ConnectionString":"",
"SomethingElse"
}
}
Use this syntax to configure connection string:
Settings__ConnectionString
I am late to the party. How about saving your connection string into an DynamoDB table and let each lambda take from it. I assume one connection string for all of it. Most of the time for SQL Server.
I have done the following...
private static IDbConnectionProvider CreateSqlConnectionProvider(DbConfig dbConfig)
{
return new QcDbConnectionProvider(() =>
{
SqlConnectionStringBuilder csBuilder = new SqlConnectionStringBuilder();
if (!string.IsNullOrEmpty(dbConfig.DataSource))
csBuilder.DataSource = dbConfig.DataSource;
if (!string.IsNullOrEmpty(dbConfig.Database))
csBuilder.InitialCatalog = dbConfig.Database;
.
.
.
.
return new SqlConnection(csBuilder.ConnectionString);
});
}
The client is using VERACODE tool for doing code analysis and the VERACODE has detected a flaw "Untrusted initialization" at
return new SqlConnection(csBuilder.ConnectionString);
Also, the dbConfig is being initialized as shown below...
DbConfig configDbConfig = new DbConfig
{
Database = codeFile.ConfigurationDb,
DataSource = codeFile.DataSource,
IntegratedSecurity = sqlCredentials.UseWindowsAuthentication ? 1 : 0,
UserId = sqlCredentials.UseWindowsAuthentication ? null : sqlCredentials.SqlUserName,
ClearTextPassword = sqlCredentials.UseWindowsAuthentication ? null : sqlCredentials.SqlUserPassword
};
What else I need to do in order to fix this flaw? Also as per this link, I am creating the connection string using the SqlConnectionStringBuilder which is safe of creating the connection string.
Thanks in advance...
Description for Untrusted initialization issue is:
Applications should be reluctant to trust variables that have been initialized outside of its trust boundary. Untrusted initialization refers to instances in which an application allows external control of system settings or variables, which can disrupt service or cause an application to behave in unexpected ways. For example, if an application uses values from the environment, assuming the data cannot be tampered with, it may use that data in a dangerous way.
In your case you're reading data for dbConfig from file:
if (TryReadCodeFile(configurationProfileFile...)) {
DbConfig configDbConfig = new DbConfig...
}
Note that warning you get should also come with a line number (to circumscribe erroneous code). Almost everything in code you posted can generate this issue (I don't see where sqlCredentials comes from but it may even be another source of security problems if they're in clear text - or code to decrypt is accessible in your application).
From cited paragraph: "...application allows external control of system settings or variables, which can disrupt service...". This is the core of this issue: if your application uses external data without a direct control over them then its behavior can be changed modifying that data. What these external data are? List is all but not exhaustive:
Environment variables (for example to resolve a path to another file or program) because user may change them. Original files aren't touched but you read something else.
Paths (to load code or data) because user may redirect to something else (again original files aren't touched but you read something else).
Support files because user can change them (in your case, for example, to point to another server and/or catalog).
Configuration files because user can change them (same as above).
Databases because they may be accessible to other users too and they may be changed (but they may be protected).
How a malicious user may use this? Imagine each user is connected to a different catalog (according to their rule in organization). This cannot be changed and it's configured during installation. If they can have access to your configuration files they may change catalog to something else. They may also change DB host name to a tunnel where they may sniff data (if they have physical access to someone else's machine).
Also note that they also say "...assuming the data cannot be tampered with, it may use that data in a dangerous way". It means if, for example, your application runs on a web server and physical access is secured then you may consider that data safe.
Be aware your application will be secure as less secure item in your whole system. Note that to make an application safe (I know, this term is pretty vague) to encrypt password is not enough.
If support files may be manipulated then best thing you can do is to encrypt them with a public/private key encryption. A less optimal solution is to calculate a CRC or hash (for example) you'll apply to configuration files before you use them (they are able to change them but your application will detect this issue).
To summarize: you can ignore this issue but you have to prove your customer that data you rely on cannot be tampered. You can reasonably prove if at least one of these conditions is satisfied:
1) System where support files reside is not accessible by anyone else than your application. Your application security cannot be higher than system security.
2) Your support files are valid per-machine (to avoid copies between different machines) and they're encrypted in a way they cannot be changed (intentionally or not) by anyone.
3) Your support files are valid per-machine and they're hashed in a way your application can detect external changes.
4) It doesn't matter what users do with your configuration files, application itself cannot change its behavior because of that (for example it's a single installation where only one DB and one catalog exist).
The most important for connection strings is how they are stored. If they are stored in plaintext, this poses a security risk. So, it is advisable to store them in encrypted format and in application decrypt and use it.
I currently run MongoDB pointing to the appropriate data directory using the command line below:
mongod --dbpath "somePath/data"
But currently this is a manual step that I run before running a particular suite of tests. Is there a way I can set the path within the code (without calling a script or batch file) using the Mongo C# driver to use a specific data directory?
Update:
To clarify, the reason I'm looking to do this isn't for use in production code, but to isolate test databases for different test suites and to point at a disposable and isolated data directory so that each server instance is clean at the time of running tests and is only populated with the data it requires for the same server settings as production.
You probably won't find any way to do that. The Mongo C# Driver is for programming a MongoClient, not a server. The documentation for C# Driver for MongoDB says - MongoClient class serves as the root object for working with a MongoDB server. When you are programming a client, you automatically would assume that the server is up and running. Whether you do it manually or you write another code for it, that is a different story.
Very rarely would you allow people to connect to a machine and let them start a server AND A CLIENT on it. And why is it rare? You may try to start a server on another machine and screw up with that machine (which may be providing some other completely different service too!). There are some ways (and there are times when it is needed) to start a server remotely, but that is not what you can do using the MongoDB C# Driver.
Now, in order to get your task done, you can try this:
Start one mongod per database on your server, and make each mongod listen to a different port. Then in your code, you can connect your MongoClient to mongod running on the concerned database's port. You can achieve this by using a simple if condition (or a switch case) and checking what database the MongoClient wants to connect to and thus finding the right port value to put in the connection string. Each mongod can serve only one database or more or whatever you want.
So if you are running three mongod's on port1, port2 and port3 and all those three are connected to their respective db paths, the code can be somewhat like this:
var DBNAME = name_of_the_db;
string connectionString;
switch (DBNAME)
{
case name_of_first_DB:
connectionString = "mongodb://[user:pass#]hostname[:port1][/[DBNAME][?options]]";
break;
case name_of_second_DB:
connectionString = "mongodb://[user:pass#]hostname[:port2][/[DBNAME][?options]]";
break;
case name_of_third_DB:
connectionString = "mongodb://[user:pass#]hostname[:port3][/[DBNAME][?options]]";
break;
default:
Console.WriteLine("Invalid DB Name");
}
Answering the updated part of the question:
You can start mongod's on different partitions of the server. Even start the daemons from different drives altogether and make them listen to different ports. Goes without saying that the dbpaths should not be pointing to the same drive for any two databases to at least pretty closely mimic what you wanted.
Just to complete this answer I am adding what #Schaliasos has mentioned in comments.. Consider installing mongo as a window service.
I have a developer tool that I want to run from an internal site. It scans source code of a project and stores the information in a DB. I want user to be able to go to the site, chose their project, and hit run.
I don't want the code to be uploaded to the site because the projects can be large. I want to be able to run my assembly locally on their machine. Is there an easy way to do this?
EDIT: I should note, for the time being, this needs to be accomplished in VS2005.
EDIT 2: I am looking for similar functionality to TrendMicro's Housecall. I want the scan to run locally, but the result to be displayed in the web page
You could use a ClickOnce project (winform/wpf) - essentially a regular client app, deployed via a web-server. At the client, it can do whatever it needs. VS2005/VS2008 have this (for winform/wpf) as "Publish" - and results in a ".application" file that is recognised by the browser (or at least, some browsers ;-p).
You might be able to do the same with Silverlight, but that has a stricter sandbox, etc. It would also need to ask the web-server to do all the db work on its behalf.
I want to be able to run my assembly
locally on their machine
Sounds like you want them to download the tool and run it from their local machine, does that work for you?
Any code can scan files given the location and permissions. For a website to open an exe on a different machine and permit that to run and get access to the files contained on the web server would require a horrifically low level of security that would mean the entire system is practically completely open to attack. If your system is completely behind a firewall and hence protected from outside intererance then you want to look more at the permissions and less at the code.
To run an exe on a machine try following notepad example, though you may have to use a specified directory as well
ProcessStartInfo psi = new ProcessStartInfo("notepad.exe");
psi.WindowStyle = ProcessWindowStyle.Hidden;
Process p = new Process();
p.EnableRaisingEvents = true;
p.Exited += new EventHandler(ExitHandlerToKillProcess);
p.StartInfo = psi;
p.Start();
and when done dont forget to kill the Process. Alternately use javascript. Either way watch the security permissions and remember the risks of doing this.
I would probably write some sort of command line tool or service that does the processing and extraction of project data. Then I would use a page to update/register projects that the web server and the command line tool both have common access to. then at specified times either manually or via cron or similar mechanisms extract the data to your database. once you have this, you just use the website to display last extraction times and the extracted data.
if the projects/end users are on a different subnet etc, then you will need the end users to run the tool and then have it post the data into the database.