We have a custom automation framework in place. So for running scripts we have a windows application for trigger some test case. Test cases are simple C# methods and we invoke them using Reflection.
Now My requirement is to integrate this framework with MTM. and would like to have continuous integration in process. This means I want if some build comes then some of the test case will automatically start running to secure our code.
How can I achieve this?
More specifically my requirement is I pushed some code related to some PBI, So the testcase related to that requirement should automatically run.
Generally the CI (continuous integration) is against the build process in TFS.
If you mean run the tests using custom test framework, then you need to create a Test Adapter accordingly, so that VS and the Build server can pick up the tests. Here is a good article for your reference on how to do this: Authoring a new Visual studio unit test adapter
Actually you can run the tests in build process, with the CI set, the new build will be triggered once a new change checked in, then the tests will run automatically. Refer to below articles for details:
vNext build - Testing in Continuous Integration and Continuous
Deployment Workflows
Xaml build - TFS, Automated Testing and Continuous Integration
If you still want to use MTM, you can reference below articles:
Automate a test case in Microsoft Test Manager
Continuous Delivery with TFS: Running Automated Web Tests with
MTM
Related
I was wondering if it would be possible to either write or deploy an existing test runner along side my MVC application. The tests could be referenced from the test Runner application and deployed separately from the main application.
The aim is to run the tests remotely on client site as a sort of diagnostics tool.
It would be great if we could have a little more information when things go wrong.
Is this possible?
I have NUnit selenium tests that i have embedded into Unit test and i am trying to run this using TFS Build 2015. All the tests run perfectly fine locally but not from the TFS Server. I enabled code coverage and found out that "Module unittests.dll" works fine and most of the code is covered but for "Seleniumtest.exe" 0 lines covered.
I have read a lot on this problem and came across the idea of enabling the TFS Agent to run in interactive mode but i do not know how to do this from the TFS Server and most of the documentation online is not good enough.Thanks
If you're using the Hosted agent queue, you can't. If you have your own on-premise build agent, you just need to run the agent from a command prompt (as opposed to as a service). That's all that "interactive" means.
You can also use the Test Agent tasks to deploy a test agent to a target machine and invoke the tests on them. This will also allow you to run UI tests even if you're using the hosted agent queue -- you'll be passing off the running of the tests to an on-premise test agent.
On a side note, you should generally not run UI tests as part of a build process, but rather as a step in a release pipeline after the application has been released to a QA environment. UI tests are slow and brittle, and you don't want your CI process to be bogged down with running UI tests.
I have been trying for a while and I am not able to run Load Tests with my Build Agent. I have a solution with different projects, running Coded UI Tests, Web Performance Tests; all of them running fine on the Build Agent. The problem is that I can't run Load tests.
I started thinking that it was not possible, but I found this at Visual Studio online documentation:
The build agent can run:
Code Coverage
Coded UI Tests (Requires a build server that is running in interactive
mode)
Database Test Data Generation
Database Unit Tests
Generic Tests
Load Tests
Unit Tests
Ordered Tests
Test Impact Analysis
Web Tests
So I guess it is possible somehow. Does anybody know how to do it?
I have a bunch of C# unit tests on my teams Team Foundation Server. Normally we Run these tests using Jetbrains test-suite. However, I've like to make these tests available to the QA department, through a browser.
Is there any way to easily make C# unit test runnable through the web? Anybody who have experience with this?
I have googled it and i'm only able to find client applications to run unittests.
Thanks in advance!
Your Unit Test should be for developers and Continuos Integration process, QA departement should run UI/Web Coded/Performance Test by VS Test Manager; anyway if you configure the execution of the test suite in the build configuration, test result will be available also from TFS Web Access
Run tests in your build process
I'm running some automated Continuous Integration tests through a TFS build. The tests are written using the Microsoft UI Automation Framework on MSTest, and are configured to run locally on the build agent, without any controller or agent.
Now, the tests themselves execute well. The Startup method of the test suite launches the compiled application's executable and the process shows up in the server's task manager. However, since this is done through the Team Foundation Build Service, the UI isn't actually shown, as services are not allowed to interact with the active desktop.
Is there a way to force the application's UI to appear on the active desktop? I'm aware that I can install a test controller and agent that allow tests interaction with the desktop, however this creates additional overhead and some technical problems, as we're using VS2012 against TFS2010, which goes directly to the VS2010 MSTest and thus to the VS2010 test controller & agent, which I cannot configure through VS2012.
I ended up setting up a test controller a test agent. Apparently, with VS2010 it's not possible to install a test agent on the same server with the build controller, so I went for a separate machine altogether.
The tests run nicely now!
With what account are you logged on to the build server?
Only if you are logged on with the build account you will see the interaction with the desktop/UI.