The purpose of this is to be able to set the project's .exe file to automatically run on startup and have the tests run. The tests are integrated into BrowserStack where I can view the results. Thanks for any help!
Yes, with the latest changes to the framework, it's possible to do this. Some features may not be available to you. For example, you probably will not be able to recover warning results, which don't throw an exception, or use multiple asserts for the same reason.
Most recently, before the latest release, this was broken btw, so be sure to get the latest.
Related
I'm currently porting a huge project from VS2010 to VS2017 and have troubles running the C# Tests via mstest.exe on Jenkins.
I want to run multiple test.dlls (multiple testcontainer) from different test projects in one run. In earlier versions of VS I could use a testlist in the form of a .vsmdi file. Using this file is deprecated for some versions now. It can stil be used, but it cannot be properly maintained anymore with VS2017:
/testcontainer:"...\Test.dll" vs /testmetadata:..Tests.vsmdi
(containing the paths and metadata of multiple testprojects)
By using giyf I was not able to find another way to do this. Has anyone an idea?
I hope I can keep using mstest.exe, since my whole test setup is using it. If it is not possible, I'm open for alternatives as well.
With MSTest: it is possible specify multiple test containers (mstest /testcontainer:Test1.dll /testcontainer:Test2.dll).
Alternatively, I recomend switching to VSTest.console, which is the successor of MSTest executor since VS2012. See https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options for more information.
I have a UWP project which fails to run unit tests via R#.
I know this was very cumbersome in the beginning of UWP, but I was able to get the unit tests going if I restarted Visual Studio a couple of times. A year later, and a major release further in VS2017 and R# I am still not able to run unit tests...
I'm using:
VS2017 Pro (15.4.3)
R# 2017.2.2 ultimate
UWP solution with target project Creators update (15063)
MS test project (15063)
When I open my solution, it doesn't matter which unit test I run, every test will be inconclusive.
Does anybody recognize these kind of issues and have a workaround for this? Any way to at least allow some kind of TDD way of working?
PS: Of course there isn't an actual stack-trace or inner exception in an output window which I can use for troubleshooting. It's just this pointless message without any useful suggestion how to move further
We have 10K+ unit tests in C# solution which are passed when running in local and in TFS.
Now, we are setting up Jenkins for our solution stack and we are facing issue of around 250 Unitests failing consistently.
The same unit tests are passed when i tried running them in jenkins setup server by using Visual Studio and Commnad prompt(MSTest).
What do you think is the issue? Any leads to look at this issue will be helpful.
Edit 1:
I did the research and not able to find anything as the problem itself is strange one. If you are not clear please raise questions instead of Down voting.
Edit 2:
I am able to find out the issue. It is with the unittests dll config file. When i executed MSTest in server by removing the config file, i am seeing same set of tests failing which are failing with Jenkins setup.
I guess we need to modify the steps configure in Jenkins portal to load the unittests dll config file.
My guess would be you have tests that are not actually Unit tests, but instead are integration tests or worse, and they fail non-deterministically. Other than that, you're asking people to do the impossible here without source code posted.
Either post source, or hire a consultant who knows about Jenkins IMO.
There might be issues with conditional compilation symbols (e.g. DEBUG vs. RELEASE code): in VS you normally run the tests on a DEBUG build, on the CI server on a release build.
Also look at some global state not being cleaned up correctly. Some threads which may still be running after a test has seemingly finished may corrupt later tests even when those later tests are located in a different test dll. That can sometimes be detected if test failure depends on the order of tests run.
Another issue often faced is dependency on test data in files: the files may be missing in the virtualized environment where the test is actually run. Use the Deployment attribute.
I had a similar problem with NCrunch. Maybe you can check in VS your "Build"-tag the Platform target. It should be the same like Jenkins is configured. For example "Platform target" is x64, it should be used in Jenkins as well.
I have this project which requires to run a InstallerClass as part of the installation.
I added it as a Custom Action, and it has been working like that for ages. We did some maintenance to this class, cleaned the solution, rebuild the solution, and now the Custom Actions are not being triggered.
I know they are not being triggered, because I had Debugger.Break() calls working prior to the cleaning, and the fact the registry entries that are meant to be created are not created anymore.
What is going on? What can I do to obtain more info from the MSI installer?
For what is worth, my machine is 64 bit, but the project is mean for and built as 32 bit (x86). I used to build it as Any CPU before, now it doesn't matter what I built it with, no results.
I've been done some more research and found that if Debugger.Break(); is no longer working could be due to a dependency issue, I got no error or warnings on the project though. I ran the MSI on verbose, but I can't make a thing out of it either.
After getting lost on the logs I got nothing useful. I started over again and created a new Installer class with a pop window. that worked. Added a Break() afterwards, it worked too, added the using Process statement I wanted to run and it worked... once.
After that I found that: If I leave the Debugger.Break() it gets skipped, but if I remove it it works...
Yes, this is THAT random... What's the deal?
It would be worth first checking in source control what changes were made to the installer.
In any case, I have seen various weird cases where the installers acted weird. A few suggestions:
Ensure base.Install is last.
Run VS as Administrator.
Recreate the installer project (they can get corrupted).
Try the MSI on another machine - if it works then nothing wrong with MSI and problem is with your machine. It is possible one MSI execution may have left some remnants.
Uninstall the product from your machine and then try MSIZap with the product code. This will cleanup your machine for any remnants that uninstall may have left behind.
Using Visual Studio and TFS & preferably Specflow or standard unit test.
I want devs to run ALL unit test as a policy before check in. If a unit test breaks, then vS should stop them from checking in, just like when running across a merge conflict.
I know there're post build scripts that will do this, but really if unit test breaks, I rather that it doesn't get into source control at all. Plus the turn around is rather slow to wait for the full build. And then there's the bickering on who breaks whose stuff.
So no, I want unit test to pass locally before a check in. How would I do that? Yes they can just hit the button, but I like to get them a bit more "incentive" than that.
It sounds like what you're after is a TFS Gated Check-in. This can ensure that the code builds, merges and that tests run successfully prior to committing the check-in. You can read more about it here:
An introduction to gated
check-in
It's worth noting that it's a much slower process than CI builds, so depending on how many check-ins your developers are doing you may be better off looking at a CI build with 'Create Work Item on Failure' enabled and a Project Alert set up to notify the developer that they broke the build.
The TeamCity Visual Studio plugin supports pre-tested commits. I can't speak for TFS, however.