I have a UWP project which fails to run unit tests via R#.
I know this was very cumbersome in the beginning of UWP, but I was able to get the unit tests going if I restarted Visual Studio a couple of times. A year later, and a major release further in VS2017 and R# I am still not able to run unit tests...
I'm using:
VS2017 Pro (15.4.3)
R# 2017.2.2 ultimate
UWP solution with target project Creators update (15063)
MS test project (15063)
When I open my solution, it doesn't matter which unit test I run, every test will be inconclusive.
Does anybody recognize these kind of issues and have a workaround for this? Any way to at least allow some kind of TDD way of working?
PS: Of course there isn't an actual stack-trace or inner exception in an output window which I can use for troubleshooting. It's just this pointless message without any useful suggestion how to move further
Related
I installed VSCode for Unity game development a few weeks ago and have been using it as an alternative to my usual setup of VS + Resharper. I enjoy VSCode quite a bit, especially given how fast and lightweight it is.
One problem I have, however, is writing tests. It's standard practice at our studio to create unit tests for certain features we implement, and we so we have an assembly for running tests in a Unity project.
There's a testing .asmdef that references the NUnit framework DLL and the assembbly that has the code I want to test. In VS, if I navigate through the code, it all looks fine. If I switch to VSCode, however, it tells me there's compilation errors all over the place, because VSCode can't see any of the NUnit classes or the classes from the assembly I want to test (in Unity, I get 0 compilation errors even when writing new code).
Is there any VSCode setup I can do to get rid of the compilation errors and write tests normally, or do I have to switch to VS to write tests?
UPDATE: A work colleague helped me out and linked me this extension which clears up the compilation errors.
It looks like it lets me run tests from the IDE, too, but I haven't set it up yet as I just run them from Unity's Test Runner window. This isn't a priority for me, so I'll mark this as the answer.
I'm trying to run unit tests from the command line.
For this, I need the test dlls, as per this:
https://msdn.microsoft.com/en-us/library/ms182490.aspx
In the Prerequisites section on that page, it says I need to "Run a Unit Test and Fix Your Code"
Question 1: Do I need to run tests from VS to be able to run them from command line? Surely not?
Now to the more important bit. When I build my solution, regardless of whether it's via VS or command line using MSBuild, I do not get any dlls generated for my tests.
I know this is for C++ but thought might still be relevant https://social.msdn.microsoft.com/Forums/vstudio/en-US/a89c2173-90e6-47b2-af8e-48865969cbca/msbuild15-does-not-create-a-dll-file-after-building-the-c-native-unit-test-project?forum=msbuild. (not that it helped of course).
Question 2: Why don't my unit test dlls get generated when building?
UWP is a bit different from other projects when it comes to Unit Testing. Universal Windows Apps run in a sandbox to make sure they don't do things they don't have permissions for. For this reason, the unit test project is not a simple DLL only, but in fact a full-fledged UWP app (and the generated executable has appx extension), that is launched and performs the tests.
That said, you can still launch the unit test project from the console using a special command as you can see in this SO answer.
vstest.console.exe /Platform:x64 AppPackages\UnitTestProject1_1.0.0.0_x64_Debug_Test\UnitTestProject1_1.0.0.0_x64_Debug.appx
Over the last 6 months our test team have been using selenium webdriver to test our web based products. We have had great success with it and continue to use it on a daily basis. We use visual studio since we are a .net shop to write our c# unit tests. We don't use any other testing frameworks for .net.
We have up until recently been running our automation tests manually through the test explorer window in visual studio (2013), checking on the results and then logging them into a spread sheet. I know this isn't ideal but we don't have that many tests so this has suited us fine thus far. However, the number of automation tests we will be required to write and maintain is due to rapidly increase over the next few months.
I have recently been playing around with creating batch files for calling vstest.console.exe and its various commands and then adding those logs to a server. Not ideal. I know I can still do so much more, specifically integrate some sort of CI server.
We are already using team foundation server and have various virtual servers (all running windows 8.1) at our disposal so I thought about taking advantage of this so I began looking into Jenkins. Trouble is, I'm not finding much information regarding Jenkinks and c#. It looks primarily geared to a java setup. Am I missing something? What little information I have found is seriously outdated and didn't work for me.
I got as far as setting it up and installing the vstest.console.exe plugin but couldn't get a simple test to run. A current step by step guide that doesn’t pre-date 2012 would be great :)
Do you guys think Jenkins is the way to go for c# and the .net framework? Is there a "standard" used within the c# community? I have heard of cruise control and I’m going to check that out. Is it a viable alternative? Easier to use with .net?
Here is essentially what we need:
Continue writing our tests inside visual studio and creating c# unit tests
Schedule a run of our unit tests on a remote / local server
Write out a result / log file - nice reporting features on fails / passes would be great
Email said file to qa / dev teams
I'm hoping some of you guys have been down this road once and can share some insights
It is possible to use Jenkins to run tests via batch scripts, reporting back to Jenkins via the NUnit or MSTest plugins. To do this, simply call the test runner from a Jenkins-executed script (see links below). The primary reason for doing this in my shop is that Jenkins is used to automate the build process, and automated tests are run every time new code is promoted. If you don't use Jenkins for build automation and reporting - i.e. you just want scheduling - the most basic solution would be Task Scheduler (as John O indicated). Plus, if you are using MSTest rather than NUnit then, as others have suggested, it is better to have a look at TFS.
If you really want to use Jenkins with MSTest, the following links may be useful from a configuration perspective:
Error trying to run mstest on jenkins - 2012
Example of running MSTest from Jenkins from above link:
"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe" /testcontainer:Example.Tests\bin\Debug\Example.Tests.dll /resultsfile:TestResults.trx
Jenkins MSTestRunner plugin is unable to launch mstest.exe - 2014
If you can/want to use NUnit, check out the following:
How do you run NUnit tests from Jenkins?
Execute NUnit with Jenkins
Actually, for reporting purposes, logging to Excel or something similar isn't a horrible idea (particularly if your organization uses Sharepoint). Worst comes to the worst and you can't get anything to work, then one solution would be automating this reporting process by using Excel's COM Object Model to directly write results to the spreadsheet.
Would still suggest that TFS is your best bet, however.
I am going to give you some information on how the test can be handle from CI not exactly how to configure Jenkins or other CI server. This might be a partial answer to your question but at least something. Keep that in mind selenium tests always need a live site to point to and before you kick off the tests site deployment should happen(either manually or automatic). I prefer running the test suite once at night and let it go for whole night. You can simply accomplish this with the help of NUnit console, batch and windows task scheduler. See my answer here how to do that. Sure enough this can be accomplish from any other CI server. With NUnit Command switch you can export result automatically also.
I am trying to use xUnit.net to test a project I wrote in Mono on OSX. I wannt good way to run the tests in MonoDevelop or at least a way to create a test suite and run these tests from a main method of the project. Has anyone tried doing this? I remember in NUnit I would be able to create test suites and run them myself but I am not able to figure out how to do the equivalent in xUnit.net.
Let me know if anyone has found a good way to run tests without using a runner in xUnit.net or a better way to run tests in OSX MonoDevelop.
You can run your tests using xunit gui (use mono xunit.gui.flavor from terminal) - I'm not aware of an xunit addin for MonoDevelop that could integrate an xunit test runner into the MonoDevelop IDE (another reference: http://xunit.codeplex.com/discussions/439410).
To debug your unit tests, you can attach the debugger to the xunit gui. To make this simpler, you can customize the projects run configuration: http://www.grumpydev.com/2011/06/30/debugging-xunit-tests-using-monodevelop/
It's a decision you have to make whether you want to jump through a few hoops to use xunit or just use NUnit that's integrated well with the platform.
Is there a tool for automated test discovery for .NET. I am using the Visual Studio unit testing stuff and wanted functionality similar to Python Nose. I need to have a tool automatically discover all the unit test available and run for example the ones "marked" as unit and in different scenarios run the tests "marked" as Integration and so on. I have found an individual that has created his own implementation of the MSBuild test task and an considering creating my own with annotation attributes to do what Nose does but wanted to see if anyone was aware of an existing tool that could work.
Thanks
Visual Studio integrated test framework does exactly that when running from the IDE.
If you need a command line tool that does exactly the same functionality (finds all the tests in a specific directory/solution) I guess you have to write something.
Because MSTest command line needs at least the assembly to be specified. I suggest you write a short script that iterates all the assemblies and find if they have tests in them and then run each assembly using MSTest.exe
Update:
I've just published a new CodePlex project called #Nose that does exactly what you need. Currently it only supports NUnit but I plan on adding VSTest as well.
Try ReSharper from JetBrains. It does what you want with unit tests plus a whole lot more.