We are developing and maintaining a legacy winforms application. The application uses a domain model which ist quite separate from the GUI layer, but because of many dependecies, it is hard to unit test. At least we have a build and an installation script which we can use to automatically build and deploy the application.
Now if we could only have some automated system tests...
We came up with the following idea:
First, we could write our tests in script files like this:
public class Test
{
public void TestSaveACustomer()
{
var c = new Customer();
c.Name="Miller";
Mapper.Save(c);
Assert.IsTrue(c.CustomerId >0);
}
}
Then, after building and deploying the application, we could run the scripts with a script runner that is baked into our application binary. We imagine we could use the following commandline
myapp.exe /runTest c:\myScript.cs
We could start to build our own Testing framework, wouldn't that be cool? ... er... no. It would be much less work if we could use an existing testing framework like nUnit, xUnit or MSTest from our application .
I guess I need to
Start a UnitTestRunner
Compile the Script
tell the runner to run the tests in the compiled script assembly.
I have no idea where to start, because I have only used UnitTests from VisualStudio.
Would that be possible with any existing testing framework? Which are the main classes I would use?
Here a existing answer how to run NUnit tests by yourself.
Question is, what does you prevent from running the tests regularly?
To get to run myApp.exe c:\test\test.cs you would also need to compile the test, then run it. You still need to compile it against your app?
Can you debug your application from let's say Visual Studio? Right? So, you could add a 'sub project', which points to the main project for the dependencies and your code. Then you add your tests there, using NUnit regularly.
Or what does prevent that?
Related
I have a C# application built over .NET 4.6.1 (VS2017).
I am now building a test environment and would like to include user base tests.
The user provide a folder of files, each one is a seperate user test i need to run against.
I would like to create a TestMethod for each file to run against my application within VS2017.
Is there a way to automate such task? or the only way is to write a script to create the TestClass implementing a uniqueTestMethod for each file
You can make use of the xUnit tests using parameterised tests with InlineData. Article with examples
Is it possible to embed test code (based on one of the MSTEST, NUNIT, or XUNIT) inside a dotnet core console app?
Based on the tutorials like the one below, it seems there is the requirement that the test code has always to be a separate project than the actual project.
I understand some of the cons of embedding testing code in the actual project, but I really don’t want to turn many of the internal classes to public, or adding the InternalTo attribute everywhere, just to make them accessible to the test code.
https://learn.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-mstest
I always create new project for Unit Tests, but you can have Unit Tests in your Console Application.
Just for demo purposes, I've added all the classes in one single file.
If you don't want your tests on production, you can surround your test classes with the #if DEBUG and #endif.
This way, it will compile on debug mode and be ignored on release mode.
Hope this helps.
You can also use NUnitLite to create an executable test program, which runs your tests. Link to both the nunit framework and the corresponding version of nunitlite and write a Main following these instructions: https://github.com/nunit/nunit/blob/master/src/NUnitFramework/nunitlite/AutoRun.cs#L34
This is a common question, it's really quite astounding to me how difficult such a simple task is turning out to be ...
I have a Visual Studio (formerly Xamarin) C# cross-platform library. It has some code in the shared "abstract" library, and a few machine-specific implementation classes for Android, iOS, and UWP libraries. It also has a few automated (non-UI) unit tests that use the Microsoft.VisualStudio.TestTools.UnitTesting package. This is a great start, when Visual Studio 2017 is feeling generous it runs my unit tests and life is good.
Now I want to run the unit tests on simulators of the actual devices, from a command line (so I can run them from Jenkins and capture the output). So, for example, I want to run my unit tests on a simulator of an iPad. From various web sites I get the feeling that I can't use the Microsoft package for this, so I need to switch to either Xunit or NUnit. I gave up on Xunit earlier today, here's how NUnit went.
Go into the Unit Test project and remove the NuGet packages for Microsoft.NET.Test.Sdk, MSTest.TestFramework, and MSTest.TestAdapter.
Add NUnit instead (version 3.7.0).
Go into each CS file, change to "using NUnit.Framework", change [TestMethod] to [Test], and [TestClass] to [TestFixture].
Project:Manage NuGet Packages, add the NUnit3TestAdapter.
Test Explorer: Run All
At this point I get this weird error for the UWP machine-specific library:
The "XamlCTask" task could not be initialized with its input parameters
The "DebugType" parameter is not supported by the "XamlCTask" task. Verify the parameter exists on the task, and it is a settable public instance property.
That's a really strange message, and if I Rebuild the UWP library, it rebuilds without errors. Not sure why it's building the UWP library as part of the Unit Tests, the only thing listed under "Projects" is the abstract (non-machine-specific) library.
Also strange that building the Unit Test library works, it's only when I do the Test Explorer: Run All that I get these errors.
OK fine, it's a known error.
Close VS2017, delete bin and obj folders, reopen, no better. Close VS2017, hack the Xamarin.form.targets as listed. Same error. Realize error is in the global Xamarin.Forms.targets, not the one in my solution. Hack it. Close & reopen solution, rebuild. No error this time!
Test Explorer: Run all. Nothing. Output:Build shows no failures, Output:Tests is completely empty. Close & reopen solution. Get new errors:
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_5937a670-0e60-4077-877b-f7221da3dda1.cs'.
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_E7A71F73-0F8D-4B9B-B56E-8E70B10BC5D3.cs'.
[Failure] Could not find file 'C:\Workspace\PopComm.iOS\obj\Debug\TemporaryGeneratedFile_036C0B5B-1481-4323-8D20-8F5ADCB23D92.cs'.
Test Explorer: Run all. This time get more interesting errors:
------ Discover test started ------
NUnit Adapter 3.7.0.0: Test discovery starting
Assembly contains no NUnit 3.0 tests:
C:\Workspace\PopComm.iOS\bin\iPhone\Debug\PopComm.dll
NUnit Adapter 3.7.0.0: Test discovery complete
========== Discover test finished: 0 found (0:00:00.0530024) ==========
Of course that assembly doesn't contain any unit tests! It's not my unit test project!
Sigh.
OK, create a new project of type "NUnit 3 Unit Test Project". Move my test classes into this new project and add a dependency to my non-machine-specific library.
Test Explorer: Run all - Tests! There are tests! And they even run successfully.
Close VS2017, open again, Test Explorer: Run all. No tests. Output:Tests is empty.
Close project, delete all bin and obj folders, Rebuild Solution, Test Explorer: Run All. No tests.
Push code to git, pull on the Mac.
Open solution in Visual Studio for the Mac.
Use the mostly-hidden View:Test feature to bring up the Unit Tests window, run the tests. Tests found and run. Yippee!
Now I want to run my unit tests on an iPad simulator, not inside VSMac.
Add a new project to my solution, type iOS:Tests:Unified API:Unit Test App.
Wizard asks what is the minimum iOS version I want to support. I figure I should match the iOS version that my machine-specific library targets, so I go looking in the VSMac options for the project. The iOS target version isn't shown anywhere. Use the default (10.3) for the Unit Test App.
Do I have to copy all of my unit test CS files into this new app now? Or can I somehow reference my cross-platform unit test project in this Unit Test App?
Pushing everything and pulling on the Windows machine, I try adding a reference to my unit test library but get an error:
The project 'NUnit.Tests' cannot be referenced. The referenced project is targeted to a different framework family (.NETFramwork)
Hum. I wonder what framework the new NUnit.Tests.iOS application is targeting. Check its options - it doesn't say anywhere. Remove the reference from the Unit Test App back to the unit test files, and just copy the unit test source files over (non-optimal).
Run the application in the iPad simulator, it finds & runs the tests, but I don't know how to (1) run it from the command line, and (2) capture the unit test statuses in a file.
So after all this, I still don't know how to run my unit tests (1) from the command line, (2) on an iPad simulator, and (3) capturing the output in a text file.
I have a test suite, comprised of both unit tests and integration tests, in a project using C# on .NET 4.0 with Visual Studio 2010. The test suite uses MSTest. When I run all tests in solution (either by hitting the button in the testing toolbar or using the Ctrl-R A shortcut chord) all of the tests, integration and unit, pass successfully.
When I either attempt to run the same tests from the command line with mstest (explicitly using the only .testsettings file present) or attempt to run them from the Test List Editor or using the .vsmdi file the integration tests fail.
The integration tests test the UI and so have dependencies on deployment items and such, whereas the unit tests do not. However, I cannot seem to pin down what is actually different between these two methods of running the tests.
When I inspect the appropriate Out directories from the test run, not all of the files are present.
What would cause some of the files that deploy correctly in one situation from Visual Studio to not deploy correctly in another?
The static content started being copied shortly after I wrote the comments above. The other major issue I ran into was that the integration test project referenced libraries that were dependencies of the system-under-test (with copy-local set to true) in order to ensure that the DLLs would be present when they were needed. For some reason, these stubbornly refused to copy when the tests were run through Test List or mstest.
What I eventually did to work around it was include [DeploymentItem] attributes for the DLLs that I needed. This got things working no matter how the tests were run. What I am still unclear on, that may have answered the underlying solution, or provided a better solution, is how Test View/mstest differ from the regular test runner (assuming that the correct .settings file was passed to mstest.). I'm putting these notes/workarounds in an answer, but I'll leave the question open in case anyone can address the underlying cause for how the different test execution paths differ.
I would like to run automated Silverlight unit tests from a Hudson build server. It seems there are two options:
Use Statlight, although it seems to be designed for TeamCity rather than Hudson, so it would involve a bit of hacking to get it to work.
Use NUnit Silverlight tests.
Can anyone recomend either of these options? Or is there a better alternative?
You can try using Lighthouse Silverlight Unit Test Runner, it works with every Build Server including Hudson, TeamCity and CCNet because it by default produces NUnit compatible xml results file:
http://lighthouse.codeplex.com/
In our company we are using NUnit with Hudson for automatized unit testing. It is simple to setup and execute.
Just download and unzip latest nunit somewhere on Hudson host.
Add Windows batch command as last buildstep with content like:
C:\NUnit\bin\net-2.0\nunit-console.exe "%WORKSPACE%\src\Test\AllTests.nunit" /config=Release /xml="%WORKSPACE%\src\Test\TestResults.xml"
This will execute tests as defined in "AllTests.nunit" file. It is possible tu point just to one assembly (.dll).
To populate test results within Hudson Job page, you would need to install Hudson NUnit plugin. Its possible directly from Hudson plugin management.
After instalation there will be new Post build action: Publish NUnit test result report.
If you check it, you've got line to enter path to test result report. Corresponding path for example above is:
src/Test/TestResults.xml
Hope it helps you to decide ;-)