Currently I am executing some test in my dll via command line like so
dotnet test UnitTest123/bin/Debug/UnitTest123.dll --logger html
After executing I get my results in my TestResults folder where my project is in my local machine..
I want to give my report some meaning full name like UnitTest123 but I get
TestResult_MachineName-WS102_20201008_184559
Another Thing I wanted to know was how can I log to the report.. example if I have to log a message so it will show in the report how can I do this? I have tried TestContext but it did not work..
Thanks for any help :)
As far as I can see, you can't change the file name for the html report, only the folder:
dotnet test -l:html -r htmlresultsfolder
Using non html format you can do something like this:
dotnet test -l:trx;LogFileName=C:\temp\TestOutput.xml
Or:
dotnet test --logger "console;verbosity=detailed" > WHATEVER_FILENAME_YOUWANT.log
Alternatively (based on the fact you said you're using NUnit), I found (but never used) this logger: https://github.com/spekt/nunit.testlogger Which seems to let you specify the output file name:
dotnet test --test-adapter-path:. --logger:"nunit;LogFilePath=test-result.xml"
Without seeing your cude, I think you can use this to capture output: https://xunit.net/docs/capturing-output
using Xunit;
using Xunit.Abstractions;
public class MyTestClass
{
private readonly ITestOutputHelper output;
public MyTestClass(ITestOutputHelper output)
{
this.output = output;
}
[Fact]
public void MyTest()
{
var temp = "my class!";
output.WriteLine("This is output from {0}", temp);
}
}
I am currently working with NUnit to write test cases for an ASP.NET application.
I have created a sample test project and have added NUnit latest version (3.6.1) and NUnitTestAdapter 2.1.1.
If I run the application by clicking on menu Test → Windows → Text Explorer, I am getting a blank screen in the Output Window. Why?
My code is:
[TestFixture]
public class SampleTest
{
[Test]
public void StringCheck()
{
string str = "Hello";
Assert.That(str, Is.EqualTo("Hello"));
}
[Test]
public void EmptyCheck()
{
string str = "siva";
Assert.That(str, Is.EqualTo(string.Empty));
}
[Test]
public void NumberCheck()
{
int i = 0;
Assert.That(i, Is.EqualTo(0));
}
}
You are using the adapter for the NUnit 2.x series, rather than the NUnit 3 adapter. NUnit versions 2 and 3 differ substantially, and there are two separate Visual Studio adapters to run the tests.
This is the one you'll need to run NUnit 3.6.1 tests:
NUnit 3 Test Adapter
I found the solution. The problem was with NUnit version (3.6.1). Now I am updated with NUnit 2.6.4. It is working for me.
I have a requirement to run Coded UI test that is data driven from console application. If I run the Coded UI test as a standalone, then [DataSource] can access the values from the CSV file. Whereas if I call Coded UI from the console app, I get the unhandled exception: System.NullReferenceException: Object reference not set to an instance of an object. - since TestContext.DataRow is null.
Here’s snippet from the code
Program.cs (console app):
public class Program
{
static void Main(string[] args)
{
Playback.Initialize();
CodedUITestWarmup test = new CodedUITestWarmup();
test.WarmUp();
Playback.Cleanup();
}
}
CodedUITestWarmup.cs (coded ui test):
public class CodedUITestWarmup
{
[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", #"|DataDirectory|\DataFiles\warmup.csv", "warmup#csv", DataAccessMethod.Sequential)]
public void WarmUp()
{
InitializeVendorTest();
...
}
private void InitializeVendorTest()
{
caseV = new CaseVariables(TestContext);
...
}
}
class CaseVariables
{
public string lastNameID;
...
public CaseVariables(TestContext TestContext)
{
lastNameID = TestContext.DataRow["lastNameID"].ToString();
...
}
}
Could you please provide some inputs what can be done in this regard?
You cannot run codded ui test without vstestconsole.exe or from visual studio.
You can try run vstestconsole.exe with parameter (path to your test dll)
eg.
C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstestconsole.exe
Dll file will be created during each build the codedui test project
Why do you need the test to run from a console application?
I believe you can use either vstestconsole.exe or mstest.exe
vstestconsole is command line tool that replaces MStest. But in this case I thing any of them can do wat you want!
A Coded UI Test or Unit Test will allwais need to have the TestContext initialized, and the test engine is responsible to do that, that is why you get an exception.
In my PC the mstest executable is in "c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\"
So I can start the command prompt, navigate to the mstest.exe folder (the one abouve) and use a command like the following one:
mstest.exe /testcontainer:"c:/TestFolder/testassembly.dll" /test:"TestNamespace.MyTestToExecute_TestMethod1"
Reading through https://msdn.microsoft.com/en-us/library/jj635153.aspx I have created a .RunSettings files with a few parameters similar to the example:
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="webAppUserName" value="Admin" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
I plan on having a .RunSettings file for each of our environments with appropriate URLs and credentials for running a CodedUI test on the specified RunSettings file's environment.
I can see that from command line to reference the Settings file I can run:
vstest.console myTestDll.dll /Settings:Local.RunSettings /Logger:trx
vstest.console myTestDll.dll /Settings:QA.RunSettings /Logger:trx
etc...
But I don't see any way that calls out how to actually utilize the TestRunParameters from within the codedUI test.
What I would like to do is set up test initializers that use the TestRunParameters to determine where to log in, and what credentials to use. Something like this:
[TestInitialize()]
public void MyTestInitialize()
{
// I'm unsure how to grab the RunSettings.TestRunParameters below
string entryUrl = ""; // TestRunParameters.webAppUrl
string userName = ""; // TestRunParameters.webAppUserName
string password = ""; // TestRunParameters.webAppPassword
LoginToPage(entryUrl, userName, password);
}
public void LoginToPage(string entryUrl, string userName, string password)
{
// Implementation
}
Information on how to reference the TestRunParameters is greatly appreciated!
EDIT
/// <summary>
/// Summary description for CodedUITest1
/// </summary>
[CodedUITest]
public class CodedUITest1
{
public static string UserName = string.Empty;
[ClassInitialize]
public static void TestClassInitialize(TestContext context)
{
UserName = context.Properties["webAppUserName"].ToString();
Console.WriteLine(UserName);
}
[TestMethod]
public void CodedUITestMethod1()
{
this.UIMap.RecordedMethod1();
// To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items.
}
// Rest of the default class - TestContext instantiation, UI map instantiation, etc
}
The exception I'm getting when running:
NullReference Exception
#williamfalconeruk I have updated my test class as above, but I am still getting the same error, any idea what I'm doing wrong?
For those that use Resharper with this issue, I discovered the fix (no need to disable Resharper):
Go to Visual Studio top menu -> Resharper -> Options
Find the Tools section, expand "Unit Testing"
Click on "MsTest". The checkbox should be on enabled, but the Test Settings file path below that may be blank. If it is, click on browse and select the runsettings file you want to use.
Click save, rebuild and try to run the tests, the parameters should now work.
Not sure why but simply selecting the Test Settings file from the Tests menu -> Test Settings does not actually work when using Resharper, so this file needs to be explicitly pointed to directly in the Resharper options.
I also came across this recently, as we wanted to move away from legacy environment variable usage. The solution provided below was suitable for our needs, but there may be a better one...
It turns out you can access these in the TestContext of a ClassInitialize method of your Test fixture. There is a Properties dictionary which has these parameters, and you could access the values here:
[ClassInitialize]
public static void TestClassinitialize(TestContext context)
{
var webAppUrl = context.Properties["webAppUrl"].ToString();
//other settings etc..then use your test settings parameters here...
}
Note: these are static so if you need access to this you may need to set up static properties to access within your Test code.
An alternative suggested is using a Data Driven Test approach.
Here's some basic info on data driven tests here which may also help:
https://msdn.microsoft.com/en-us/library/ms182527.aspx
As I said, the above solution suited our needs at a basic level.
UPDATE: see image below in response to test settings returning null...
This works for me (VS2017-pro):
namespace TestApp.Test
{
[TestClass]
public class UnitTest1
{
// This enables the runner to set the TestContext. It gets updated for each test.
public TestContext TestContext { get; set; }
[TestMethod]
public void TestMethod1()
{
// Arrange
String expectedName = "TestMethod1";
String expectedUrl = "http://localhost";
// Act
String actualName = TestContext.TestName;
// The properties are read from the .runsettings file
String actualUrl = TestContext.Properties["webAppUrl"].ToString();
// Assert
Assert.AreEqual(expectedName, actualName);
Assert.AreEqual(expectedUrl, actualUrl);
}
[TestMethod]
public void TestMethod2()
{
// Arrange
String expectedName = "TestMethod2";
// Act
String actualName = TestContext.TestName;
// Assert
Assert.AreEqual(expectedName, actualName);
}
}
}
Make sure to select the runsettings file you wish to use, here: Test -> Test Settings.
An alternative to disable Resharper is to enable MSTest support and select the test setting file on Resharper Options dialog (->Tools->Unit Testing->MsTest).
I was trying to do this exact thing as well. As many of you may know, running tests through MTM exposes some additional properties to the TestContext, including the name of the Run Settings used. I used this property as a "Foreign Key" of sorts for our test data, allowing us to specify environment URLs etc. without hardcoding them or using the incredibly lackluster "Data Driving" tools that come with out of the box testing.
Of course, there's no way to expose any run-time properties when executing tests as part of a BDT or release workflow besides what #kritner is attempting which microsoft describes HERE. However if you read the comments of that link you'll discover what you may be able to infer here:
You need to use VS 2013 R5 or VS 2015 to use this solution
It will only work for Unit Tests!
Those of us who are trying to execute UI or Load tests as part of a CI or CD workflow are completely screwed. You don't get any additional properties in testContext, even when executing a Plan/Suite with certain test configurations (not settings) created in MTM. #Adam may have been able to get this to work when running vs debugging, but that may have only worked with unit tests. Through CodedUI I've been unable to retrieve the properties without getting a NullReferenceException. Here's an example of the janky code I was using to investigate:
if (testContextInstance.Properties["__Tfs_TestConfigurationName__"] != null) //Exposed when run through MTM
{
TFSTestConfigurationName = testContextInstance.Properties["__Tfs_TestConfigurationName__"].ToString();
}
else TFSTestConfigurationName = "Local"; //Local
var configName = testContextInstance.Properties["configurationName"] ?? "No Config Found";
Trace.WriteLine("Property: " + configName);
And the XML of my .runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Parameters used by tests at runtime. These are required as a substitute for TFS/MTM test settings.-->
<!-- File instructions: https://msdn.microsoft.com/en-us/library/jj635153.aspx#example -->
<!-- TFS instructions: https://blogs.msdn.microsoft.com/visualstudioalm/2015/09/04/supplying-run-time-parameters-to-tests/ -->
<TestRunParameters>
<Parameter name="configurationName" value="Local" />
</TestRunParameters>
</RunSettings>
And an excerpt from the .trx produced by the BDT workflow:
Property: No Config Found
For the NullReferenceException issue:
I was also facing the same issue recently and the solution to it is to have the latest update of Visual Studio 2013. Right now the latest update is Update 5. I am not sure which particular update fixes this issue. I applied Update 5 and was able to successfully access the TestRunParameters in the ClassInitialize method.
You can find the updates # https://support.microsoft.com/en-us/kb/2829760
So I had two machines, on one it was all working fine and on the other I was getting the exception. I investigated that the only difference is the Update of VS; applied it and that solved the problem. :)
With NUnit 3, I was able to find the properties in the runsettings file, using
TestContext.Parameters
So in this case it would be:
string entryUrl = TestContext.Parameters["webAppUrl"];
string username = TestContext.Parameters["webAppUserName"];
string password = TestContext.Parameters["webAppPassword"];
I was able to resolve this for Unit tests by disabling Resharper. Wish I could say the same for Coded UI tests.
Why dont you use Application settings?
you can then read them anywhere like
var yesICan= Properties.Settings.Default.IToldYou;
you can create Properties out from them, pretty much you can do a lot.
public string URL_WEBOFFICE
{
get
{
return Properties.Settings.Default.WEBOFFICE_URL.Replace("***", Properties.Settings.Default.SiteName);
}
}
I have the following code:
[TestFixture]
public class LexicalTests
{
[Test]
public void LexicalTest1()
{
TestContext.CurrentContext.TestDirectory;
}
}
CurrentContext throws an exception while attempting to get TestDirectory or WorkingDirectory property.
How can I solve this problem?
P.S.: On my home PC tests work perfectly (without strange exceptions).
It seems that some applications that offer the functionality to run NUnit unit tests have a problem with the TestContext class.
The test in class below should pass:
using NUnit.Framework;
namespace UnitTests
{
[TestFixture]
public class UnitTests
{
[Test]
public void CurrentContextTest()
{
Assert.IsNotNull(TestContext.CurrentContext);
Assert.IsNotNull(TestContext.CurrentContext.TestDirectory);
Assert.IsNotNull(TestContext.CurrentContext.WorkDirectory);
}
}
}
If the test doesn't pass then, as Dmitry wrote in his comment above, change the NUnit version in the ReSharper menu. From within Visual Studio, go to ReSharper -> Options -> Tools -> NUnit. Click the Specified NUnit installation radio button and ensure that a folder with nunit.core.dll, nunit.core.interfaces.dll and nunit.util.dll is specified. An error will be displayed if the listed files cannot be found.
Once the NUnit version has been changed, re-run the test and it should pass.