I have a requirement to run Coded UI test that is data driven from console application. If I run the Coded UI test as a standalone, then [DataSource] can access the values from the CSV file. Whereas if I call Coded UI from the console app, I get the unhandled exception: System.NullReferenceException: Object reference not set to an instance of an object. - since TestContext.DataRow is null.
Here’s snippet from the code
Program.cs (console app):
public class Program
{
static void Main(string[] args)
{
Playback.Initialize();
CodedUITestWarmup test = new CodedUITestWarmup();
test.WarmUp();
Playback.Cleanup();
}
}
CodedUITestWarmup.cs (coded ui test):
public class CodedUITestWarmup
{
[TestMethod]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", #"|DataDirectory|\DataFiles\warmup.csv", "warmup#csv", DataAccessMethod.Sequential)]
public void WarmUp()
{
InitializeVendorTest();
...
}
private void InitializeVendorTest()
{
caseV = new CaseVariables(TestContext);
...
}
}
class CaseVariables
{
public string lastNameID;
...
public CaseVariables(TestContext TestContext)
{
lastNameID = TestContext.DataRow["lastNameID"].ToString();
...
}
}
Could you please provide some inputs what can be done in this regard?
You cannot run codded ui test without vstestconsole.exe or from visual studio.
You can try run vstestconsole.exe with parameter (path to your test dll)
eg.
C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstestconsole.exe
Dll file will be created during each build the codedui test project
Why do you need the test to run from a console application?
I believe you can use either vstestconsole.exe or mstest.exe
vstestconsole is command line tool that replaces MStest. But in this case I thing any of them can do wat you want!
A Coded UI Test or Unit Test will allwais need to have the TestContext initialized, and the test engine is responsible to do that, that is why you get an exception.
In my PC the mstest executable is in "c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\"
So I can start the command prompt, navigate to the mstest.exe folder (the one abouve) and use a command like the following one:
mstest.exe /testcontainer:"c:/TestFolder/testassembly.dll" /test:"TestNamespace.MyTestToExecute_TestMethod1"
Related
I currently have an nunit project outputting a class library 'RegressionTests.dll' that opens the Selenium WebDriver and runs a few dozen UI tests. I have created a WinForm app with a button 'Run Tests'. When clicking this button, I want to execute a series of n-unit tests from RegressionTests.dll.
I had gotten this to work on my local machine using Process.Start("nunit3-console.exe, nunit-console RegressionTests.dll"), but realized that it would only work on my local if I had installed nunit3-console as a standalone app. After realizing this, I dug more into the n-unit documentation and discovered the n-unit engine. I have tried leveraging the n-unit3 Engine in order to run it internally but have faced issues with implementation of the ITestEventListener in the WinForm project. I've attached the code to my button here:
Form1.cs
private void btnRun_Click(object sender, EventArgs e)
{
TestRunner.Run();
}
Inside TestRunner.cs, we have this code:
[Extension(Description = "Test Reporter Extension", EngineVersion = "3.11")]
public class TestRunner : ITestEventListener
{
public static void Run()
{
ITestEngine engine = TestEngineActivator.CreateInstance();
TestPackage package = new TestPackage("RegressionTests.dll");
ITestEventListener testListener = new TestRunner();
using (ITestRunner runner = engine.GetRunner(package))
{
XmlNode result = runner.Run(testListener, TestFilter.Empty);
}
}
public void OnTestEvent(string report)
{
throw new NotImplementedException();
}
}
Currently, the solution layout is as follows.
Solution
Regression (project)
RegressionTests.dll
TestRunner.cs (file that contains my code linked above)
SeleniumFormApp
Form1.cs (contains button that, upon click, should run Selenium test cases)
How can I leverage n-units Nuget packages to accomplish what I want to here? Is n-unit engine the proper one? If so, how should the ITestEventListener be implemented to accomplish this?
Thank you - please let me know if this is unclear.
I have two projects in Visual Studio, the core project which is a Windows Service Executable, and the Unit Test Project.
The core project has two original files broken out like this
File1.cs:
using static Proj.ConfigHelper;
namespace Proj
{
class MyClass
{
(lots of code)
}
}
File2.cs looks something like this.
namespace Proj
{
static class ConfigHelper
{
public static NameValueCollection AppSettings { get { return ConfigurationManager.AppSettings; } }
public static NameValueCollection CustomSection { get { return ConfigurationManager.GetSection("CustomSection") as NameValueCollection; } }
}
}
Both of those classes are internal and made visible to the unit test project via InternalsVisibleToAttribute.
Inside of the UnitTest project which is a discrete project within the same solution (and so it has its own app.config), calling ConfigHelper.AppSettings results in a 0-item collection, and calling ConfigHelper.CustomSection results in null. If I attempt to unit test a method within File1.cs that depends on those settings, they run as default values as if they were not configured at all. I don't quite understand why this is happening. Can anyone help me understand what I did wrong? It seems as though the ConfigHelper is not loading the App.Config for its own project.
The app.config for the Windows Service Project is set to "always copy" and the app.config for the unit test project is set to "never copy"
The test will use its own config so you need to mirror it. There are work runarounds: Can a unit test project load the target application's app.config file?
I'm trying to execute test program which uses selenium web driver in custom test runner.
In the test runner, selenium web driver equiped test method is invoked by reflection.
When the test program is run by Visual Studio Test Explorer, it works fine.
Problem occurs when it is run dynamically by reflection.
The test program is as follows.
namespace TrialWebUnitTest
{
public class WebDriverTest01
{
private IWebDriver driver;
[TestMethod]
public void NavigateToSeleniumHQByChrome()
{
string TargetUrl = "https://www.seleniumhq.org/projects/webdriver/";
this.driver = new ChromeDriver();
driver.Manage().Window.Size = new System.Drawing.Size(1000, 800); // <- driver throws exception here.
this.driver.Navigate().GoToUrl(TargetUrl);
this.driver.Dispose();
}
}
}
The custom test runner's core test executing method is as follows.
namespace TrialWebUnitTestRunner
{
public partial class TestForm : Form
{
// test button click's event handler.
private void TestButton_Click(object sender, EventArgs e)
{
string retErrorMsg = string.Empty;
if (!ExecTestDynamically(ref retErrorMsg))
{
// show error information in UI textbox.
this.this.ResultMessage.Text = retErrorMsg;
}
}
internal bool ExecTestDynamically(ref string retErrorMsg)
{
var target = new TrialWebUnitTest.WebDriverTest01();
System.Type targetType = typeof(TrialWebUnitTest.WebDriverTest01);
var method = targetType.GetMethod("NavigateToSeleniumHQByChrome");
try
{
method.Invoke(target, null);
return true
}
catch (Exception exp)
{
retErrorMsg = exp.Message + Environment.NewLine + exp.StackTrance;
return false;
}
}
}
}
Exception information
System.InvalidOperationException:
disconnected: unable to connect to renderer
(Session info: chrome=65.0.3325.181)
(Driver info: chromedriver=2.31.488763
Sample program
I've written a sample program to reproduce the error.
Please download it from my dropbox url: https://db.tt/HqUTMOKWBl
You can click the link and download 'TrialWebTestForInspection.zip'.
Please extract it in any arbitrary folder and find TrialWebTestForInspection.sln.
The solution consists of two projects, "TrialWebUnitTest" and "TrialWebUnitTestRunner".
The first one is MS Unit Test, and you can run 3 test methods from Visual Studio Test Explorer.
The 3 test methods are very simple. They just launch the browser correspond to the webdriver, and navigate to Selenium HQ site.
The other project is a WindowsForm application which provide a very simple test runner.
It kicks the test methods in previous test project.
When you choose the test method using IE or FireFox driver, it works fine.
On the other hand, when you choose Chrome driver test, it thows exception which I mentioned above.
Things I'd like to know.
First I'd like to know, if it is a bug of current Chrome driver version, or it is a part of specification.
Then I'd like to know, if there is a way to avoid this problem or not.
What I'm afraid of is the possibility that IWebDriver specification originally does not support correct action whent it is run by reflection.
spec of sample program
.NET Framework version 4.6.1
nuget package
MSTest.TestFramework.1.2.0 MSTest.TestAdapter.1.2.0
Selenium.WebDriver.3.11.0 Selenium.WebDriver.ChromeDriver.2.37.0
Selenium.WebDriver.IEDriver.3.11.1 Selenium.Firefox.WebDriver.0.20.0
Chrome Browser version 65.0.3325.181（Official Build）(64 bit)
The problem was solved. Please refer to this.
The custom TestRunner application didn't refer to the latest chrome-driver version. I had to install selenium.webdrivers not only to the testclass, but also to the testrunner program.
For further information, please refer to the following issue in GitHUB.
https://github.com/SeleniumHQ/selenium/issues/5705
For years we have used the following code to setup databases in a base class for our functional tests for our DAL, and this has worked extremely well for us.
/// <summary>
/// Initializes the test class by creating the integration database.
/// </summary>
[TestInitialize]
public virtual void TestInitialize()
{
DataContext = new DataContext(ConnectionString);
CleanupPreviousTestRunDatabases();
if (DataContext.Database.Exists())
{
DataContext.Database.Delete();
}
DataContext.Database.Create();
DataContext.Database.ExecuteSqlCommand(String.Format(Strings.CreateLoginCommand, DatabaseUserName, DatabasePassword));
DataContext.Database.ExecuteSqlCommand(String.Format("CREATE USER {0} FOR LOGIN {0}", DatabaseUserName));
DataContext.Database.ExecuteSqlCommand(String.Format("EXEC sp_addrolemember 'db_owner', '{0}'", DatabaseUserName));
}
However, using Entity Framework does not setup all components of a database and we would like to catch discrepancies between our EF DAL model and the actual database.
We use the SSDT tools / Visual Studio Database Project for all of our database work, and I know you can write SQL unit tests, and in those SQL unit tests, I have seen the ability to setup and create a database based on the database project itself. This is what I would like to do, but from our other functional test libraries.
I can reference the libraries and write some of the setup code, but what I'm looking for is:
a) How do I provide which Database project to use to deploy?
b) How can I specify connection string in code rather than an app.config, such as using localdb instead with a dynamically named database?
namespace Product.Data.Tests
{
using Microsoft.Data.Tools.Schema.Sql.UnitTesting;
using Microsoft.VisualStudio.TestTools.UnitTesting;
[TestClass]
public class FunctionalTest
{
[TestInitialize]
public virtual void TestInitialize()
{
SqlDatabaseTestClass.TestService.DeployDatabaseProject();
SqlDatabaseTestClass.TestService.GenerateData();
}
}
}
The app.config in a SQL Unit Test Project doesn't contain any reference back to the original Database project used to create it, and decompiling some of the test code and seeing how it works, I don't see any indication. Does it assume there is only one database project in the solution?
With some direction from the links #Ed Elliott posted, I was able to make this happen. You will need to add Microsoft.SqlServer.Dac as a assembly reference from C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\SQLDB\DAC\120\Microsoft.SqlServer.Dac.dll (Visual Studio 2015). It is part of the SSDT tooling, so I'm sure the path may be different for earlier versions.
[TestClass]
public class DatabaseTest
{
protected string DatabaseConnectionString = $#"Data Source=(localdb)\v11.0; Integrated Security=True";
protected DatabaseContext DatabaseContext;
protected string DatabaseName = $"UnitTestDB_{Guid.NewGuid().ToString("N").ToUpper()}";
public TestContext TestContext { get; set; }
[TestInitialize]
public virtual void TestInitialize()
{
var instance = new DacServices(DatabaseConnectionString);
var path = Path.GetFullPath(Path.Combine(TestContext.TestDir,
#"..\..\..\Build\Database\Database.dacpac"));
using (var dacpac = DacPackage.Load(path))
{
instance.Deploy(dacpac, DatabaseName);
}
DatabaseContext = new DatabaseContext(DatabaseConnectionString);
}
[TestCleanup]
public virtual void TestCleanup()
{
DeleteDatabase(DatabaseName);
}
}
Then how it would be used for a functional test in a unit test project.
[TestClass]
public class CustomerTypeTests : DatabaseTest
{
private CustomerType customerType;
[TestInitialize]
public override void TestInitialize()
{
base.TestInitialize();
customerType = new CustomerType
{
Name = "Customer Type"
};
}
[TestMethod]
public void AddOrUpdateCustomerType_ThrowExceptionIfNameIsNull()
{
ExceptionAssert.Throws<ArgumentNullException>(() => DatabaseContext.AddOrUpdateCustomerType(customerType));
}
}
Just a note to others, you should also setup your Build Dependencies so that your unit test project depends on the database project, ensuring it is built first and produces the correct dacpac file.
What this solves for us, is this gives us a true database, not one just based on Entity Framework's model, which lacks quite a lot of SQL constructs (to be expected), especially default constraints, indexes, and other important elements of a database. At our DAL layer, this is essential for us.
I think the process you have is a little over complicated (if I understand it correctly which I might not have!).
What I do for unit testing in ssdt is to:
Build the solution
Deploy each dacpac that I need to my dev instance
Run the tests
To deploy a project there are a few ways, you can:
Create a "Publish Profile" for each project and run that
Right click on the project and choose publish
Use a powershell script (or do it in code in your test initialize) to do a publish of the dacpac.
Once it is deployed run your tests, doing a publish of a dacpac (project) is pretty simple from code or a script, you can either:
call sqlpackage.exe to do it for you
use the dacfx api's to do the deploy (http://blogs.msmvps.com/deborahk/deploying-a-dacpac-with-dacfx-api/)
If you control the publish yourself then it gives you a lot more control plus when you deploly using this you are testing the same deployment system that you will use in other environments (assuming you use dacpac's to deploy).
ed
I have the following code:
[TestFixture]
public class LexicalTests
{
[Test]
public void LexicalTest1()
{
TestContext.CurrentContext.TestDirectory;
}
}
CurrentContext throws an exception while attempting to get TestDirectory or WorkingDirectory property.
How can I solve this problem?
P.S.: On my home PC tests work perfectly (without strange exceptions).
It seems that some applications that offer the functionality to run NUnit unit tests have a problem with the TestContext class.
The test in class below should pass:
using NUnit.Framework;
namespace UnitTests
{
[TestFixture]
public class UnitTests
{
[Test]
public void CurrentContextTest()
{
Assert.IsNotNull(TestContext.CurrentContext);
Assert.IsNotNull(TestContext.CurrentContext.TestDirectory);
Assert.IsNotNull(TestContext.CurrentContext.WorkDirectory);
}
}
}
If the test doesn't pass then, as Dmitry wrote in his comment above, change the NUnit version in the ReSharper menu. From within Visual Studio, go to ReSharper -> Options -> Tools -> NUnit. Click the Specified NUnit installation radio button and ensure that a folder with nunit.core.dll, nunit.core.interfaces.dll and nunit.util.dll is specified. An error will be displayed if the listed files cannot be found.
Once the NUnit version has been changed, re-run the test and it should pass.