I know that all test should be independent and run in the random order.
But here is my situation:
I need to test my library which is working with external web resource. I have one test which checks if the web resource available and I have many tests which check the data I get from this resource.
If the web resource is not available I have all my tests failed and I think it is difficult to interpret this result. Is there any way to run all tests which check the data I get from resource if and only if the first test (which checks resource availability) was not failed?
I think availability of web resource should be a precondition for tests of your library. It should not be a separate test itself.
So, you need to check connection in TestFixtureSetup. If connection fails, all tests from fixture will not run. All tests in this test fixture will be marked as failed (which is true if there is no connection) and you will receive message
TestFixtureSetUp failed in YourLibtraryTests
Unfortunately NUnit will not show additional information if you will fail test with some message
Assert.NotNull(connection, "Cannot establish connection to remote service");
But it will be easy to find the reason if you know that TestFixtureSetUp failed.
Related
Here's the gist of my issue. I have a C# program that is calling several PowerShell cmdlets.
All the calls work, except when I get to "Connect-AzureAD -TenantId XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
But the way it "fails" is odd. It only lets me enter my Microsoft Account, but not the password. IF I RUN IT AGAIN, I can log in correctly and it works.
I thought I was hitting a timeout of some kind, but each call bellow results in a Null Reference Exception. You can see my comments of frustration.
// All of these are set to null...so what gives?
// all generate NRE's
//Debug.Print("MaxIdle Timeout is |{0}|", runspace.ConnectionInfo.MaxIdleTimeout.ToString());
//Debug.Print("Current Idle Timeout is |{0}|", runspace.ConnectionInfo.IdleTimeout.ToString());
//Debug.Print("Operation Timeout is |{0}|", runspace.ConnectionInfo.OperationTimeout.ToString());
Thoughts, ideas, comments?
I can post more of the code, if needed, but everything else works all the time.
Per a comment, I added a CR, but it appeared to do nothing (I will try a CR+LF, because I have no idea what's actually broken here, so there's no harm in trying).
So...after the PowerShell invoke call, I collect the results as follows:
foreach (PSObject outputObject in results)
and my results collection has these four errors in it:
{One or more errors occurred.: The browser based authentication dialog failed to complete. Reason: The server or proxy was not found.}
{One or more errors occurred.}
{ The browser based authentication dialog failed to complete. Reason: The server or proxy was not found.}
{One or more errors occurred.: The browser based authentication dialog failed to complete. Reason: The server or proxy was not found.}
It's telling me what I already know (that auth didn't work), but I'm still clueless as to what the actual issue is.
Adding some more detail
So...I've tried a few things. I added my Microsoft account to the connect. The result was even more odd -- a long delay, no password screen, and failure to log in.
Then I pivoted to setting up an application principal as detailed here: "Using a Service Principal to connect to a directory in PowerShell" https://learn.microsoft.com/en-us/powershell/azure/active-directory/signing-in-service-principal?view=azureadps-2.0
I got it all set up and verified in PowerShell that I can connect, then I placed the exact same (I copied and pasted it) Connect-AzureAD with the tenant, application ID, and certificate thumbprint that I used in PowerShell. It failed with these "new" errors -- still four somehow:
One or more errors occurred.
An error occurred while sending the request.
The remote name could not be resolved: 'login.microsoftonline.com'
One or more errors occurred.
Tomorrow, I'll fire up Wireshark. I'm getting the feeling that these errors are all misleading, although the fact that it works on the second try is just mind boggling.
I am using the Moq testing framework in C#, and I trying to create a unit test where I have a method which calls out to a service on a remote server over a unreliable network.
In side the method I am trying to unit test I have the following code:
Task.Factory.StartNew(() => this.remoteClient.SomeMethod(userDetails)).Wait(Timespan.FromSeconds(10));
The aim of the unit test is to mock out the timeout where the service failed to respond within 10 seconds.
So far I have the unit test as follows, which does work, but I am feeling that it is not quite right.
mockRemoteClientService.Setup(r =>r.SomeMethod(It.IsAny<string>())).Callback(() => Thread.Sleep(11000));
Can anyone advise me whether the approach I am taking is correct or is there another way to do this?
You can use a cancellation token to setup the actual timeout.
When you want to mock out the service call to simulate a timeout, you can use Throws (or ThrowsAsync in case of an async method) to throw a TaskCancelledException or TimeOutException like so:
mockRemoteClientService.Setup(x => x.SomeMethod(It.IsAny<string>())).Throw(new TaskCancelledException());
Then Assert that you handle the response correctly.
Trying to do unit testing using MSTest in VS2015 with the Microsoft Band nuGet package and running into the following error
"Microsoft.Band.BandIOException: An error occurred while attempting to acquire the Bluetooth device service.
This error can occur if the paired device is unreachable or has become unpaired from the current host.
System.InvalidOperationException: A method was called at an unexpected time. (Exception from HRESULT: 0x8000000E)".
Code runs fine when run inside the application. It fails on the line to call BandClientManager.Instance.ConnectAsync.
The exception and error message are not helpful here, but you must establish Bluetooth connections on a UI thread. This is because the app might prompt the user and ask if they want to allow access to the Bluetooth device.
For example, in a UWP app, you can do the following to ensure UI thread execution:
await Windows.ApplicationModel.Core.CoreApplication.MainView.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
{
IBandClient client = await BandClientManager.Instance.ConnectAsync(...);
...
});
Alternatively, if you have access to a UI control, you can use its Dispatcher directly.
Any code that ultimately calls BluetoothLEDevice.FromBluetoothAddressAsync must do it on a UI thread. The Bluetooth access prompt will happen whenever the app package manifest (.appxmanifest) changes.
I can't imagine this fix being dependable for unit tests since there's no UI. I'm not sure what the intended fix is besides mocking the client interfaces and just avoiding Bluetooth altogether.
If it matters I'm using:
xUnit for the test runner
C# 4.5.1 (so that I can take advantage of TransactionScopeAsyncFlowOption.Enabled and just rollback a transaction across threads - [AutoRollback] didn't work)
Protractor.NET for the browser/driver (the client uses Angular)
Owin to self-host the site (if that's the right term)
EntityFramework to set up the test data before each test and in my app to do most data access
When I run each test individually by right-clicking it, everything works fine. However whenever I run them all using the Test Runner/Test Explorer all but the first throw exceptions.
I'm following the Arrange-Act-Assert pattern. During "Arrange" I call DbContext.SaveChanges(). That's when this error is thrown:
Result Message:
System.Exception : The underlying provider failed on Open.
---- System.Data.Entity.Core.EntityException : The underlying provider failed on Open.
-------- System.Transactions.TransactionException : The operation is not valid for the state of the transaction.
------------ System.Transactions.TransactionPromotionException : Failure while attempting to promote transaction.
---------------- System.Data.SqlClient.SqlException : There is already an open DataReader associated with this Command which must be closed first.
-------------------- System.ComponentModel.Win32Exception : The wait operation timed out
I've tried googling several of the errors. Enabling MARS in my connection string didn't fix it. Extending the Timeout in my connection string didn't fix it.
I figured: maybe someone knows something about this pattern of nested exceptions and could shed some light on a precise issue.
If not, what are my next diagnostic steps to figure out why the exception is being thrown?
I was explicitly creating a SqlConnection/SqlDataAdapter in a WebApi endpoint of my code that creates a SQL query dynamically that tests 2 and 3 were calling and test 1 wasn't. The outermost rollback transaction I described in my question along with the SqlConnection were causing the connection.Open() call in the WebApi endpoint to promote to being handled by the DTC.
I discovered this by putting this in my bootstrap code:
TransactionManager.DistributedTransactionStarted += TransactionManager_DistributedTransactionStarted;
// ...
void TransactionManager_DistributedTransactionStarted(object sender, TransactionEventArgs e)
{
throw new Exception("DistributedTransactionStarted");
}
For the time being (I'll only do the following during these E2E tests ultimately) I rewrote my Ninject code to use singletons for SqlConnections and made it so the DbContext does not own the connection. This solved the exception I was getting above so therefore answers the question (even though I'm getting a different error now). (I think I also ended up putting MARS back in my connection string, but that may have been for a different exception - can't remember.)
I want to run Fiddler to help debug a portable API client that I'm developing.
In Visual Studio, on the Windows Store Unit Test project property page > Debug tab I have checked the 'Allow Local Network Loopback' option.
Every web request run during a store unit test fails with a System.Net.Sockets.SocketException : A connection attempt failed because the connected party did not properly respond after a period of time or established connection failed because connected host has failed to respond 127.0.0.1:8888.
The Uri I'm connecting to is not local - but another machine on the local network - so this fiddler loopback address is indeed coming from the global proxy setting.
I know that for Windows Store Apps you have to enable the loopback network isolation exemptions and have ticked every app container listed in the UI for that (despite none of them being related to VS or this unit test projects) - to no avail.
I've also tried disabling IPv6 in Fiddler - don't know why I thought that might work, but it was worth a punt - it didn't work either.
Anyone got any ideas!?
Okay - whilst I was on the right track, thinking that there must be some app container registered for the unit test project - what I didn't think was that it might only be active while MSTest is running.
You need to debug the Windows Store unit test (any of the tests in the project) and stick a breakpoint in (to halt execution of the test runner) so it will then appear in the list of AppContainers within Fiddlers' Loopback Extensions tool.
So - given this test:
[TestMethod]
public async Task Example()
{
var result = await GetSomeData(); //<-- breakpoint
Assert.IsNotNull(result);
}
private async string GetSomeData()
{
//TODO something that makes a web request with, say, HttpClient
}
All you do is breakpoint the line shown above - and then launch the 'Win8 Config' option from Fiddler - and you'll see that your unit tests project appears in the list of AppContainers.
Enable loopback for that app container, save the changes and then continue in your unit test - you should see the traffic correctly captured.
After doing this once it would seem that you don't have to do it again - I've closed down fiddler and restarted it and the traffic is still captured correctly.
One word of warning, however, once you do this the Loopback exemptions tool will moan about an exemption being defined for a SID for which there is no AppContainer if it is launched - unless you happen to be running a unit-test again.