Using VS2010's load test feature with recorded webtests.
I have problems with cascading errors in my recorded webtests. That is, if one request fails, several other requests will also fail. This creates a lot of clutter in the logs, since usually only the first error is relevant.
Is there a way to make a failed validation rule terminate the web test at the point of failure, and NOT run the rest of the requests?
(to be clear, I still want to continue with the load test in general, just stop that specific iteration of that specific test case)
Here is some sample code that demonstrates what I'm trying to do:
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;
namespace TestPlugInLibrary
{
[DisplayName("My Validation Plugin")]
[Description("Fails if the URL contains the string 'faq'.")]
public class MyValidationPlugin : ValidationRule
{
public override void Validate(object sender, ValidationEventArgs e)
{
if (e.Response.ResponseUri.AbsoluteUri.Contains("faq"))
{
e.IsValid = false;
// Want this to terminate the running test as well.
// Tried throwing an exception here, but that didn't do it.
}
else
{
e.IsValid = true;
}
}
}
}
I found a good solution to this. There's a blog here that details it in full, but the short version is to use e.WebTest.Stop(). This aborts the current iteration of the current test, while leaving the rest of the run intact as needed.
Use the Assert.Fail(). This will stop the Test and will throw an AssertFailedException, just like in any failed assertion.
if (e.Response.ResponseUri.AbsoluteUri.Contains("faq"))
{
e.IsValid = false;
Assert.Fail("The URL contains the string 'faq'.");
}
This will stop only the specific test. At the end of the load test you can see the total number of tests failed with this exception.
Related
As above when I run all tests the only that passes is the 1st one,
I get the following stack trace.
Message: Initialization method QtpTest.ChromeTest7.Initiliize threw
exeption.OpenQa.selenium.WebdriverException:
OpenQa.selenium.WebDriverException: invalid session id
TestBase.cs
namespace QtpTest
{
[TestClass]
public class TestBase
{
[TestInitialize()]
public void Initialize()
{
Browser.Initialize();
}
[TestCleanup]
public void Cleanup()
{
Browser.Close();
}
}
}
I think That after the 1st test when it goes back to test initioalize the session id is either wrong or missing.
Though there is little to go off of here, I have had this issue in the past when running test in parallel.
We were experiencing the same issue you are facing where the first test would run and the rest would fail.
This was due to sharing the chromedriver.exe having access issues (file in use by another program).
I have a suggestion to try and resolve this since i am unsure if you are running in parallel or not as well as how you are creating your driver.
Try and not only do Browser.Close(), but due Browser.Quit();
Browser.Close() simply closes the browser window. Browser Quit, quits the task as well(session) and allows you to instantiate a new instance of the browser for the subsequent test which gives you a new session.
What i think is happening is by just doing browser.Quit() your session id is set to the first test, and when you go onto the next test a new session id is generated but they do not match causing the executing test to have invalid session id.
Similar to what is being outline in this post
https://developer.mozilla.org/en-US/docs/Web/WebDriver/Errors/InvalidSessionID'
I use SeleniumWebdriver C# to build automated tests.
I also use NUnit to mark test methods (so I can run them).
In each of the tests there are several verification and when the first verification fails then the test stops executing (an exception is thrown).
I want the test to continue executing so more errors could be found!
Please guys, give me a clue how to do it right.
I think about something like that:
Instead of throwing Exception in verification I'll add an error to a stack
When a test ends I check if my stack is not empty
If the stack is not empty I push to console all errors and fail the test
If the stack is empty then the test passed successfully.
Are these steps good? Is there a better way?
I think this should work for you.
class Asserts
{
private static StringBuilder _stack;
[SetUp]
public void SetUp()
{
_stack = new StringBuilder();
}
[TearDown]
public void TearDown()
{
if (_stack.Lenght != 0) Assert.Fail(_stack.ToString());
}
[Test]
public void Test()
{
AssertHelper(() => Assert.AreEqual(0, 0));
AssertHelper(() => Assert.IsNotNull(null));
AssertHelper(() => Assert.AreEqual(3, 4));
AssertHelper(() => Assert.AreEqual(1, 1));
}
private static void AssertHelper(Action assert)
{
try
{
assert();
}
catch (AssertionException e)
{
_stack.Append(e.Message);
}
}
}
But the good idea is to keep one verification per one test.
Generally you want your tests to only be verifying one thing at a time. If there's 2 steps to an action that require asserting, considering having tests like:
UserForm_FirstNameMissing_ThrowsException
UserForm_SecondNameMissing_ThrowsException
UserForm_AgeTooLarge_ThrowsException
instead of having a single test called something like:
UserForm_TestValidation
This means that - like you wanted - if one stage of validation fails, you can continue to test the rest. Your best bet for this may be a singular test with a dataset, however keeping the answer simple yet relevant, a test per failure state is good enough for what you want.
I solved my problem in the next way:
I've created a set of methods that are performed once prior to executing any of the tests ([TestFixtureSetUpAttribute] in NUnit);
In this set of methods I do the main actions results of which I'll check (in my case I buy a lottery ticket);
In each test ([Test] in NUnit) I check one assertion.
So now I have a main action that doesn't include any verification. After this actions is completed I check all needed assertions.
I have an application for smoke testing several key services. I want these tests to be easily written and self discovering. To this end, I have created an attribute for each method in the style of [TestMethod] from MSTest. I have also created a object that will find all these methods and execute them within a try-catch. If the method throws an exception I report as a failure, otherwise as a success.
This is all very unit test familiar and that is the intention. So a test would look like...
[MyTestAttribute]
public void TestingTimesAhead()
{
var d = DateTime.MaxValue.AddDays(1);
}
The magic happens in the test object, this has an Action property and a Run method...
public Action TestAction { get; private set; }
public override ITestResult RunTest()
{
try
{
this.TestAction.Invoke();
return new BaseTestResult() { Result = TestResultStatus.Success };
}
catch(Exception ex)
{
return new BaseTestResult() { Result = TestResultStatus.Failure, FailureException = ex};
}
}
When ran in isolation the above test, being wrapped as the Action, will cause an exception and the test fails. Perfect.
However, when I wrap the attributes and test running object up into a dll and reference from a new project, VS Debugger breaks for the exception.
I am presented with an option in that dialog to toggle 'break when this exception type is user-unhandled'.
I can suppress exceptions on a type by type basis, but that isn't going to work for reuse. Also, if I run the release build from the BIN output, the expected behaviour is there - it's just a really awkward development experience.
Is it possible to manage this exception setting from my code instead? Or if anyone can suggest a way around the problem, I'm open to ideas.
I have a situation where an error getting thrown by a process run on my server for one user and is affecting another process being run by for another user.
Basically, it looks like the other process stop working because of the error, even though the two process are totally unrelated and the pages on which my two users are not the same.
I know this is occuring because my second user has confirmed that he sees the error message even though it isn't related to him or his session.
Has anyone every experience this problem before ?
Thanks.
UPDATE:
The function that generate an alert looks like this, it is called directly in my Catch(Exception ex)
public static class CommonAlert
{
public static void GenerateAlert(Page CurrentPage, string strAlertMessage)
{
string script = #"ApplyJqueryUI();embedHtml({0});
function embedHtml(result)
{{
alert(result)
}};";
ScriptManager.RegisterStartupScript(CurrentPage,
CurrentPage.GetType(),
"embedHtml",
string.Format(script,EncodeJsString(strAlertMessage)), true);
}
}
How are you handling your exceptions? As Ryan McDonough mentioned above, the app pool will recycle after a specified numbers of errors occur.
Are you using static variables anywhere to capture the errors before showing them? Without seeing code this is just a guess, but if you potentially have something like the following going on, then users may see the wrong messages:
public static Exception _lastError;
void Application_Error(object sender, EventArgs e)
{
_lastError = Server.GetLastError();
}
And then in the request, if _lastError is not null, you may be showing it. Again, just a guess...
If this is the case, consider [ThreadStatic] or not using static variables.
Otherwise, post an example of how you're handling your exceptions and you'll get a better answer!
Good luck!
I have built a windows service to populate a database with my email inbox every 5 minutes.
I used a class inside my windows service the class gets my emails and writes them to my database, the class has been tested and works.
All i need the windows service to do is use a timer and call the class every 5 minutes, but i have no idea whats going on as i cant even test my windows service.
Please someone tel me what to do to test, if there is a way to test, or just blink luck and pray it works lol.
Also do u have to uninstall and re-install every time you want to test the service or is there an update service option? Please answer this i'm really interested even tho its not my main question.
This is my windows service, if u can point out any errors that would be amazing since i cant test for them. I think my timer might be wrong if some one could look at it?
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Linq;
using System.ServiceProcess;
using System.Text;
using System.Timers;
namespace EmailWindowsService
{
public partial class MyEmailService : ServiceBase
{
private Timer scheduleTimer1 = null;
private DateTime lastRun;
private bool flag;
public MyEmailService()
{
InitializeComponent();
if (!System.Diagnostics.EventLog.SourceExists("MySource"))
{
System.Diagnostics.EventLog.CreateEventSource(
"MySource", "MyNewLog");
}
eventLogEmail.Source = "MySource";
eventLogEmail.Log = "MyNewLog";
scheduleTimer1 = new Timer();
scheduleTimer1.Interval = 5 * 60 * 1000;
scheduleTimer1.Elapsed += new ElapsedEventHandler(scheduleTimer_Elapsed);
}
protected override void OnStart(string[] args)
{
flag = true;
lastRun = DateTime.Now;
scheduleTimer.Start();
eventLogEmail.WriteEntry("Started");
}
protected override void OnStop()
{
scheduleTimer.Stop();
eventLogEmail.WriteEntry("Stopped");
}
protected override void OnPause()
{
scheduleTimer.Stop();
eventLogEmail.WriteEntry("Paused");
}
protected override void OnContinue()
{
scheduleTimer.Start(); ;
eventLogEmail.WriteEntry("Continuing");
}
protected override void OnShutdown()
{
scheduleTimer.Stop();
eventLogEmail.WriteEntry("ShutDowned");
}
protected void scheduleTimer_Elapsed(object sender, ElapsedEventArgs e)
{
RetriveEmailClass Emails = new RetriveEmailClass();
if (flag == true)
{
eventLogEmail.WriteEntry("In getting Email Method");
Emails.ServiceEmailMethod();
lastRun = DateTime.Now;
flag = false;
}
else if (flag == false)
{
if (lastRun.Date < DateTime.Now.Date)
{
Emails.ServiceEmailMethod();
eventLogEmail.WriteEntry("In getting Email Method");
}
}
}
}
}
Surely you can test it. All you need is
start up the service
observe that it triggers the expected call after 5 minutes
(observe that it triggers the expected call every 5 minutes for a couple more times)
You can test this manually, or (preferably) create/use an automated test harness which allows you to test repeatedly and reliably, as many times as you want. This is possible even using a simple batch file.
To detect that the timer works correctly, you can inspect its log file. It also helps of course if you make the called class method configurable instead of hardcoding it. So you can run your automated tests using a dummy worker class which does not flood your inbox :-)
To make it even more testable, you can extract the timing logic from your service class too, so that it becomes runnable from a regular application. Then you can test it even easier, even using a unit test framework such as NUnit. This allows you to do more thorough testing, using different timing intervals etc. And the service class itself becomes an almost empty shell whose only job is to launch and call the other classes. If you have verified that all the classes containing real program logic (i.e. all the code which can fail) is unit tested and works fine, you can have much greater confidence in that your whole app, when integrated from its smaller parts, works correctly too.
Update
Looking through your code, it seems that you don't initialize flag anywhere, so its default value will be false. You should initialize it to true in the constructor, otherwise your email retriever won't ever get called even if the timer fires properly.
To set the interval to 1 minute, my first guess would be
scheduleTimer1.Interval = 1 * 60 * 1000;
James Michael Hare has on his blog written about a really nice template/framework he has made, making it lot easier to develop (and debug) Windows Services: C# Toolbox: A Debuggable, Self-Installing Windows Service Template (1 of 2)
It provides you with all the basics you need to quickly get started. And best of all, it give you a really nice way to debug your service as if it was a regular console application. I could also mention that it provides out of the box functionality to install (and uninstall) your service. Part two of the post can be found at this link.
I've used this myself a couple of times, and can really recommend it.
Refactor you logic in another class.
Write a simple console application invoking this class
Test it like a normal application.
Once it runs standalone, it should run as a service.
Beware on permissions and service registration, there are a couple of issues there (like having a sys user, or a desktop session).
A good practice is to use system logs (e.g. the ones you can inspect with eventvwr)
1.add this line to the place you want to break, then you can debug your service.
System.Diagnostics.Debugger.Break();
or
2.try to attach to your service progress from process explorer, then you can also debug your service.
or
3.use a log file to log what your service is doing.
You can attach a debugger to you running service instance from Visual Studio. Click "Debug" in the main menu, "Attach to Process...", select your service process from the list and click "Attach".
If you need to debug the startup of your service, you need to use System.Diagnostics.Debugger.Break().