We are developing some functional tests using Selenium 2 framework on a FireFox 10 environment and we are in trouble with this special functionality:
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(5));
We are trying to setup the implicitly wait time to avoid sleep statements into our page objects. We we run the tests local or using a remote web driver everything works flawlessly, but when executing inside the TeamCity 7.0.4 server we got this unexpected modal:
SetUp method failed. SetUp : System.InvalidOperationException : Modal dialog present (UnexpectedAlertOpen)
at ... [Stack trace] ...
Unfortunately we don't have access to see what modal is really showing up. If we turn back using sleep timers in place of the driver configuration, everything works.
Obs: We are sure that the TeamCity environment is setup just like in our local machines. Including the FireFox version.
Any ideas?
Thank you!
Start TeamCity Agent as normal process under your account (and not windows service) and wait what kind of dialog you will see.
Related
currently in my 2nd week of trying to work out why a certain test runs perfectly locally but fails when being kicked off by a VSTS agent.
Program flow:
Select an row from a table
Click Verify button
CSS overlay darkens the screen, spinner in the middle
Toaster pop up in the bottom left saying Verification starting
About 1.5 minutes of time passes, spinner spins
CSS overlay removed
Toaster pop up bottom left, successful verification
I have screenshots and debug output on everystep of the way and the failure is at the "Waiting for CSS overlay to be removed" stage. It times out waiting for it, now matter how long the timeout.
Screenshot shows the CSS overlay and spinner still spinning even though the verification process completed successfully. Its almost as if it's stuck in time.
Scenarios and results:
Run from Local Visual Studio - PASS
VSTS Agent - FAIL
Manually run on VSTS Agent by remoting in and passing the exact commandline as the build - PASS (This is what infuriates me!!)
Downgrading / upgrading Webdriver - no difference
Headless / full mode - no difference
Using different types of waits - no difference
Waiting for Toaster pop ups instead of CSS - no difference
One thing to note from the documentation:
Agents that are configured to run as service can run Selenium tests
only with headless browsers. If you are not using a headless browser,
or if you are running UI tests for desktop apps, Windows agents must
be configured to run as an interactive process with auto-logon
enabled.
The fact that you also tried to run Headless might mean that even if you are running as a service and still got errors, this might not be your issue. In some cases, I have had issues with remoting to the machine messing up future test runs or video recording by logging out the user or changing the screen resolution. The documentation continues about ways to prevent this:
If you use Remote Desktop to access the computer on which an agent is
running with auto-logon, simply disconnecting the Remote Desktop
causes the computer to be locked and any UI tests that run on this
agent may fail. To avoid this, use the tscon command on the remote
computer to disconnect from Remote Desktop. For example:
%windir%\System32\tscon.exe 1 /dest:console
In this example, the number '1' is the ID of the remote desktop
session. This number may change between remote sessions, but can be
viewed in Task Manager. Alternatively, to automate finding the current
session ID, create a batch file containing the following code:
for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do (
%windir%\System32\tscon.exe %%s /dest:console
)
Save the batch file and create a desktop shortcut to it, then change the shortcut
properties to 'Run as administrator'. Running the batch file from this
shortcut disconnects from the remote desktop but preserves the UI
session and allows UI tests to run.
I'll answer my own question: We created a new agent which was an auto login. We watched the test run and pressed F12 and there, in the console, was an error.
It was trying to trim() on the username, the test account looked like it had a username but I guess it didn't really. So the system, when run by the agent, was throwing an error and not handling it.
Handled the error and everything went through swimmingly. Very long and frustrating road to get to here! Hope this helps someone in the future.
Please check Azure DevOps Hands-On-Labs to get started with Automating Selenium Tests in Azure Pipelines. Hope this helps...
VSTS will run Selenium and the browser as local admin. The browser will refuse that when started in sandbox mode.
Check for articles related to running Selenium as local admin
I have been able to run Selenium test locally on my test machine using a simple webdriver setup such as IWebDriver driver = new FirefoxDriver(); driver.Url = "http://www.testedApp.com"; and running the tests through Microsoft.VisualStudio.TestTools.UnitTesting;, but when I deploy the test through Team Foundation Server vnext Build step "Test: Run Functional Tests" https://www.visualstudio.com/en-gb/docs/build/steps/test/run-functional-tests I get various errors that all seem to relate back to not being able to create a driver in the first place.
I am deploying the selenium tests onto remote machines (either windows Server 2008 R2 or 2012) that have the application to be tested already installed. It doesn't seem to matter if I put the url to driver.Url = "http://www.testedApp.com"; or driver.Url = "http://www.localhost/testedApp.com"; the machine (which looks to still be running from the TFS server machine, not the machine I have deployed the tests onto) appears to be failing at timeouts which led me to this answer on Stack Overflow
Selenium Error - The HTTP request to the remote WebDriver timed out after 60 seconds
After trying all the options there, looking further, I discovered the Selenium Server and RemoteWebdriver (the RemoteWebdriver actually being in my error very deep down)
I am also running the test agent on the remote machine under an Administrator account, so would assume the UI access wouldn't be a problem, but the error also persists if I am using Phantomjs as a browser.
So, I guess my real question is, in order to run a test successfully on a remote machine from the TFS machine, will I need to install and run a Selemium Server (and therefore create my drivers through a RemoteWebdriver)?
If so, would this Selenium Server go on the TFS machine, or the remote machine I am attempting to run the tests on?
I just wanted to update this with my solution.
I managed to get the vnext Build step "Test: Run Functional Tests" working without using remoteWebDriver or running a Selenium Server with nodes.
I had all the correct trust levels and ports opened, my problem was that I didn't realise that I needed to have the agent running the tests being Authenticated with a PAT Token (in other words, it requires ssl to operate successfully) (DOH!)
I also ran into a problem where the domain policy kept resetting the Group Policy for autologon to 0, which I have got around by running a script to reset before I kick the tests off.
Hope this helps somebody else.
And I apologise in advance if my question initially led to some false assumptions.
But, needless to say, I am grateful that I am not required to maintain the selenium servers (or Octopus deploy), and that Vnext build is adaptable enough to handle this all in house.
Vnext still has a way to go to become completely adaptable to my needs, but it is a great start (imo)
You will need a selenium server or node on every machine running a browser.
I'm supposed to test the reaction of a page when net is disconnected while on the page. (errors get thrown).
My Method does the following - the page loads, i disconnect net, verify if the right actions take place and then connect the net back up and continue the next set of actions. This method is a part of an ordered test.
I have the code that makes this possible, and works as expected while running the test on Visual Studio (VS 2013).
I tried this in two ways using C# (Coded UI) - by disconnecting the LAN(bad option) and by disconnecting the WAN(by disabling the DNS which poses an issue since MTM tries to connect with my system, but fails since theres no DNS).
Both of these are not really helpful while i try to run my ordered test on MTM.
When I try and run the ordered test on MTM, it just does not work! I get the errors 'Your Team Foundation Server is currently unavailable' or 'An error occurred while communicating with Agent'.
The destination for the test run is my own system.
Is there anyway i could disconnect the internet AND not affect MTM while this happens ?
Any ideas/alternate solutions will be much appreciated !!
Thanks!
I'm having some troubles with building reliable environment for our Selenium based UI tests. We have tried Selenium Grid + latest Firefox (+ 41.x), Latest Chrome as well but they are always some problems with hanging browser and failing tests because of that (or builds). SeleniumDriver is latest as well (nuget 2.49.0).
I can mention also some crazy things like even with all this we still can't make browsers go full screen if test are started from Team City job against Selenium grid:
firefoxDriver.Manage().Window.Maximize();
//Make Firefox fullscreen for TeamCity agent's runs
((IJavaScriptExecutor)firefoxDriver).ExecuteScript(
"window.resizeTo(screen.width, screen.height)");
firefoxDriver.Manage().Window.Position = new Point(0, 0);
firefoxDriver.Manage().Window.Size = new Size(Screen.PrimaryScreen.WorkingArea.Width, Screen.PrimaryScreen.WorkingArea.Height);
firefoxDriver.FindElement(By.CssSelector("body")).SendKeys(Keys.F11);
We also tried to run the test locally by installing team city agent on the virtual machine (we use Azure) - still same problem - maybe even worse...
Before each test we start new browser and after it we stop it with:
Driver.Close();
Driver.Dispose();
Driver.Quit();
but still they are some unstopped/hang browsers...
When executed locally from Visual Studio 1 by 1 (not as a suite) everything is OK.
Any suggestions what we might have done wrong?
Is it entirely browser fault? And if yes what we can use? I would like to try PhantomJS (Ghostdriver) but it's kind a outdated and have another set of problems with it. Any other option? IE 11 also have a problem -> Selenium Server error: Unable to get browser which did not quite work for us.
Not to mention some stuffs like 5 dropdowns can be automated but the 6th one is not working in Firefox, but works on Chrome and vice versa.
Also I have a feeling the Azure VM is really slow maybe real machine would help (keep in mind that even on my machine if I run it with team city commands browsers hangs as well).
I'm pretty sure someone else also face all this problems and hopefully have some guidance to share which I think would be helpful to others as well in the road to stable automation environment for Selenium UI tests.
Try to see the change Log under selenium official website, and use the browser and selenium which are compatible to each other. After that you can use the code for setup and login.
I have created a web application in ASP.NET, with C#. I am getting the following message in my browser window when accessing the application:
---------------------------
Windows Internet Explorer
---------------------------
Stop running this script?
A script on this page is causing Internet Explorer to run slowly.
If it continues to run, your computer might become unresponsive.
I researched into the problem and found the following link, which is proposed as a solution to this problem:
http://support.microsoft.com/kb/175500
I want to execute this setup when a client accesses the web site for the first time. How can I run this setup on the client side?
It is not possible to fully automatically run some program from a website as #cHao said, this is what all browsers prevent.
But you may look at ClickOnce deployment, it allows for very fast and easy installation of your program literally in one click.
http://msdn.microsoft.com/en-us/library/t71a733d(v=vs.80).aspx