I'm having some troubles with building reliable environment for our Selenium based UI tests. We have tried Selenium Grid + latest Firefox (+ 41.x), Latest Chrome as well but they are always some problems with hanging browser and failing tests because of that (or builds). SeleniumDriver is latest as well (nuget 2.49.0).
I can mention also some crazy things like even with all this we still can't make browsers go full screen if test are started from Team City job against Selenium grid:
firefoxDriver.Manage().Window.Maximize();
//Make Firefox fullscreen for TeamCity agent's runs
((IJavaScriptExecutor)firefoxDriver).ExecuteScript(
"window.resizeTo(screen.width, screen.height)");
firefoxDriver.Manage().Window.Position = new Point(0, 0);
firefoxDriver.Manage().Window.Size = new Size(Screen.PrimaryScreen.WorkingArea.Width, Screen.PrimaryScreen.WorkingArea.Height);
firefoxDriver.FindElement(By.CssSelector("body")).SendKeys(Keys.F11);
We also tried to run the test locally by installing team city agent on the virtual machine (we use Azure) - still same problem - maybe even worse...
Before each test we start new browser and after it we stop it with:
Driver.Close();
Driver.Dispose();
Driver.Quit();
but still they are some unstopped/hang browsers...
When executed locally from Visual Studio 1 by 1 (not as a suite) everything is OK.
Any suggestions what we might have done wrong?
Is it entirely browser fault? And if yes what we can use? I would like to try PhantomJS (Ghostdriver) but it's kind a outdated and have another set of problems with it. Any other option? IE 11 also have a problem -> Selenium Server error: Unable to get browser which did not quite work for us.
Not to mention some stuffs like 5 dropdowns can be automated but the 6th one is not working in Firefox, but works on Chrome and vice versa.
Also I have a feeling the Azure VM is really slow maybe real machine would help (keep in mind that even on my machine if I run it with team city commands browsers hangs as well).
I'm pretty sure someone else also face all this problems and hopefully have some guidance to share which I think would be helpful to others as well in the road to stable automation environment for Selenium UI tests.
Try to see the change Log under selenium official website, and use the browser and selenium which are compatible to each other. After that you can use the code for setup and login.
Related
currently in my 2nd week of trying to work out why a certain test runs perfectly locally but fails when being kicked off by a VSTS agent.
Program flow:
Select an row from a table
Click Verify button
CSS overlay darkens the screen, spinner in the middle
Toaster pop up in the bottom left saying Verification starting
About 1.5 minutes of time passes, spinner spins
CSS overlay removed
Toaster pop up bottom left, successful verification
I have screenshots and debug output on everystep of the way and the failure is at the "Waiting for CSS overlay to be removed" stage. It times out waiting for it, now matter how long the timeout.
Screenshot shows the CSS overlay and spinner still spinning even though the verification process completed successfully. Its almost as if it's stuck in time.
Scenarios and results:
Run from Local Visual Studio - PASS
VSTS Agent - FAIL
Manually run on VSTS Agent by remoting in and passing the exact commandline as the build - PASS (This is what infuriates me!!)
Downgrading / upgrading Webdriver - no difference
Headless / full mode - no difference
Using different types of waits - no difference
Waiting for Toaster pop ups instead of CSS - no difference
One thing to note from the documentation:
Agents that are configured to run as service can run Selenium tests
only with headless browsers. If you are not using a headless browser,
or if you are running UI tests for desktop apps, Windows agents must
be configured to run as an interactive process with auto-logon
enabled.
The fact that you also tried to run Headless might mean that even if you are running as a service and still got errors, this might not be your issue. In some cases, I have had issues with remoting to the machine messing up future test runs or video recording by logging out the user or changing the screen resolution. The documentation continues about ways to prevent this:
If you use Remote Desktop to access the computer on which an agent is
running with auto-logon, simply disconnecting the Remote Desktop
causes the computer to be locked and any UI tests that run on this
agent may fail. To avoid this, use the tscon command on the remote
computer to disconnect from Remote Desktop. For example:
%windir%\System32\tscon.exe 1 /dest:console
In this example, the number '1' is the ID of the remote desktop
session. This number may change between remote sessions, but can be
viewed in Task Manager. Alternatively, to automate finding the current
session ID, create a batch file containing the following code:
for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do (
%windir%\System32\tscon.exe %%s /dest:console
)
Save the batch file and create a desktop shortcut to it, then change the shortcut
properties to 'Run as administrator'. Running the batch file from this
shortcut disconnects from the remote desktop but preserves the UI
session and allows UI tests to run.
I'll answer my own question: We created a new agent which was an auto login. We watched the test run and pressed F12 and there, in the console, was an error.
It was trying to trim() on the username, the test account looked like it had a username but I guess it didn't really. So the system, when run by the agent, was throwing an error and not handling it.
Handled the error and everything went through swimmingly. Very long and frustrating road to get to here! Hope this helps someone in the future.
Please check Azure DevOps Hands-On-Labs to get started with Automating Selenium Tests in Azure Pipelines. Hope this helps...
VSTS will run Selenium and the browser as local admin. The browser will refuse that when started in sandbox mode.
Check for articles related to running Selenium as local admin
I have been able to run Selenium test locally on my test machine using a simple webdriver setup such as IWebDriver driver = new FirefoxDriver(); driver.Url = "http://www.testedApp.com"; and running the tests through Microsoft.VisualStudio.TestTools.UnitTesting;, but when I deploy the test through Team Foundation Server vnext Build step "Test: Run Functional Tests" https://www.visualstudio.com/en-gb/docs/build/steps/test/run-functional-tests I get various errors that all seem to relate back to not being able to create a driver in the first place.
I am deploying the selenium tests onto remote machines (either windows Server 2008 R2 or 2012) that have the application to be tested already installed. It doesn't seem to matter if I put the url to driver.Url = "http://www.testedApp.com"; or driver.Url = "http://www.localhost/testedApp.com"; the machine (which looks to still be running from the TFS server machine, not the machine I have deployed the tests onto) appears to be failing at timeouts which led me to this answer on Stack Overflow
Selenium Error - The HTTP request to the remote WebDriver timed out after 60 seconds
After trying all the options there, looking further, I discovered the Selenium Server and RemoteWebdriver (the RemoteWebdriver actually being in my error very deep down)
I am also running the test agent on the remote machine under an Administrator account, so would assume the UI access wouldn't be a problem, but the error also persists if I am using Phantomjs as a browser.
So, I guess my real question is, in order to run a test successfully on a remote machine from the TFS machine, will I need to install and run a Selemium Server (and therefore create my drivers through a RemoteWebdriver)?
If so, would this Selenium Server go on the TFS machine, or the remote machine I am attempting to run the tests on?
I just wanted to update this with my solution.
I managed to get the vnext Build step "Test: Run Functional Tests" working without using remoteWebDriver or running a Selenium Server with nodes.
I had all the correct trust levels and ports opened, my problem was that I didn't realise that I needed to have the agent running the tests being Authenticated with a PAT Token (in other words, it requires ssl to operate successfully) (DOH!)
I also ran into a problem where the domain policy kept resetting the Group Policy for autologon to 0, which I have got around by running a script to reset before I kick the tests off.
Hope this helps somebody else.
And I apologise in advance if my question initially led to some false assumptions.
But, needless to say, I am grateful that I am not required to maintain the selenium servers (or Octopus deploy), and that Vnext build is adaptable enough to handle this all in house.
Vnext still has a way to go to become completely adaptable to my needs, but it is a great start (imo)
You will need a selenium server or node on every machine running a browser.
I have an application that is accessed by many users. It's a .net 4.0 C# web forms application, and they requested me to automate a task that they have to do almost daily. The task I should automate is to access a link outside my domain and fill it up with some info, log-in, and other stuff like dates.
So, after many researching and reading, I've managed to do it locally with Selenium. I have the entire process automated and I call it from a link inside my website. Locally it works like a charm.
Here's the code:
public void AutomateTask()
{
var url = new Uri("http://127.0.0.1:4545/wd/hub");
var capability = DesiredCapabilities.Chrome();
var driver = new RemoteWebDriver(url, capability);
driver.Navigate().GoToUrl("url");
//do all the stuff I need to do
driver.Close();
}
I start my server using this command:
java -jar C:\...\selenium-server-standalone-2.43.1.jar -role hub -port 4545
The main problem is that I have to publish my website for the users and they should be able to use the automated process without having to start a selenium node on their machines or anything else. Locally I have a node running for each browser, is it possible for them to consume this webdrivers? I mean, users should not worry about installing or anything, they should just click the link and see their browsers acting till the task is over. When I click at the link on my application(from my local IIS) on another computer, a new window of the request browser is opened on server machine, not on the machine which clicked the link.
I've looking everywhere for a solution for this but none was found.
So:
I have a website that access a third website.
I need to automate this process.
Users must only click the link, wait the new browser window open, wait for some fields to be filled automatic and then fill the captcha and press "continue", then enjoy their browser working.
Is it possible with selenium? If not, which framework or how can I achieve this?
Let me know if any further details are required.
If you don't need to show the browser to the users you could just put a link on your webapp which runs selenium on the machine hosting your webapp. That way you only need to have selenium installed and running on the webserver hosting your webapp and not all the end user machines. If you do it this way you should probably use the HtmlUnitDriver because its headless and you really aren't going to be watching these selenium tests executing.
I am starting to write tests for a project that is in its 8th agile sprint. I was brought on the team about one week ago.
Yesterday my Selenium tests were passing against the git master branch. I was accessing the http localhost address, opening it, running a test, and then closing the browser all from the Selenium code.
My team changed the access (somehow, don't quite understand) and the http localhost address changed. This was yesterday evening. Pulled the new code this morning from git.
I found the new address, and replaced it in my localhost constant. Ran the tests. Now the tests open the browser, no localhost info is entered into the url, test times out, test fails. All I changed was the localhost address.
When I just run start from the Visual Studio ribbon start button, the web app will open two windows in the browser (Firefox default). The team lead says that both windows are needed right now.
I am fairly new to Selenium (1 month and learning) and very new to .NET/Visual Studio/C# (day 4).
Researched the issue, only thing I have found that might make sense is creating an ASP.NET HTTP Module-- And I have read up on it, but don't know if it is THE solution.
If anybody has any other ideas, please let me know.
thx
a
EDIT : CODE
string URL = "http://localhost:54879/"; //unable to connect
....
driver.Navigate().GoToUrl(URL);
Had our local Microsoft guru look at what was happening. Tests run/console connects if I run after selecting start without debugging. I am v. new to Visual Studio, so I guess it was set up wrong in my tray and this makes it run properly.
In my case, the problem was that there was i needed to first run the programme without debuggen ( Debug > Start without Debugging).
We are developing some functional tests using Selenium 2 framework on a FireFox 10 environment and we are in trouble with this special functionality:
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(5));
We are trying to setup the implicitly wait time to avoid sleep statements into our page objects. We we run the tests local or using a remote web driver everything works flawlessly, but when executing inside the TeamCity 7.0.4 server we got this unexpected modal:
SetUp method failed. SetUp : System.InvalidOperationException : Modal dialog present (UnexpectedAlertOpen)
at ... [Stack trace] ...
Unfortunately we don't have access to see what modal is really showing up. If we turn back using sleep timers in place of the driver configuration, everything works.
Obs: We are sure that the TeamCity environment is setup just like in our local machines. Including the FireFox version.
Any ideas?
Thank you!
Start TeamCity Agent as normal process under your account (and not windows service) and wait what kind of dialog you will see.