currently in my 2nd week of trying to work out why a certain test runs perfectly locally but fails when being kicked off by a VSTS agent.
Program flow:
Select an row from a table
Click Verify button
CSS overlay darkens the screen, spinner in the middle
Toaster pop up in the bottom left saying Verification starting
About 1.5 minutes of time passes, spinner spins
CSS overlay removed
Toaster pop up bottom left, successful verification
I have screenshots and debug output on everystep of the way and the failure is at the "Waiting for CSS overlay to be removed" stage. It times out waiting for it, now matter how long the timeout.
Screenshot shows the CSS overlay and spinner still spinning even though the verification process completed successfully. Its almost as if it's stuck in time.
Scenarios and results:
Run from Local Visual Studio - PASS
VSTS Agent - FAIL
Manually run on VSTS Agent by remoting in and passing the exact commandline as the build - PASS (This is what infuriates me!!)
Downgrading / upgrading Webdriver - no difference
Headless / full mode - no difference
Using different types of waits - no difference
Waiting for Toaster pop ups instead of CSS - no difference
One thing to note from the documentation:
Agents that are configured to run as service can run Selenium tests
only with headless browsers. If you are not using a headless browser,
or if you are running UI tests for desktop apps, Windows agents must
be configured to run as an interactive process with auto-logon
enabled.
The fact that you also tried to run Headless might mean that even if you are running as a service and still got errors, this might not be your issue. In some cases, I have had issues with remoting to the machine messing up future test runs or video recording by logging out the user or changing the screen resolution. The documentation continues about ways to prevent this:
If you use Remote Desktop to access the computer on which an agent is
running with auto-logon, simply disconnecting the Remote Desktop
causes the computer to be locked and any UI tests that run on this
agent may fail. To avoid this, use the tscon command on the remote
computer to disconnect from Remote Desktop. For example:
%windir%\System32\tscon.exe 1 /dest:console
In this example, the number '1' is the ID of the remote desktop
session. This number may change between remote sessions, but can be
viewed in Task Manager. Alternatively, to automate finding the current
session ID, create a batch file containing the following code:
for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do (
%windir%\System32\tscon.exe %%s /dest:console
)
Save the batch file and create a desktop shortcut to it, then change the shortcut
properties to 'Run as administrator'. Running the batch file from this
shortcut disconnects from the remote desktop but preserves the UI
session and allows UI tests to run.
I'll answer my own question: We created a new agent which was an auto login. We watched the test run and pressed F12 and there, in the console, was an error.
It was trying to trim() on the username, the test account looked like it had a username but I guess it didn't really. So the system, when run by the agent, was throwing an error and not handling it.
Handled the error and everything went through swimmingly. Very long and frustrating road to get to here! Hope this helps someone in the future.
Please check Azure DevOps Hands-On-Labs to get started with Automating Selenium Tests in Azure Pipelines. Hope this helps...
VSTS will run Selenium and the browser as local admin. The browser will refuse that when started in sandbox mode.
Check for articles related to running Selenium as local admin
Related
I have been able to run Selenium test locally on my test machine using a simple webdriver setup such as IWebDriver driver = new FirefoxDriver(); driver.Url = "http://www.testedApp.com"; and running the tests through Microsoft.VisualStudio.TestTools.UnitTesting;, but when I deploy the test through Team Foundation Server vnext Build step "Test: Run Functional Tests" https://www.visualstudio.com/en-gb/docs/build/steps/test/run-functional-tests I get various errors that all seem to relate back to not being able to create a driver in the first place.
I am deploying the selenium tests onto remote machines (either windows Server 2008 R2 or 2012) that have the application to be tested already installed. It doesn't seem to matter if I put the url to driver.Url = "http://www.testedApp.com"; or driver.Url = "http://www.localhost/testedApp.com"; the machine (which looks to still be running from the TFS server machine, not the machine I have deployed the tests onto) appears to be failing at timeouts which led me to this answer on Stack Overflow
Selenium Error - The HTTP request to the remote WebDriver timed out after 60 seconds
After trying all the options there, looking further, I discovered the Selenium Server and RemoteWebdriver (the RemoteWebdriver actually being in my error very deep down)
I am also running the test agent on the remote machine under an Administrator account, so would assume the UI access wouldn't be a problem, but the error also persists if I am using Phantomjs as a browser.
So, I guess my real question is, in order to run a test successfully on a remote machine from the TFS machine, will I need to install and run a Selemium Server (and therefore create my drivers through a RemoteWebdriver)?
If so, would this Selenium Server go on the TFS machine, or the remote machine I am attempting to run the tests on?
I just wanted to update this with my solution.
I managed to get the vnext Build step "Test: Run Functional Tests" working without using remoteWebDriver or running a Selenium Server with nodes.
I had all the correct trust levels and ports opened, my problem was that I didn't realise that I needed to have the agent running the tests being Authenticated with a PAT Token (in other words, it requires ssl to operate successfully) (DOH!)
I also ran into a problem where the domain policy kept resetting the Group Policy for autologon to 0, which I have got around by running a script to reset before I kick the tests off.
Hope this helps somebody else.
And I apologise in advance if my question initially led to some false assumptions.
But, needless to say, I am grateful that I am not required to maintain the selenium servers (or Octopus deploy), and that Vnext build is adaptable enough to handle this all in house.
Vnext still has a way to go to become completely adaptable to my needs, but it is a great start (imo)
You will need a selenium server or node on every machine running a browser.
Context: Azure; C#; .NET 4.5.2; Selenium 2.53
I'm running a Selenium-based scraper written in C# from a Scheduled Task. ChromeDriver.exe runs silently and launches Chrome.exe silently: neither interact with the logged-on user's screen.
I've been asked to make a video of the interaction between the C# app, ChromeDriver and Chrome. It's proving to be a bit harder than I thought.
I've tried Screencastify on the expectation that, because it's running as a Chrome extension, that it would "see" the screen and thus be able to record it. However, Screencastify is proving to be quite hard to drive, not least because I'm having to send keystrokes to it rather than being able to control it via an API.
I noticed that Screencastify does have an API but that using it presupposes that you've got it installed in your own site. That is, it's not scraper friendly.
There are other screen capture tools out there like ScreenRecorder. I'm not at all sure if it will work in this situation, there being no screen as such (unless one launches it with the scraper.)
In this situation is Chrome just doing something within itself and ignoring the lack of an output device? That is, is it running in a kind of pseudo-headless-browser state?
Is making a video of a scheduled scrape by definition impossible?
You have to set the task to only run when the user is logged in otherwise it will start hidden and you won't be able to see it. Also make sure hidden isn't checked. You can set a default user to login when the machine starts and then have the task kick off when that user is logged in. That way everything will be visible on the desktop.
We are developing some functional tests using Selenium 2 framework on a FireFox 10 environment and we are in trouble with this special functionality:
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(5));
We are trying to setup the implicitly wait time to avoid sleep statements into our page objects. We we run the tests local or using a remote web driver everything works flawlessly, but when executing inside the TeamCity 7.0.4 server we got this unexpected modal:
SetUp method failed. SetUp : System.InvalidOperationException : Modal dialog present (UnexpectedAlertOpen)
at ... [Stack trace] ...
Unfortunately we don't have access to see what modal is really showing up. If we turn back using sleep timers in place of the driver configuration, everything works.
Obs: We are sure that the TeamCity environment is setup just like in our local machines. Including the FireFox version.
Any ideas?
Thank you!
Start TeamCity Agent as normal process under your account (and not windows service) and wait what kind of dialog you will see.
I have created a web application in ASP.NET, with C#. I am getting the following message in my browser window when accessing the application:
---------------------------
Windows Internet Explorer
---------------------------
Stop running this script?
A script on this page is causing Internet Explorer to run slowly.
If it continues to run, your computer might become unresponsive.
I researched into the problem and found the following link, which is proposed as a solution to this problem:
http://support.microsoft.com/kb/175500
I want to execute this setup when a client accesses the web site for the first time. How can I run this setup on the client side?
It is not possible to fully automatically run some program from a website as #cHao said, this is what all browsers prevent.
But you may look at ClickOnce deployment, it allows for very fast and easy installation of your program literally in one click.
http://msdn.microsoft.com/en-us/library/t71a733d(v=vs.80).aspx
Is there a way for a program to be executed without clicking on it? Is it possible for it to be executed the minute you receive the file?
I want to build a c# application that executes any file sent to the machine from another machine connected to it. I want to do this without remote desktop, so I would check a checkbox that says "execute when finished sending" and when it is received, it would execute the program. So I would essentially be able to remote desktop, without actually remote desktoping.
Thanks in advance
Do you realize what you are proposing? If this was possible, every machine out there would be infested with the nastiest viruses.
Why don't you build a server part that you leave running on your remote machine and then have it execute whatever your heart pleases.
There is a program out there (currently in Beta) called SkyNet that does this. You can drag and drop a URL or file onto one of your "Contacts" and (for a URL) it will open an new tab and navigate to that site on your friend's computer. For files, it automatically drops them on that person's desktop.
So it is definitely possible.
Perhaps you want to use secure shell. There are clients and servers for various platforms (you didn't specify) and you can run programs remotely. This is useful for e.g. automated testing.