Develop Reference

ChromeDriver getting detected after first request

I'm using Selenium ChromeDriver to navigate to pages and it works fine, but on second request, I get intercepted by Incapsula.
If I dispose of the driver everytime, it works though.
Here's the current code:
var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments(new List<string>() { "headless" });
var chromeDriverService = ChromeDriverService.CreateDefaultService();
ChromeDriver driver = new ChromeDriver(chromeDriverService, chromeOptions);
The code below is in a loop which iterates over many records
//extract json variable from page output
ResultModel resultModel = new ResultModel();
driver = new ChromeDriver(chromeDriverService, chromeOptions);
driver.Navigate().GoToUrl($"https://www.website.ca{resultUrl}");
var modelString = driver.ExecuteScript("return JSON.stringify(window.the_variable);", new object[] { });
if (modelString != null)
resultModel = JsonConvert.DeserializeObject<ResultModel>(modelString.ToString());
driver.Dispose();
So this works, but disposing and re-creating the driver everytime slows the process quite a bit.
When I try to simply Navigate to the next page, after the first request, I get intercepted.
What is happening exactly when I'm disposing and recreating ? Could I spoof that without actually doing this ?

Clearing the cookies seemed to have helped:
driver.ExecuteChromeCommand("Network.clearBrowserCookies", new Dictionary<string, object>() );

C# Selenium Chrome change homepage

i am trying to change chrome default homepage (google tabs) but i didn' t find a working solution.
What i have tried:
var _options = new ChromeOptions();
_options.AddUserProfilePreference("homepage", "http://www.example.com");
_options.AddUserProfilePreference("homepage_is_newtabpage", true);
_options.AddUserProfilePreference("session.restore_on_startup", 4);
_options.AddUserProfilePreference("session.startup_urls", new List<string>() { "http://in.gr"});
_options.AddArgument("--homepage=http://in.gr");
var _driver = new ChromeDriver(_options);

You can navigate to the page you want to after initializing the WebDriver before you begin the rest of your web automation.
...
var _driver = new ChromeDriver(_options);
_driver.Navigate().GoToUrl("http://in.gr");
Hope that is of some help.

How to start ChromeDriver in headless mode

I want to try out headless chrome, but I am running into this issue, that I can't start the driver in headless mode. I was following google documentation. am I missing something ? The code execution gets stuck in var browser = new ChromeDriver(); line
Here is my code:
var chromeOptions = new ChromeOptions
{
BinaryLocation = #"C:\Users\2-as Aukstas\Documents\Visual Studio 2017\Projects\ChromeTest\ChromeTest\bin\Debug\chromedriver.exe",
DebuggerAddress = "localhost:9222"
};
chromeOptions.AddArguments(new List<string>() {"headless", "disable-gpu" });
var browser = new ChromeDriver(chromeOptions);
browser.Navigate().GoToUrl("https://stackoverflow.com/");
Console.WriteLine(browser.FindElement(By.CssSelector("#h-top-questions")).Text);

UPDATE
Chrome version 60 is out so all you need to do is to download Chromdriver and Selenium via Nuget and use this simple code and everything works like a charm. Amazing.
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
...
var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments("headless");
using (var browser = new ChromeDriver(chromeOptions))
{
// add your code here
}
DATED
There is a solution until the official release of Chrome 60 will be released. You can download Chrome Canary and use headless with it. After installation set BinaryLocation to point to chrome canary also comment out the DebuggerAddress line(it forces chrome to timeout):
var chromeOptions = new ChromeOptions
{
BinaryLocation = #"C:\Users\2-as Aukstas\AppData\Local\Google\Chrome SxS\Application\chrome.exe",
//DebuggerAddress = "127.0.0.1:9222"
};
chromeOptions.AddArguments(new List<string>() { "no-sandbox", "headless", "disable-gpu" });
var _driver = new ChromeDriver(chromeOptions);

For you that did not get reference for ChromeDriver.
Use this step :
Download the dll from this: http://seleniumtestings.com/selenium-download/
Extract, and you should see: Selenium.WebDriverBackedSelenium.dll, ThoughtWorks.Selenium.Core.dll, WebDriver.dll and WebDriver.Support.dll
Add those files via "Add Reference"
Now you can use it:
String url = "http://www.google.com";
var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments(new List<string>() {
"--silent-launch",
"--no-startup-window",
"no-sandbox",
"headless",});
var chromeDriverService = ChromeDriverService.CreateDefaultService();
chromeDriverService.HideCommandPromptWindow = true; // This is to hidden the console.
ChromeDriver driver = new ChromeDriver(chromeDriverService, chromeOptions);
driver.Navigate().GoToUrl(url);
====
If after you run, you are still facing error about no ChromeDriver.exe file, try to add the Selenium.WebDriver.ChromeDriver, WebDriver.ChromeDriver, WebDriver.ChromeDriver.win32, Selenium.Chrome.WebDriver via nuget.

As alternative:
Add 2 libraries via NuGet like below picture.
Try below Code:
String url = "http://www.google.com";
var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments(new List<string>() { "headless" });
var chromeDriverService = ChromeDriverService.CreateDefaultService();
ChromeDriver driver = new ChromeDriver(chromeDriverService, chromeOptions);
driver.Navigate().GoToUrl(url);

What OS you're running? I see on developers.google.com/web/updates/2017/04/headless-chrome that headless won't be available on Windows until Chrome 60.

Below i have given how to set the headless to true for firefox and chrome browsers.
FirefoxOptions ffopt = new FirefoxOptions();
FirefoxOptions option = ffopt.setHeadless(true);
WebDriver driver = new FirefoxDriver(option);
ChromeOptions coptions = new ChromeOptions();
ChromeOptions options = coptions.setHeadless(true);
WebDriver driver = new ChromeDriver(options);

How can I add profile preferences to Chrome for Selenium Grid 2 in C#?

This is how I add profile preferences to Chrome for Local Auto-test runs and TeamCity(CI):
Capabilities = DesiredCapabilities.Chrome();
var chromeOptions = new ChromeOptionsWithPrefs();
chromeOptions.AddUserProfilePreference("download.default_directory", DownloadPath);
chromeOptions.AddUserProfilePreference("intl.accept_languages", "nl");
chromeOptions.AddUserProfilePreference("disable-popup-blocking", "true");
return new ChromeDriver(chromeDriverPath, chromeOptions);
But when I create new 'RemoteWebDriver', I must send it a hub URL and 'Capabilities', this way I send profile preferences to Firefox (to RemoteWebDriver):
var profile = new FirefoxProfile();
Capabilities = DesiredCapabilities.Firefox();
profile.SetPreference("browser.helperApps.alwaysAsk.force", false);
profile.SetPreference("browser.download.useDownloadDir", true);
profile.SetPreference("browser.download.folderList", 2);
profile.SetPreference("browser.download.dir", DownloadPath);
profile.SetPreference("browser.helperApps.neverAsk.saveToDisk",
"application/zip, application/octet-stream");
Capabilities.SetCapability(FirefoxDriver.ProfileCapabilityName, profile.ToBase64String());
return Capabilities;
Can someone help me, I need to do the same thing to Chrome as I did to Firefox. Basically what I need, is that I could change the default path for downloading files.

You'll want to do something like the following:
var chromeOptions = new ChromeOptions();
chromeOptions.AddUserProfilePreference("download.default_directory", DownloadPath);
chromeOptions.AddUserProfilePreference("intl.accept_languages", "nl");
chromeOptions.AddUserProfilePreference("disable-popup-blocking", "true");
IWebDriver driver = new RemoteWebDriver(new Uri("http://path/to/selenium/server"), chromeOptions.ToCapabilities());

C# Selenium WebDriver FireFox Profile - using proxy with Authentication

When you set proxy server parameter in the code below if your proxy server requires authentication then FireFox will bring Authentication dialog and basically you can't fill it in automatically.
So is there is anyway to set USERNAME and PASSWORD ?
FirefoxProfile profile = new FirefoxProfile();
String PROXY = "192.168.1.100:8080";
OpenQA.Selenium.Proxy proxy = new OpenQA.Selenium.Proxy();
proxy.HttpProxy=PROXY;
proxy.FtpProxy=PROXY;
proxy.SslProxy=PROXY;
profile.SetProxyPreferences(proxy);
FirefoxDriver driver = new FirefoxDriver(profile);
If you try to format proxy string to something like that http://username:pass#192.168.1.1:8080
You get error that string is invalid. So I wonder there is must be a way of achieving this.
Any help would be appreciated.

String PROXY = "http://login:pass#proxy:port";
ChromeOptions options = new ChromeOptions();
options.AddArguments("user-data-dir=path/in/your/system");
Proxy proxy = new Proxy();
proxy.HttpProxy = PROXY;
proxy.SslProxy = PROXY;
proxy.FtpProxy = PROXY;
options.Proxy = proxy;
// Initialize the Chrome Driver
using (var driver = new ChromeDriver(options))

You can write own firefox extension for proxy, and launch from selenium. You need write 2 files and pack it.
background.js
var proxy_host = "YOUR_PROXY_HOST";
var proxy_port = YOUR_PROXY_PORT;
var config = {
mode: "fixed_servers",
rules: {
singleProxy: {
scheme: "http",
host: proxy_host,
port: proxy_port
},
bypassList: []
}
};
function proxyRequest(request_data) {
return {
type: "http",
host: proxy_host,
port: proxy_port
};
}
browser.proxy.settings.set({value: config, scope: "regular"}, function() {;});
function callbackFn(details) {
return {
authCredentials: {
username: "YOUR_USERNAME",
password: "YOUR_PASSWORD"
}
};
}
browser.webRequest.onAuthRequired.addListener(
callbackFn,
{urls: ["<all_urls>"]},
['blocking']
);
browser.proxy.onRequest.addListener(proxyRequest, {urls: ["<all_urls>"]});
manifest.json
{
"name": "My Firefox Proxy",
"version": "1.0.0b",
"manifest_version": 2,
"permissions": [
"browsingData",
"proxy",
"storage",
"tabs",
"webRequest",
"webRequestBlocking",
"downloads",
"notifications",
"<all_urls>"
],
"background": {
"scripts": ["background.js"]
},
"browser_specific_settings": {
"gecko": {
"id": "myproxy#example.org"
}
}
}
Next you need packed this files to zip archive in DEFLATED mode with .xpi at end like my_proxy_extension.xpi.
You have two choices:
Sign your extension Here you can read more about verify extension and extension's structure
OR
Run unsigned. For this step:
Open firefox flags at about:config and set options xpinstall.signatures.required to false
OR
Update firefox profile in:
Windows: C:\Program Files\Mozilla Firefox\defaults\pref\channel-prefs.js
Linux: /etc/firefox/syspref.js
Add next line to end of file:
pref("xpinstall.signatures.required",false);
After this steps run selenium and install this extension:
FirefoxProfile profile = new FirefoxProfile();
profile.addExtension(new File("path/to/my_proxy_extension.xpi"));
driver = new FirefoxDriver(profile);

What you can do is to create a profile and save the authentication data in it.
If your profile is called "webdriver" you can select it from your code in the initialization:
ProfilesIni allProfiles = new ProfilesIni();
FirefoxProfile profile = allProfiles.getProfile("WebDriver");
profile.setPreferences("foo.bar",23);
WebDriver driver = new FirefoxDriver(profile);

Did it with MS UI Automation without AutoIt:
public void AuthInProxyWindow (string login, string pass)
{
var proxyWindow = AutomationElement.RootElement
.FindFirst(TreeScope.Subtree,
new PropertyCondition(AutomationElement.ClassNameProperty, "MozillaDialogClass"));
var edits = proxyWindow.FindAll(TreeScope.Subtree,
new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Edit));
var unamePoint = edits[1].GetClickablePoint();
Mouse.MoveTo(new Point((int) unamePoint.X, (int) unamePoint.Y));
Mouse.Click(MouseButton.Left);
SendKeys.SendWait(login);
var pwdPoint = edits[2].GetClickablePoint();
Mouse.MoveTo(new Point((int) pwdPoint.X, (int) pwdPoint.Y));
Mouse.Click(MouseButton.Left);
SendKeys.SendWait(pass);
Keyboard.Press(Key.Return);
Logger.Debug("Authefication in Firefox completed succesfully");
}
Mouse moves by Microsoft.TestApi

To stop firefox from giving you the auth pop up simple make sure you set your proxy URL to include the auth details in the setup stage as below:
var myProxy = user + ":" + pass + "#" + proxyIP + ":" + proxyPORT;
options.SetPreference("network.proxy.type", 1);
options.SetPreference("network.proxy.http", myProxy);
options.SetPreference("network.proxy.http_port", proxyPORT);
options.SetPreference("general.useragent.override", useragent);
driver = new FirefoxDriver(driverService, options);