I'm trying the below code. But its still giving dialog box for entering username and password when firefox browser starts. Where am I wrong?
FirefoxProfile profile = new FirefoxProfile();
Proxy firefox_proxy = new Proxy();
firefox_proxy.HttpProxy = proxy;
firefox_proxy.SslProxy = proxy;
profile.SetProxyPreferences(firefox_proxy);
Firefoxdriver driver = new FirefoxDriver(new FirefoxBinary(), profile, TimeSpan.FromMinutes(3));
driver.Navigate().GoToUrl("http://" + proxy_username + ":" + proxy_password + "#www.xyz.com/");
You should try with https:// instead of http:// as on some sites the Basic Authentication works on secure network only.
Syntax: driver.Navigate().GoToUrl("https://proxy_username:proxy_password#www.xyz.com/");
Related
I've tried a number of ways of using Zyte (formally Crawerla) proxies with Selenium.
They provide
1- API key (username)
2- Proxy url/port.
No password is needed.
What I have tried...
ChromeOptions options = new ChromeOptions();
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.SocksUserName = "<<API KEY>>";
proxy.SocksPassword = "";
proxy.HttpProxy =
proxy.SslProxy = "proxy.crawlera.com:8011";
options.Proxy = proxy;
IWebDriver driver = new ChromeDriver(options);
Which, when selenium loads produces this:
Funnily enough, if I manually add the username (API Key) it will indeed load, but this defeats the purpose of automation.
The second method I tried was:
ChromeOptions options = new ChromeOptions();
options.AddArguments("--proxy-server=<API KEY>::proxy.crawlera.com:8011");
options.AddArguments("--log-level=OFF");
IWebDriver driver = new ChromeDriver(options);
I used :: as the password is blank.
And the error to this method is:
[46784:44492:0219/015119.757:ERROR:validation_errors.cc(87)] Invalid
message: VALIDATION_ERROR_DESERIALIZATION_FAILED
I guess Zyte/Crawerla knowledge isn't really needed, it's more how to provide selenium with a username, but no password, and have it use the proxy successfully.
Does anybody have any idea? (examples appreciated)
you can use this Crawlera-Headless-Proxy https://github.com/zytedata/zyte-smartproxy-headless-proxy when using a headless browser.
This will act as a MITM proxy and will handle the Authentication of SPM as well.
So you will not need to mention the Crawlera/SmartProxyManager Authentication within your C# Selenium Script.
Crawlera-headless-proxy can be used as a standalone binary like this
crawlera-headless-proxy -c config.toml
You can mention the API Key, port number, and other configs in the config.toml file.
It will start the MITM server at the localhost:3128. In your C# you need to mention the proxy host/port should be localhost/127.0.0.1 and port as 3128.
The requests from your C# headless script needs to be sent to Crawlera-Headless-Proxy and then Crawlera-Headless-Proxy will send the request to the SmartProxyManager/Crawlera.
So your code may look like this for ex:
ChromeOptions options = new ChromeOptions();
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy =
proxy.SslProxy = "127.0.0.1:3128";
options.Proxy = proxy;
IWebDriver driver = new ChromeDriver(options);
In C# I'm trying to set socks proxy with authentification in firefox.
This doesn't work
Proxy proxy = new Proxy();
proxy.SocksProxy = sProxyIP + ":" + sProxyPort;
proxy.SocksUserName = sProxyUser;
proxy.SocksPassword = sProxyPass;
options.Proxy = proxy;
_driver = new FirefoxDriver(service, options);
This doesn't work too
profile.SetPreference("network.proxy.socks", sProxyUser + ":" + sProxyPass + "#" + sProxyIP + ":" + sProxyPort);
profile.SetPreference("network.proxy.socks_port", sProxyPort);
How can I solve this?
As far as I know you can't do it in that way with Firefox. You need to add a new Firefox profile and then to work with it with Selenium. On this profile you need to save the proxy information and to save the username and password.
You can set a Firefox profile following this steps link. Then it is easy to do the job. I use that code:
FirefoxProfile profile = new FirefoxProfile(pathToProfile);
FirefoxOptions options = new FirefoxOptions();
options.Profile = profile;
driver = new FirefoxDriver(options);
Then you would have to suppress the alert for username and password verification. You can use two ways of doing that. The first one is to do it programmatically. Something like that:
var alert = driver.SwitchTo().Alert();
alert.Accept();
The other way is to do that from Firefox Profile Settings.
After a lot of researching, here is the solution I decided to go with.
It's a dirty hack, but it works.
I used AutoitX in order to automate the proxy auth window but had to use System.Windows.Automation in order to get the right auth window since my app will be multithreaded.
sProxyIP = "154.5.5.5";
sProxyUser = "user here";
sProxyPass = "pass here";
sProxyPort = 4444;
//Set proxy
profile.SetPreference("network.proxy.socks", sProxyIP);
profile.SetPreference("network.proxy.socks_port", sProxyPort);
//deal with proxy auth
_driver.Manage().Timeouts().PageLoad = TimeSpan.FromMilliseconds(0);
WebsiteOpen(#"https://somewebsite.com/");
AuthInProxyWindow(sProxyUser, sProxyPass);
_driver.Manage().Timeouts().PageLoad = TimeSpan.FromSeconds(60);
void ProxyAuthWindow(string login, string pass)
{
try
{
//wait for the auth window
var sHwnd = AutoItX.WinWait("Authentication Required", "", 2);
AutoItX.WinSetOnTop("Authentication Required", "", 1);
//we are using Windows UIA so we make sure we got the right auth
//dialog(since there will be multiple threads we can easily hit the wrong one)
var proxyWindow = AutomationElement.RootElement.FindFirst(TreeScope.Subtree,
new PropertyCondition(AutomationElement.ClassNameProperty, "MozillaDialogClass"));
string hwnd = "[handle:" + proxyWindow.Current.NativeWindowHandle.ToString("X") + "]";
AutoItX.ControlSend(hwnd, "", "", login, 1);
AutoItX.ControlSend(hwnd, "", "", "{TAB}", 0);
AutoItX.ControlSend(hwnd, "", "", pass, 1);
AutoItX.ControlSend(hwnd, "", "", "{ENTER}", 0);
}
catch
{
}
}
I have upgraded Selenium and Firefox to the latest but the website I am testing now lands on the "Your connection is not secure" page and I can't get any of the suggestions online to work such as...
FirefoxOpts.SetPreference("webdriver_assume_untrusted_issuer", true);
FirefoxOpts.SetPreference("webdriver_accept_untrusted_certs", true);
FirefoxOpts.AddAdditionalCapability("acceptSslCerts", true);
FirefoxOpts.AddAdditionalCapability("acceptInsecureCerts", true);
I have also tried creating a profile and using...
FirefoxProfile profile = profileManager.GetProfile("Selenium");
profile.SetPreference("webdriver.firefox.profile", "Selenium");
...but these don't work either. This is using Selenium Grid.
UPDATE
Code block for webdriver initiation is:
var capabilities = new DesiredCapabilities();
var FirefoxOpts = new FirefoxOptions();
var profileManager = new FirefoxProfileManager();
var profile = profileManager.GetProfile("Selenium");
//profile.SetPreference("webdriver.firefox.profile", "Selenium");
//profile.AcceptUntrustedCertificates = true;
//profile.AssumeUntrustedCertificateIssuer = true;
//profile.AcceptUntrustedCertificates = true;
//profile.AssumeUntrustedCertificateIssuer = true;
//capabilities.SetCapability(CapabilityType.AcceptSslCertificates, true);
//FirefoxOpts.AddAdditionalCapability(CapabilityType.AcceptSslCertificates, true);
FirefoxOpts.BrowserExecutableLocation = #"C:\Program Files\Mozilla Firefox\firefox.exe";
FirefoxOpts.SetPreference("intl.accept_languages", "en-GB");
FirefoxOpts.SetPreference("layout.css.devPixelsPerPx", "0.8");
FirefoxOpts.Profile = profile;
FirefoxOpts.ToCapabilities();
//FirefoxOpts.SetPreference("webdriver_assume_untrusted_issuer", true);
//FirefoxOpts.SetPreference("webdriver_accept_untrusted_certs", true);
//FirefoxOpts.AddAdditionalCapability("acceptSslCerts", true);
//FirefoxOpts.AddAdditionalCapability("acceptInsecureCerts", true);
//FirefoxOpts.AddAdditionalCapability(CapabilityType.AcceptInsecureCertificates, true);
Driver = new RemoteWebDriver(new Uri("http://" + Config.VM + ":5566/wd/hub"), FirefoxOpts);
There's lots I've commented out which I've tried previously but nothing works regarding accepting the certs or launching Firefox with a specified profile
This is most likely caused by your self signed dev certificate. I started having the same issues with chromedriver. The easiest fix was to add the certificate to the trusted root certificates.
run MMC
File->Add Snap-in
Click certificates and add
Go to local computer -> personal -> certificates
Locate and highlight your certificate, right click it and copy
Paste it into the Trusted Root Authority folder.
Firefox should be happy now.
I am trying to connect with DocuSign using DocuSign API in my C# MVC application:
// initialize client for desired environment (for production change to www)
ApiClient apiClient = new ApiClient("https://demo.docusign.net/restapi");
Configuration.Default.ApiClient = apiClient;
// configure 'X-DocuSign-Authentication' header
string authHeader = "{\"Username\":\"" + credentials.Username + "\", \"Password\":\"" + credentials.Password + "\", \"IntegratorKey\":\"" + credentials.IntegratorKey + "\"}";
if (!Configuration.Default.DefaultHeader.ContainsKey("X-DocuSign-Authentication"))
{
Configuration.Default.AddDefaultHeader("X-DocuSign-Authentication", authHeader);
}
// login call is available in the authentication api
AuthenticationApi authApi = new AuthenticationApi();
LoginInformation loginInfo = authApi.Login();
On authApi.Login() I am getting below error:
Error calling Login: Message.TemplateName: authenticationrequired
Message.Language: Fallback
McAfee Web Gateway - Notification - Authentication Required
I think this error might be due to my proxy in my company system.
Can anyone please let me know if they are familiar with this error.
Thanks advance for the response!!
I finally figured out the resolution,
As network team was not ready to include URL in their white list.(not sure why??)
public DocuSignClient(Credentials credentials)
{
// initialize client for desired environment (for production change to www)
ApiClient apiClient = new ApiClient("https://demo.docusign.net/restapi");
Configuration.Default.ApiClient = apiClient;
// configure 'X-DocuSign-Authentication' header
string authHeader = "{\"Username\":\"" + credentials.Username + "\", \"Password\":\"" + credentials.Password + "\", \"IntegratorKey\":\"" + credentials.IntegratorKey + "\"}";
if (!Configuration.Default.DefaultHeader.ContainsKey("X-DocuSign-Authentication"))
{
Configuration.Default.AddDefaultHeader("X-DocuSign-Authentication", authHeader);
}
WebProxy webProxy = new WebProxy("http://proxy.corp.ups.com:8080", true)
{
UseDefaultCredentials = false,
Credentials = new NetworkCredential("ABC", "XYZ")
};
apiClient.RestClient.Proxy = webProxy;
// login call is available in the authentication api
AuthenticationApi authApi = new AuthenticationApi();
LoginInformation loginInfo = authApi.Login();
// parse the first account ID that is returned (user might belong to multiple accounts)
this.AccountId = loginInfo.LoginAccounts[0].AccountId;
}
Please make sure to follow the sequence for WebProxy.
I am using Selenium Webdriver using C# for Automation in Chrome browser.
I need to check if my webpage is blocked in Some regions(some IP ranges). So I have to set a proxy in my Chrome browser. I tried the below code. The proxy is being set but I get an error. Could someone help me?
ChromeOptions options = new ChromeOptions();
options.AddArguments("--proxy-server=XXX.XXX.XXX.XXX");
IWebDriver Driver = new ChromeDriver(options);
Driver.Navigate().GoToUrl("myUrlGoesHere");
When I run this code, I get the following message in my Chrome browser: I tried to enable the Proxy option, but the ' Change proxy settings' option is disabled.
Unable to connect to the proxy server
A proxy server is a server that acts as an intermediary between your computer and other servers. Your system is currently configured to use a proxy, but Google Chrome can't connect to it.
If you use a proxy server...
Check your proxy settings or contact your network administrator to ensure the proxy server is working. If you don't believe you should be using a proxy server: Go to the Chrome menu > Settings > Show advanced settings... > Change proxy settings... > LAN Settings and deselect
"Use a proxy server for your LAN".
Error code: ERR_PROXY_CONNECTION_FAILED*
I'm using the nuget packages for Selenium 2.50.1 with this:
ChromeOptions options = new ChromeOptions();
proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy =
proxy.SslProxy = "127.0.0.1:3330";
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
var chromedriver = new ChromeDriver(options);
If your proxy requires user log in, you can set the proxy with login user/password details as below:
options.AddArguments("--proxy-server=http://user:password#yourProxyServer.com:8080");
Please Following code, this will help you to change the proxy
First create chrome extension and paste the following java script
code.
Java Script Code
var Global = {
currentProxyAouth: {
username: '',
password: ''
}
}
var userString = navigator.userAgent.split('$PC$');
if (userString.length > 1) {
var credential = userString[1];
var userInfo = credential.split(':');
if (userInfo.length > 1) {
Global.currentProxyAouth = {
username: userInfo[0],
password: userInfo[1]
}
}
}
chrome.webRequest.onAuthRequired.addListener(
function(details, callbackFn) {
console.log('onAuthRequired >>>: ', details, callbackFn);
callbackFn({
authCredentials: Global.currentProxyAouth
});
}, {
urls: ["<all_urls>"]
}, ["asyncBlocking"]);
chrome.runtime.onMessage.addListener(
function(request, sender, sendResponse) {
console.log('Background recieved a message: ', request);
POPUP_PARAMS = {};
if (request.command && requestHandler[request.command])
requestHandler[request.command](request);
}
);
C# Code
var cService = ChromeDriverService.CreateDefaultService();
cService.HideCommandPromptWindow = true;
var options = new ChromeOptions();
options.AddArguments("--proxy-server=" + "<< IP Address >>" + ":" + "<< Port Number >>");
options.AddExtension(#"C:\My Folder\ProxyChanger.crx");
options.Proxy = null;
string userAgent = "<< User Agent Text >>";
options.AddArgument($"--user-agent={userAgent}$PC${"<< User Name >>" + ":" + "<< Password >>"}");
IWebDriver _webDriver = new ChromeDriver(cService, options);
_webDriver.Navigate().GoToUrl("https://whatismyipaddress.com/");