I had asked a question some time back and I am still facing random errors trying to parse a webpage.
The scenario is that the system goes to https://www.sprouts.com/store/tx/plano/plano/ clicks on "VIEW THIS STORE’S SPECIALS" navigates to https://shop.sprouts.com/shop/flyer and extracts the stores specials. Currently the code below works 10% or 20% of the times only as it is unable to find the button to click and navigate to the next page.
What am i doing wrong?
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.IO;
using System.Linq;
using System.Reflection;
using SeleniumExtras.WaitHelpers;
[TestClass]
public class UnitTest1
{
ChromeDriver driver;
WebDriverWait webDriverWait;
[TestInitialize]
public void Startup()
{
var chromeOptions = new ChromeOptions();
//chromeOptions.AddArguments("headless");
chromeOptions.AddArguments("--proxy-server='direct://'");
chromeOptions.AddArguments("--proxy-bypass-list=*");
//chromeOptions.AddUserProfilePreference("disable-popup-blocking", "true");
//chromeOptions.AddArguments("--disable-extensions");
chromeOptions.AddArguments("--start-maximized");
var path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
ChromeDriverService chromeDriverService = ChromeDriverService.CreateDefaultService(path);
driver = new ChromeDriver(chromeDriverService, chromeOptions);
webDriverWait = new WebDriverWait(driver, TimeSpan.FromSeconds(30));
}
[TestCleanup]
public void CleanUp()
{
driver.Quit();
}
[TestMethod]
public void GetSproutsWeeklyAdDetails()
{
try
{ driver.Navigate().GoToUrl("http://www.sprouts.com/store/tx/plano/plano/");
}
catch (TimeoutException timeoutException)
{
driver.Navigate().Refresh();
}
webDriverWait.Until(driver1 => ((IJavaScriptExecutor)driver).ExecuteScript("return document.readyState").Equals("complete"));
**var elements1 = webDriverWait.Until(ExpectedConditions.VisibilityOfAllElementsLocatedBy(
By.XPath("//div[#class='cell small-6 divider']/button")));
elements1.First().Click();**
<= the system is unable to find the button 80% of the times
webDriverWait.Until(driver1 => ((IJavaScriptExecutor)driver).ExecuteScript("return document.readyState").Equals("complete"));
var elements2 = webDriverWait.Until(ExpectedConditions.VisibilityOfAllElementsLocatedBy(
By.XPath("//li[#class='cell-wrapper' and #ng-repeat='item in items track by $index']")));
//More code below
}
}
}
source code of area where button that is not clickable:
<div class="cell small-6 divider">
<button onclick="viewStoreFlyer(event, 101)">
<img src="https://www.sprouts.com/wp-content/themes/FoundationPress/dist/assets/images/weekly-specials-stores-icon.svg" width="32" alt="" role="presentation">
<br>
View this store’s specials
</button>
</div>
In my perpesctive the code is almost good, except two things. You are waiting for all elements to be visible and then immediately want to click on the first of them. Firstly on the website is only one element with following xPath, so there is no need to locate a list. The second is - when element is visible, it does not means it is clickable, so the better way to deal with this would be:
webDriverWait.Until(ExpectedConditions.ElementToBeClickable(By.XPath("//div[#class='cell small-6 divider']/button")));
PS: the answer #DebanjanB has provided is very good in that case, since you just wanted to extract the text withoun interacting with element. In our case we have to wait until elelment will be ready to recieve a click, that's why ElementToBeClickable is better in this case.
I was able to get this working by adding a thread sleep just before the button was clicked.
var elements1 = webDriverWait.Until(ExpectedConditions.ElementToBeClickable(
By.XPath("//div[#class='cell small-6 divider']/button")));
Thread.Sleep(2000);
elements1.Click();
Related
Hello everyone I need some help. There is URL: http://lisans.epdk.org.tr/epvys-web/faces/pages/lisans/petrolBayilik/petrolBayilikOzetSorgula.xhtml. As you can see in screenshot I need to click checkbox Captcha.
https://i.stack.imgur.com/xjXaA.png
Here is my code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
namespace AkaryakitSelenium
{
class Program
{
private static string AkaryakitLink = "http://lisans.epdk.org.tr/epvys-web/faces/pages/lisans/petrolBayilik/petrolBayilikOzetSorgula.xhtml";
static void Main(string[] args)
{
IWebDriver driver = new ChromeDriver();
IJavaScriptExecutor js = driver as IJavaScriptExecutor;
driver.Navigate().GoToUrl(AkaryakitLink);
var kategoriCol = driver.FindElements(By.CssSelector(".ui-selectonemenu-trigger.ui-state-default.ui-corner-right"));
var x = kategoriCol[3];
x.Click();
var deneme = driver.FindElement(By.Id("petrolBayilikOzetSorguKriterleriForm:j_idt52_1"));
deneme.Click();
var check = driver.FindElement(By.Id("recaptcha-anchor"));
check.Click();
}
}
}
And lastly this error that I am facing:
"OpenQA.Selenium.NoSuchElementException: 'no such element: Unable to
locate element: {"method":"css
selector","selector":"#recaptcha-anchor"}"
Thank you for your help.
The element you are looking for is inside an iframe :
//iframe[#title='reCAPTCHA']
first you need to switch to iframe like this :
new WebDriverWait(driver, TimeSpan.FromSeconds(3)).Until(ExpectedConditions.FrameToBeAvailableAndSwitchToIt(By.XPath("//iframe[#title='reCAPTCHA']")));
then you can perform a click on it :
var check = driver.FindElement(By.Id("recaptcha-anchor"));
check.Click();
PS : captchas are not meant to be automated. since Captcha stands for CAPTCHA stands for the Completely Automated Public Turing test to tell Computers and Humans Apart.
You can not bypass captcha with Selenium.
It is designed to avoid automated access to web pages as described here and in many other places.
I am trying to make a bot that goes through drop-downs and makes selections but it's having trouble finding the element. The code is below.
using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.Extensions;
using OpenQA.Selenium.Support.UI;
namespace selenium_test
{
class Program
{
static void Main(string[] args)
{
IWebDriver driver;
driver = new ChromeDriver("C:\\");
Console.WriteLine("ChromeDriver Initialized");
driver.Url = "https://odyssey.gwinnettcourts.com/Portal/Home/Dashboard/26";
IWebElement typeElement = driver.FindElement(By.XPath(".//option[#id='cboHSSearchBy']"));
var typeSelect = new SelectElement(typeElement);
typeSelect.SelectByValue("JudicialOfficer");
typeElement.Click();
}
}
}
what exactly is the error? I can only assume that it is taking time to find element.
add wait time maybe? also, why using xpath when you have id of controls? id is quicker!
new WebDriverWait(driver, TimeSpan.FromSeconds(10)).Until(SeleniumExtras.WaitHelpers.ExpectedConditions.ElementExists((By.Id("cboHSSearchBy"))));
I am new to selenium, currently exploring on how it works. Started using it for ASP.NET application, i am using C# Selenium driver, IE Driver server ( 32 bit as it's faster than 64 bit)
I navigated to a application there I am clicking a link which should take me to ANOTHER WEBSITE where I have to find a textbox and clear it and enter some text (SendKeys) and then click a button.
When it goes to a another website from main website, It's unable to find the element ( I tried using by.ID and by.Name). I made sure element is available on the webpage. As recommened I used ImplicitlyWait but no luck, tried thread.sleep() no lucK.. Does the test needs to be on the same website which launched initially ?.. Below is my code snippet.. Please help me..
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using OpenQA.Selenium;
using OpenQA.Selenium.IE;
using OpenQA.Selenium.Support.UI;
using System.Threading;
namespace mySelenium
{
class Program
{
private static void Main(string[] args)
{
IWebDriver driver = new InternetExplorerDriver(#"C:\Users\msbyuva\Downloads\IEDriverServer_Win32_2.45.0\");
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));
driver.Navigate().GoToUrl("http://MyorgName.org/Apps/Sites/2015/login.aspx");
IWebElement userNameTxtBox = driver.FindElement(By.Id("ContentPlaceHolder1_Login1_UserName"));
userNameTxtBox.SendKeys("MSBYUVA");
IWebElement passwordTxtBox = driver.FindElement(By.Id("ContentPlaceHolder1_Login1_Password"));
passwordTxtBox.SendKeys("1234");
var myButton = driver.FindElement(By.Id("ContentPlaceHolder1_Login1_LoginButton"));
myButton.Click();
var EMailLink = driver.FindElement(By.LinkText("Email Testing Link"));
EMailLink .Click();
//Thread.Sleep(10000);
// -- HERE IT IS THROWING ERROR (ANOTHER WEBSITE AFTER CLICKING HYPERLINK)
var toEmailAddress = driver.FindElement(By.Name("ctl00$ContentPlaceHolder1$txtTo"));
toEmailAddress.Clear();
toEmailAddress.SendKeys("msbyuva#gmail.com");
var chkEmailAttachment = driver.FindElement(By.Name("ctl00$ContentPlaceHolder1$ChkAttachMent"));
chkEmailAttachment.Click();
var sendEmailButton = driver.FindElement(By.Id("ctl00_ContentPlaceHolder1_BtnSend"));
sendEmailButton.Click();
}
}
}
You need to switchTo newly opened window and set focus to it in order to send any commands to it
string currentHandle = driver.CurrentWindowHandle;
driver.SwitchTo().Window(driver.WindowHandles.ToList().Last());
After you done with newly opened window do(as need)
driver.Close();
driver.SwitchTo().Window(currentHandle );
More perfectly use PopupWindowFinder class
string currentHandle = driver.CurrentWindowHandle;
PopupWindowFinder popUpWindow = new PopupWindowFinder(driver);
string popupWindowHandle = popUpWindow.Click(EMailLink );
driver.SwitchTo().Window(popupWindowHandle);
//then do the email stuff
var toEmailAddress = driver.FindElement(By.Name("ctl00$ContentPlaceHolder1$txtTo"));
toEmailAddress.Clear();
toEmailAddress.SendKeys("msbyuva#gmail.com");
var chkEmailAttachment = driver.FindElement(By.Name("ctl00$ContentPlaceHolder1$ChkAttachMent"));
chkEmailAttachment.Click();
var sendEmailButton = driver.FindElement(By.Id("ctl00_ContentPlaceHolder1_BtnSend"));
sendEmailButton.Click();
}
}
}
//closing pop up window
driver.Close();
driver.SwitchToWindow(currentHandle);
Due to the lack of proper documentation, I'm not sure if HtmlAgilityPack supports screen capture in C# after it loads the html contents.
So is there a way I can more or less grab a screenshot using (or along with) HtmlAgilityPack so I can have a visual clue as to what happens every time I do page manipulations?
Here is my working code so far:
using HtmlAgilityPack;
using System;
namespace ConsoleApplication4
{
class Program
{
static void Main(string[] args)
{
string urlDemo = "https://htmlagilitypack.codeplex.com/";
HtmlWeb getHtmlWeb = new HtmlWeb();
var doc = getHtmlWeb.Load(urlDemo);
var sentence = doc.DocumentNode.SelectNodes("//p");
int counter = 1;
try
{
foreach (var p in sentence)
{
Console.WriteLine(counter + ". " + p.InnerText);
counter++;
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
Console.ReadLine();
}
}
}
Currently, it scrapes and output all the p of the page in the console but at the same time I want to get a screen grab of the scraped contents but I don't know how and where to begin.
Any help is greatly appreciated. TIA
You can't do this with HTML Agility Pack. Use a different tool such as Selenium WebDriver. Here is how to do it: Take a screenshot with Selenium WebDriver
Could you use Selenium WebDriver instead?
You'll need to add the following NuGet packages to your project first:
Selenium.WebDriver
Selenium.Support
Loading a page and taking a screenshot is then as simple as...
using System;
using System.Drawing.Imaging;
using System.IO;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Support.UI;
namespace SeleniumTest
{
class Program
{
static void Main(string[] args)
{
// Create a web driver that used Firefox
var driver = new FirefoxDriver(
new FirefoxBinary(), new FirefoxProfile(), TimeSpan.FromSeconds(120));
// Load your page
driver.Navigate().GoToUrl("http://google.com");
// Wait until the page has actually loaded
var wait = new WebDriverWait(driver, new TimeSpan(0, 0, 10));
wait.Until(d => d.Title.Contains("Google"));
// Take a screenshot, and saves it to a file (you must have full access rights to the save location).
var myDesktop = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
((ITakesScreenshot)driver).GetScreenshot().SaveAsFile(Path.Combine(myDesktop, "google-screenshot.png"), ImageFormat.Png);
driver.Close();
}
}
}
I'm looking to write a program that opens up an instance of Firefox, namely the default instance of Firefox that contains my various login information, and then simply switch through a few sites. I'm sort of able to do this, using the following code:
System.Diagnostics.Process.Start("firefox.exe", "thisIsMyURL");
However, as I'm sure you're mostly aware, this simply opens a new Firefox process with the given URL as a default site to open to. In order to do what I want, I'd essentially have to open a new Firefox process, do what I need to on the page, kill the process, and repeat this for each page I need. This is less than ideal. So, I'm hoping that someone will know of a way to programmatically control Firefox, through an API or library or something. I've searched on Google, and so far have only found outdated solutions that didn't really solve my problem in the first place.
As always, thanks for your help! Everything you can offer is appreciated.
You can use Selenium WebDriver for C #.
This is a cross-platform API that allows you to control various browsers using APIs for Java, C#, among others.
Attachment of a code C # with Selenium WebDriver tests.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium;
using OpenQA.Selenium.Interactions;
using OpenQA.Selenium.Interactions.Internal;
using OpenQA.Selenium.Support.UI;
using OpenQA.Selenium.IE;
using NUnit.Framework;
using System.Text.RegularExpressions;
namespace sae_test
{ class Program
{ private static string baseURL;
private static StringBuilder verificationErrors;
static void Main(string[] args)
{ // test with firefox
IWebDriver driver = new OpenQA.Selenium.Firefox.FirefoxDriver();
// test with IE
//InternetExplorerOptions options = new InternetExplorerOptions();
//options.IntroduceInstabilityByIgnoringProtectedModeSettings = true;
//IWebDriver driver = new OpenQA.Selenium.IE.InternetExplorerDriver(options);
SetupTest();
driver.Navigate().GoToUrl(baseURL + "Account/Login.aspx");
IWebElement inputTextUser = driver.FindElement(By.Id("MainContent_LoginUser_UserName"));
inputTextUser.Clear();
driver.FindElement(By.Id("MainContent_LoginUser_UserName")).Clear();
driver.FindElement(By.Id("MainContent_LoginUser_UserName")).SendKeys("usuario");
driver.FindElement(By.Id("MainContent_LoginUser_Password")).Clear();
driver.FindElement(By.Id("MainContent_LoginUser_Password")).SendKeys("123");
driver.FindElement(By.Id("MainContent_LoginUser_LoginButton")).Click();
driver.Navigate().GoToUrl(baseURL + "finanzas/consulta.aspx");
// view combo element
IWebElement comboBoxSistema = driver.FindElement(By.Id("MainContent_rcbSistema_Arrow"));
//Then click when menu option is visible
comboBoxSistema.Click();
System.Threading.Thread.Sleep(500);
// container of elements systems combo
IWebElement listaDesplegableComboSistemas = driver.FindElement(By.Id("MainContent_rcbSistema_DropDown"));
listaDesplegableComboSistemas.FindElement(By.XPath("//li[text()='BOMBEO MECANICO']")).Click();
System.Threading.Thread.Sleep(500);
IWebElement comboBoxEquipo = driver.FindElement(By.Id("MainContent_rcbEquipo_Arrow"));
//Then click when menu option is visible
comboBoxEquipo.Click();
System.Threading.Thread.Sleep(500);
// container of elements equipment combo
IWebElement listaDesplegableComboEquipos = driver.FindElement(By.Id("MainContent_rcbEquipo_DropDown"));
listaDesplegableComboEquipos.FindElement(By.XPath("//li[text()='MINI-V']")).Click();
System.Threading.Thread.Sleep(500);
driver.FindElement(By.Id("MainContent_Button1")).Click();
try
{ Assert.AreEqual("BOMBEO MECANICO_22", driver.FindElement(By.XPath("//*[#id=\"MainContent_RejillaRegistroFinanciero_ctl00_ctl04_LabelSistema\"]")).Text);
}
catch (AssertionException e)
{ verificationErrors.Append(e.Message);
}
// verify coin format $1,234,567.89 usd
try
{ Assert.IsTrue(Regex.IsMatch(driver.FindElement(By.XPath("//*[#id=\"MainContent_RejillaRegistroFinanciero_ctl00_ctl04_labelInversionInicial\"]")).Text, "\\$((,)*[0-9]*[0-9]*[0-9]+)+(\\.[0-9]{2})? usd"));
}
catch (AssertionException e)
{ verificationErrors.Append(e.Message);
}
try
{ Assert.IsTrue(Regex.IsMatch(driver.FindElement(By.XPath("//*[#id=\"MainContent_RejillaRegistroFinanciero_ctl00_ctl04_labelCostoOpMantto\"]")).Text, "\\$((,)*[0-9]*[0-9]*[0-9]+)+(\\.[0-9]{2})? usd"));
}
catch (AssertionException e)
{ verificationErrors.Append(e.Message);
}
try
{ Assert.IsTrue(Regex.IsMatch(driver.FindElement(By.XPath("//*[#id=\"MainContent_RejillaRegistroFinanciero_ctl00_ctl04_labelCostoEnergia\"]")).Text, "\\$((,)*[0-9]*[0-9]*[0-9]+)+(\\.[0-9]{2})? usd"));
}
catch (AssertionException e)
{ verificationErrors.Append(e.Message);
}
try
{ Assert.IsTrue(Regex.IsMatch(driver.FindElement(By.XPath("//*[#id=\"MainContent_RejillaRegistroFinanciero_ctl00_ctl04_labelcostoUnitarioEnergia\"]")).Text, "\\$((,)*[0-9]*[0-9]*[0-9]+)+(\\.[0-9]{2})? usd"));
}
catch (AssertionException e)
{ verificationErrors.Append(e.Message);
}
// verify number format 1,234,567.89
try
{ Assert.IsTrue(Regex.IsMatch(driver.FindElement(By.XPath("//*[#id=\"MainContent_RejillaRegistroFinanciero_ctl00_ctl04_labelConsumo\"]")).Text, "((,)*[0-9]*[0-9]*[0-9]+)+(\\.[0-9]{2})?"));
}
catch (AssertionException e)
{ verificationErrors.Append(e.Message);
}
System.Console.WriteLine("errores: " + verificationErrors);
System.Threading.Thread.Sleep(20000);
driver.Quit();
}
public static void SetupTest()
{ baseURL = "http://127.0.0.1:8081/ver.rel.1.2/";
verificationErrors = new StringBuilder();
}
protected static void mouseOver(IWebDriver driver, IWebElement element)
{ Actions builder = new Actions(driver);
builder.MoveToElement(element);
builder.Perform();
}
public static void highlightElement(IWebDriver driver, IWebElement element)
{ for (int i = 0; i < 2; i++)
{ IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
js.ExecuteScript("arguments[0].setAttribute('style', arguments[1]);",
element, "color: yellow; border: 2px solid yellow;");
js.ExecuteScript("arguments[0].setAttribute('style', arguments[1]);",
element, "");
}
}
}
}
download at http://vidadigital.com.mx/publicacion/source/Program.cs
I was reading MSDN magazine a while back, and I saw an article about a project called "Watir" that piqued my interested, because I was doing a lot of automated testing at the time. I looked into it and found there is actually a project called WatiN which is .NET based. Check it out I think it is exactly what you're looking to do.
http://watin.org/
http://watir.com/
First start the process using :
Process.Start("firefox.exe", "www.example.com");
Then to kill it you need to do next:
Process[] processes = Process.GetProcessesByName("firefox.exe");
foreach (Process p in processes)
{
if (p.ProcessName.Equals("firefox.exe", StringComparison.OrdinalIgnoreCase))
{
p.Kill();
}
}