I'm trying to use Selenium 4 to log requests during manual usage of Chrome browser.
The issue is that request interception stops after around 40 seconds of usage (approximately).
I've tried to change commandTimeout but it didn't change anything.
Also I've tried to look into chromedriver logs but I didn't find anithing there.
Here's my code:
static void Main(string[] args)
{
// Enable chromedriver logging
var service = ChromeDriverService.CreateDefaultService();
service.LogPath = AppDomain.CurrentDomain.BaseDirectory + "chromedriver.log";
service.EnableVerboseLogging = true;
var options = new ChromeOptions();
var webDriver = new ChromeDriver(service, options);
var devToolsSession = webDriver.CreateDevToolsSession();
devToolsSession.Network.Enable(new EnableCommandSettings());
EventHandler<RequestInterceptedEventArgs> requestIntercepted = (sender, e) =>
{
Console.WriteLine(e.Request.Url);
};
RequestPattern requestPattern = new RequestPattern();
requestPattern.InterceptionStage = InterceptionStage.Request;
requestPattern.ResourceType = ResourceType.Image;
var setRequestInterceptionCommandSettings = new SetRequestInterceptionCommandSettings();
setRequestInterceptionCommandSettings.Patterns = new RequestPattern[] { requestPattern };
devToolsSession.Network.SetRequestInterception(setRequestInterceptionCommandSettings);
devToolsSession.Network.RequestIntercepted += requestIntercepted;
while (true)
{
webDriver.Url = "https://translate.google.com/";
Thread.Sleep(5000);
webDriver.Navigate().Refresh();
}
}
As of Selenium 4 beta-1, you can use the Fetch API and retrieve the same resluts (SetRequestIntercepted is deprecated in ChromeTools as late as December 29, 2020).
In either case, the key point for both RequestIntercepted (prior to deprecation) and Fetch to be able to continue after intercepting the request is to capture the RequestId.
With Fetch this can be done in ContinueRequest():
fetch.ContinueRequest(new Fetch.ContinueRequestCommandSettings()
{
RequestId = e.RequestId
});
Below is your code updated to use Fetch which allowed the query to run for 2+ minutes and 5+ minutes before I manually stopped it:
(Note: ensure all of your DevTools as using the same version (i.e. 89 in my case) or you will get errors):
using V89 = OpenQA.Selenium.DevTools.V89;
using V89Net = OpenQA.Selenium.DevTools.V89.Network;
using OpenQA.Selenium.DevTools.V89.Log;
var service = ChromeDriverService.CreateDefaultService();
service.LogPath = AppDomain.CurrentDomain.BaseDirectory + "chromedriver.log";
service.EnableVerboseLogging = true;
var options = new ChromeOptions();
new DriverManager().SetUpDriver(new ChromeConfig(), VersionResolveStrategy.MatchingBrowser);
var driver = new ChromeDriver(service, options);
IDevTools devTools = driver as IDevTools;
var devToolsSession = devTools.GetDevToolsSession();
var fetch = devToolsSession.GetVersionSpecificDomains<V89.DevToolsSessionDomains>()
.Fetch;
var enableCommandSettings = new V89.Fetch.EnableCommandSettings();
var requestPattern = new V89.Fetch.RequestPattern();
requestPattern.RequestStage = V89.Fetch.RequestStage.Response;
requestPattern.ResourceType = V89Net.ResourceType.Document;
enableCommandSettings.Patterns = new V89.Fetch.RequestPattern[] { requestPattern };
fetch.Enable(enableCommandSettings);
void RequestIntercepted(object sender, V89.Fetch.RequestPausedEventArgs e)
{
e.Request.Url.Dump();
fetch.ContinueRequest(new V89.Fetch.ContinueRequestCommandSettings()
{
RequestId = e.RequestId
});
}
fetch.RequestPaused += RequestIntercepted;
while (true)
{
driver.Url = "https://translate.google.com/";
Thread.Sleep(5000);
driver.Navigate().Refresh();
}
Screenshot (From left-to-right): LinqPad output Console.Write.Url equivalent, Above code in LINQPad, and view of Chrome log file)
Related
I would like to use a c# language server client to connect and communicate with a language server. The language server client I found is here (https://github.com/OmniSharp/csharp-language-server-protocol). In the examples I found code to create a client.
protected virtual ILanguageClient CreateClient(Action<LanguageClientOptions> clientOptionsAction = null)
{
_client = LanguageClient.Create(
options => {
var (reader, writer) = SetupServer();
options
.WithInput(reader)
.WithOutput(writer)
.WithLoggerFactory(TestOptions.ClientLoggerFactory)
.WithAssemblies(TestOptions.Assemblies)
.WithAssemblies(typeof(LanguageProtocolTestBase).Assembly, GetType().Assembly)
.ConfigureLogging(x => x.SetMinimumLevel(LogLevel.Trace))
.WithInputScheduler(options.InputScheduler)
.WithOutputScheduler(options.OutputScheduler)
.WithDefaultScheduler(options.DefaultScheduler)
.Services
.AddTransient(typeof(IPipelineBehavior<,>), typeof(SettlePipeline<,>))
.AddSingleton(Events as IRequestSettler);
clientOptionsAction?.Invoke(options);
}
);
Disposable.Add(_client);
return _client;
}
But it is not clear to me how I can establish a connection to a language server (that is, to a language-server.exe). I found this example here (https://gist.github.com/tintoy/a2ec9424d17fe9ef17db0621479a7b43) but I think it's never has been working or with a very old version of the omnisharp language server client. But in general this is what I want to do. May be someone can give me a working example that can do the same thing like the example just mentioned.
Best regards,
Basti
I found the solution myself. With this example you can request code completion.
ILanguageClient _client;
ProcessStartInfo info = new ProcessStartInfo();
var programPath = #"path\to\language-server.exe";
info.FileName = programPath;
info.WorkingDirectory = Path.GetDirectoryName(programPath);
info.RedirectStandardInput = true;
info.RedirectStandardOutput = true;
info.UseShellExecute = false;
Process process = new Process
{
StartInfo = info
};
process.Start();
_client = LanguageClient.Create(
options =>
{
.WithInput(process.StandardOutput.BaseStream)
.WithOutput(process.StandardInput.BaseStream)
.WithCapability(
new CompletionCapability
{
CompletionItem = new CompletionItemCapabilityOptions
{
DeprecatedSupport = true,
DocumentationFormat = new Container<MarkupKind>(MarkupKind.Markdown, MarkupKind.PlainText),
PreselectSupport = true,
SnippetSupport = true,
TagSupport = new CompletionItemTagSupportCapabilityOptions
{
ValueSet = new[] { CompletionItemTag.Deprecated }
},
CommitCharactersSupport = true
}
}
);
}
);
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
await _client.Initialize(cancellationTokenSource.Token);
try
{
var actualCompletions = await _client.TextDocument.RequestCompletion(
new CompletionParams
{
TextDocument = #"path\to\someScript.py",
Position = (23, 11),
}, cancellationTokenSource.Token
);
System.Threading.Thread.Sleep(1000);
var items = actualCompletions.Items;
Console.WriteLine("items.Count: {0}", items?.Count());
Console.WriteLine("actualCompletions: {0}", string.Join(",", items?.Select(p => p.Label)));
}
catch (Exception ex)
{
Console.WriteLine("Exception thrown: {0}", ex.Message);
}
I have problem getting output from html2canvas JS library within Chrome automated by selenium.
Response is always null, but I can see in the Chrome console that code executed successfully and screenshot has been encoded
public byte[] TakeScreenshot(string fileName)
{
this.logger.Debug($"Taking screenshot");
var seleniumDownloadPath = this.seleniumEngine.GetDownloadPath();
IJavaScriptExecutor js = Driver as IJavaScriptExecutor;
var html2canvasJs = System.IO.File.ReadAllText(Path.Combine(Path.GetDirectoryName(new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath), this.seleniumEngine.GetHtml2CanvasPath()));
var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(60));
var response = new object { };
js.ExecuteScript(html2canvasJs);
string generateScreenshotJS =
var canvasImgContentDecoded;
#"function genScreenshot () {
html2canvas(document.body).then(function(canvas) {
window.canvasImgContentDecoded = canvas.toDataURL(""image/png"");
console.log(window.canvasImgContentDecoded);
return window.canvasImgContentDecoded;
});
}
genScreenshot();";
response = js.ExecuteScript(generateScreenshotJS);
}
I also tried solution from this Here but the behavior was unstable (e.g. when running realtime i got error of nulls, but if running using breakpoints, I got result sometime)
Finally found the solution after finding out that execution of script takes more time and variable was null. So added wait function to retrieve once its filled - window.canvasImgContentDecoded
public byte[] TakeScreenshot(string fileName)
{
this.logger.Debug($"Taking screenshot");
var seleniumDownloadPath = this.seleniumEngine.GetDownloadPath();
IJavaScriptExecutor js = Driver as IJavaScriptExecutor;
var html2canvasJs = System.IO.File.ReadAllText(Path.Combine(Path.GetDirectoryName(new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath), this.seleniumEngine.GetHtml2CanvasPath()));
var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(60));
var response = new object { };
js.ExecuteScript(html2canvasJs);
string generateScreenshotJS =
#"
var canvasImgContentDecoded;
function genScreenshot () {
html2canvas(document.body).then(function(canvas) {
window.canvasImgContentDecoded = canvas.toDataURL(""image/png"");
console.log(window.canvasImgContentDecoded);
});
}
genScreenshot();";
response = js.ExecuteScript(generateScreenshotJS);
string getSCreenShot = "return window.canvasImgContentDecoded;";
var encodedPngContent = new object { };
/*ADDED WAIT FUNCTION*/
wait.Until(
wd =>
{
encodedPngContent = js.ExecuteScript(getSCreenShot);
if (encodedPngContent != null)
{
return true;
}
return false;
});
string pngContent = encodedPngContent.ToString();
pngContent = pngContent.Replace("data:image/png;base64,", string.Empty);
string fileSavePath = this.seleniumEngine.GetDownloadPath() + fileName;
File.WriteAllBytes(fileSavePath, Convert.FromBase64String(pngContent));
byte[] fileByte = System.IO.File.ReadAllBytes(fileSavePath);
File.Delete(fileSavePath);
return fileByte;
}
I'm trying to extract the JWT within requests we have in our system.
When I'm running locally it works great, the code intercepting all requests...
When code runs by Azure Pipeline, no requests were intercepted, empty.
private static void GetJwtFromRequest(WebDriver driver)
{
string jwt = "";
FetchAdapter fetchAdapter = null;
if (!Requests.Any())
Requests = new List<Request>();
fetchAdapter = GetBreezeNetworkRequests(driver);
var isRequestExist = driver.GetWebDriverWait(30)
.Until(x =>
{
var temp = Requests;
return temp.Any(x => x.Headers.Any(y => y.Key == "Authorization"));
});
if (isRequestExist)
{
Request wantedRequest = Requests.First(x => x.Headers.Any(y => y.Key == "Authorization"));
jwt = wantedRequest.Headers["Authorization"].Replace("Bearer ", "");
}
UserFactory.CurrentUser.Jwt = jwt;
}
Network interception by Selenium:
public static List<Request> Requests = new List<Request>();
private static FetchAdapter GetBreezeNetworkRequests(WebDriver driver)
{
IDevTools devTools = driver.OriginDriver as IDevTools;
DevToolsSession session = devTools.GetDevToolsSession();
FetchAdapter fetchAdapter = session.GetVersionSpecificDomains<OpenQA.Selenium.DevTools.V95.DevToolsSessionDomains>().Fetch;
var enableCommandSettings = new OpenQA.Selenium.DevTools.V95.Fetch.EnableCommandSettings();
var requestPattern = new OpenQA.Selenium.DevTools.V95.Fetch.RequestPattern();
requestPattern.RequestStage = RequestStage.Request;
requestPattern.ResourceType = ResourceType.XHR;
enableCommandSettings.Patterns = new OpenQA.Selenium.DevTools.V95.Fetch.RequestPattern[] { requestPattern };
fetchAdapter.Enable(enableCommandSettings);
EventHandler<OpenQA.Selenium.DevTools.V95.Fetch.RequestPausedEventArgs> requestIntercepted = (sender, e) =>
{
Requests.Add(e.Request);
fetchAdapter.ContinueRequest(new OpenQA.Selenium.DevTools.V95.Fetch.ContinueRequestCommandSettings()
{
RequestId = e.RequestId
});
};
fetchAdapter.RequestPaused += requestIntercepted;
return fetchAdapter;
}
.netcore 3.1
chrome/edge V95
Selenium 4.0.1
Any insights?
Thanks.
I'm using the Saucelabs Selenium implementation to automate my testing across multiple devices and platforms.
Using the demo code from SauceLabs (below) doesn't work behind a proxy.
I've tried adding the proxy details to the DesiredCapibilities but this doesn't seem to do anything
[TestMethod]
public void TestSauceLabs()
{
DesiredCapabilities caps = new DesiredCapabilities();
caps.SetCapability("browserName", "Safari");
caps.SetCapability("platform", "macOS 10.13");
caps.SetCapability("version", "11.1");
caps.SetCapability("username", _sauceUserName);
caps.SetCapability("accessKey", _sauceAccessKey);
caps.SetCapability("name", _name);
var tags = new List<string> {"demoTest", "sauceDemo"};
caps.SetCapability("tags", tags);
caps.SetCapability("maxDuration", 3600);
caps.SetCapability("commandTimeout", 600);
caps.SetCapability("idleTimeout", 1000);
caps.SetCapability("build", "SauceDemo");
/****************************************
* Edited demo code here
* Added proxy config to DesiredCapabilities **
*/
var proxy = new Proxy
{
IsAutoDetect = false,
HttpProxy = $"{_proxyScheme}://{_proxyHost}:{_proxyPort}",
SslProxy = $"{_proxyScheme}://{_proxyHost}:{_proxyPort}",
FtpProxy = $"{_proxyScheme}://{_proxyHost}:{_proxyPort}"
};
caps.SetCapability(CapabilityType.Proxy, proxy);
/*
*****************************************/
var uri = new Uri("https://ondemand.eu-central-1.saucelabs.com/wd/hub");
_driver = new RemoteWebDriver(uri,
caps, TimeSpan.FromSeconds(600));
_javascriptExecutor = ((IJavaScriptExecutor) _driver);
_javascriptExecutor.ExecuteScript("sauce:context=Open SauceDemo.com");
_driver.Navigate().GoToUrl(_url);
_javascriptExecutor.ExecuteScript("sauce:context=Sleep for 10000ms");
Thread.Sleep(10000);
Assert.IsTrue(true);
var passed = true;
_javascriptExecutor.ExecuteScript("sauce:job-result=" + (passed ? "passed" : "failed"));
_driver?.Quit();
}
Found the solution was to use the HttpCommandExecutor when using the RemoteWebDriver behind a proxy.
Here is my example code:
[TestMethod]
public void TestSauceLabs_Chrome()
{
var remoteOptions = new Dictionary<string, object>
{
{ "username", _sauceUserName },
{ "accessKey", _sauceAccessKey },
{ "name", _name },
{ "maxDuration", 3600 },
{ "commandTimeout", 600 },
{ "idleTimeout", 1000 }
};
var options = new ChromeOptions()
{
PlatformName = "Windows 10",
BrowserVersion = "latest"
};
//Remote options need to be global
options.AddAdditionalCapability("sauce:options", remoteOptions, true);
var caps = options.ToCapabilities();
/*
Using the HttpCommandExecutor persists the proxy details
and allows you to pass in credentials if required
*/
var executor = new HttpCommandExecutor(
new Uri("https://ondemand.eu-central-1.saucelabs.com/wd/hub"),
TimeSpan.FromSeconds(600))
{
Proxy = GenerateProxy()
};
_driver = new RemoteWebDriver(executor, caps);
_javascriptExecutor = ((IJavaScriptExecutor)_driver);
_javascriptExecutor.ExecuteScript($"sauce:context=Open {_url}");
_driver.Navigate().GoToUrl(_url);
_javascriptExecutor.ExecuteScript("sauce:context=Sleep for 10000ms");
Thread.Sleep(10000);
Assert.IsTrue(true);
var passed = true;
_javascriptExecutor.ExecuteScript("sauce:job-result=" + (passed ? "passed" : "failed"));
_driver?.Quit();
}
public WebProxy GenerateProxy()
{
var proxy = new WebProxy
{
Address = new Uri($"{_proxyScheme}://{_proxyHost}:{_proxyPort}"),
BypassProxyOnLocal = false,
UseDefaultCredentials = _networkCredential != null,
Credentials = _networkCredential
};
return proxy;
}
I'm trying to use UI automation with ChromiumWebBrowser, so I managed to make it work by passing force-renderer-accessibility command line option to settings. It got available on inspect.exe tool but I still couldn't find it by code. Could anyone just give me a code sample?
Here's how I initialize the ChromiumWebBrowser:
void InitializeChromiumWebBrowser()
{
var settings = new CefSettings()
{
CefCommandLineArgs = {
new KeyValuePair<string, string>("force-renderer-accessibility", "true")
},
MultiThreadedMessageLoop = false
};
Cef.Initialize(settings);
m_chromeBrowser = new ChromiumWebBrowser("http://127.0.0.1/calc.html");
m_chromeBrowser.Name = "chromiumWebBrowser";
var t = new Timer { Interval = 5 };
t.Start();
t.Tick += (s, e) => BeginInvoke((Action)(() => Cef.DoMessageLoopWork()));
m_chromeBrowser.LoadingStateChanged += M_chromeBrowser_LoadingStateChanged;
browser_tabPage.Controls.Add(m_chromeBrowser);
}
I'm trying to manipulate them with UI Automation, but I couldn't find even the top window:
using (var proc = Process.GetCurrentProcess())
{
var root = AutomationElement.FromHandle(proc.MainWindowHandle);
var browser = root.FindFirst(TreeScope.Descendants,
new PropertyCondition(AutomationElement.ClassNameProperty, "CefBrowserWindow")); // Always null
}
But browser always is null. What am I missing?