I have an issue bugging me for the past few weeks.
What I'm trying to accomplish: I need a webbrowser control with the ability to change user agent (once at start) and referrer. But most important The ability to see the urls responses. What I mean by that for example if you navigate to a website you get back Images/Javascripts files in response I need access to those urls (Regular Webbrowser Control will not show you those & you can't access it in any way).
I was able to do that with webbrowser + fiddlercore I can see and do what ever with those urls addresses. The problem was if you run few instances of this program (or sometimes once if the program has some automation to work with the url responses) It gets stuck or doesn't work. I tried fixing it and making it work but it's kind of a hacky solution that doesn't work right. I need a simple way to access those urls just as if you used httpwebrequest but as a webbrowser. Why I need it as a webbrowser? The way I work I need the execution of all the tracking pixels and scripts and images etc.. a normal webbrowser behaivor in httpwebrequest if you want google analytics you have to create a code to re create it, you can't just navigate and all the scripts will be execute as webbrowser, or can you?
can you guys can offer me some help or guide me in what direction should I look? Thanks a lot!
Mind to Replacing with some like geckofx ( mozilla firefox ENGINE)?
Start at this:
GeckoFX
Event now you can using xulrunner 16 ( main engine of mozilla firefox 16 ).
New GeckoFX
and
XulRunner
Here article about useragent in xulrunner
I think this is what you are looking for:
Replacing .NET WebBrowser control with a better browser, like Chrome?
However there are many other "browser wrapper" implementations out there, and if nothing on that answer looks good to you let me know and I will try and recommend another.
Probably not the best solution, but you can handle DocumentCompleted event of standard WebBrowser control. Every time it fires you can get access to Image tags in WebBrowser.Document.Images and get their urls and parse WebBrowser.DocumentText for .js urls using Regex or use something like Html Agility Pack on WebBrowser.Document to get all interesting elements...
Related
I tried WebBrowser but any solution related to WebBrowser I find uses the exact same codes, WebBrowser.DrawToBitmap
Problem with that is, it gives out inaccurate results depending on URL, to what page actually would look like, sometimes even blank pages.
So I am looking for any alternate solution to WebBrowser if there is any.
It needs to run in background, as in not open any browser on screen, render it in background with all scripts and get an image.
If I understand what you're trying to do, you might be interested in a framework called PhantomJS, which is a WebKit "browser" engine which runs the pages without visually rendering them. It can be used to capture screens.
Now this technique requires JavaScript, but there is something called Selenium WebDriver to help you wrapping that. Users here at SO posted a simple example and this comment looks useful as it contains the list of required Packages.
I haven't looked into it very much but am struggling to find relevant information on the topic. I basically want to create a browser that applies a filter to a webpage by changing colors in a webpage. My guess is that I will have to change the html once loaded or something, would this work? Do I have other options?
PS. I don't just want to make every color darker, I would more like to invert the colors.
Edit:
If any you were wondering, I am talking about the XAML browser component that can be used in a Windows Phone application.
I think the simplest way to do that is to inject some Javascript into your page once it has loaded.
To do that, you need to set the IsScriptEnabled to true on your WebBrowser control and then subscribe to the Navigated event.
When that event occurs you can inject some JS codeby using the WebBrowser.InvokeScript method.
Here is an example of JS code that darken the page : JavaScript: Invert color on all elements of a page
If you are talking about in a PC internet browser, you can find an add-on to execute Javascript automatically, such as Greasemonkey for Firefox. If you are talking about Windows Phone's Internet Explorer, I don't really know what you could do there, as I don't think they allow add-ons.
There is a website that was created using ColdFusion (not sure if this matters or not). I need to interact with this web site. The main things I need to do are navigate to different pages and click buttons.
I have come up with two ideas on how to do this. The first is to use the WebBrowser control. With this, I could certainly navigate pages, and click buttons (According to This).
The other way is to interact with the html directly. Not sure exactly how to do this, but I am assuming I could click buttons or use HTML requests to interact with the page.
Does anyone have a recommendation on which way is better? Is there a better way that I haven't thought of?
I'd use Html AgilityPack to parse the html and then do POSTs and GETs appropriately with HttpWebRequest.
While it may be possible to use the WebBrowser control to simulate clicks and navigation you get more control with Html AgilityPack and HttpWebRequest regarding what gets sent
Did you consider Selenium? The WebDriver API is quite good, and permits a lot of things in terms of Website automation.
why not submit directly the url? that's what the button click will do.
using WebRequest.Create you can submit directly to the url. no need to load, parse and "click" the button.
HtmlAguilityPack is useful for pulling the web elements and finding tags easily. If you need to remotely "steer" a web session, though, I prefer to use WatiN. It bills itself as a web unit testing framework, but it's very useful anytime you need to fake a browser section. Further, it can remote control different browsers well enough for most tasks you'll need (like finding a button and pushing it, or a text field and filling in text if you need a login).
I've been working on a project that requires that I can go around webpages with different proxies, user-agents, and clear cookies. Now after looking all around the net, it looks like there are some solutions for each of these, but I can never get them working. I was wondering if there was a wrapper for this control that fixed all of these problems or even just a different control I could include. Thanks.
Edit: I tried using HTTPWebRequest, it has everything I need -Javascript
Maybe the GeckoFX Engine is Right for you....
Click ME
I have a class of library code which uses a System.Windows.Forms.WebBrowser control to load a web page then extract information from the DOM tree, etc... I've found this to be the only reliable way to be able to parse information loaded onto a page with javascript, etc...
This worked fine when testing with console app, but when I call the same method from an asp.net page (ASP.NET MVC Controller) it crashes with a "ActiveX control '8856f961-340a-11d0-a96b-00c04fd705a2' cannot be instantiated because the current thread is not in a single-threaded apartment."... i've tried adding [STAThread] to my method but that doesn't seem to help!
Any ideas?
Thanks
Update:
I need to allow a page to complete load and execute all its javascript, then search it for text that matches a template I have already stored... Don't think HtmlAgilityPack will work? Does WatIn run the javascript?
Save yourself a lot of headache and use WatiN instead of a WebBrowser control.
See this question: Best method for Website Automation?
I believe it is impossible to get IIS to run in single thread apartment model (STA). Instead of using the WebBrowser control, you could take a look at the HTML Agility Pack. It will let your read in a web page and poke around the DOM. The WebBrowser control uses lots of computer resources that you won't have to worry about it make the switch.
Everything in the System.Windows is intended for desktop, not web stuff. I agree with Jake Pearson that you should use something else. But if you want you can try forcing the page into STA mode using this directive:
<%#Page AspCompat="true"%>
http://msdn.microsoft.com/en-us/library/zwk9h2kb.aspx