I've been working on a project that requires that I can go around webpages with different proxies, user-agents, and clear cookies. Now after looking all around the net, it looks like there are some solutions for each of these, but I can never get them working. I was wondering if there was a wrapper for this control that fixed all of these problems or even just a different control I could include. Thanks.
Edit: I tried using HTTPWebRequest, it has everything I need -Javascript
Maybe the GeckoFX Engine is Right for you....
Click ME
Related
I have an issue bugging me for the past few weeks.
What I'm trying to accomplish: I need a webbrowser control with the ability to change user agent (once at start) and referrer. But most important The ability to see the urls responses. What I mean by that for example if you navigate to a website you get back Images/Javascripts files in response I need access to those urls (Regular Webbrowser Control will not show you those & you can't access it in any way).
I was able to do that with webbrowser + fiddlercore I can see and do what ever with those urls addresses. The problem was if you run few instances of this program (or sometimes once if the program has some automation to work with the url responses) It gets stuck or doesn't work. I tried fixing it and making it work but it's kind of a hacky solution that doesn't work right. I need a simple way to access those urls just as if you used httpwebrequest but as a webbrowser. Why I need it as a webbrowser? The way I work I need the execution of all the tracking pixels and scripts and images etc.. a normal webbrowser behaivor in httpwebrequest if you want google analytics you have to create a code to re create it, you can't just navigate and all the scripts will be execute as webbrowser, or can you?
can you guys can offer me some help or guide me in what direction should I look? Thanks a lot!
Mind to Replacing with some like geckofx ( mozilla firefox ENGINE)?
Start at this:
GeckoFX
Event now you can using xulrunner 16 ( main engine of mozilla firefox 16 ).
New GeckoFX
and
XulRunner
Here article about useragent in xulrunner
I think this is what you are looking for:
Replacing .NET WebBrowser control with a better browser, like Chrome?
However there are many other "browser wrapper" implementations out there, and if nothing on that answer looks good to you let me know and I will try and recommend another.
Probably not the best solution, but you can handle DocumentCompleted event of standard WebBrowser control. Every time it fires you can get access to Image tags in WebBrowser.Document.Images and get their urls and parse WebBrowser.DocumentText for .js urls using Regex or use something like Html Agility Pack on WebBrowser.Document to get all interesting elements...
I'd like to use C# to interact with the IE browser.
I have a feeling that shdocvw.dll will be involved, but there are so many classes in there that I don't know where to start, and maybe it's not even necessary to use it.
The goal here is to interact with a website, visiting it's pages and "warming it up," not unlike as described here by Kenneth Scott. The thing is, javascript is getting executed as you interact with a website, so it would be nice just to be able to login / submit forms exactly as you would on the website itself.
Plus it would be nice to be able to create a program that records my actions in IE, and then be able to slightly automate and slightly modify them.
Additionally, it would be nice if it could do all this in the background, without having to display the webpage at all.
I'm not looking for third party solutions, I want to do this myself (with your advice of course.)
Thanks.
You said you're not looking for a third party solution, however, we have used WatiN in work with great success for automated UI testing.
It's open source, so if you want to see how they do it, you can.
Things like selenium and watin are very mature frameworks for doing exactly what you ask. Unless the point is to learn for yourself how to do this I would use one of them.
Watin is also a great way to learn how to do this in c# as it is an open source c# project.
I have a .NET ActiveX/COM Object which has a WebBrowser control (among other controls such has buttons, etc) inside it.
This object seems to load fine, the buttons are displayed and working but, for some reason, the WebBrowser control doesn't show up!
Is there some limitation for this in IE (e.g., is IE afraid of something recursive, etc.)? Is there a way that I can overcome it?
Thanks in advance,
Zubrowka
FireBreath has a library called WebView that does this; I've tested it both in NPAPI based browsers and in ActiveX and it works just fine. There are many reasons why you may want to do this.
Since I don't know what you're doing, I can't tell you what you're doing wrong... but I can show you the source code that FireBreath uses to do it:
https://github.com/firebreath/FireBreath/blob/master/src/libs/WebView/Win/WebViewWin.cpp
This is in C++, but you might be able to find something useful from it. I don't use .net in the browser -- too much overhead -- but theoretically the same principles should apply.
I am making a multi-threaded [workers] application. Each thread should have it's own Non-GUI WebBrowser that Navigates to a web page and writes data to fields and click a button. I also need each WebBrowser to have it's own proxy. I tried the classic Windows.Forms.WebBrowser but I got stuck at the proxy part as it depends on IE global settings which won't work in my case. Any recommendations are welcome.
note: I tried doing it through HttpWebRequest/Response but it will never work as the data to be passed to the page contains a field called [ab_test_data] which gets its value from javascript code that calculate the value according to AB testing which I don't even fully understand. So a WebBrowser would be my best solution, unless someone can tell me how to convert that Javascript code that calculates ab_test_data to C# code. The algorithm used by the page I am trying to access is really sophisticated.
note2: ab_test_data value depends on Window.Event and Timestamp which can't be simulated on a httpWebRequest/Response.
note3: I tried Gecko, But it won't let me do anything to the webPage unless GeckoWebBrowser is drawn on the form (which I don't want).
Any solutions are welcome.
edit: If you know any WebBrowser that works like I want in any different language (Maybe Java) I would like to know.
Thanks in advance.
CefSharp: .Net binding for the Chromium Embedded Framework
use http://webkitdotnet.sourceforge.net/
As a question that may help, I wonder why browsers don't allow a proxy per (say) window/tab? I think a lot of it is because of lack of usefulness with respect to development time.
It may also be because the browsers [presumably] have centralized engines for things like web requests and caches, etc... Perhaps, allowing a proxy per window and/or tab would fundamentally alter the design of the modern browser and or have negative performance impacts. I don't really know. To illustrate the point further, consider things like Incognito mode and Private Browsing. In these cases, the browsers have, at least, conceptually made separate caches per windows...but I still bet an Incognito window and a standard window (in Chrome) use the same underlying web request engine.
Right now there are so many people who want a JavaScript and DOM parser and interpreter. Projects like the HtmlAgility Pack and Jint are helping, but there doesn't seem to be a unified and standard solution; at least not one with the simplicity of a web browser.
[rant below]...
Unfortunately, projects like Jint and HtmlAgility are worrisome. For one, they're not IE, Chrome, Safari or FireFox. You don't exactly know what you're getting yourself into. For instance, you know that in Chrome page xyz.com loads and renders perfectly. You can fire up FireFox and see that maybe something is not quite the same and so on with the other browsers. But, with these libraries you don't really know what if not everything is working right (there's no visual display to do a quick check). Plus, who knows what pace they're being developed at. Do they keep up with HTML5? Do they lag behind the major browsers? What about performance? Even more so, browsers already have things like caching and performance enhancements, which I doubt you'll get with individual libraries.
The best browser control would of course be something like:
IWebBrowser browser = new IE();
IWebBrowser browser = new Chrome();
IWebBrowser browser = new Safari();
IWebBrowser browser = new FireFox();
I think that is a dream, unfortunately. For one, what if you ever wanted to load plug-ins with these? What about user profiles, user logins, and so on? I think most of us just want the muscle of the browsers without these extras.
I really do hope that you find a good Chrome solution. I don't know what, if any, luck you'll have in the FireFox realm - maybe you can keep us updated? These solutions are evolving so quickly - I had never even heard of CefSharp or WebKit.NET before today and I looked for the same thing (Chrome and/or FireFox .NET browsers) several months ago for my own use. It would be great if a lot of people got together, made a standard interface and then each company built their embedded browser against the spec. Here's to wishing.
My whole websites makes use of the ASP.NET(2.0) repeater control and I have been getting some complains from people that they cant view my site as the controls are overlapping.
This sounds like a browser issue to me as I never see it in IE8 or IE7 or Chrome or Firefox.
Any idea why this is happening? What should I do to prevent this from happening?
Sounds like a Microsoft issue?
As far as the browser is concerned, there's no such thing as an ASP.NET repeater control (indeed, to the browser, there's no such thing as ASP.NET); the browser cares only about the resultant HTML and CSS.
The first thing I would do is to make sure the HTML is valid, while browsers will do incorrect things with valid HTML, testing its validity is a quick way to catch a lot of errors that can cause problems (in particular, where the tags aren't correctly nested). I use this firefox plugin that is imperfect but quick to use.
This isn't guaranteed to be your issue, but it's definitely the place to start.
The only way you are going to find out what is happening is to check your site in various browsers. You should at least be testing in Firefox, Chrome and IE (arguably 6 and above).
The issue is most likely css markup not being cross browser compliant.
A lot of developers go with process of get it working in Firefox, and then fix any quirks in IE. The Firefix Firebug plugin is ideally suited for testing and modifying css. IETester is also handy for testing various versions of IE in one application.