I want to open some websites which may contain javascript code for example google analytics or piwik. I think the using a webclient is the easiest way to visit websites but does the webclient run javascript code in the background automaticly or how could I get javascriptcode running by visiting a website in C#?
Have you considered using a headless browser like PhantomJS.
If you are looking for something with a UI interface, look at the WebBrowser Control.
If you need to just get the DOM or underlying elements of the DOM, I would suggest Watin. It is very mature and works well and its fast.
WebClient only loads data from the web. It does not interpret it in any way.
If you need to treat the page the way a web browser would, then you need to use a Web Browser control.
in C# there is an Internet Explorer control. You can execute javascript code on a client PC by setting web URL link to this control. Internet Explorer control is a fully functional browser which executes all client code.
Related
I have a TCP/IP based windows form chat application. I've been using a web browser to display the communication because the HTML is easier than windows form drawitem.
I have a regex, what recognize the links, and put them into an tag. I'd like to open these links with the default browser.
How can I do this?
I found a code on this page, but it doesn't work.
this was the wrong solution
webBrowser1.Url = new Uri("https://www.google.com/");
Obviously your link is going to look different than that, but that code will change the url of the webBrowser.
I'm developing a software on C# which has to get info from a website which the user opens in chrome, the user has to input some data and then the website returns a list of different items.
What I want is a way to be able to access to the source code of the page in order to get the info, I cant open the web myself as it doesnt show anything because I didnt input any data, so I need to get it directly from chrome.
How can I achieve this ? A chrome extension ? Or can I access to chrome directly from my software ?
Off the top of my head, I don't know any application that gets data directly from an open instance of Chrome. You'd have to write your own Chrome extension.
Alternatively, you can open the web browser from your application initially.
You can look into these libraries for doing so:
Watin (My personal favourite)
Selenium
Awesomium (You'd have to roll out your own UI, it's invisible)
Cef
Essential Objects Web Browser
EDIT: I didn't think about using QA tools as the actual browser hook as #TheAnathema mentions. That would probably work for your needs.
You're going to need to create it as Chrome extension if you must be dependent on the user actually going to a specific web site (i.e. not being able to do the requests yourself with either Selenium or standard web requests in Python).
The reason why a Chrome extension would be required is because think of how bad it could be for any software to easily read the pages you browse. Banking, medical, email, etc. could all be accessed anonymously from any process if Google allowed any outside process to tap into the web page.
Even Chrome extensions have to ask for permission to be able to do what they want, but at least it is software the user knowingly installed and agreed to the permissions.
A quick search yielded this example of modifying a page's HTML with a Chrome extension: https://blog.lateral.io/2016/04/create-chrome-extension-modify-websites-html-css/
It sounds like you want to do web scraping. Here's a good tutorial to get you started: HTML Scraping.
And this answer has a good example of how to scrape data from a website where you need to submit a form to get access to the data.
I'm creating a web browser that has an automatic loading of specific web pages, but the problem is that the browser that i created using c# in visual studio wont load javascipt, the browser that i created only load html file, but don't support java script. can anyone help me on how to add some functionality on my custom web browser that will support javascript.
WebKit DotNet is the best port of WebKit powerful browser engine into DotNetFramework.
It has nice and easy tutorials and properties and methods to customize.
It has JavaScript activated by default.
http://webkitdotnet.sourceforge.net/ is the official website.
http://webkitdotnet.sourceforge.net/basics.php is where you can find the basic tutorial
Webbrowser control is really crappy. I'm assuming you're trying to scape some kind of website. For this use HttpWebRequest instead.
If you`re trying create your own webbrowser: don't, or use Webkit or Gecko instead.
In case you're using the webbrowser control you will have to enable JavaScript in your Internet Explorer settings, because the webbrowser control is the Internet Explorer or at least the engine of it. IE has local JavaScript disabled by default, so this could be your problem. As user #user3855678 said I would recommend using Webkit etc too.
Suppose several websites are filtered in my region, and in order to get some specific information on that pages, i need to use a proxy website such as (Thisone) to bypass the filtering.
How can i send my URL to these kind of sites and get the result?
How can i search in the resulting content for specific term or object (image, flash, etc)?
It looks like the site you mentioned does not provide any API of any kind that you can call from C#. Probably your best option is to automate manual entry of the url using a tool like Selenium or Watin.
The WebClient and WebBrowser components in .NET automatically use the system Internet Explorer network and proxy settings (set via the Control Panel > Internet). If you set them there then you're all set.
If you use WebBrowser I don't know (off my head) if you can manually override the proxy settings, but with WebClient you set the WebClient.Proxy property.
This website has a custom google search box:
http://ezinearticles.com/
The search results are generated by a piece of JS code. How would I access these results using wget and/or C#'s WebClient?
It looks like the searches on that page are normal google site searches. Try wget with the following url, where 'asdf' is your search
wget http://www.google.com/search?&q=site:ezinearticles.com+asdf
You need to to what your web browser does - render the page. Maybe you can extract the js call to the webservice providing the results and just execute this request and parse the output directly.
You need to access it with a programmable browser supporting JavaScript.
The HtmlUnit library for Java does this, and runs fine headless.
You can automate a real web browser, e.g. with WatiN on Windows, and access the page's content. This requires a GUI desktop though, because a real browser window is opened.