Is there an example of adding plugins to Google IE browser that supports language c# or .net I want to run files js - json - hrml popup
Microsoft just released Blazor, which allows browsers to run Razor / C# code on the client side. You could research and experiment with that.
However, you don't need that to run js - json - html popup. Use something like ASP.Net MVC to run any C# code on the server. The server generates your HTML and Javascript and stuff. Browsers know how to work with all of that stuff without any plugins.
This site will work better when you have specific questions and you can show your code and what you have tried. For a question like this, Google will work better for you! Good luck.
Related
I'm trying to scrape a particular webpage which works as follows.
First the page loads, then it runs some sort of javascript to fetch the data it needs to populate the page. I'm interested in that data.
If I Get the page with HtmlAgilityPack - the script doesn't run so I get what it essentially a mostly-blank page.
Is there a way to force it to run a script, so I can get the data?
You are getting what the server is returning - the same as a web browser. A web browser, of course, then runs the scripts. Html Agility Pack is an HTML parser only - it has no way to interpret the javascript or bind it to its internal representation of the document. If you wanted to run the script you would need a web browser. The perfect answer to your problem would be a complete "headless" web browser. That is something that incorporates an HTML parser, a javascript interpreter, and a model that simulates the browser DOM, all working together. Basically, that's a web browser, except without the rendering part of it. At this time there isn't such a thing that works entirely within the .NET environment.
Your best bet is to use a WebBrowser control and actually load and run the page in Internet Explorer under programmatic control. This won't be fast or pretty, but it will do what you need to do.
Also see my answer to a similar question: Load a DOM and Execute javascript, server side, with .Net which discusses the available technology in .NET to do this. Most of the pieces exist right now but just aren't quite there yet or haven't been integrated in the right way, unfortunately.
You can use Awesomium for this, http://www.awesomium.com/. It works fairly well but has no support for x64 and is not thread safe. I'm using it to scan some web sites 24x7 and it's running fine for at least a couple of days in a row but then it usually crashes.
I have a good understanding of DOM+HTML etc but I'm new to c#, whats the best way currently of downloading then rendering (executing all javascript + DOM changes etc) and simulating user interaction with a webpage in c#?
I've seen HTML agility pack mentioned quite a few times but it doesn't look like its been updated since August 2012? Has anyone used this recently and encountered any problems? Does c# have anything built in for this?
Thanks!
First of all HTMLAgilityPack it's not for simulating user interaction in a web page, HTMLAgilityPack is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT (you actually don't HAVE to understand XPATH nor XSLT to use it, don't worry...).
HTMLAgilityPack not support JavaScript, it's a very important step, because many developers get trouble with the full load of the page in the browser and the request made by HTMLAgilityPack or any library you use to make the request.
For user interaction, full load of the web page, web testing I strongly recommend you Selenium, Selenium automates browsers. Selenium has support for several programming languages (Java, C#, Ruby, Python, etc), you can read more in the above link with a very good documentation.
The only drawback of Selenium is its open a browser to make the work, but it can be simulated in some environments to run headless browser, you can read more about this in the following links :
Selenium Headless Automated Testing in Ubuntu
Headless Browser and scraping - solutions
I hope this help you
Could anyone please advise me what is the best framework/library for web browser automatisation? The task is to open web browsers page, sign in, perform some long searches, and save gathered information to excel. Now I'm using IE references in C#, but at work I could use only IE8. If I've upgraded it to IE9, but some scripts on target sites started working with errors.
I tried to use awesomium, but I couldn't parse page with help of it, as I understand. Are there any variants to do this with high speed? Size of libs - doesn't matter.
If there are any solutions compatible with Scala it would be great.
As om-nom-nom hinted already, your best bet is probably a webdriver implementation like selenium webdriver. It has bindings for c# and java and can use IE, FF, Chrome, phantomjs (great if you want to go headless) and others.
Note that it might be not the best idea to do also the gathering of information directly with the webdriver, especially if the site content is changing fast. In such cases it might be useful to save the html page source with webdriver and then switch to some more efficient library for static content, like JSoup.
This will probably sound dumb, but I need to execute a C# code from my html file. For example I just want to execute this
System.Diagnostics.Process.Start(#"D:\Movies\HurtLocker.avi");
Not any server side code.
I can't create aspx page, because to open an aspx page in a browser it needs to be hosted in IIS.
You can't execute server side code from a client side page. If your page is aspx you can use a webservice or click a serverside button from javascript.
Edit: If you want to embed a video player please check this link. You don't need server side code for this. You'll be able to do it with javascript.
Where do you want the c# code to run?
If you want it to run in the browser that is being used to render the html then I'd say that was basically not possible. Something, presumably javascript code in the html page, would have to somehow instantiate a .net clr and pass the c# code to it for execution. The clr does have a COM-based hosting interface that would allow instantiation, but even if you could call this from javascript I think that any sensible browser security settings would prevent it.
If you want the c# to run on the server supplying the html page then you should use asp.net.
EDIT
Ok, you want to run it in the browser. I'm not aware of any examples for hosting a clr in the browser process, sorry.
You can create activex/com objects in js using something like var obj=new ActiveXObject("<comclassname>");, and you might be able to create a CLR that way by instantiating one of the COM classes (maybe CLRRuntimeHost) listed on this page. You could then pass your c# code to your clr for execution. More info here and here. I'm really not sure if that would work, though. I've never used the hosting api, I just know it exists!
Seems like an interesting project to try if you are curious, but deploying this in a real environment would likely present lots of problems. Good luck!
Not C#, but how about .NET dynamic language in the browser with Gestalt? http://gestalt.codeplex.com/
You can create a code block in your ASP file but if this is simple HTML file this is not possible.
At least you will have some application that will read the content of the page compile it and execute.
ASP code block
As Pabuc mentioned you can't execute server side code on the client machine in HTML. If you were to use Silverlight you could execute the code client side, but then the client need to have Silverlight installed and it is not strictly HTML anymore.
Silverlight could be used to play movies client side with C#.
The only way to have C# on client side is Silverlight application [update] or any other browser plugin as #kenny mentioned.
I have tried to find a software for this some time. I have software, which has needs, that are hard to make with traditional Web programming.
Now I have made simple demo, how to create HTML5 online application with C# or VB.NET.
It is Scot library which translates C# to Javascript on time when executing .NET application. It also supports events on Browsers, which is executed in c# code.
To original question:
On the Html page you will need to add single line after :
<script src="myclass.cs"> </script>`
to connect .Net class:
using Scot;
//..
public myclass:Document
{
protected override OnConnect()
{
Elements["mybutton"].OnClick+=new JsInputEventHandler(click);
//your initialization //....
}
private void click(object sender, JsInputEventArgs e)
{
Window.Alert("Click()");
}
}
Demos are quite simple, but actually I needed this library for another project.
It would be nice to have any feedback.
I have been given a task to crawl / parse and index available books on many library web page. I usually use HTML Agility Pack and C# to parse web site content. One of them is the following:
http://bibliotek.kristianstad.se/pls/bookit/pkg_www_misc.print_index?in_language_id=en_GB
If you search for a * (all books) it will return many lists of books, paginated by 10 books per page.
Typical web crawlers that I have found fail on this website. I have also tried to write my own crawler, which would go through all links on the page and generate post/get variables to dynamically generate results. I havent been able to do this as well, mostly due to some 404 errors that I get (although I am certain that the links generated are correct).
The site relies on javascript to generate content, and uses a mixed mode of GET and POST variable submission.
I'm going out on a limb, but try observing the JavaScript GETs and POSTs with Fiddler and then you can base your crawling off of those requests. Fiddler has FiddlerCore, which you can put in your own C# project. Using this, you could monitor requests made in the WebBrowser control and then save them for crawling or whatever, later.
Going down the C# JavaScript interpreter route sounds like the 'more correct' way of doing this, but I wager it will be much harder and frought with errors and bugs unless you have the simplest of cases.
Good luck.
FWIW, the C# WebBrowser control is very, very slow. It also doesn't support more than two simultaneous requests.
Using SHDocVw is faster, but is also semaphore limited.
Faster still is using MSHTML. Working code here: https://svn.arachnode.net/svn/arachnodenet/trunk/Renderer/HtmlRenderer.cs Username/Password: Public (doesn't have the request/rendering limitations that the other two have when run out of process...)
This is headless, so none of the controls are rendered. (Faster).
Thanks,
Mike
If you use the WebBrowser control in a Windows Forms application to open the page then you should be able to access the DOM through the HtmlDocument. That would work for the HTML links.
As for the links that are generated through Javascript, you might look at the ObjectForScripting property which should allow you to interface with the HTML page through Javascript. The rest then becomes a Javascript problem, but it should (in theory) be solvable. I haven't tried this so I can't say.
If the site generates content with JavaScript, then you are out of luck. You need a full JavaScript engine usable in C# so that you can actually execute the scripts and capture the output they generate.
Take a look at this question: Embedding JavaScript engine into .NET -- but know that it will take "serious" effort to do what you need.
AbotX does javascript rendering for you. Its not free though.