I am trying to evaluate javascript from html page based on HTML (I mean some of the variables in javascript function are using html tag id )
I need to use it multiple threads (Not UI) so this means that i can not use WebBroser.
I can not use Eval.JScriptEvaluate because the javascript depends on the html
Is there is any way to achieve this with this limitations
Maybe there is web Service that revives html as string and lets execute some specific script.
You'll need to parse and execute the script - there's a number of projects out there to help with this see:
Are there any .NET CLR/DLR implementations of ECMAScript?
Interpreting JavaScript outside of the browser?
Can you give more detail in your question. e.g. Do you have control over this page that you are trying to run (in which case, maybe you could solve your multiple thread requirements with HTML5 webworkers) and why do you need multiple threads?
Here is one link you might find interesting which, with some work on your side would allow you to load web pages into Rhino (a server side, javascript interpreter) and execute the javascript.
Another link to check out is HTMLunit, which also allows you to simulate a browser that can execute javascript code from within Java.
Related
I'm trying to scrape a particular webpage which works as follows.
First the page loads, then it runs some sort of javascript to fetch the data it needs to populate the page. I'm interested in that data.
If I Get the page with HtmlAgilityPack - the script doesn't run so I get what it essentially a mostly-blank page.
Is there a way to force it to run a script, so I can get the data?
You are getting what the server is returning - the same as a web browser. A web browser, of course, then runs the scripts. Html Agility Pack is an HTML parser only - it has no way to interpret the javascript or bind it to its internal representation of the document. If you wanted to run the script you would need a web browser. The perfect answer to your problem would be a complete "headless" web browser. That is something that incorporates an HTML parser, a javascript interpreter, and a model that simulates the browser DOM, all working together. Basically, that's a web browser, except without the rendering part of it. At this time there isn't such a thing that works entirely within the .NET environment.
Your best bet is to use a WebBrowser control and actually load and run the page in Internet Explorer under programmatic control. This won't be fast or pretty, but it will do what you need to do.
Also see my answer to a similar question: Load a DOM and Execute javascript, server side, with .Net which discusses the available technology in .NET to do this. Most of the pieces exist right now but just aren't quite there yet or haven't been integrated in the right way, unfortunately.
You can use Awesomium for this, http://www.awesomium.com/. It works fairly well but has no support for x64 and is not thread safe. I'm using it to scan some web sites 24x7 and it's running fine for at least a couple of days in a row but then it usually crashes.
I am using the Nancy Web Framework in my C# Console Application to basically create a Web Administration panel for my software. I have opted to use the Spark View Engine, as it is basically just HTML. I basically want to create a chatbox, except pull the data written to my application's console every X seconds and display it in a box instead.
I have very little experience with JQuery and AJAX, but they aren't overly complicated from the examples I have seen. The issue I am running into is that ALL of the chatbox and shoutbox examples use PHP.
I basically just need something like this...
The only difference is I need to pull the information from my application instead. I can use basic C# methods inside of the HTML (and probably inside of javascript but I haven't tried this). What would be the best way to do this, and are there any examples floating around that don't use PHP?
This was completed using AJAX and JSON.
Well, to use HTML for styling inside some PC program is just not wise. It has much better UI engines, though. But for your information here is nice jQuery shoutbox tutorial, but well, you only need to handle data input and output with C#, so actually I see no problems. The engine which you are using should have some kind of data stream, or requests handler (bla://program/???)
I would like to know how the HTML source of ajax based sites can be read using HttpWebRequest / HttpWebResponse (That is reading the contents of a website at server side). The problem that I'm facing is that I'm unable to read parts of the webpage which uses Ajax or stuffs like UpdatePanel.
My application is in ASP.NET / C#, so can't think of using stuffs like Browser control or mshtml.dll since I would not be able to serve multiple requests.
Thanks in advance.
this is going to be difficult.
I know you said you don't want to use Browser control, but I'm going to say it anyway. You will most probably be better off using a Browser control. The reasons are as follow:
AJAX sites make multiple calls from the browser to the server to obtain the required view.
The multiple calls are being performed via JavaScript
The data returned from the server may be reformatted by JavaScript before being updated onto the view.
If you are going to do this using HttpWebXyz functions, you will have to do the following:
Make the relevant calls to get the initial page source.
Parse the page for JavaScript.
Evaluate/execute the JavaScript. This may include providing the relevant implementation for functions such as alert and making subsequent calls to the server.
Depending on the complexity of the AJAX site, you may want to reconsider using the browser control. Complex sites are easier process by the control. If the site is simple enough, you may survive parsing and executing the required JavaScript.
This example uses a deprecated class to parse JavaScript.
You may want to explore ICodeCompiler and its relevant classes for the new approach.
Good luck.
This will probably sound dumb, but I need to execute a C# code from my html file. For example I just want to execute this
System.Diagnostics.Process.Start(#"D:\Movies\HurtLocker.avi");
Not any server side code.
I can't create aspx page, because to open an aspx page in a browser it needs to be hosted in IIS.
You can't execute server side code from a client side page. If your page is aspx you can use a webservice or click a serverside button from javascript.
Edit: If you want to embed a video player please check this link. You don't need server side code for this. You'll be able to do it with javascript.
Where do you want the c# code to run?
If you want it to run in the browser that is being used to render the html then I'd say that was basically not possible. Something, presumably javascript code in the html page, would have to somehow instantiate a .net clr and pass the c# code to it for execution. The clr does have a COM-based hosting interface that would allow instantiation, but even if you could call this from javascript I think that any sensible browser security settings would prevent it.
If you want the c# to run on the server supplying the html page then you should use asp.net.
EDIT
Ok, you want to run it in the browser. I'm not aware of any examples for hosting a clr in the browser process, sorry.
You can create activex/com objects in js using something like var obj=new ActiveXObject("<comclassname>");, and you might be able to create a CLR that way by instantiating one of the COM classes (maybe CLRRuntimeHost) listed on this page. You could then pass your c# code to your clr for execution. More info here and here. I'm really not sure if that would work, though. I've never used the hosting api, I just know it exists!
Seems like an interesting project to try if you are curious, but deploying this in a real environment would likely present lots of problems. Good luck!
Not C#, but how about .NET dynamic language in the browser with Gestalt? http://gestalt.codeplex.com/
You can create a code block in your ASP file but if this is simple HTML file this is not possible.
At least you will have some application that will read the content of the page compile it and execute.
ASP code block
As Pabuc mentioned you can't execute server side code on the client machine in HTML. If you were to use Silverlight you could execute the code client side, but then the client need to have Silverlight installed and it is not strictly HTML anymore.
Silverlight could be used to play movies client side with C#.
The only way to have C# on client side is Silverlight application [update] or any other browser plugin as #kenny mentioned.
I have tried to find a software for this some time. I have software, which has needs, that are hard to make with traditional Web programming.
Now I have made simple demo, how to create HTML5 online application with C# or VB.NET.
It is Scot library which translates C# to Javascript on time when executing .NET application. It also supports events on Browsers, which is executed in c# code.
To original question:
On the Html page you will need to add single line after :
<script src="myclass.cs"> </script>`
to connect .Net class:
using Scot;
//..
public myclass:Document
{
protected override OnConnect()
{
Elements["mybutton"].OnClick+=new JsInputEventHandler(click);
//your initialization //....
}
private void click(object sender, JsInputEventArgs e)
{
Window.Alert("Click()");
}
}
Demos are quite simple, but actually I needed this library for another project.
It would be nice to have any feedback.
I have been given a task to crawl / parse and index available books on many library web page. I usually use HTML Agility Pack and C# to parse web site content. One of them is the following:
http://bibliotek.kristianstad.se/pls/bookit/pkg_www_misc.print_index?in_language_id=en_GB
If you search for a * (all books) it will return many lists of books, paginated by 10 books per page.
Typical web crawlers that I have found fail on this website. I have also tried to write my own crawler, which would go through all links on the page and generate post/get variables to dynamically generate results. I havent been able to do this as well, mostly due to some 404 errors that I get (although I am certain that the links generated are correct).
The site relies on javascript to generate content, and uses a mixed mode of GET and POST variable submission.
I'm going out on a limb, but try observing the JavaScript GETs and POSTs with Fiddler and then you can base your crawling off of those requests. Fiddler has FiddlerCore, which you can put in your own C# project. Using this, you could monitor requests made in the WebBrowser control and then save them for crawling or whatever, later.
Going down the C# JavaScript interpreter route sounds like the 'more correct' way of doing this, but I wager it will be much harder and frought with errors and bugs unless you have the simplest of cases.
Good luck.
FWIW, the C# WebBrowser control is very, very slow. It also doesn't support more than two simultaneous requests.
Using SHDocVw is faster, but is also semaphore limited.
Faster still is using MSHTML. Working code here: https://svn.arachnode.net/svn/arachnodenet/trunk/Renderer/HtmlRenderer.cs Username/Password: Public (doesn't have the request/rendering limitations that the other two have when run out of process...)
This is headless, so none of the controls are rendered. (Faster).
Thanks,
Mike
If you use the WebBrowser control in a Windows Forms application to open the page then you should be able to access the DOM through the HtmlDocument. That would work for the HTML links.
As for the links that are generated through Javascript, you might look at the ObjectForScripting property which should allow you to interface with the HTML page through Javascript. The rest then becomes a Javascript problem, but it should (in theory) be solvable. I haven't tried this so I can't say.
If the site generates content with JavaScript, then you are out of luck. You need a full JavaScript engine usable in C# so that you can actually execute the scripts and capture the output they generate.
Take a look at this question: Embedding JavaScript engine into .NET -- but know that it will take "serious" effort to do what you need.
AbotX does javascript rendering for you. Its not free though.