I have to make a console application in C# which retrieve some data from webpages.
I have downloaded the HTML code from the main page of a website.
WebClient client = new WebClient();
String htmlCode= client.DownloadString(linkToWebpage);
I have verified the string and it is good.
After this part, I have searched for a specific line in the html code which contains a button and a link.
<a rel="nofollow" class="link" onclick="loadornot()" href="http://aaaaa.com/D?WUrtlC1" target="_blank">Click to read more</a>
Now i am trying to download html code from the ancored link (the one from the href), but I am redirected to the main page and I am not sure why. Even if I copy the link from href and paste it into a webbrowser, I am redirected to the main page.
I believe that this happens because the button call a function onclick="loadornot()". That's why it doesn't work the way I have tried? And if yes, how could I call that function from my c# application to continue my app?
Thank you.
Edit:
I have found out that I need some cookies, more exactly, sessioncode, to make that link work. How can I do that?
You can't run javascript code from web page without browser. So, if you really need to execute that function in downloaded page, use some kind of headless browser, like those: webkitdotnet or awesomium
Related
I have a dll that is being called like this:
http://xxxx/dllFile.dll
for security reasons, I want to hide the dll call from the browser, I asked the company that developed that page to hide that dll call and this is their answer:
There’s no development to hidden the url for the “chat.dll”.
The most simple is making an alternative start page. On this page have an “IFrame” and inside will contain the complete chat. Thus hidden all chats urls, and show only the URL from the start page.
so I started working on their suggesting.
I have iframe like this:
<iframe>
</iframe>
and my question is how to call that dll file from this iframe?
thanks
To tell the iframe to load its content from a url, just set its src attribute:
<iframe src="http://xxxx/dllFile.dll">
We have tried using selenium for testing, but it has numerous setbacks, delays and sudden crashes.
Jquery sounds a good alternative, but the challenge is how to jquerify every page load on the browser.
Brandon Martinez here has an example of how to add jquery to the console of chrome to jquerify a page:
var element1 = document.createElement("script");
element1.src = "http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js";
element1.type="text/javascript";
document.getElementsByTagName("head")[0].appendChild(element1);
we want that code to automatically be available in every browser page without the need to manually click a bookmark link on every page.
If we get around that then we can use C# code to:
Process.Start("chrome", #"target site");
and since jquery is already available for every page it will do the population and submit we want.
How can I automatically include jquery for every page that gets loaded on the browser? Is it possible to do that via a chrome plugin; jquery or C# code!? Is it at all possible?
I've decided to use Fiddler to modify response body before being displayed on the browser. Now I can jquerify all pages comes to the browser. Look at this link for a detailed example.
I have a web page with this part of code:
<form action="/?cid=2000" id="queryForm">
<div><button type="submit" name="computeTrip" value="true" class="button">compute trip </button></div></form>
I need to run this in my c# app. I tried a POST method but it doesnt work. (I think I could do it wrong).
To achive next page I wanted to download code in first page,to make changes in http adress and next run second page. But I have to alllow cookies.
I use Http Client sample from msdn.
Thanks for help!
I've application that uses another web sites data so how can i get it because it uses some JavaScript functions to get that data and it not show in page view-source.
Check the NET tab in firebug, XHR and check the resource that is requested, and request the same resource.
Basically you have to render the webpage and ensure the javascript functions are run (evaluated). You could do this by "borrowing" their javascript files (by linking to them from your own page), but this may not work as you don't know what's in those files - they could be accessing DOM elements that you don't have in your page, or calling to other domains which may prevent them from working correctly.
The easiest way to show the same data is to just host the page inside an iframe on your own page. If you are looking to do this from a normal client application (i.e. not a web app) then you will need a browser control that you navigate to the target page. If the browser control is invisible you could then scrape values from it and show them in your app, although this is a very clumsy way to do it, and it's debatable about how ethical it is.
If you want the another web site view source use the HTTPWebRequest to get the response stream in c#.
I have a C# app that uses a web browser control to display some HTML and do some JScript in the background.
How do I integrate this with other system calls? Do I call other C# code from JScript/VBScript? Do I need to do it from the form itself?
When I need javascript hosted in a webbrowser control to call back to the parent, I have it try to navigate the page to another URL. Then in the webbrowser's "BeforeNavigate" event, I get the URL the page was trying to go to, parse out any arguments, dispatch the request to the other C# code, then cancel the original navigate request.