I'm using WebBrowser in C# to display the website that contains login form. After I logged in successfully some parts of the website are not shown in the WebBrowser but in the default browser the same site is showing correctly.
Another issue is when I click on the link in WebBrowser it opens up in the default browser like (Firefox,chrome...etc ).
My application set the username and password automatically for login .
ChangeUserAgent();
var request = (HttpWebRequest)WebRequest.Create(loadLinks(namewebsite));
string useragent = request.UserAgent = "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393";
webBrowser2.Navigate(request.RequestUri,null,null, "User-Agent:Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393");
loadauthintication(namewebsite, _username, _password);
it is the method loadauthintication
using (var webClient = new System.Net.WebClient())
{
var json = webClient.DownloadString(#"http://example.com/getauthintication.php?name="+name+"&user="+user+"&pass="+pass);
// Now parse with JSON.Net
// MessageBox.Show(json);
string[] array = json.Split('~');
username = array[0];
password = array[1];
}
method
public void ChangeUserAgent()
{
ua = "Googlebot/2.1 (+http://www.google.com/bot.html)";
UrlMkSetSessionOption(URLMON_OPTION_USERAGENT_REFRESH, null, 0, 0);
UrlMkSetSessionOption(URLMON_OPTION_USERAGENT, ua, ua.Length, 0);
}
The problem with the WebBrowser control is that by default it renders output in IE7 rendering mode. Rick Strahl has written a very detailed post that covers some of the ways you can work around this.
The way I've usually used is to modify the registry by creating a new key at HKEY_CURRENT_USER\SOFTWARE\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION which is named for the executable for the application (e.g. ConsoleApplication1.exe) and setting the value to be that of the version of Internet Explorer I know will be installed in the target environment - for me that's always been IE11 - a value of 11001.
NOTE: As Rick mentions in his post, bear in mind that when run through Visual Studio, the name of your exe may/will be ConsoleApplication1.vshost.exe so you'll need to add a registry key for that
Related
I'm trying to get a Url full path after it's doing redirect, simply here is the code:
var documentx = new HtmlWeb().Load(textBox1.Text);
Where the textbox1.text value is "https://xxxx.org/file/download"
so after i run that code in real it's redirect and change the structure to:
https://xxxx.org/file/ur344333kd/45rrreew
so how i can get the new url path? using HtmlAgilityPack C# Winform. Thanks
By setting web.CaptureRedirect to true, and by querying web.ResponseUri,
You can get the Url of the final request which actually downloaded the document:
Note: I am sending this UserAgent string, just like my Chrome Browser because server behavior may change depending on it.
HtmlWeb web = new HtmlWeb();
web.UserAgent = "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36";
web.CaptureRedirect = true;
HtmlDocument doc = web.Load("http://www.google.com");
Console.WriteLine("Response retrieved from: {0}", web.ResponseUri);
The output is:
Response retrieved from: https://www.google.com/?gws_rd=ssl
Trying to automate actions on a website. I followed a tutorial and still no luck, no matter what website, tag or class I try it always gets null.
Saw a similar problem somewhere else where they suggested the website hadn't loaded so should add "while" section, which still didn't work for me.
webBrowser1.Navigate("https://www.simplesite.com/");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
{
Application.DoEvents();
}
var myElement = webBrowser1.Document.GetElementById("_ctl0_Header2017_btnLogin");
myElement.InvokeMember("Click");
This is just a simple windows forms application. I've tried webBrowser1_Navigated and webBrowser1_DocumentCompletedmthods too.
Inspect element
The site is responding based on how it sees your browser's capabilities, which it determines based on the User Agent value passed in the request header. You need to set the User Agent string. The one the web browser control is sending is different than your Chrome/Firefox/IE sends.
I've added the Chrome User Agent string... you can find other User Agent strings here, under the Software section...
https://developers.whatismybrowser.com/useragents/explore/
// call navigate and pass the latest Chrome User Agent string
webBrowser1.Navigate("https://www.simplesite.com/", null, null,
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36\r\n");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
{
Application.DoEvents();
// let's just wait a few milliseconds but it would be better
// if we used the DocumentCompleted event
System.Threading.Thread.Sleep(100);
}
var myElement = webBrowser1.Document.GetElementById("_ctl0_Header2017_btnLogin");
myElement.InvokeMember("Click");
im trying to scrape data from Facebook, but i have problems with login.
I'm using Html Agility Pack and I found almost working code. First im trying to login to mobile version of Facebook (it's simpler) and next scrape messages from other site (https://m.facebook.com/messages/?no_hist=1). It's working, but after login I got message "unsupported browser" and I need to set User Agent. I tried to set HttpClient DefaultRequestHeaders but when I added this to my code, then the login stops working and a message appears "Login to Facebook".
Here is my login method:
async private void login(string username, string password)
{
HttpClient hc = new HttpClient();
hc.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36");
HttpResponseMessage resultLogin = await hc.PostAsync("https://m.facebook.com/", new StringContent("login="+username+"&password="+password, Encoding.UTF8, "application/x-www-form-urlencoded"));
HttpResponseMessage resultPlaylist = await hc.GetAsync("https://m.facebook.com/messages/?no_hist=1");
Stream stream = await resultPlaylist.Content.ReadAsStreamAsync();
HtmlDocument doc = new HtmlDocument();
doc.Load(stream);
string webContent = doc.DocumentNode.InnerHtml;
HtmlNode[] nodes = doc.DocumentNode.SelectNodes("//div").ToArray();
foreach (HtmlNode item in nodes)
{
Console.WriteLine(item.InnerHtml);
}
}
This code should find all and show their contents in the console and it does, but i get data from Facebook Login site instead of Messenger site.
Have you tried hc.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36");
Edit: Also download https://www.telerik.com/fiddler and use your browser to login and see if the fiddler shows the same.
My code is :
string result = new WebClient().DownloadString("https://www.facebook.com/profile.php?id=123456789");
the result giving me supported browser.
body class=\"unsupportedBrowser
What i intend to do:
Download a source code from facebook for the particular page.
Problem encounter :
I get the stream from facebook, facebook block me since i access from the apps due to this is not a valid browser.
Expectation : How i can submit the browser type like chrome to cheat the facebook as this is a valid browser.
You can add headers to the webclient object. However, I prefer to go the HttpWebRequest/HttpWebResponse method of scraping since I believe it gives more options.
Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
Headers.Add("Accept-Encoding", "gzip, deflate");
Headers.Add("Accept-Language", "en-US,en;q=0.5");
Headers.Add("Cookie", "has_js=1");
Headers.Add("DNT", "1");
Headers.Add("Host", host);
Headers.Add("Referer", url);
Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0");
I'm trying to change the User Agent of a Webview in a Windows 8 app. At the moment, I'm currently doing this:
[DllImport("urlmon.dll", CharSet = CharSet.Ansi)]
private static extern int UrlMkSetSessionOption(int dwOption,
string pBuffer, int dwBufferLength, int dwReserved);
const int URLMON_OPTION_USERAGENT = 0x10000001;
Then if I needed to change the UA to the iPhone for example:
string ua = "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X)" +
"AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25";
UrlMkSetSessionOption(URLMON_OPTION_USERAGENT, ua, ua.Length, 0);
This works perfectly. But unfortunately, if I'm going to submit this app to the Windows Store, I won't be able to use this method since UrlMkSetSessionOption is not officially supported in Windows 8 apps.
Upon searching for alternatives I came across this. However, although that works, the page renders horribly, with barely any images or other elements being shown.
So I hoped that another way to do this would be by using the WebView.InvokeScript method and injecting some JavaScript to change the UA. The problem being, to be quite honest, is that I'm a complete beginner when it comes to JavaScript.
I did a bit more searching and apparently something like this (found here) would work:
navigator.__defineGetter__('userAgent', function(){
return 'Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25' // customized user agent
});
navigator.userAgent; // 'foo'
But like I said, I'm a complete beginner with Javascript. So when exactly should I inject the code? Do I need to listen for an event in JavaScript that runs this code when it fires?
it's easy.
string ua = "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X)" + "AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25";
Uri targetUri = new Uri(url);
HttpRequestMessage hrm=new HttpRequestMessage(HttpMethod.Get,targetUri);
hrm.Headers.Add("User-Agent",ua);
webView1.NavigateWithHttpRequestMessage(hrm);