Develop Reference

How to have C# Webbrowser handle webpage login popup for webscraping

I'm trying to programmatically login to a site like espn.com. The way the site is setup is once I click on the Log In button located on the homepage, a Log In popup window is displayed in the middle of the screen with the background slightly tinted. My goal is to programmatically obtain that popup box, supply the username and password, and submit it -- hoping that a cookie is returned to me to use as authentication. However, because Javascript is used to display the form, I don't necessarily have easy access to the form's input tags via the main page's HTML.
I've tried researching various solutions such as HttpClient and HttpWebRequest, however it appears that a Webbrowser is best since the login form is displayed using Javascript. Since I don't necessarily have easy access to the form's input tags, a Webbrowser seems the best alternative to capturing the popup's input elements.
class ESPNLoginViewModel
{
private string Url;
private WebBrowser webBrowser1 = new WebBrowser();
private SHDocVw.WebBrowser_V1 Web_V1;
public ESPNLoginViewModel()
{
Initialize();
}
private void Initialize()
{
Url = "http://www.espn.com/";
Login();
}
private void Login()
{
webBrowser1.Navigate(Url);
webBrowser1.DocumentCompleted +=
new WebBrowserDocumentCompletedEventHandler(webpage_DocumentCompleted);
Web_V1 = (SHDocVw.WebBrowser_V1)this.webBrowser1.ActiveXInstance;
Web_V1.NewWindow += new SHDocVw.DWebBrowserEvents_NewWindowEventHandler(Web_V1_NewWindow);
}
//This never gets executed
private void Web_V1_NewWindow(string URL, int Flags, string TargetFrameName, ref object PostData, string Headers, ref bool Processed)
{
//I'll start determing how to code this once I'm able to get this invoked
}
private void webpage_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
HtmlElement loginButton = webBrowser1.Document.GetElementsByTagName("button")[5];
loginButton.InvokeMember("click");
//I've also tried the below InvokeScript method to see if executing the javascript that
//is called when the Log In button is clicked, however Web_V1_NewWindow still wasn't called.
//webBrowser1.Document.InvokeScript("buildOverlay");
}
}
I'm expecting the Web_V1_NewWindow handler to be invoked when the InvokeMember("click") method is called. However, code execution only runs through the webpage_DocumentCompleted handler without any calls to Web_V1_NewWindow. It might be that I need to use a different method than InvokeMember("click") to invoke the Log In button's click event handler. Or I might need to try something completely different altogether. I'm not 100% sure the Web_V1.NewWindow is the correct approach for my needs, but I've seen NewWindow used often when dealing with popups so I figured I should give it a try.
Any help would be greatly appreciated as I've spent a significant amount of time on this.

I know it is the late answer. But it will help someone else.
You can extract the value from FRAME element by following
// Get frame using frame ID
HtmlWindow frameWindow = (from HtmlWindow win
in WbBrowser.Document.Window.Frames select win)
.Where(x => string.Compare(x.WindowFrameElement.Id, "frm1") == 0)
.FirstOrDefault();
// Get first frame textbox with ID
HtmlElement txtElement = (from HtmlElement element
in frameWindow.Document.GetElementsByTagName("input")
select element)
.Where(x => string.Compare(x.Id, "txt") == 0).FirstOrDefault();
// Check txtElement is nul or not
if(txtElement != null)
{
Label1.Text = txtElement.GetAttribute("value");
}
For more details check
this article

disable browser close button in asp.net application?

I am doing online Exam application using asp.net in this i have to disable the titlebar so that the user has no option to exit with in the time period.So please help with this one

Its not good practice to force user to stay on the page if they don't wish to, but you can have some work around if you want to confirm the close event before they leave the browser tab
function internalHandler(e) {
return "Please don't leave the page you can be fail in exams!";
}
if (window.addEventListener) {
window.addEventListener('beforeunload', internalHandler, true);
} else if (window.attachEvent) {
window.attachEvent('onbeforeunload', internalHandler);
}
If you prevent user to close it any way you don't have control over ALT + F4 or closes it from Task Manager
you can do it using javascript like this
var message = "You have not completed exam. Are you sure you want to leave?";
window.onbeforeunload = function(event) {
var e = e || window.event;
if (e) {
e.returnValue = message;
}
return message;
};
and you can unload it when user finish the exam
window.onbeforeunload = null;
or you can create your own browser application using c# windows forms. where you can set this custom option without having close button. You load your web application form in windows forms application easily.

onbeforeunload & onunload will help you out. You can't disable but you can show user an alert.
var showMsgTimer;
window.onbeforeunload = function(evt) {
var message = 'Don't Discard';
showMsgTimer = window.setTimeout(showMessage, 500);
evt = evt || window.evt;
evt.returnValue = message;
return message;
}
window.onunload = function () {
clearTimeout(showMsgTimer);
}
function showMessage() {
alert("You're Right!");
}
If this is not the one you expect. Then please try https://eureka.ykyuen.info/2011/02/22/jquery-javascript-capture-the-browser-or-tab-closed-event/

Using Window.Open instead of Response.Redirect to open new window?

My page code looks like this:
<asp:Button ID="btnSearch" runat="server" Text="Search" onclick="btnSearch_Click"/>
My method looks like this:
protected void btnSearch_Click(object sender, EventArgs e)
{
var value = lblGraphicNameValue.Text.ToString();
Response.Redirect("Search.aspx?txtGraphicName=" +
value);
}
Currently, when the user press the 'Search' button the page refreshes and loads the Search.aspx page. What I'd like to happen is have the Search.aspx open in a new window, instead. I've looked at using Window.Open, but I'm not sure if this is the correct route, or if I can use the same method of passing in my variable (querystring). Can someone point me in the right direction? What I have works, I just want it to open in a new page while leaving the prior page alone.
EDIT: I should mention that I cannot use javascript (secure environment, every browser has javascript disabled).
From what I'm reading, it seems to indicate that opening a new web page from within an asp.net page and having parms passed in is not do-able without javascript? Is this correct?

This code below ultimately does exactly what I needed it to:
<a href="<%= this.ResolveUrl("Search.aspx?id=" + lblGraphicNameValue.Text.Remove(lblGraphicNameValue.Text.Length -4)) %>"
target="_blank">Search Related</a>
This code does three things:
1) Opens Search in new page.
2) Truncates the search value by four
characters (I only needed part of the search string)
3) Passes in
parameter to new page.
This accomplished exactly what I needed without resorting to custom classes or javascript, although it did make me have to use a link instead of a button.

Use this class.
ResponseHelper .Redirect("popup.aspx", "_blank", "menubar=0,width=100,height=100");
public static class ResponseHelper {
public static void Redirect(string url, string target, string windowFeatures) {
HttpContext context = HttpContext.Current;
if ((String.IsNullOrEmpty(target) ||
target.Equals("_self", StringComparison.OrdinalIgnoreCase)) &&
String.IsNullOrEmpty(windowFeatures)) {
context.Response.Redirect(url);
}
else {
Page page = (Page)context.Handler;
if (page == null) {
throw new InvalidOperationException(
"Cannot redirect to new window outside Page context.");
}
url = page.ResolveClientUrl(url);
string script;
if (!String.IsNullOrEmpty(windowFeatures)) {
script = #"window.open(""{0}"", ""{1}"", ""{2}"");";
}
else {
script = #"window.open(""{0}"", ""{1}"");";
}
script = String.Format(script, url, target, windowFeatures);
ScriptManager.RegisterStartupScript(page,
typeof(Page),
"Redirect",
script,
true);
}
}
}

I think your on the right track, but you're confusing server side code, and client side code. window.open is a Javascript function which works on the client side. So you'll need to render some Javascript from C# to make the window popup. Try:
protected void btnSearch_Click(object sender, EventArgs e)
{
var value = lblGraphicNameValue.Text.ToString();
ClientScript.RegisterStartupScript(this.GetType(), "newWindow", String.Format("<script>window.open('Search.aspx?txtGraphicName={0}');</script>", value));
}
That will re-render the page, and then add a script on pageload that will popup the window. A little warning, this will probably be blocked by a browser popup blocker. If you want to get around that, you can probably achieve this without posting back to the server by using Javascript.

A better option would be to create a javascript function like:
function PreviewPOSTransaction(Id)
{
if (Id != null)
{
window.open('POSTransReport.aspx?TransID=' + Id);
return true;
}
}
</script>
and call this function on button "OnClientClick" event like:
OnClientClick="PreviewPOSTransaction(1);

Changing the value of a hidden field without postbacking

I have a link button, that calls the c# function 'send_Click' when clicked. here is the function:
void send_Click(object sender, EventArgs e)
{
Page.ClientScript.RegisterStartupScript(this.GetType(), "Call my function", "text_alert()", true);
Response.Write(hidAcrobat.Value);
}
as you can see, it calls a javascript function "text_alert()" which looks like this:
function text_alert() {
var person = prompt("Additional Comments:", "");
if (person != null && person != "") {
document.getElementById('hidAcrobat').value = person;
}
}
which pops a prompt box accepting a user input and setting the value of the hidden field 'hidAcrobat' to this value.
then back to c#, the next line is Response.write(hidAcrobat.Value);
It writes the default value of the hidAcrobat and NOT the new value which was assigned to it in the prompt box.
I assume its because the page isn't postbacked,
How can I solve this?

This is easily implemented by Jquery Like that
<script type="text/javascript" language="javascript">
$(document).ready(function() {
$('#MainContent_lnkAddToList').live("click", function() {
var person = prompt("Additional Comments:", "");
$('[id*="hidAcrobat"]').val(person);
});
});
After that you can use your hidden field value in any server side event.
For Jquery add reference
Hope It helps you.

Handle onclick event of link button on client side (using javascript) and call text_alert() method from the handler.
After executing the client-side event handler, the page posts back and on the server side you can access the hidden variable as you are doing in send_Click event handler.

You can easily create a prompt like effect using ModalPopUp and then easily grab the value on server side.

Disable Submit button in case of multiple click.. (C#)

My Problem is ,I have a simple web form, which contains two textboxes and a button.there are some asp.net validator controls on page.so i want client side disabling of button when all validation is done.and also after disabling of button, i am executing some server side code.All of this is working fine but, in case when I set postback url of button it gets fail. bellow is some part of coding that will give you some brief idea. Any hint will be highly appreciated.......
I wanted to make this functionality in composite control.
here is button class
public class MyButton : Button
{
protected override void OnPreRender(EventArgs e)
{
if (this.CausesValidation)
{
if (!String.IsNullOrEmpty(this.ValidationGroup))
{
this.Attributes.Add("onclick", #"javascript:
if (typeof(Page_ClientValidate) == 'function')
{
if (Page_ClientValidate('" + this.ValidationGroup + "')){" + this.ClientID + ".disabled=true;" + Page.GetPostBackEventReference(this) +
"}else{return;}} else{" + this.ClientID + ".disabled=true;" + Page.GetPostBackEventReference(this) + "}");
}
else
{
this.Attributes.Add("onclick", #"javascript:
if (typeof(Page_ClientValidate) == 'function')
{
if (Page_ClientValidate()){" + this.ClientID + ".disabled=true;" + Page.GetPostBackEventReference(this) +
"}else{return;}} else{" + this.ClientID + ".disabled=true;" + Page.GetPostBackEventReference(this) + "}");
}
}
else
this.Attributes.Add("onclick", "javascript:" + this.ClientID + ".disabled=true;" + Page.GetPostBackEventReference(this));
base.OnPreRender(e);
}

This is the correct and simple way to do this:
Create a helper method in your application (say in a Utlity Namespace):
Public Shared Sub PreventMultipleClicks(ByRef button As System.Web.UI.WebControls.Button)
button.Attributes.Add("onclick", "this.disabled=true;" & button.Page.ClientScript.GetPostBackEventReference(button, String.Empty).ToString)
End Sub
Now from the code behind of each of your web pages you can simply call:
Utility.PreventMultipleClicks(button1)
where button1 is the the button you want to prevent multiple clicks.
What this does is simply sets the on click handler to: this.disabled=true
and then appends the buttons own post back handler, so we get:
onclick="this.disabled=true";__doPostBack('ID$ID','');"
This does not break the default behaviour of the page and works in all browsers as expected.
Enjoy!

If you disable the button then form submission will not happen. Correct way would be to set timer to disable the button. I would also suggest to use submit behavior instead of putting post-back event ref. For example,
function clickHandler(id, validate, validationGroup) {
var isValid = true;
if (validate && typeof(Page_ClientValidate) == 'function') {
isValid = validationGroup? Page_ClientValidate(validationGroup): Page_ClientValidate();
}
if (isValid)
{
// set timer to disable the button
var b = document.getElementById(id);
var f = function() { b.disabled = 'disabled'; };
setTimeout(f, 100);
return true;
}
return false;
}
And now attach function to your button
protected override void OnPreRender(EventArgs e)
{
this.Attributes.Add("onclick",
string.Format("return clickHandler('{0}', {1}, '{2}')",
this.ClientID, this.CausesValidation ? "true" : "false",
this.ValidationGroup));
}

If you disable a submit button once it is clicked, then it won't postback. I've been researching this for many many hours now, and the best solution I've seen is here. (The best solution is at the bottom of the page)
I'm now writing a custom server control to add to the toolbox that extends from Button, and uses a slightly modified version of this code. (overriding the OnLoad method)
I'm allowing the user the ability to change the 'processing...' text for something else and may make an attribute that allows the text to change to something else when submission is complete(which would be produced when the postback returns)

I have come up with the solution. Just hide the button after you click it. The postback will take place as usual. After completion of the postback, you will get your button as it is! See the detailed steps on how to disable asp.net button on postback and master the art!

Assuming this is asynchronous, and that you validate prior to saving, you can try as pseudo-follows:
bool hasSavedAlready = false;
savedata(){
if (!hasSavedAlready){
//normal saving code
...
//after success
hasSavedAlready = true;
}
}
That's the simplest solution I can think of.

I had this problem and none of the previous solutions worked for me but after some messing around I used:
<asp:LinkButton runat="server" OnClientClick="this.setAttribute('disabled','disabled'); this.text = 'Submit in progress...';" UseSubmitBehavior="false" ID="btnSubmit" ValidationGroup="formSubmit"</asp:LinkButton>

You need to consider that if the same session/user opens a page (example.whatever) on a window or tab lets say window1 ,and he again opens the same page (example.whatever) on window2.
When the user clicks the button on window1 and before 30 secs ends he clicks that button on window2 the server will think you clicked that button twice.
To avoid this in asp.net, i use :
Application["isButtonGettingClickedByAnyOne"] this an array that is shared bettwen all Sessions (all the users of your website).
or
Session["isButtonGettingClickedByThisUserInAnotherTab"] you can see it as shared bettwen all tabs (of the same user).
And make the server checks if they are reserved by some one or not.
if yes then wait 30sec + you turn ...
if no then you can 1st reserve it, and then execute your function, and when you finish the function free it so that others can use it again.
I hop this was useful and not painful to read.