I need to post data to a website. So I created a small app in C#.net where I open this website and fill in all the controls (radio buttons, text boxes, checkboxes etc) with the values from my database. I also have a click event on the SUBMIT button. The app then waits for 10-15 seconds and then copies the response from the webpage to my database.
As you can see, this is really a hectic process. If there are thousands of records to upload, this app takes much longer (due to fact that it waits 15s for the response).
Is there any other way to post data? I am looking for something like concatenating all the fields with its value and uploading it like a stream of data. How will this work if the website is https and not http?
You can use HttpWebRequest to do this, and you can concatenate all the values you want to post into a single string for the request. It could look something like this:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.yoursite.com");
request.Method = "POST";
formContent = "FormValue1=" + someValue +
"&FormValue2=" + someValue2 +
"&FormValue=" + someValue2;
byte[] byteArray = Encoding.UTF8.GetBytes(formContent);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
WebResponse response = request.GetResponse();
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = HttpUtility.UrlDecode(reader.ReadToEnd());
//You may need HttpUtility.HtmlDecode depending on the response
reader.Close();
dataStream.Close();
response.Close();
This method should work fine for http and https.
MSDN has a great article with step-by-step instructions detailing how you can use the WebRequest class to send data. Link below:
http://msdn.microsoft.com/en-us/library/debx8sh9.aspx
Yes, there is a WebClient class. Look into documentation. There're some usful method to make GET and POST requests.
Related
I am working on getting information that is behind a log in page, and using this as my starting point.
Looking at the Network tab, I looked at the form data and saw there were 3 additional values than just client/password (csrf, time, hash).
I attempted to log into the site as follows.
string formUrl = "mysite_loginaction";
string formParams = string.Format("client_id={0}&password={1}", "client", "password");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
When I print out the resp to my console, it shows my the log in page, when i was expecting the next page after login (google 2f page).
Do I need to post a csfr, time, and hash values as well to get a successful login?
Like it has been mentioned in your link, there is a concept of sessionid token. If you do want to stay logged in, you need to pass that token everytime for the following http requests.
Also, the CSRF token will always be different each time you do the request, but you do need to pass it along your next request to be successful.
To know more about CSRF, I should redirect you to this link
You're going to have to mess around with it. Most of the time you don't need all the headers, but I would assume that hash is required.
I appreciate this is a little bit niche, but I thought I would ask anyway. I'm writing a small c# application to utilise the HMRC web portal and electronically submit VAT returns in XML format. According to the HMRC specification it is just a simple Http 1.1 POST action required, and retrieving the response in XML. The application is built, however, I am having trouble with this code. I get an "OK" HttpWebResponse, but the HMRC server returns this spurious error message which they can't seem to tell me what it means. Here is my code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(destinationUrl);
byte[] bytes;
bytes = System.Text.Encoding.ASCII.GetBytes(requestXml);
request.ContentType = "text/xml; charset='utf-8'";
request.ContentLength = bytes.Length;
request.Method = "POST";
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
HttpWebResponse response;
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
string responseStr = new StreamReader(responseStream).ReadToEnd();
return responseStr;
}
And the error is:
1001 - The submitted XML document either failed to validate against the GovTalk schema for this class of document or its body was badly formed.
I know the XML is ok, because when I test it using a third-party tool like "Postman" it submits 100% and the Transaction Engine returns no errors, so it must be my code. Does anything look glaringly wrong to post an XML? I have tried different Content/MIME Types and also I have confirmed that 'utf-8' is the correct encoding.
I was just wondering if any developers out there had worked on the Transaction Engine and could share their submit/post code?
The answer is that the code converting the file contents into the Byte Array was actually converting the file name.
I changed this line:
bytes = System.Text.Encoding.ASCII.GetBytes(requestXml);
To this:
bytes = File.ReadAllBytes(requestXml);
how would I be able to do something like http://myanimelist.net/modules.php?go=api#verifycred
aka "curl -u user:password http://myanimelist.net/api/account/verify_credentials.xml"
I wish to option the id
my code so far is
string url = "http://myanimelist.net/api/account/verify_credentials.xml";
WebRequest request = WebRequest.Create(url);
request.ContentType = "xml";
request.Method = "GET";
request.Credentials = new NetworkCredential(username, password);
byte[] buffer = Encoding.GetEncoding("UTF-8").GetBytes("xml/user/id"); // i think this line?
Stream reqstr = request.GetRequestStream();
reqstr.Write(buffer, 0, buffer.Length);
reqstr.Close();
but I get a error on "reqstr.Write(buffer, 0, buffer.Length)"
Cannot send a content-body with this verb-type, I have tried googling but with no prevail, I am using c#
Your snippet is trying to send a GET request with request data (you're calling GetRequestStream and then writing some data to the request stream). The HTTP protocol does not allow this - you can only send data with POST request.
However, the API that you are trying to call is actually doing something different - you do not need to send it the XML data. The XML data (with user ID and user name) is the response that you get when you successfully login.
So, instead of calling GetRequestStream and writing the XML data, you need to call GetResponse and then GetResponseStream to read the XML data!
I have a silverlight application which calls a web page, on clikc of a button. I want to send some parameters to the page on click of the button.
Now I can send it via query string but I don't want to do it as I might want to send a list of users which can go lenghty.
Also using sessions is not an option as these are two different applications. Also on click of the button we have to do some operations and display the results in the web page.
Is there a way I can call the web method in the page - Do my operation and then then show the details of the operation on my web page.
Maybe you could use something like the WebRequest class to create a POST request and then use the HttpWebResponse to get the details.
var request = (HttpWebRequest) WebRequest.Create(Uri);
request.Method = "POST";
var postData = string.Format("param1={0}¶m2={1}", "value1", "value2");
var data = Encoding.UTF8.GetBytes(postData);
request.ContentLength = data.Length;
var requestStream = request.GetRequestStream();
requestStream.Write(data, 0, data.Length);
requestStream.Close();
var response = request.GetResponse() as HttpWebResponse;
I found this snippet of code here that allows you to log into a website and get the response from the logged in page. However, I'm having trouble understanding all the part of the code. I've tried my best to fill in whatever I understand so far. Hope you guys can fill in the blanks for me. Thanks
string nick = "mrbean";
string password = "12345";
//this is the query data that is getting posted by the website.
//the query parameters 'nick' and 'password' must match the
//name of the form you're trying to log into. you can find the input names
//by using firebug and inspecting the text field
string postData = "nick=" + nick + "&password=" + password;
// this puts the postData in a byte Array with a specific encoding
//Why must the data be in a byte array?
byte[] data = Encoding.ASCII.GetBytes(postData);
// this basically creates the login page of the site you want to log into
WebRequest request = WebRequest.Create("http://www.mrbeanandme.com/login/");
// im guessing these parameters need to be set but i dont why?
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
// this opens a stream for writing the post variables.
// im not sure what a stream class does. need to do some reading into this.
Stream stream = request.GetRequestStream();
// you write the postData to the website and then close the connection?
stream.Write(data, 0, data.Length);
stream.Close();
// this receives the response after the log in
WebResponse response = request.GetResponse();
stream = response.GetResponseStream();
// i guess you need a stream reader to read a stream?
StreamReader sr = new StreamReader(stream);
// this outputs the code to console and terminates the program
Console.WriteLine(sr.ReadToEnd());
Console.ReadLine();
A Stream is a sequence of bytes.
In order to use text with a Stream, you need to convert it into a sequence of bytes.
This can be done manually with the Encoding classes or automatically with StreamReader and StreamWriter. (which read and write strings to streams)
As stated in the documentation for GetRequestStream,
You must call the Stream.Close method to close the stream and release the connection for reuse. Failure to close the stream causes your application to run out of connections.
The Method and Content-* properties reflect the underlying HTTP protocol.