After I create connection, I need to send IDs and then store response, then send again and again (usually ~60-100 IDs). Currently I did it this way:
public void EThread()
{
HttpWebRequest HWRequest = (HttpWebRequest)WebRequest.Create(_AjaxURL);
HWRequest.Method = "POST";
HWRequest.ContentType = "text/xml; encoding='utf-8'";
string StrID;
while(!_RequiredIDs.IsEmpty)
if (_RequiredIDs.TryDequeue(out StrID))
Extract(StrID, ref HWRequest);
}
public void Extract(string ID, ref HttpWebRequest HWRequest)
{
string data = "strID=" + ID;
byte[] Bytes = Encoding.ASCII.GetBytes(data);
using (Stream SWriter = HWRequest.GetRequestStream())
{
SWriter.Write(Bytes, 0, Bytes.Length);
SWriter.Close();
}
HttpWebResponse resp = (HttpWebResponse)HWRequest.GetResponse();
using (Stream s = resp.GetResponseStream())
using (StreamReader sr = new StreamReader(s))
_RespnseCollection.Enqueue(sr.ReadToEnd());
}
However, I got this error: System.Net.WebException: 'The request was aborted: The connection was closed unexpectedly.. If I put HttpWebRequest inside Extract method it works, but that works long. This was checked on 1 thread. What is wrong with the code or the server does not support this.
The problem is that you are re-using the HWRequest. But an HttpWebRequest is a single use item. In other words you can fire off the request only once.
So if thr _RequiredIDs contains more than one item, the first item will work properly, but change the state of the HwRequest to completed (ie it now contains a response) and the HaveResponse property will be set to true.
If you then again execute GetRequestStream()or GetResponse() it determines that, having already received a response, it does not make sense to set up the request again.
So you could modify your code as follows:
public void EThread()
{
string StrID;
while(!_RequiredIDs.IsEmpty)
if (_RequiredIDs.TryDequeue(out StrID))
{
HttpWebRequest HWRequest = (HttpWebRequest)WebRequest.Create(_AjaxURL);
HWRequest.Method = "POST";
HWRequest.ContentType = "text/xml; encoding='utf-8'";
Extract(StrID, ref HWRequest);
}
}
BUT, I suppose this is really functionally equivalent to declaring the HttpRequest and initializing inside the Extract method.
SUGGESTIION:
Try calling HWRequest.Reset():
public void EThread()
{
HttpWebRequest HWRequest = (HttpWebRequest)WebRequest.Create(_AjaxURL);
HWRequest.Method = "POST";
HWRequest.ContentType = "text/xml; encoding='utf-8'";
string StrID;
while(!_RequiredIDs.IsEmpty)
if (_RequiredIDs.TryDequeue(out StrID))
{
Extract(StrID, ref HWRequest);
HWRequest.Reset();
}
}
Related
I have jobs in Jenkins that i cannot access unless i log in first using a my username and password.
For example if i try to access "localhost:xxx/job/some_job_1" i will get a 404 Error unless i log in first. And i say this because i have tried the following using WebRequest class:
string formParams = "j_username=bobbyLee&j_password=SecretPassword25&from=%2F&json=%7B%22j_username%22%3A+%bobbyLee%22%2C+%22j_password%22%3A+%22SecretPassword%25%22%2C+%22remember_me%22%3A+false%2C+%22from%22%3A+%22%2F%22%7D&Submit=log+in";
// ***this is the exact string that is sent when i log in normally, obtained using Fiddler***
string formUrl = "http://serverName:PortNum/j_acegi_security_check";
// ***I have also tried http://serverName:PortNum/login***
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "http://serverName:portNum/job/some_job/";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
The response that i get back from the POST request is "HTML OK", and cookieHeader is not null. But when i then try to make a GET request to get what i want, i get a 404 error when attempting to access the job "http://serverName:portNum/job/some_job/", as if i didn't log in successfully.
So what is the correct way to log into Jenkins from c#, and get the HTML source code of the jobs that only appears after logging in?
The RESTAPI is your best friend here.
It is an incredibly rich source of information. I have written a system that will show an entire program of work on a page with full deployment traceability.
I am going to assume you have some security in place in your Jenkins instance which means requests need to be authenticated.
I use the following class for this:
using System;
using System.Net;
using System.Text;
namespace Core.REST
{
public class HttpAdapter
{
private const string ApiToken = "3abcdefghijklmnopqrstuvwxyz12345"; // you will need to change this to the real value
private const string UserName = "restapi";
public string Get(string url)
{
try
{
const string credentials = UserName + ":" + ApiToken;
var authorization = Convert.ToBase64String(Encoding.ASCII.GetBytes(credentials));
using (var wc = new WebClient())
{
wc.Headers[HttpRequestHeader.Authorization] = "Basic " + authorization;
var htmlResult = wc.DownloadString(string.Format(url));
return htmlResult;
}
}
catch (WebException e)
{
Console.WriteLine("Could not retrieve REST API response");
throw e;
}
}
}
}
restapi is a dedicated user I created. I think I gave it admin access just so I didn't have to worry about it. I was admin but all the other developers and testers in the 3 crews had highly controlled and limited access to only what they needed and nothing more. It is also better practice to have a dedicated users for functions like this.
I constructed my c# classes to consume (deserialise) data from any page that supports the api/json suffix.
I'm a bit clueless on this so please excuse my lack of code examples.
Basically I've got a requirement to pass in a json 'string' array to a php page which in turn should give me back a response.
so firstly I need to create a string and serialize (?) this into a json object and pass it to a specified url....so firstly I'm not sure that I'm going about this in the correct way and then pass this to a URL:
protected void cmdConnect_Click(object sender, EventArgs e)
{
var sig = FormsAuthentication.HashPasswordForStoringInConfigFile(DateTime.Now.ToString("yyyy-MM-dd") + "!Cv*z]&z7:c,+zW", "SHA1");
var model = "Array (shortened for ease))";
var json = JsonConvert.SerializeObject(model);
string data = HttpUtility.UrlEncode(json);
byte[] byteArray = System.Text.Encoding.ASCII.GetBytes(data);
curlPost(data);
//jsonString.Text = json;
}
protected void curlPost(string data)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://dev.com/shopping_cart.php");
request.Method = "POST";
request.ContentType = "application/json";
byte[] buffer = UTF8Encoding.UTF8.GetBytes(data);
request.ContentLength = buffer.Length;
using (Stream oStream = request.GetRequestStream())
{
oStream.Write(buffer, 0, buffer.Length);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
string myResponse = reader.ReadToEnd();
jsonString.Text = myResponse;
}
}
But although I'm getting a response (myResponse) the data that I'm passing in has no impact on it so I'm thinking that I'm either not constructing it properly or I'm not passing it in at all. What I should get back is a 'key' but what I'm actually getting back is the HTML for the php page.
I know that this is really vague but a) this is all brand new to me and b) as such I'm not sure that of the terminology involved.
Any pointers would be greatly appreciated.
Thanks,
Craig
I have the following function which makes a HTTP request and returns the response body as a string:
private string getResponseBody(string method, string partialUrl)
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(_hostPath + partialUrl);
req.ContentLength = 0;
req.KeepAlive = false;
req.Method = method;
return new StreamReader(req.GetResponse().GetResponseStream()).ReadToEnd();
}
The first parameter, method, can only have the general HTTP methods as values (GET, POST, PUT, DELETE).
To force the input to be one of these values, I know I would need to create an object, but the details are escaping me. Can anyone help out?
PS: I'm using 4.0 framework
To force the input to be one of these values, I know I would need to create an object,
How about an enum:
public enum HttpMethod
{
GET,
POST,
PUT,
DELETE
}
and then:
private string getResponseBody(HttpMethod method, string partialUrl)
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(_hostPath + partialUrl);
req.ContentLength = 0;
req.KeepAlive = false;
req.Method = method.ToString();
using (var response = req.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
return reader.ReadToEnd();
}
}
Also notice how I have fixed the flaw you had in the GetResponse code where you didn't dispose any of the IDisposable objects which could result in leaks and all kind of nasty stuff. For example if you don't dispose the response object it could result in timeouts if this method is called many times.
I'm trying to get this correct as I feel I'm missing something. I want to use the keyword using whenever I have an IDisposable object. Please note that the code works, I just want to optimize it.
I have two questions here:
1) For this code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
What does it mean to add (HttpWebRequest) like that? Am I converting WebRequest to HttpWebRequest?
Why can't I do this?
HttpWebRequest rq = new HttpWebRequest();
rq.Create(url);
2) In the functional code below, how would I go about using the keyword using where applicable?
public static int UploadFileToixLibrary(string url, string file)
{
NetworkCredential credentials = new NetworkCredential();
credentials.UserName = AppVars.Username;
credentials.Password = AppVars.Password;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Credentials = credentials;
request.Method = "POST";
request.ContentType = "image/tiff";
request.Headers.Add("X-Object-Key", Path.GetFileName(file));
byte[] bytes = File.ReadAllBytes(file);
Stream st = null;
try
{
request.ContentLength = bytes.Length;
st = request.GetRequestStream();
st.Write(bytes, 0, bytes.Length);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return 1;
}
finally
{
if (st != null)
{
st.Close();
}
}
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream());
response.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return 1;
}
return 0;
}
Question 1:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
The reason for the cast, is that the static Create method on WebRequest returns an instance of WebRequest which is most appropriate for the scheme you supply in the url (ie, http:// address will return an HttpWebRequest, ftp:// will return FtpWebRequest etc). As you know your url is http, you know you'll get back an HttpWebRequest, so you can explicitly cast to the right type to get access to the extra functionality HttpWebRequest over the abstract WebRequest.
Now, WebRequest is not IDisposable, so you cannot use it inside a using statement!
Question 2: In your functional code, the only place you could use a using statement is around the Stream access:
try
{
request.ContentLength = bytes.Length;
st = request.GetRequestStream();
st.Write(bytes, 0, bytes.Length);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return 1;
}
finally
{
if (st != null)
{
st.Close();
}
}
Could be re-written:
request.ContentLength = bytes.Length;
using(var st = request.GetRequestStream())
{
st.Write(bytes, 0, bytes.Length);
st.Close();
}
WebRequest is a abstract class, it does not expose most of the properties/methods that you're going to use. You must cast it to the appropriate type to use it. The static method Create will return the specialized WebRequest object according to your URL. As you know that this URL uses the HTTP protocol (http://), it's safe to cast it to HttpWebRequest. It could, however, be a FtpWebRequest.
You can put the GetRequestStream initilization inside using(...) statements. Looks like WebRequest (and its children) does not implement IDisposable
1.
The Webrequest.Create method returns a WebRequest reference, but the actual object has different types depending on what kind of request you want to do. In your case you are doing a HTTP request, so the actual object is an HttpWebRequest object. You cast the reference to the actual type of the object to get access to the members that are specific for the HttpWebRequest class.
2.
The HttpWebRequest is not disposable. Use a using block for the HttpWebResponse object:
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) {
...
}
I've got a problem with creating an HTTP post request in .NET. When I do this request in ruby it does work.
When doing the request in .NET I get following error:
<h1>FOXISAPI call failed</h1><p><b>Progid is:</b> carejobs.carejobs
<p><b>Method is:</b> importvacature/
<p><b>Parameters are:</b>
<p><b> parameters are:</b> vacature.deelnemernr=478
</b><p><b>GetIDsOfNames failed with err code 80020006: Unknown name.
</b>
Does anyone knows how to fix this?
Ruby:
require 'net/http'
url = URI.parse('http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature')
post_args = {
'vacature.deelnemernr' => '478',
}
resp, data = Net::HTTP.post_form(url, post_args)
print resp
print data
C#:
Uri address = new Uri(url);
// Create the web request
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
// Set type to POST
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// Create the data we want to send
StringBuilder data = new StringBuilder();
data.Append("vacature.deelnemernr=" + HttpUtility.UrlEncode("478"));
// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
// Set the content length in the request headers
request.ContentLength = byteData.Length;
// Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
result = reader.ReadToEnd();
}
return result;
Don't you need the ? after the URL in order to do a post with parameters? I think that Ruby hides this behind the scenes.
I found the problem! The url variable in the C# code was "http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature/"
It had to be "http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature" without the backslash.