I'm making a tool in Unity to retrieve data from a server. The server's interface can provide URLs that we can later click on which will return an XML or CSV file with the results of that query from that server. But, it requires Basic Authentication. When clicking the links, it simply pops up a login screen before giving me the results. If I try what I [think] I know in Unity (starting with WebRequest.GetResponse()) it simply fails and says I am not authorized. It does not show the popup for authentication. So how do I let that login popup appear when accessing with Unity and await the login results to get the file? Or is there some standardized way to provide that info in the link itself?
Here is some code that should you get started. Just fill in the request link and username, password. please see the comments in the code to see what it does.
//try just in case something went wrong whith calling the api
try
{
//Use using so that if the code end the client disposes it self
using (HttpClient client = new HttpClient())
{
//Setup authentication information
string yourusername = "username";
string yourpwd = "password";
//this is when you expect json to return from the api
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
//add the authentication to the request
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(
System.Text.ASCIIEncoding.ASCII.GetBytes($"{yourusername}:{yourpwd}")));
//api link used to make the call
var requestLink = $"apiLink";
using (HttpResponseMessage response = client.GetAsync(requestLink).Result)
{
//Make sure the request was successfull before proceding
response.EnsureSuccessStatusCode();
//Get response from website and convert to a string
string responseBody = response.Content.ReadAsStringAsync().Result;
//now you have the results
}
}
}
//Catch the exception if something went from and show it!
catch (Exception)
{
throw;
}
This is what I ended up going with after looking at the comments above. Let me know if I'm doing anything terribly inefficient!
String username = "Superman"; // Obviously handled secretly
String pw = "ILoveLex4evar!"; // Obviously handled secretly
String url = "https://www.SuperSecretServer.com/123&stuff=?uhh";
String encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(username + ":" + pw));
CookieContainer myContainer = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Headers.Add("Authorization", "Basic " + encoded);
try
{
using (WebResponse response = request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (FileStream xml = File.Create("filepath/filename.xml"))
{
byte[] buffer = new byte[BufferSize];
int read;
while ((read = responseStream.Read(buffer, 0, buffer.Length)) > 0)
{
xml.Write(buffer, 0, read);
}
}
}
}
}
Related
It seems like the RTC server doesn't like my url request. I can essentially add whatever I want after the "host" part of the url and get the same result. So I'm guessing something about what I've got is wrong. Following the answer from my previous post, I'm pretty sure I have the right url from the "services" file in the <oslc_cm:simpleQuery><dc:title>Change request queries</dc:title> tag. So I'm not sure if there's something else it doesn't like? It no longer fails authentication and I'm now using form-based rather than basic, so I don't think it's authentication-related. It just seems to ignore anything and everything but still knows my credentials aren't wrong. Any ideas?
Update: I've also tried swapping all the colons with %3A as the Jazz documentation didn't seem particularly consistent in their examples if that was necessary or not. Same results though.
string host = "https://my.host.com:9443/ccm/";
string item = host + "oslc/contexts/_MySp3ci4lK3Y/workitems?" +
"oslc.where=dcterms:identifier=%222494443%22&" +
"oslc.properties=dcterms:title,dcterms:identifier&" +
"oslc.prefix=dcterms=%3Chttp://purl.org/dc/terms/%3E";
Debug.Log("Request");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(item);
request.Accept = "application/json";
request.Headers.Add("OSLC-Core-Version", "2.0");
WebResponse response = request.GetResponse();
string AuthHeader = response.Headers["X-com-ibm-team-repository-web-auth-msg"];
//check if authentication has failed
if ((AuthHeader != null) && AuthHeader.Equals("authrequired"))
{
Debug.Log("Authentication Required");
HttpWebRequest _formPost = (HttpWebRequest)WebRequest.Create(host + "authenticated/j_security_check"); // Same response without the "authenticated/j_security_check"
_formPost.Method = "POST";
_formPost.Timeout = 30000;
_formPost.Headers.Add("OSLC-Core-Version", "2.0");
_formPost.CookieContainer = request.CookieContainer;
_formPost.Accept = "text/xml";
_formPost.ContentType = "application/x-www-form-urlencoded";
Byte[] _outBuffer = Encoding.UTF8.GetBytes(credentials); //store in byte buffer
_formPost.ContentLength = _outBuffer.Length;
Stream _str = _formPost.GetRequestStream();
_str.Write(_outBuffer, 0, _outBuffer.Length); //update form
_str.Close();
//FormBasedAuth Step2:submit the login form and get the response from the server
HttpWebResponse _formResponse = (HttpWebResponse)_formPost.GetResponse();
string _rtcAuthHeader = _formResponse.Headers["X-com-ibm-team-repository-web-auth-msg"];
//check if authentication has failed
if ((_rtcAuthHeader != null) && _rtcAuthHeader.Equals("authfailed"))
{
Debug.Log("Authentication Failed");
return;
}
else
{
//login successful
// *** Still says AuthRequired here for some reason ***
Debug.Log("Auth Header = " + _rtcAuthHeader);
_formResponse.GetResponseStream().Flush();
_formResponse.Close();
//FormBasedAuth Step3: Resend the request for the protected resource.
response = (HttpWebResponse)request.GetResponse();
}
}
else if (AuthHeader == null)
{
Debug.Log("AuthHeader Null");
}
else
{
Debug.Log("AuthHeader = " + AuthHeader);
}
Debug.Log("Response Stream");
Stream responseStream = response.GetResponseStream();
byte[] buffer = new byte[BufferSize];
int read;
while ((read = responseStream.Read(buffer, 0, buffer.Length)) > 0)
{
// Prints out an HTML Doc rather than a JSON string.
Debug.Log(Encoding.UTF8.GetString(buffer));
}
This is what I've come to understand.
The comment "// * Still says AuthRequired here for some reason *" is telling of the fact that authorization is indeed not occurring. The header value for "X-com-ibm-team-repository-web-auth-msg" will indeed be null when it is officially no longer required.
It is failing because:
The _formPost itself needs basic authentication to post the form values
The CookieContainer is null. Creating a new CookieContainer allows for the authentication to proceed.
The "authenticated/j_security_check" is not correct. It should simply be "j_security_check".
When requesting the data a second time after authenticating, a new request must be created and using the CookieContainer from the original.
I have jobs in Jenkins that i cannot access unless i log in first using a my username and password.
For example if i try to access "localhost:xxx/job/some_job_1" i will get a 404 Error unless i log in first. And i say this because i have tried the following using WebRequest class:
string formParams = "j_username=bobbyLee&j_password=SecretPassword25&from=%2F&json=%7B%22j_username%22%3A+%bobbyLee%22%2C+%22j_password%22%3A+%22SecretPassword%25%22%2C+%22remember_me%22%3A+false%2C+%22from%22%3A+%22%2F%22%7D&Submit=log+in";
// ***this is the exact string that is sent when i log in normally, obtained using Fiddler***
string formUrl = "http://serverName:PortNum/j_acegi_security_check";
// ***I have also tried http://serverName:PortNum/login***
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "http://serverName:portNum/job/some_job/";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
The response that i get back from the POST request is "HTML OK", and cookieHeader is not null. But when i then try to make a GET request to get what i want, i get a 404 error when attempting to access the job "http://serverName:portNum/job/some_job/", as if i didn't log in successfully.
So what is the correct way to log into Jenkins from c#, and get the HTML source code of the jobs that only appears after logging in?
The RESTAPI is your best friend here.
It is an incredibly rich source of information. I have written a system that will show an entire program of work on a page with full deployment traceability.
I am going to assume you have some security in place in your Jenkins instance which means requests need to be authenticated.
I use the following class for this:
using System;
using System.Net;
using System.Text;
namespace Core.REST
{
public class HttpAdapter
{
private const string ApiToken = "3abcdefghijklmnopqrstuvwxyz12345"; // you will need to change this to the real value
private const string UserName = "restapi";
public string Get(string url)
{
try
{
const string credentials = UserName + ":" + ApiToken;
var authorization = Convert.ToBase64String(Encoding.ASCII.GetBytes(credentials));
using (var wc = new WebClient())
{
wc.Headers[HttpRequestHeader.Authorization] = "Basic " + authorization;
var htmlResult = wc.DownloadString(string.Format(url));
return htmlResult;
}
}
catch (WebException e)
{
Console.WriteLine("Could not retrieve REST API response");
throw e;
}
}
}
}
restapi is a dedicated user I created. I think I gave it admin access just so I didn't have to worry about it. I was admin but all the other developers and testers in the 3 crews had highly controlled and limited access to only what they needed and nothing more. It is also better practice to have a dedicated users for functions like this.
I constructed my c# classes to consume (deserialise) data from any page that supports the api/json suffix.
I am developing a C# application which needs to use the onelogin API to retrieve a session token. I am able to authenticate and and create a token with the following code:
WebRequest Authrequest = WebRequest.Create("https://api.us.onelogin.com/auth/oauth2/token");
Authrequest.Method = "POST";
Authrequest.ContentType = "application/json";
Authrequest.Headers.Add("cache-control", "no-cache");
Authrequest.Headers.Add("Authorization: client_id:XXXXXXX7bbf2c50200d8175206f664dc28ffd3ec66eef0bfedb68c3366420dc, client_secret:XXXXXXXXXX6ba2802187feb23f6450c6812b8e6639361d24aa83f12010f ");
using (var streamWriter = new StreamWriter(Authrequest.GetRequestStream()))
{
string Authjson = new JavaScriptSerializer().Serialize(new
{
grant_type = "client_credentials"
});
streamWriter.Write(Authjson);
}
WebResponse AuthReponse;
AuthReponse = Authrequest.GetResponse();
Stream receiveStream = AuthReponse.GetResponseStream ();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader (receiveStream);
JObject incdata = JObject.Parse(readStream.ReadToEnd());
string sToken = incdata["data"][0]["access_token"].Value<string>();
AuthReponse.Close();
However, when running the Create Session Login Token with the following code, it only returns a 400 error, and the message has no detail. Just Bad Request:
//Get the session token for the specified user, using the token recieved from previous web request
WebRequest request = WebRequest.Create("https://api.us.onelogin.com/api/1/login/auth");
request.Method = "POST";
request.ContentType = "application/json";
request.Headers.Add("authorization", "bearer:" + sToken);
using (var streamWriter2 = new StreamWriter(request.GetRequestStream()))
{
string json = JsonConvert.SerializeObject(new
{
username_or_email = sUsername,
password = sPassword,
subdomain = "comp-alt-dev"
});
streamWriter2.Write(json);
}
WebResponse response;
response = request.GetResponse();
string streamText = "";
var responseStream = response.GetResponseStream();
using (responseStream)
{
var streamReader = new StreamReader(responseStream);
using (streamReader)
{
streamText = streamReader.ReadToEnd();
streamReader.Close();
//
}
responseStream.Close();
}
Any ideas?
-Thank you
Also for anyone who may be getting this error. in C# the email is case sensitive. I tried User.email.com. In onelogin it was saved as user#email.com. changing the c# to lower case fixed it.
Can you let us know what payload you're sending across the wire to the .../1/login/auth endpoint as well as the response (either as others have suggested as packet snoop, or just as a debug output from the code)
400 means either bad json or the endpoint requires MFA, so this will narrow it down.
~thanks!
Just joining the troubleshooting effort =) -- I can replicate a 400 Bad Request status code with a "bad request" message when the request body contains a username_or_email and/or subdomain value that does not exist, or if the request body is empty.
Can you post what goes over the wire to the OneLogin endpoint...
OK Thanks. So it appears your subdomain does not exist. If you give me an email in the account I can find the correct subdomain value for you.
I need some images from a portal and they are only accessible if I login to the portal.
I need to do it with a C# program. I don't know what username field and password field are, because they use POST method. After loging in I want to enter some URLs that contain the images I want.
What should I do?
For logging in I'm using:
HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(#"http://mysite.com");
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "UsernameFieldName=Something";
postData += "&PasswordFieldName=SomethingElse";
byte[] data = encoding.GetBytes(postData);
httpWReq.Method = "POST";
httpWReq.ContentType = "application/x-www-form-urlencoded";
httpWReq.ContentLength = data.Length;
using (Stream stream = httpWReq.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
HttpWebResponse response = (HttpWebResponse)httpWReq.GetResponse();
string responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
Then for downloading image which is in another page I use:
using (var client = new WebClient())
{
string FileName = #"image.jpg";
client.DownloadFile("http://mysite.com/Image?imgCode=12345", FileName);
}
I don’t have a complete solution but here are some details to get you started.
Figure out what are post field simply by looking at page source
Once you send request for login you’ll also need to find a way to accept authentication cookie and then send it in all subsequent requests because their application most probably uses cookies.
After you are logged in you can download images like this
string imageFile = #"c:\image.jpg";
using (System.Net.WebClient client = new System.Net.WebClient())
{
client.DownloadFile("http://www.somewebsite.com/someimage.jpg", imageFile);
}
Here are couple examples to get you started with http posts in C#
HTTP request with post
I am getting The remote server returned an error: (400) Bad Request error while running the following code.
I am trying to upload xml file on the http server.
My xml file contains tag for the username,password and domain and when i am trying to connect is manually i am able to connect it,but using same credentials when i am trying to connect it through this code, i am getting 400 Bad Request error.
Please suggest me how to overcome this issue.
Thanks
`
public static void UploadHttp(string xml)
{
string txtResults = string.Empty;
try
{
string url = "http://my.server.com/upload.aspx ";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.KeepAlive = false;
request.SendChunked = true;
request.AllowAutoRedirect = true;
request.Method = "Post";
request.ContentType = "text/xml";
var encoder = new UTF8Encoding();
var data = encoder.GetBytes(xml);
request.ContentLength = data.Length;
var reqStream = request.GetRequestStream();
reqStream.Write(data, 0, data.Length);
reqStream.Close();
WebResponse response = null;
response = request.GetResponse();
var reader = new StreamReader(response.GetResponseStream());
var str = reader.ReadToEnd();
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError)
{
HttpWebResponse err = ex.Response as HttpWebResponse;
if (err != null)
{
string htmlResponse = new StreamReader(err.GetResponseStream()).ReadToEnd();
txtResults = string.Format("{0} {1}", err.StatusDescription, htmlResponse);
}
}
else
{
}
}
catch (Exception ex)
{
txtResults = ex.ToString();
}
}`
Are you sure you should be using POST not PUT?
POST is usually used with application/x-www-urlencoded formats. If you are using a REST API, you should maybe be using PUT? If you are uploading a file you probably need to use multipart/form-data. Not always, but usually, that is the right thing to do..
Also you don't seem to be using the credentials to log in - you need to use the Credentials property of the HttpWebRequest object to send the username and password.
400 Bad request Error will be thrown due to incorrect authentication entries.
Check if your API URL is correct or wrong. Don't append or prepend spaces.
Verify that your username and password are valid. Please check any spelling mistake(s) while entering.
Note: Mostly due to Incorrect authentication entries due to spell changes will occur 400 Bad request.
What type of authentication do you use?
Send the credentials using the properties Ben said before and setup a cookie handler.
You already allow redirection, check your webserver if any redirection occurs (NTLM auth does for sure). If there is a redirection you need to store the session which is mostly stored in a session cookie.
//use "ASCII" or try with another encoding scheme instead of "UTF8".
using (StreamWriter postStream = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.UTF8))
{
postStream.Write(postData);
postStream.Close();
}