Once again, I am new to C# and I am forcing myself through some exercises and putting the theory into practice. I am currently working on a small app to post the contents of a large csv file (2000+ lines) to a http API. The API is in the format
https://mydomain.com//api/dump/?
t = <app_token> // provided by auth function
&[
xml = <device_data_package>
OR
csv = <device_data_package>
My question is, how would i pass the csv contents to the body of the http POST? I appreciate your feedback and help.
Like other answers have said, assuming you have the code to open your file and read the contents, your CSV file will contain a string with values separated by commas. You can add it as a url parameter to give you something like this:
https://mydomain.com//api/dump/?t=my_token&csv=my,values,from,my,csv,file,go,here
However, there are limitations to the length of urls. So if you have anything other than a small CSV file, you are better off sending the CSV data as the body of the post, like you mention in your question.
The method below might come in handy. It takes an object as a parameter and embeds it into a post request and returns the response as a string. I have been using it on a site of mine to send data to an external service. It is part of a much bigger class, so I might have too many using statements.
You would use it like this:
WebUtilities.Post("https://mydomain.com//api/dump/?t=my_token", "csv=" + contents_of_my_csv_file);
The method looks like this:
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Linq;
using System.Net;
using System.Runtime.Serialization.Formatters.Binary;
using System.Security.Cryptography;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using System.Web;
using System.Web.Script.Serialization;
using System.Xml;
public static class WebUtilities
{
public static string Post(string url, object postData)
{
HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(url);
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
bf.Serialize(ms, postData);
byte[] data = ms.ToArray();
httpWReq.Method = "POST";
httpWReq.ContentType = "application/x-www-form-urlencoded";
httpWReq.ContentLength = data.Length;
using (Stream newStream = httpWReq.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
HttpWebResponse response = (HttpWebResponse)httpWReq.GetResponse();
Stream stream = response.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader streamReader = new StreamReader(stream, encode);
string html = streamReader.ReadToEnd();
response.Close();
streamReader.Close();
return html;
}
}
The API should provide some specifications, but usually it'll just be one giant comma separated string like the following:
string foo = "bar1, bar2, bar3, soomanybars, notabar";
Related
i can past this url into my browser and get the server time, https://api.binance.je/api/v3/time
But i am unable to get a response using below code. How can i debug this
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
// Create a request for the URL.
WebRequest request = WebRequest.Create("https://api.binance.je/api/v3/time");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Display the status.
Console.WriteLine(response.StatusDescription);
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseFromServer);
// Cleanup the streams and the response.
reader.Close();
dataStream.Close();
response.Close();
}
}
}
I see that you are able to retrieve the json data from the website. Tested this on my side.
However if you are trying to get the value only you need to read the json string (responsefromServer). This can be done by using a nuget pakage called Newtonsoft.
Then you will have to add the following to your code.
Above namespace:
using Newtonsoft.Json.Linq;
In main function before closing reader enz:
//Create jsonObject object from the api call response
JObject jObject = JObject.Parse(responseFromServer);
//Read propertie called serverTime and convert this to string to match variabele set
string time = jObject["serverTime"].ToString();
//Write the given time
Console.WriteLine(time);
Your code will look like the following:
using Newtonsoft.Json.Linq;
using System;
using System.IO;
using System.Net;
namespace StackOverflow
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
// Create a request for the URL.
WebRequest request = WebRequest.Create("https://api.binance.je/api/v3/time");
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Display the status.
Console.WriteLine(response.StatusDescription);
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseFromServer);
//Create jsonObject object from the api call response
JObject jObject = JObject.Parse(responseFromServer);
//Read propertie called serverTime and convert this to string to match variabele set
string time = jObject["serverTime"].ToString();
//Write the given time
Console.WriteLine(time);
// Cleanup the streams and the response.
reader.Close();
dataStream.Close();
response.Close();
}
}
}
The problem was down to my vpn. Even though i didnt have an active connection. I killed the background service and restarted it and is now working
Similar issue
httpclient-getasync-times-out-when-connected-to-vpn
I'm building a Windows Store app, but I'm stuck at getting a UTF-8 response from an API.
This is the code:
using (HttpClient client = new HttpClient())
{
Uri url = new Uri(BaseUrl + "/me/lists");
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Add("Accept", "application/json");
HttpResponseMessage response = await client.SendRequestAsync(request);
response.EnsureSuccessStatusCode();
string responseString = await response.Content.ReadAsStringAsync();
response.Dispose();
}
The reponseString always contains strange characters which should be accents like é, and I tried using a stream, but the API I found in some examples don't exist in Windows RT.
Edit: improved code, still same problem.
Instead of using response.Content.ReadAsStringAsync() directly you could use response.Content.ReadAsBufferAsync() pointed by #Kiewic as follows:
var buffer = await response.Content.ReadAsBufferAsync();
var byteArray = buffer.ToArray();
var responseString = Encoding.UTF8.GetString(byteArray, 0, byteArray.Length);
This is working in my case and I guess that using UTF8 should solve most of the issues. Now go figure why there is no way to do this using ReadAsStringAsync :)
Solved it like this:
using (HttpClient client = new HttpClient())
{
using (HttpResponseMessage response = await client.GetAsync(url))
{
var byteArray = await response.Content.ReadAsByteArrayAsync();
var result = Encoding.UTF8.GetString(byteArray, 0, byteArray.Length);
return result;
}
}
I like El Marchewko's approach of using an extension, but the code did not work for me. This did:
using System.IO;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
namespace WannaSport.Data.Integration
{
public static class HttpContentExtension
{
public static async Task<string> ReadAsStringUTF8Async(this HttpContent content)
{
return await content.ReadAsStringAsync(Encoding.UTF8);
}
public static async Task<string> ReadAsStringAsync(this HttpContent content, Encoding encoding)
{
using (var reader = new StreamReader((await content.ReadAsStreamAsync()), encoding))
{
return reader.ReadToEnd();
}
}
}
}
Perhaps the problem is that the response is zipped. If the content type is gzip, you will need decompress the response in to a string. Some servers do this to save bandwidth which is normally fine. In .NET Core and probably .NET Framework, this will automatically unzip the response. But this does not work in UWP. This seems like a glaring bug in UWP to me.
string responseString = await response.Content.ReadAsStringAsync();
This thread gives a clear example of how to decompress the response:
Compression/Decompression string with C#
The HttpClient doesn't give you a lot of flexibility.
You can use a HttpWebRequest instead and get the raw bytes from the response using HttpWebResponse.GetResponseStream().
Can't comment yet, so I'll have to add my thoughts here.
You could try to use _client.GetStringAsync(url) as #cremor suggested, and set your authentication headers using the _client.DefaultRequestHeaders property.
Alternatively, you could also try to use the ReadAsByteArrayAsync method on the response.Content object and use System.Text.Encoding to decode that byte array to a UTF-8 string.
My approach using an Extension:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Windows.Web.Http;
namespace yourfancyNamespace
{
public static class IHttpContentExtension
{
public static async Task<string> ReadAsStringUTF8Async(this IHttpContent content)
{
return await content.ReadAsStringAsync(Encoding.UTF8);
}
public static async Task<string> ReadAsStringAsync(this IHttpContent content, Encoding encoding)
{
using (TextReader reader = new StreamReader((await content.ReadAsInputStreamAsync()).AsStreamForRead(), encoding))
{
return reader.ReadToEnd();
}
}
}
}
okay so I have a c# console source code I have created but it does not work how I want it to.
I need to post data to a URL the same as if I was going to input it an a browser.
url with data = localhost/test.php?DGURL=DGURL&DGUSER=DGUSER&DGPASS=DGPASS
Here is my c# script that does not do it the way I want to I want it to post the data as if I had typed it like above.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Collections.Specialized;
using System.Net;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string URL = "http://localhost/test.php";
WebClient webClient = new WebClient();
NameValueCollection formData = new NameValueCollection();
formData["DGURL"] = "DGURL";
formData["DGUSER"] = "DGUSER";
formData["DGPASS"] = "DGPASS";
byte[] responseBytes = webClient.UploadValues(URL, "POST", formData);
string responsefromserver = Encoding.UTF8.GetString(responseBytes);
Console.WriteLine(responsefromserver);
webClient.Dispose();
}
}
}
I have also triead another method in c# this does now work either
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Collections.Specialized;
using System.Net;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string URI = "http://localhost/test.php";
string myParameters = "DGURL=value1&DGUSER=value2&DGPASS=value3";
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "text/html";
string HtmlResult = wc.UploadString(URI, myParameters);
System.Threading.Thread.Sleep(500000000);
}
}
}
}
I have been trying to figure a way to do this in my c# console for days now
Since what you seem to want is a GET-request with querystrings and not a POST you should do it like this instead.
static void Main(string[] args)
{
var dgurl = "DGURL", user="DGUSER", pass="DGPASS";
var url = string.Format("http://localhost/test.php?DGURL={0}&DGUSER={1}&DGPASS=DGPASS", dgurl, user, pass);
using(var webClient = new WebClient())
{
var response = webClient.DownloadString(url);
Console.WriteLine(response);
}
}
I also wrapped your WebClient in a using-statement so you don't have to worry about disposing it yourself even if it would throw an exception when downloading the string.
Another thing to think about is that you might want to url-encode the parameters in the querystring using WebUtility.UrlEncode to be sure that it doesn't contain invalid chars.
How to post data to a URL using WebClient in C#: https://stackoverflow.com/a/5401597/2832321
Also note that your parameters will not appear in the URL if you post them. See: https://stackoverflow.com/a/3477374/2832321
When I execute this C# code...
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.IO;
using System.Net;
using System.Dynamic;
using HtmlAgilityPack;
namespace entropedizer
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
public String postRequest(string url, string eventTarget)
{
// A "pre-request," sent to gather SessionID and POST data parameters for the main request
HttpWebRequest prequest = (HttpWebRequest)WebRequest.Create("http://www.entropedia.info/Chart.aspx?chart=Chart");
HttpWebResponse presponse = (HttpWebResponse)prequest.GetResponse();
Stream pstream = presponse.GetResponseStream();
StreamReader psr = new StreamReader(pstream);
string phtml = psr.ReadToEnd();
Match viewstate = Regex.Match(phtml, "id=\"__VIEWSTATE\".+/>");
Match eventvalidation = Regex.Match(phtml, "id=\"__EVENTVALIDATION\".+/>");
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "__EVENTTARGET=" + eventTarget + "&__VIEWSTATE=" + Uri.EscapeDataString(viewstate.ToString().Substring(24, viewstate.Length - 28)) + "&__EVENTVALIDATION=" + Uri.EscapeDataString(eventvalidation.ToString().Substring(30, eventvalidation.Length - 34));
byte[] data = encoding.GetBytes(postData);
// The main request, intended to retreive the desired HTML
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.entropedia.info/Chart.aspx?chart=Chart");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.CookieContainer = new CookieContainer();
Cookie sessionId = new Cookie("ASP.NET_SessionId", Regex.Match(presponse.Headers.ToString(), "ASP.NET_SessionId=.+ d").ToString().Substring(18, Regex.Match(presponse.Headers.ToString(), "ASP.NET_SessionId=.+ d").Length - 21), "/", ".entropedia.info");
request.CookieContainer.Add(new Uri("http://www.entropedia.info/Chart.aspx?chart=Chart"), sessionId);
Stream stream = request.GetRequestStream();
stream.Write(data, 0, data.Length);
stream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
stream = response.GetResponseStream();
StreamReader sr = new StreamReader(stream);
return sr.ReadToEnd();
}
private void Form1_Load(object sender, EventArgs e)
{
System.Net.ServicePointManager.Expect100Continue = false;
HtmlAgilityPack.HtmlDocument hChart = new HtmlAgilityPack.HtmlDocument();
hChart.LoadHtml(postRequest("http://www.entropedia.info/Chart.aspx?chart=Chart", "ctl00%24ContentPlaceHolder1%24DG1%24ctl19%24ctl05"));
HtmlNodeCollection chartStrings = hChart.DocumentNode.SelectNodes("/");
if (chartStrings != null)
{
foreach (HtmlNode i in chartStrings)
{
System.IO.File.WriteAllText("C:/Users/Admin/Desktop/WholeDocument.txt", i.OuterHtml);
}
}
else
{
MessageBox.Show("Error: Null item list.");
}
}
}
}
...the following HTML is written to a text file.
http://pastebin.com/FALerBWR
When I change the line in my C# code to HtmlNodeCollection chartStrings = hChart.DocumentNode.SelectNodes("/html/body"); the 400+ lines of HTML within the body is written to the text file instead.
When I change the line to HtmlNodeCollection chartStrings = hChart.DocumentNode.SelectNodes("/html/body/form"); only a single line of code (the form opening tag with its attributes) is written to the text file. It should write many lines (most of the document). I believe HtmlAgilityPack becomes confused due to the malformed HTML tags. Is there a way to programatically work around this? I do not want to correct the HTML manually every time I run the program!
This is a "by design" behavior. FORM is, by default, considered as an empty HTML element. The reasons are explained here on SO (check my answer): HtmlAgilityPack -- Does <form> close itself for some reason?
But this is also configurable, you just need to instruct the parser to behave differently, like this:
HtmlAgilityPack.HtmlDocument hChart = new HtmlAgilityPack.HtmlDocument();
// remove all specific behaviors for the `FORM` element
HtmlAgilityPack.HtmlNode.ElementsFlags.Remove("form");
hChart.LoadHtml(postRequest("http://www.entropedia.info/Chart.aspx?chart=Chart", "ctl00%24ContentPlaceHolder1%24DG1%24ctl19%24ctl05"));
HtmlNodeCollection chartStrings = hChart.DocumentNode.SelectNodes("/");
If you believe it is caused by faulty html, you tidy up the html first. Here's something you can use...
https://github.com/markbeaton/TidyManaged
So I am trying to create a simple class which I could use to consume REST web services. However I am having some troubles with HttpWebRequest object. Here is my code:
using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Ink;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;
namespace RichardKnop.Utils
{
public class REST
{
public void POST(string Uri)
{
}
public void GET(string Uri)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
// Set some reasonable limits on resources used by this request
request.MaximumAutomaticRedirections = 4;
request.MaximumResponseHeadersLength = 4;
// Set credentials to use for this request.
request.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine("Content length is {0}", response.ContentLength);
Console.WriteLine("Content type is {0}", response.ContentType);
// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
Console.WriteLine("Response stream received.");
Console.WriteLine(readStream.ReadToEnd());
response.Close();
readStream.Close();
}
}
}
However I am getting several errors - for example:
Error 4 'System.Net.HttpWebRequest' does not contain a definition for 'GetResponse' and no extension method 'GetResponse' accepting a first argument of type 'System.Net.HttpWebRequest' could be found (are you missing a using directive or an assembly reference?) C:\Users\Richard\Documents\Visual Studio 2010\Projects\RichardKnop\RichardKnop\Utils\REST.cs 31 65 RichardKnop
How is that possible that it does not contain definition of the GetResponse method when I can clearly see in the documentation that it does have a method like that? Here it is:
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.getresponse.aspx
Sorry if this is something trivial but I am new to .NET.
As far as i know Silverlight allows only asynchronous calls
try using BeginGetResponse. MSDN link