Develop Reference

Trouble making POST request from c# [.NET]

We have a created an API for the application which takes the image via POST request process it and sends the result in JSON format.
We tried calling API from different sources like python, postman app, c#. We can successfully call end point using python and postman app but with c# getting error
c# code [Not working]
byte[] img_data = System.IO.File.ReadAllBytes(#"file_path");
string url_ep = "http://ip:port/get";
Dictionary<string, byte[]> fl_image = new Dictionary<string, byte[]>();
fl_image.Add("image", img_data);
string data = JsonConvert.SerializeObject(fl_image);
var dataToSend = Encoding.UTF8.GetBytes(data);
var request = HttpWebRequest.Create(url_ep);
request.ContentType = "application/json";
request.ContentLength = dataToSend.Length;
request.Method = "POST";
request.GetRequestStream().Write(dataToSend, 0, dataToSend.Length);
var response = request.GetResponse();
System.IO.Stream dataStream = response.GetResponseStream();
System.IO.StreamReader reader = new System.IO.StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
Console.WriteLine(responseFromServer);
python code [working]
import requests
url = 'http://ip:port/get'
fl_image = {'image': open('file_path', 'rb')}
res = requests.post(url, files=fl_image)
print(res.json())
API Endpoint
from flask import Flask, request
import numpy as np
import cv2 as cv
#app.route('/get', methods = ['POST'])
def get_image():
if request.method == 'POST':
file = request.files['image']
# Read file
f = file.read()
# convert string of image data to uint8
f1 = np.fromstring(f, np.uint8)
# decode image
f2 = cv.imdecode(f1,cv.IMREAD_COLOR)

There are several issues with the way you are posting data from C#. The most relevant one is that you are trying to post a file as a JSON object, with file contents as string.
This cannot work: your python server is clearly expecting multipart/form-data as content-type.
I also strongly recommend you to use HttpClient and not the old HttpWebRequest class to send HTTP Requests.
var filePath = #"file_path";
var url = "http://ip:port/get";
using (var client = new HttpClient())
using (var content = new MultipartFormDataContent())
using (var fileStream = File.OpenRead(filePath))
{
var imageContent = new StreamContent(fileStream);
// NOTE: the line below is not required, but useful when you know the media type
imageContent.Headers.ContentType = MediaTypeHeaderValue.Parse("image/jpeg");
content.Add(imageContent, "image", Path.GetFileName(filePath));
var response = await client.PostAsync(url, content);
var stringResponse = await response.Content.ReadAsStringAsync();
// do what you need with the response
}
Other minor issues:
Do not read the entire file in memory (using File.ReadAllBytes), but open a stream for reading instead.
Use async/await when possible, do not block on async code (do not use .Result, .Wait() or .GetAwaiter().GetResult() on Task or Task<T>)
Always call Dispose() on IDisposable objects when you have finished using them (wrapping them inside a using block)

You need to dispose the connections
reader.Close();
dataStream.Close();
response.Close();
Hope this helps
Or try using HttpClient for .net within the using block

C# reading data off html

I've been trying to see if I could get timetable data of a school website, and make a little application of it. At the moment this is what I have :
string userInput = "/*My username will be here*/";
string passInput = "/*My password will be here */";
string formUrl = "https://portal.gc.ac.nz/student/index.php/process-login";
string formParams = string.Format("username={0}&password={1}", userInput, passInput);
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "https://portal.gc.ac.nz/student/index.php/timetable";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
I couldn't find a way to check if above code works, however my question is:
How can you access the data(texts) you want from the page? I want to get the subject names. Part of the html looks like this :

There are a few ways to do this: one would be regexp matching and taking the contents of the tags and another would be to just use HtmlAgilityPack library.
If you don't need to do it in C# I would strongly recommend a different language like Python or Perl. It seems to me that you are trying to scrape the data and in this case I strongly recommend to use the Scrapy framework from Python if possible. It's the best tool I encountered for scraping and you can use XPath to get your data easily. Here is the link to Scrapy's website.

Post JSON to Azure website

I have a string that uses JsonTextWriter to create a JSON formatted-string. How do I interact with it if I want to store it in an Azure website? I was thinking of using an httpWebRequest like
string webAddr = "http://{url to website}/test.json";
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(webAddr);
httpWebRequest.ContentType = "application/json; charset=utf-8";
httpWebRequest.Method = "POST";
StringWriter strwriter = new StringWriter();
JsonTextWriter writer = new JsonTextWriter(strwriter);
writer.WriteStartObject();
writer.WritePropertyName("id");
writer.WriteValue(v.id);
writer.WriteEndObject();
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
string json = strwriter.ToString();
streamWriter.Write(json);
}
But I can't seem to figure out how to actually post the JSON to a file. Am I missing anything?
I think that it would be fine to store on the local storage of the VM/website to avoid a CORS issue, unless there is something that Blob storage would benefit over local storage.

You're part way there, next you need to actually dispatch the request:
HttpWebResponse httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
And you can read through the returned body (if required):
using (httpWebResponse)
{
StreamReader reader = new StreamReader(httpWebResponse.GetResponseStream());
string response_body = reader.ReadToEnd();
}
Assuming you just wanted to dump the body to the file system (for arguments sake):
using (httpWebResponse)
{
...
using (var file = new FileStream("some\path\to\file.json",FileMode.Create,FileAccess.Write,FileShare.None))
{
StreamWriter writer = new StreamWriter(file,Encoding.UTF8);
writer.Write(response_body);
writer.Flush();
}
}
However this won't work for non VM websites, for that you'd probably have to shun the file off to blob storage.

json.Dumps - C# equivalent?

I have the following API - documentation which seems to be written in Python:
request = {
"username": "johndoe47",
"password": "secret12345"
}
apiConnection.request("POST", "/api/session?sid=" + sid, json.dumps(request))
response = json.loads(apiConnection.getresponse().read())
I need to access this REST-Service via C#
I am a bit unsure, what "json.dumps" does. I tried the following:
data="{\"Username\":\"johndoe47\",\"password\":\"secret12345\"}";
HttpWebRequest request=HttpWebRequest.Create("http://example.com/api/session?sid=workingsid");
request.Method = "POST";
request.ContentType = "text/plain;charset=utf-8";
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
byte[] bytes = encoding.GetBytes(data);
request.ContentLength = bytes.Length;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(bytes, 0, bytes.Length);
}
request.BeginGetResponse((x) =>
{
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(x))
{
StreamReader reader = new StreamReader(response.GetResponseStream());
retval = reader.ReadToEnd();
}
}, null);
I expect whatever json.dumps() does is not the same as I have done with "data=" as the server response is "missing username"

In python json.dump(s)(..) serializes a given Python data structure to JSON. The API provides two different calls, the first is the plane json.dump(..). This one writes the output of the serialization process to a file like object. The other one, json.dumps(..) returns the serialized data structure as a string. So in your example above, the content of the request is a string representing of a JSON serialized Python dictionary.

HTTP POST in .NET doesn't work

I've got a problem with creating an HTTP post request in .NET. When I do this request in ruby it does work.
When doing the request in .NET I get following error:
<h1>FOXISAPI call failed</h1><p><b>Progid is:</b> carejobs.carejobs
<p><b>Method is:</b> importvacature/
<p><b>Parameters are:</b>
<p><b> parameters are:</b> vacature.deelnemernr=478
</b><p><b>GetIDsOfNames failed with err code 80020006: Unknown name.
</b>
Does anyone knows how to fix this?
Ruby:
require 'net/http'
url = URI.parse('http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature')
post_args = {
'vacature.deelnemernr' => '478',
}
resp, data = Net::HTTP.post_form(url, post_args)
print resp
print data
C#:
Uri address = new Uri(url);
// Create the web request
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
// Set type to POST
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// Create the data we want to send
StringBuilder data = new StringBuilder();
data.Append("vacature.deelnemernr=" + HttpUtility.UrlEncode("478"));
// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
// Set the content length in the request headers
request.ContentLength = byteData.Length;
// Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
result = reader.ReadToEnd();
}
return result;

Don't you need the ? after the URL in order to do a post with parameters? I think that Ruby hides this behind the scenes.

I found the problem! The url variable in the C# code was "http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature/"
It had to be "http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature" without the backslash.