Webrequest GET Body ViolationException - c#

I'm using SolrExpress to search and index documents within c# (dotnet core). Inserting (indexing) documents works fine since this is a nice post request.
However when i'm trying to do a select query (to retrieve documents) i'm getting aggregation exceptions. By digging down the source in SolrExpress i came upon the following source:
private WebRequest Prepare(SecurityOptions options, string requestMethod, string handler, string data)
{
var baseUrl = $"{this.HostAddress}/{handler}";
var encoding = new UTF8Encoding();
var bytes = encoding.GetBytes(data);
var request = WebRequest.Create(baseUrl);
if (options.AuthenticationType == AuthenticationType.Basic)
{
var encoded = Convert.ToBase64String(Encoding.GetEncoding("ISO-8859-1").GetBytes(options.UserName + ":" + options.Password));
request.Headers[HttpRequestHeader.Authorization] = "Basic " + encoded;
}
request.Method = requestMethod;
request.ContentType = "application/json";
#if NET451
request.ContentLength = bytes.Length;
#endif
#if NETCORE
var taskStream = request.GetRequestStreamAsync();
taskStream.Wait();
var stream = taskStream.Result;
stream.Write(bytes, 0, bytes.Length);
#else
var stream = request.GetRequestStream();
stream.Write(bytes, 0, bytes.Length);
stream.Close();
#endif
return request;
}
The method calling this method is the following for GET:
public string Get(SecurityOptions options, string handler, string data)
{
var request = this.Prepare(options, "GET-X", handler, data);
#if NETCORE
var task = this.ExecuteAsync(request, data);
task.Wait();
return task.Result;
#else
return this.Execute(request, data);
#endif
}
This Get method caused an error by using a request method GET-X which Solr itself (6.4.1) did not understand. I've changed this to a normal request method: GET therefore solving the error on solr's side.
However currently i'm getting a System.Net.ProtocolViolationException with the message: Cannot send a content-body with this verb-type. This is happening when waiting for the taskStream to finish and write its result to the request-body.
My question:
How would one send a GET-request with a body (in json format (as string)) within dotnet core?
Since RFC2616 says it's not forbidden i'd like to use this 'feature' as answered in the following question
See RFC2616 - Hypertext Transfer Protocol -- HTTP/1.1, section 4.3 "Message Body":
A message-body MUST NOT be included in a request if the specification of the > > request method (section 5.1.1) does not allow sending an entity-body in requests.
In section 9.3 "GET" including an entity-body is not forbidden.
So, yes, you are allowed to send an entity-body with a HTTP GET request.

Related

Microsoft Translator Bad request (400) issue

I've seen threads on this issue but my problem is particularly confusing. I have a free 2 million character subscription, a valid client id and secret. When I run my code I get to call the API a few times successfully (the most I've seen is 75 consecutive successful calls). Then every other call returns a Bad request response: The remote server returned an error: (400) Bad Request.
I create the token once with my credentials and never create it again. I loop through a file, parse it, and submit every parsed string for translation by calling the API. It seems that I reach some sort of limit that I'm now aware of.
When looking at my account, it doesn't seem to be discounting the characters that I've translated already which would make me highly suspicious that I have the wrong credentials when creating the token. I quadruple-checked that and everything seems to be ok.
Any guidance on what I may be missing here would be much appreciated.
Here's the code that creates the token. I do think though that there may be an unknown limitation that I'm not aware of with the free subscription.
static void gettoken()
{
//Get access token
string clientID = "my client id";
string clientSecret = "my secret";
String strTranslatorAccessURI = "https://datamarket.accesscontrol.windows.net/v2/OAuth2-13";
String strRequestDetails = string.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=http://api.microsofttranslator.com", clientID, clientSecret);
System.Net.WebRequest webRequest = System.Net.WebRequest.Create(strTranslatorAccessURI);
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(strRequestDetails);
webRequest.ContentLength = bytes.Length;
using (System.IO.Stream outputStream = webRequest.GetRequestStream())
{
outputStream.Write(bytes, 0, bytes.Length);
}
System.Net.WebResponse webResponse = webRequest.GetResponse();
System.Runtime.Serialization.Json.DataContractJsonSerializer serializer = new System.Runtime.Serialization.Json.DataContractJsonSerializer(typeof(AdmAccessToken));
AdmAccessToken token = (AdmAccessToken)serializer.ReadObject(webResponse.GetResponseStream());
MyGlobals.headerValue = "Bearer " + token.access_token;
}
And here's the code that calls the API itself. I call the API method from a loop.
static void RunBing(string sterm)
{
//Submit the translation request
string txtToTranslate = sterm;
string uri = "http://api.microsofttranslator.com/v2/Http.svc/Translate?text=" + txtToTranslate + "&from=en&to=es";
System.Net.WebRequest translationWebRequest = System.Net.WebRequest.Create(uri);
translationWebRequest.Headers.Add("Authorization", MyGlobals.headerValue);
System.Net.WebResponse response = null;
try {
response = translationWebRequest.GetResponse();
}
catch (Exception e)
{
Console.WriteLine("Term failed: " + sterm);
Console.WriteLine(e);
return;
}
System.IO.Stream stream = response.GetResponseStream();
System.Text.Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
System.IO.StreamReader translatedStream = new System.IO.StreamReader(stream, encode);
System.Xml.XmlDocument xTranslation = new System.Xml.XmlDocument();
xTranslation.LoadXml(translatedStream.ReadToEnd());
MyGlobals.xlation = xTranslation.InnerText;
}
After several successful calls to the API, I start to get the following message:
System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at Translate.TranslateText.Program.RunBing(String sterm)

C# Web API REST Service POST

I originally asked a question regarding a WCF web service that I was trying to write and then found that the ASP.net web API was more appropriate to my needs, due to some feedback on here.
I've now found a good tutorial that tells me how to create a simple REST service using Web API which works well pretty much out of the box.
My question
I have a POST method in my REST service server:
// POST api/values/5
public string Post([FromBody]string value)
{
return "Putting value: " + value;
}
I can POST to this using POSTER and also my C# client code.
However the bit I don't understand is why I have to prepend an '=' sign to the POST data so that it reads: "=Here is my data which is actually a JSON string"; rather than just sending: "Here is my data which is actually a JSON string";
My C# Client that talks to the REST service is written as follows:
public string SendPOSTRequest(string sFunction, string sData)
{
string sResponse = string.Empty;
// Create the request string using the data provided
Uri uriRequest = GetFormRequest(m_sWebServiceURL, sFunction, string.Empty);
// Data to post
string sPostData = "=" + sData;
// The Http Request obj
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriRequest);
request.Method = m_VERB_POST;
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
Byte[] byteArray = encoding.GetBytes(sPostData);
request.ContentLength = byteArray.Length;
request.ContentType = m_APPLICATION_FORM_URLENCODED;
try
{
using (Stream dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
sResponse = reader.ReadToEnd();
}
}
}
catch (WebException ex)
{
//Log exception
}
return sResponse;
}
private static Uri GetFormRequest(string sURL, string sFunction, string sParam)
{
StringBuilder sbRequest = new StringBuilder();
sbRequest.Append(sURL);
if ((!sURL.EndsWith("/") &&
(!string.IsNullOrEmpty(sFunction))))
{
sbRequest.Append("/");
}
sbRequest.Append(sFunction);
if ((!sFunction.EndsWith("/") &&
(!string.IsNullOrEmpty(sParam))))
{
sbRequest.Append("/");
}
sbRequest.Append(sParam);
return new Uri(sbRequest.ToString());
}
Is anybody able to explain why I have to prepend the '=' sign as in the above code (string sPostData = "=" + sData;)?
Many thanks in advance!
The content type x-www-form-urlencoded is a key-value format. With form bodies you are only able to read a single simple type from a request body. As a name is expected, but in this case not allowed, you have to prefix the equal sign to indicate that there is no name with the followed value.
However, you should lean away from accepting simple types within the body of your web-api controller actions.
You are limited to only a single simple type if you attempt to pass data in the body of an HttpPost/HttpPut without directly implementing your own MediaTypeFormatter, which is unlikely to be reliable. Creating a light-weight complex type is generally much more preferable, and will make interoperating with other content-types, like application/json, much easier.

request.Method = "POST" working but NOT request.ContentLength

Some times the simple things can stump you and here is one for me.
I want to do a simple web request to verify a username and password. Its working just fine in Windows Phone 8 but I can not seem to get the same code to work on Windows 8.
I understand I can not do a GetResponse as I do with Windows Phone so I am using GetResponseAsync and that part if working fine. But the response from the Server is that it did not get the "POST" component in the header.
Here is the Windows Phone 8 code that is working fine on my Phone version
private async void VerifyUser()
{
//System.Diagnostics.Debug.WriteLine("aa");
loginParams = "username=" + username + "&password=" + password;
string teamResponse = "https://mysite.com/mystuff/LoginApp?" + loginParams;
var request = HttpWebRequest.Create(teamResponse) as HttpWebRequest;
request.Method = "POST";
request.Accept = "application/json;odata=verbose";
var factory = new TaskFactory();
var task = factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
//System.Diagnostics.Debug.WriteLine("bb");
try
{
var response = await task;
System.IO.Stream responseStream = response.GetResponseStream();
string data;
using (var reader = new System.IO.StreamReader(responseStream))
{
data = reader.ReadToEnd();
}
responseStream.Close();
//System.Diagnostics.Debug.WriteLine("cc");
webData = data;
//MessageBox.Show(data);
}
catch (Exception e)
{
MessageBox.Show("There was a network error. Please check your network connectivty and try again " + e);
}
// System.Diagnostics.Debug.WriteLine(webData);
JToken token = JObject.Parse(webData);
string success = (string)token.SelectToken("success");
Here is what I have for Windows 8 using Visual Studio 2013
private async void VerifyUser()
{
string data;
loginParams = "username=" + logInUserIdString + "&password=" + logInPasswordString;
string teamResponse = "https://mysite.com/mystuff/LoginApp?" + loginParams;
Debug.WriteLine(teamResponse);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(teamResponse);
HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync();
using (var sr = new StreamReader(response.GetResponseStream()))
{
data = sr.ReadToEnd();
}
Debug.WriteLine(data);
}
That works but I get back a simple response advising only that the user is logged in or not logged in. The Server chap says that the request did not have "POST" in the header.
SO I added the following code:
request.Method = "POST";
request.Accept = "application/json;odata=verbose";
And here is the full code:
private async void VerifyUser()
{
string data;
loginParams = "username=" + logInUserIdString + "&password=" + logInPasswordString;
string teamResponse = "https://mysite.com/mystuff/LoginApp?" + loginParams;
Debug.WriteLine(teamResponse);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(teamResponse);
request.Method = "POST";
request.Accept = "application/json;odata=verbose";
HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync();
using (var sr = new StreamReader(response.GetResponseStream()))
{
data = sr.ReadToEnd();
}
Debug.WriteLine(data);
}
And then it just throws the following:
'web1.exe' (CLR v4.0.30319: Immersive Application Domain): Loaded 'C:\Windows\system32\WinMetadata\Windows.Foundation.winmd'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
A first chance exception of type 'System.Net.WebException' occurred in mscorlib.dll
An exception of type 'System.Net.WebException' occurred in mscorlib.dll but was not handled in user code
Additional information: The remote server returned an error: (411) Length Required.
So why can I not include the "POST" in the header as the doco says I can? Any help would be much appreciated.
UPDATE: I now know its an issue with the length of the loginParams. In my IOS, Android and WindowsPhone apps I did not have to specify the length and it works great, but the Visual Studio 2013 Windows App does not accept setting the content length for some reason.
Here is the error:
System.Net.HttpWebRequest does not contain a definition for ContentLength and no extension method ContentLength accepting a fist argument of type System.Net.HttpWebRequest could be found (are you missing a using directive or an assembly reference?)
So why can I not add request.ContentLength???
IN Response to Domniks request for the code I have attached a image of the block of code with the error message. Thanks again for helping.
I have attached an image of my screen where it wont acc
You are not really posting information since all your data is in the URL. You can try changing the method to GET, because that is what you are doing. Or you can write the post data to the request object's request stream and really POST. See here for quick example.
GET Parameter are URL-encoded like your URL: "https://mysite.com/mystuff/LoginApp?" + loginParams
That means you are always sending GET Parameter, just changing the Method do POST wont change anything.
If you want to send POST Parameters, do the following:
byte[] bytes = Encoding.UTF8.GetBytes(loginParams);
request.ContentLength = bytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(bytes, 0, bytes.Length);
stream.Close();
Solved!!!
The correct process to do Http web requests and response with Visual Studio 2013 is to use the new HttpClient class. I suspected it would be some new class like this but just could not find it. 2 days of my life have been wasted!!
So here is the correct code to do a Http request to for example log in or in my case to just verify the user is a valid user based on the userID and password.
private async void VerifyUser()
{
loginParams = "username=" + logInUserIdString + "&password=" + logInPasswordString;
string teamResponse = "https:// mySite.com?" + loginParams;
Debug.WriteLine(teamResponse);
HttpClient client = new HttpClient();
try
{
HttpResponseMessage response = await client.PostAsync(new Uri(teamResponse), null);
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
Debug.WriteLine(responseBody);
}
catch (HttpRequestException e)
{
Debug.WriteLine("\nException Caught!");
Debug.WriteLine("Message :{0} ", e.Message);
}
And thanks heaps to dbugger and Dominik for your help as your comments moved me in the direction that got me to this solution.

C# api responce and request

I currently have the code
try
{
string url = "http://myanimelist.net/api/animelist/update/" + "6.xml";
WebRequest request = WebRequest.Create(url);
request.ContentType = "xml/text";
request.Method = "POST";
request.Credentials = new NetworkCredential("username", "password");
byte[] buffer = Encoding.GetEncoding("UTF-8").GetBytes("<episode>4</episode>");
Stream reqstr = request.GetRequestStream();
reqstr.Write(buffer, 0, buffer.Length);
reqstr.Close();
MessageBox.Show("Updated");
}
catch (Exception s)
{
MessageBox.Show(s.Message);
}
I am trying do send data to myanimelist.net
The code they have written for is this
URL: http://myanimelist.net/api/animelist/update/id.xml
Formats: xml
HTTP Method(s): POST
Requires Authentication:true
Parameters:
id. Required. The id of the anime to update.
Example: http://myanimelist.net/api/animelist/update/21.xml
data. Required. A parameter specified as 'data' must be passed. It must contain anime values in XML format.
Response: 'Updated' or detailed error message.
The usage code example the have stated is this, does anyone know how to do this in c# or what was wrong with my original code?
Usage Examples:
CURL: curl -u user:password -d data="XML" http://myanimelist.net/api/animelist/update/21.xml
edit: When i lauch myanimelist.net it shows that it has not been updated, i am sure that my username and password credentials are correct
Edit 2 : I have now added a response which comes up with the error
"The remote server returned an error: (501) Not Implemented."
You're not actually performing the request, so once you're done writing to the request stream itself, perform the actual web request:
string result;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
}
Also, the content type should be text/xml or application/xml - the API may be complaining about that. Read the documentation for their API carefully and ensure what you're sending is correct.

Http Post for Windows Phone 8

I am new to C# so I was wondering if someone can help me out on this. I am trying to send HttpPost from Windows Phone 8 to the server. I found two examples that I would like to combine.
The first one is an example of sending Http Post (http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetrequeststream.aspx). The problem with this one is that it is not support by Windows Phone 8.
The second example is using the BeginGetResponse (http://msdn.microsoft.com/en-us/library/windowsphone/develop/system.net.httpwebrequest(v=vs.105).aspx). This supports windows phone 8.
I need to convert the second example into a BeginGetRequestStream() like the first example. I will try to figure out this myself, but I am posting online if someone already knows how to do this. I am sure this will be helpful for other WP8 developers.
Update
I am now trying to get response from the server. I have started a new question. Please follow this link (Http Post Get Response Error for Windows Phone 8)
I am also currently working on a Windows Phone 8 project and here is how I am posting to a server. Windows Phone 8 sort of has limited access to the full .NET capabilities and most guide I read say you need to be using the async versions of all the functions.
// server to POST to
string url = "myserver.com/path/to/my/post";
// HTTP web request
var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.ContentType = "text/plain; charset=utf-8";
httpWebRequest.Method = "POST";
// Write the request Asynchronously
using (var stream = await Task.Factory.FromAsync<Stream>(httpWebRequest.BeginGetRequestStream,
httpWebRequest.EndGetRequestStream, null))
{
//create some json string
string json = "{ \"my\" : \"json\" }";
// convert json to byte array
byte[] jsonAsBytes = Encoding.UTF8.GetBytes(json);
// Write the bytes to the stream
await stream.WriteAsync(jsonAsBytes, 0, jsonAsBytes.Length);
}
I propose a more generic asynchronous approach supporting success and error callbacks here:
//Our generic success callback accepts a stream - to read whatever got sent back from server
public delegate void RESTSuccessCallback(Stream stream);
//the generic fail callback accepts a string - possible dynamic /hardcoded error/exception message from client side
public delegate void RESTErrorCallback(String reason);
public void post(Uri uri, Dictionary<String, String> post_params, Dictionary<String, String> extra_headers, RESTSuccessCallback success_callback, RESTErrorCallback error_callback)
{
HttpWebRequest request = WebRequest.CreateHttp(uri);
//we could move the content-type into a function argument too.
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
//this might be helpful for APIs that require setting custom headers...
if (extra_headers != null)
foreach (String header in extra_headers.Keys)
try
{
request.Headers[header] = extra_headers[header];
}
catch (Exception) { }
//we first obtain an input stream to which to write the body of the HTTP POST
request.BeginGetRequestStream((IAsyncResult result) =>
{
HttpWebRequest preq = result.AsyncState as HttpWebRequest;
if (preq != null)
{
Stream postStream = preq.EndGetRequestStream(result);
//allow for dynamic spec of post body
StringBuilder postParamBuilder = new StringBuilder();
if (post_params != null)
foreach (String key in post_params.Keys)
postParamBuilder.Append(String.Format("{0}={1}&", key, post_params[key]));
Byte[] byteArray = Encoding.UTF8.GetBytes(postParamBuilder.ToString());
//guess one could just accept a byte[] [via function argument] for arbitrary data types - images, audio,...
postStream.Write(byteArray, 0, byteArray.Length);
postStream.Close();
//we can then finalize the request...
preq.BeginGetResponse((IAsyncResult final_result) =>
{
HttpWebRequest req = final_result.AsyncState as HttpWebRequest;
if (req != null)
{
try
{
//we call the success callback as long as we get a response stream
WebResponse response = req.EndGetResponse(final_result);
success_callback(response.GetResponseStream());
}
catch (WebException e)
{
//otherwise call the error/failure callback
error_callback(e.Message);
return;
}
}
}, preq);
}
}, request);
}

Categories

Resources