I must be missing something very obvious, but I can't tell what. I have a DoLoginAsync like so:
private async Task DoLoginAsync(bool force = false)
{
try
{
if (client.Cookies.ContainsKey("user_credentials") && !force)
{
return;
}
var html = client.Request("login").GetStringAsync().Result;
var doc = new HtmlDocument();
doc.LoadHtml(html);
var csrf_token = doc.DocumentNode.SelectNodes("//meta[#name='csrf-token']").First().GetAttributeValue("content", string.Empty);
var values = new Dictionary<string, string>
{
{ "user_session[email]", user },
{ "user_session[password]", password },
{ "authenticity_token", csrf_token }
};
var result = await client.Request("user_session").PostUrlEncodedAsync(values);
}
catch (Exception e)
{
}
When I run this code in a test with a breakpoint in the catch clause I get an exception
Call failed with status code 404 (Not Found): GET http://www.whatever.com/user_session
WTF? I'm expecting PostUrlEncodedAsync to do a POST, not a GET. Anybody have an idea why this can happen?
The Flurl client is instantiated as client = new FlurlClient(BASE_URL).EnableCookies();
UPDATE
Tried the following test which fails with the same exception
[TestMethod]
public async Task TheTest()
{
var message = "";
try
{
var client = new FlurlClient("http://www.slimmemeterportal.nl/").EnableCookies();
var html = await client.Request("login").GetStringAsync();
var doc = new HtmlDocument();
doc.LoadHtml(html);
var csrf_token = doc.DocumentNode.SelectNodes("//meta[#name='csrf-token']").First().GetAttributeValue("content", string.Empty);
var values = new Dictionary<string, string>
{
{ "user_session[email]", "******" },
{ "user_session[password]", "******" },
{ "commit", "inloggen" }, // Not sure if this is actually needed, but it is in the website's request parameters.
{ "authenticity_token", csrf_token }
};
var result = await client.Request("user_session").PostUrlEncodedAsync(values);
}
catch (FlurlHttpException ex)
{
message = ex.Message;
}
Assert.AreEqual("Call failed with status code 404 (Not Found): POST http://www.slimmemeterportal.nl/user_session", message);
}
Mystery solved: As it turns out after some debugging with Wireshark, the website was returning HTTP status code 301. As explained here the default action is to follow the URI in the response's location header using a GET even if the original request was a POST.
Related
I am using RabbitMQ.Client for messaging and using dead letter pattern to implement retry mechanism, everything works fine with RabbitMQ.Client with version :5.1.1 .
The pseudo-code is below：
public async Task Handle(BasicDeliverEventArgs basicDeliverEventArgs)
{
try
{
var bodyBytes = basicDeliverEventArgs.Body.ToArray();
var bodyString = Encoding.UTF8.GetString(bodyBytes);
// extension method with newtonsoft.json behind
bodyString.TryDeserializeObject<T>(out var message);
// do somesting
}
catch (Exception ex)
{
var properties = basicDeliverEventArgs.BasicProperties;
if (properties.Headers == null)
{
properties.Headers = new Dictionary<string, object>();
}
var bodyBytes = basicDeliverEventArgs.Body.ToArray();
var bodyString = Encoding.UTF8.GetString(bodyBytes);
// retrive the x-death-count by header
var retryTimes = GetRetryTimes(properties.Headers);
if (retryTimes <= _retryConfiguration.MaxCount)
{
using var channel = _conn.CreateModel();
var delay = _retryConfiguration.GetNextDelay(retryTimes);
properties.Expiration = delay.TotalMilliseconds.ToString();
string deadLetterExchangeName = $"ErrorExchange_{_rabbitSubscriptionConfiguration.Queue}";
string deadLetterQueueName = $"{_rabbitSubscriptionConfiguration.Queue}.Deadletter";
string deadLetterRoutingKey = $"{_rabbitSubscriptionConfiguration.RoutingKey}.Deadletter";
channel.ExchangeDeclare(deadLetterExchangeName, "direct", true, true);
channel.QueueDeclare(deadLetterQueueName, true, false, false, new Dictionary<string, object>()
{
{ "x-dead-letter-exchange", deadLetterExchangeName },
{ "x-dead-letter-routing-key",_rabbitSubscriptionConfiguration.RoutingKey}
});
channel.QueueBind(_rabbitSubscriptionConfiguration.Queue, deadLetterExchangeName, _rabbitSubscriptionConfiguration.RoutingKey);
channel.QueueBind(deadLetterQueueName, deadLetterExchangeName, deadLetterRoutingKey);
channel.BasicPublish(deadLetterExchangeName, deadLetterRoutingKey, false, properties, bodyBytes);
}
else
{
var messageStorage = _serviceProvider.GetService<IMessageStorage>();
await messageStorage.StoreDeadLetter(bodyString, ex);
}
}
}
However, when I upgrade the RabbitMQ.Client to 6.2.4,there is some probability that the body bytes of message unable deserialize to origin object when I receiving the message again from dead letter queue.
When the retry ended,I inspect the dead letter in the database,some of them failed to convert to a normal string
By the way,the RabbitMQ Server is 3.8.5
I've created this C# method to perform a JSON request getting the place_id from a text search. As far as I can tell the request itself works since I get a nullreferenceexception when trying to access non existent values where the message variable is set for example(Initially wrote "candidats" instead of "candidates"). However after correcting the typo it seems to only return from the catch block and I can't figure out why, what am I doing wrong here?
public async Task<string> GetPlaceAsync(string cityname, string countrycode)
{
string thereturn = "";
var client = new HttpClient();
var uri = new Uri("https://maps.googleapis.com/maps/api/place/findplacefromtext/json?input="+ cityname + "," + countrycode + "&inputtype=textquery&key="+apikey);
try
{
var response = await client.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
var result = await response.Content.ReadAsStringAsync();
var jsonData = (JObject)JsonConvert.DeserializeObject(result);
var status = jsonData["status"].Value<string>();
if (status != "ZERO_RESULTS")
{
var message = jsonData["candidates"]["place_id"].Value<string>();
System.Diagnostics.Debug.WriteLine(message);
return message;
}
else
thereturn = status;
}
else thereturn = "fail";
}
catch { thereturn = "Connection-error"; }
return thereturn;
}
edit: Since i sort of figured out what the problem is using David's answer of printing out the exception, we can move on to the next one which is not finding the place_id, here's the JSON data:
{
"candidates": [
{
"place_id": "ChIJFWGA6nAUz0URs78HFrUdA7c"
}
],
"status": "OK"
}
I have a lambda built with ApiGateway / Serverless.
It will execute a MySQL query. The MySQL Server is up and running and performing other queries with no issues, it seems that this is the only Lambda that returns an empty object.
It accepts a post request with a ID number in the body and then queries the DB and returns the object/data. Like before the server is running correctly. Just this returns and empty object.
I started to think that this maybe completing before the results were returned, I tried running this both sync and aync with the same results.
I am newish to c# so forgive if the code could be more efficient.
No clue as to why this returns a empty object everytime.
namespace Aws.Lambda
{
public class SearchHandler:BaseHandler
{
public async Task<APIGatewayProxyResponse> getId(LambdaRequest request)
{
var error = new Errorhandle();
var connect = new Connection(); //THIS IS THE DATABASE CONNECTION CLASS (WORKS)
var result = new List<ReturnTypes>();
var number = JsonConvert.DeserializeObject<bodyType>(request.body);
bool boolNumber = String.IsNullOrWhiteSpace(number.factura);
try
{
if(boolNumber)
{
return error.errorApiResponse("empty");
}
else
{
await Task.Yield(); // TRYING TO ASYNC / AWAIT THIS,
string x = $"SELECT * FROM Table1 WHERE ID_NUM ='{number}'";
Console.WriteLine(x); //LOGGING THE QUERY FOR ERRORS (NONE FOUND COPY AND PASTED QUERY AND IT WORKED IN WORKBENCH).
var results = connect.getData(x); //QUERY EXECUTE PASS TO FUNCTION MADE IN CLASS
if(!results.Read())
{
return error.errorApiResponse("reader is empty");
}
while (results.Read())
{
resultList.Add(insertData(results));
}
return new APIGatewayProxyResponse
{
StatusCode = 200,
Body = JsonConvert.SerializeObject(resultList),
Headers = headerList = new Dictionary<string, string>
{
{ "Content-Type", "application/json" },
{ "Access-Control-Allow-Origin", "*" },
{"Access-Control-Allow-Methods","OPTIONS,GET"}
};
};
}
} catch (Exception e)
{
return error.errorApiResponse("exception error");
throw e;
}
}
private static ReturnTypes insertData(System.Data.IDataReader results)
{
return new ReturnTypes { Num = results["ID"], Body = results["UUID"];
}
}
}
Solved!!! - See last edit.
In my MVC app I make calls out to a Web API service with HMAC Authentication Filterign. My Get (GetMultipleItemsRequest) works, but my Post does not. If I turn off HMAC authentication filtering all of them work. I'm not sure why the POSTS do not work, but the GETs do.
I make the GET call from my code like this (this one works):
var productsClient = new RestClient<Role>(System.Configuration.ConfigurationManager.AppSettings["WebApiUrl"],
"xxxxxxxxxxxxxxx", true);
var getManyResult = productsClient.GetMultipleItemsRequest("api/Role").Result;
I make the POST call from my code like this (this one only works when I turn off HMAC):
private RestClient<Profile> profileClient = new RestClient<Profile>(System.Configuration.ConfigurationManager.AppSettings["WebApiUrl"],
"xxxxxxxxxxxxxxx", true);
[HttpPost]
public ActionResult ProfileImport(IEnumerable<HttpPostedFileBase> files)
{
//...
var postResult = profileClient.PostRequest("api/Profile", newProfile).Result;
}
My RestClient builds like this:
public class RestClient<T> where T : class
{
//...
private void SetupClient(HttpClient client, string methodName, string apiUrl, T content = null)
{
const string secretTokenName = "SecretToken";
client.BaseAddress = new Uri(_baseAddress);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
if (_hmacSecret)
{
client.DefaultRequestHeaders.Date = DateTime.UtcNow;
var datePart = client.DefaultRequestHeaders.Date.Value.UtcDateTime.ToString(CultureInfo.InvariantCulture);
var fullUri = _baseAddress + apiUrl;
var contentMD5 = "";
if (content != null)
{
var json = new JavaScriptSerializer().Serialize(content);
contentMD5 = Hashing.GetHashMD5OfString(json); // <--- Javascript serialized version is hashed
}
var messageRepresentation =
methodName + "\n" +
contentMD5 + "\n" +
datePart + "\n" +
fullUri;
var sharedSecretValue = ConfigurationManager.AppSettings[_sharedSecretName];
var hmac = Hashing.GetHashHMACSHA256OfString(messageRepresentation, sharedSecretValue);
client.DefaultRequestHeaders.Add(secretTokenName, hmac);
}
else if (!string.IsNullOrWhiteSpace(_sharedSecretName))
{
var sharedSecretValue = ConfigurationManager.AppSettings[_sharedSecretName];
client.DefaultRequestHeaders.Add(secretTokenName, sharedSecretValue);
}
}
public async Task<T[]> GetMultipleItemsRequest(string apiUrl)
{
T[] result = null;
try
{
using (var client = new HttpClient())
{
SetupClient(client, "GET", apiUrl);
var response = await client.GetAsync(apiUrl).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
await response.Content.ReadAsStringAsync().ContinueWith((Task<string> x) =>
{
if (x.IsFaulted)
throw x.Exception;
result = JsonConvert.DeserializeObject<T[]>(x.Result);
});
}
}
catch (HttpRequestException exception)
{
if (exception.Message.Contains("401 (Unauthorized)"))
{
}
else if (exception.Message.Contains("403 (Forbidden)"))
{
}
}
catch (Exception)
{
}
return result;
}
public async Task<T> PostRequest(string apiUrl, T postObject)
{
T result = null;
try
{
using (var client = new HttpClient())
{
SetupClient(client, "POST", apiUrl, postObject);
var response = await client.PostAsync(apiUrl, postObject, new JsonMediaTypeFormatter()).ConfigureAwait(false); //<--- not javascript formatted
response.EnsureSuccessStatusCode();
await response.Content.ReadAsStringAsync().ContinueWith((Task<string> x) =>
{
if (x.IsFaulted)
throw x.Exception;
result = JsonConvert.DeserializeObject<T>(x.Result);
});
}
}
catch (HttpRequestException exception)
{
if (exception.Message.Contains("401 (Unauthorized)"))
{
}
else if (exception.Message.Contains("403 (Forbidden)"))
{
}
}
catch (Exception)
{
}
return result;
}
//...
}
My Web API Controller is defined like this:
[SecretAuthenticationFilter(SharedSecretName = "xxxxxxxxxxxxxxx", HmacSecret = true)]
public class ProfileController : ApiController
{
[HttpPost]
[ResponseType(typeof(Profile))]
public IHttpActionResult PostProfile(Profile Profile)
{
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
GuidValue = Guid.NewGuid();
Resource res = new Resource();
res.ResourceId = GuidValue;
var data23 = Resourceservices.Insert(res);
Profile.ProfileId = data23.ResourceId;
_profileservices.Insert(Profile);
return CreatedAtRoute("DefaultApi", new { id = Profile.ProfileId }, Profile);
}
}
Here is some of what SecretAuthenticationFilter does:
//now try to read the content as string
string content = actionContext.Request.Content.ReadAsStringAsync().Result;
var contentMD5 = content == "" ? "" : Hashing.GetHashMD5OfString(content); //<-- Hashing the non-JavaScriptSerialized
var datePart = "";
var requestDate = DateTime.Now.AddDays(-2);
if (actionContext.Request.Headers.Date != null)
{
requestDate = actionContext.Request.Headers.Date.Value.UtcDateTime;
datePart = requestDate.ToString(CultureInfo.InvariantCulture);
}
var methodName = actionContext.Request.Method.Method;
var fullUri = actionContext.Request.RequestUri.ToString();
var messageRepresentation =
methodName + "\n" +
contentMD5 + "\n" +
datePart + "\n" +
fullUri;
var expectedValue = Hashing.GetHashHMACSHA256OfString(messageRepresentation, sharedSecretValue);
// Are the hmacs the same, and have we received it within +/- 5 mins (sending and
// receiving servers may not have exactly the same time)
if (messageSecretValue == expectedValue
&& requestDate > DateTime.UtcNow.AddMinutes(-5)
&& requestDate < DateTime.UtcNow.AddMinutes(5))
goodRequest = true;
Any idea why HMAC doesn't work for the POST?
EDIT:
When SecretAuthenticationFilter tries to compare the HMAC sent, with what it thinks the HMAC should be they don't match. The reason is the MD5Hash of the content doesn't match the MD5Hash of the received content. The RestClient hashes the content using a JavaScriptSerializer.Serialized version of the content, but then the PostRequest passes the object as JsonMediaTypeFormatted.
These two types don't get formatted the same. For instance, the JavaScriptSerializer give's us dates like this:
\"EnteredDate\":\"\/Date(1434642998639)\/\"
The passed content has dates like this:
\"EnteredDate\":\"2015-06-18T11:56:38.6390407-04:00\"
I guess I need the hash to use the same data that's passed, so the Filter on the other end can confirm it correctly. Thoughts?
EDIT:
Found the answer, I needed to change the SetupClient code from using this line:
var json = new JavaScriptSerializer().Serialize(content);
contentMD5 = Hashing.GetHashMD5OfString(json);
To using this:
var json = JsonConvert.SerializeObject(content);
contentMD5 = Hashing.GetHashMD5OfString(json);
Now the sent content (formatted via JSON) will match the hashed content.
I was not the person who wrote this code originally. :)
Found the answer, I needed to change the SetupClient code from using this line:
var json = new JavaScriptSerializer().Serialize(content);
contentMD5 = Hashing.GetHashMD5OfString(json);
To using this:
var json = JsonConvert.SerializeObject(content);
contentMD5 = Hashing.GetHashMD5OfString(json);
Now the content used for the hash will be formatted as JSON and will match the sent content (which is also formatted via JSON).
I have this async method inside ASP.NET MVC 4 WEB API Controller that I got from this blog:
http://www.strathweb.com/2012/04/html5-drag-and-drop-asynchronous-multi-file-upload-with-asp-net-webapi/
public async Task<IList<RecivedFile>> Post()
{
List<RecivedFile> result = new List<RecivedFile>();
if (Request.Content.IsMimeMultipartContent())
{
try
{
MultipartFormDataStreamProvider stream = new MultipartFormDataStreamProvider(HostingEnvironment.MapPath("~/USER-UPLOADS"));
IEnumerable<HttpContent> bodyparts = await Request.Content.ReadAsMultipartAsync(stream);
IDictionary<string, string> bodyPartFiles = stream.BodyPartFileNames;
IList<string> newFiles = new List<string>();
foreach (var item in bodyPartFiles)
{
var newName = string.Empty;
var file = new FileInfo(item.Value);
if (item.Key.Contains("\""))
newName = Path.Combine(file.Directory.ToString(), item.Key.Substring(1, item.Key.Length - 2));
else
newName = Path.Combine(file.Directory.ToString(), item.Key);
File.Move(file.FullName, newName);
newFiles.Add(newName);
}
var uploadedFiles = newFiles.Select(i =>
{
var fi = new FileInfo(i);
return new RecivedFile(fi.Name, fi.FullName, fi.Length);
}).ToList();
result.AddRange(uploadedFiles);
}
catch (Exception e)
{
}
}
return result;
}
My question is why exactly does this method have a return type of Task? It is not clear "where to" or "to whom" it returns this task? It's like there is no one that waits for/receives the returned object.
I wonder what will be the implications if I return void like this:
EDIT:
I have tried the code below and it completely breaks the program. it's like the runtime looses the reference to the code, the code itself doesn't finish running.
public async void Post()
{
List<RecivedFile> result = new List<RecivedFile>();
if (Request.Content.IsMimeMultipartContent())
{
try
{
MultipartFormDataStreamProvider stream = new MultipartFormDataStreamProvider(HostingEnvironment.MapPath("~/USER-UPLOADS"));
IEnumerable<HttpContent> bodyparts = await Request.Content.ReadAsMultipartAsync(stream);
IDictionary<string, string> bodyPartFiles = stream.BodyPartFileNames;
IList<string> newFiles = new List<string>();
foreach (var item in bodyPartFiles)
{
var newName = string.Empty;
var file = new FileInfo(item.Value);
if (item.Key.Contains("\""))
newName = Path.Combine(file.Directory.ToString(), item.Key.Substring(1, item.Key.Length - 2));
else
newName = Path.Combine(file.Directory.ToString(), item.Key);
File.Move(file.FullName, newName);
newFiles.Add(newName);
}
}
catch (Exception e)
{
}
}
}
The ASP.NET runtime waits for it. You may find this video useful.
Most examples for async assume a UI context. ASP.NET also provides a context, but it's a "request" context - one for each HTTP request. My async/await post gives an overview of this "context", and the async/await FAQ goes into much more detail.
With return type void you don't wait for an object to return.. You wait for an operation to finish. You wait for your Task to finish.
If you return void, you will be returning 204 "No Content" Response message immediately regardless of the completion status of your asynchronous operation. This is done by the help of VoidResultConverter.
Note: On RC, you will see that it returns 200 "OK" response but with
the RTM, it will return 204 "No Content" response.