Develop Reference

How to do a durable post call in an ServiceBusTrigger azure function?

Hey I'm using a ServiceBusTrigger azure function to get the messages received in a queue, and then send them to my webapi which is going to do some stuff with that content
[FunctionName("MyAzureFunction")]
public async void Run(
[ServiceBusTrigger("<MyQueue>", Connection = "<MyConnectionString>")] Message myQueueItem, ILogger log)
{
log.LogInformation($"C# ServiceBus queue trigger function processed message: {myQueueItem.ToString()}");
var client = new HttpClient();
// Retrieving the string content from the message
var bodyMessage = Encoding.UTF8.GetString(myQueueItem.Body);
// Calling my API to do something based on the message content
var response = await client.PostAsync("<MyAPIUrl>", new StringContent(bodyMessage, Encoding.UTF8, "application/json"));
// doing something based on the response
}
I've been reading about azure functions and in order to it gets cheaper I read about durable functions, I'm looking forward how to use them so I can take decisions based on my response and I can get it working with this ServiceBusTrigger

Needs to be changed the current ServiceBusTrigger function so it calls another Function that will actually do the job:
[FunctionName("MyAzureFunction")] public async void Run(
[ServiceBusTrigger("<MyQueue>", Connection = "<MyConnectionString>")] Message myQueueItem,
[DurableClient] IDurableOrchestrationClient orchestratorClient,
ILogger log) {
log.LogInformation($"C# ServiceBus queue trigger function processed message: {myQueueItem.ToString()}");
// Here is where you need to specify in the first parameter the name of the function to be called
// and the last parameter are the params you'll send to that one
var instanceId = await orchestratorClient.StartNewAsync("MyPostFunction", null, myQueueItem);
log.LogInformation($"C# ServiceBus queue trigger function created an async instance of 'MyPostFunction' with the ID: {instanceId}");
}
Then is needed to create another function that will be OrchestrationTrigger type, that will look like this:
[FunctionName("MyPostFunction")] public async void RunOrchestrator([OrchestrationTrigger] IDurableOrchestrationContext context) {
// using the context can be retrieved the parammeters passed in the function above
// in this case I just specify the type of that one and that's it
var myQueueItem = context.GetInput<Message>();
var bodyMessage = Encoding.UTF8.GetString(myQueueItem.Body);
// Create a URI of your API url
var postUri = new Uri($"<MyAPIUrl>");
// depending on your WebAPI you'll need to specify the content type in the headers
var headers = new Dictionary<string, StringValues>() { { "Content-Type", "application/json" } };
// creating durable http request
var request = new DurableHttpRequest(HttpMethod.Post, postUri, headers, bodyMessage);
// Doing the http call async, in this context you'll save money since your function will not be completely waiting for a response
// this one will keep just checking to see if there's a response available or not
var response = await context.CallHttpAsync(request);
// do your stuffs depending in the response
}
In my case I had to specicify the headers in the request, otherwise I used to get 415 Unsupported Media Type can be done in that way, or just creating the request without specifying any header at the begining and then adding those like this:
var request = new DurableHttpRequest(HttpMethod.Post, postUri, null, bodyMessage);
request.Headers.Add("Content-Type", "application/json");
Both options work

Unclear about what to pass through a given Action<T,R>

I am attempting to implement the library RestSharp.Core in a .NET Core application; But it has no documentation.
I need to implement this method;
public virtual RestRequestAsyncHandle ExecuteAsyncPost(IRestRequest request, Action<IRestResponse, RestRequestAsyncHandle> callback, string httpMethod)
I cannot figure out what to pass for the second parameter. This is the code I was using in normal RestSharp before I had to start converting it to .NET Core.
var httpResponse = await httpClient.ExecutePostTaskAsync(httpRequest);
var deserializer = new RestSharp.Deserializers.JsonDeserializer();
return deserializer.Deserialize<Dictionary<string, string>>(httpResponse);
Any ideas? I'm really confused.

RestSharp uses a callback-style kind of asynchrony which is very common in other platforms (e.g., Node), but never really caught on in .NET.
In order to implement ExecuteAsyncPost, you will need to take the request and httpMethod parameters to start an asynchronous post and then return a RestRequestAsyncHandle representing that operation.
Then, when the post completes, you should build an IRestResponse instance and pass that along with the RestRequestAsyncHandle to the callback. The RestRequestAsyncHandle you pass to callback should be the same instance as the one you already returned from ExecuteAsyncPost.
Since RestRequestAsyncHandle appears to be built around HttpWebRequest, you should be able to do something like:
public override RestRequestAsyncHandle ExecuteAsyncPost(IRestRequest request, Action<IRestResponse, RestRequestAsyncHandle> callback, string httpMethod)
{
HttpWebRequest webRequest = /* Construct webRequest from request/httpMethod */
var result = new RestRequestAsyncHandle(webRequest);
DoWebRequest(webRequest, result, callback);
}
private async void DoWebRequest(HttpWebRequest webRequest, RestRequestAsyncHandle result, Action<IRestResponse, RestRequestAsyncHandle> callback)
{
IRestResponse response;
try
{
var webResponse = await webRequest.GetResponseAsync().ConfigureAwait(false);
response = /* Construct an IRestResponse using webResponse */
}
catch (Exception ex)
{
response = /* Construct an IRestResponse with error information */
}
callback(response, result);
}
Note that I'm purposely using async void here because we're implementing a callback-based asynchronous system, and I want exceptions from the callback method to be handled at a top level. In the vast majority of normal async/await usage, you should avoid async void.

The second parameter is an Action (method) that takes a RestRequestAsyncHandle and returns a IRestResponse.
In other words, this method is looking for a method to call as a call back when it's done.

Asynchronous Service Layer underneath Web API 2.0 Controllers (.NET)

In the context of .NET...
For a web application I'm developing, I have the following architecture (from topmost to bottommost, loosely speaking):
Web API 2.0 controllers in the Web layer
Service layer (more or less one-to-one with controllers; e.g. ProjectsController calling into a ProjectsService)
Repository layer, which incorporates the Model layer and references to the .NET RavenDB Client
Ninject for DI
All calls by the controllers into the service layer return something: Create returns the object POST-ed; Update returns the object PUT-ed; Delete returns a copy of the object DELETE-ed, etc.)
All calls from the client (javascript in the browser) into the Web API are AJAX
Focusing now on one controller in particular and two services, I have the following: A ProjectsController ("PC") calling into the ProjectsService ("PS"), which has a SmartyStreetsGeocodingService ("SSGS") injected into it and that the ProjectsService calls. I wrote SSGS as a synchronous service initially just for testing purposes. But now the time has come to convert the sync call out to the Smarty Streets RESTful API to an async call. I'm having a timing issue right now that is related to synchronicity, and the GeocodingResult it processes too soon.
My question really revolves around architecture. Also, I would like to stay within the confines of just converting from sync to async (I'm aware of other approaches, such as .NET Service Bus, SignalR, etc., all of which I'll look at later).
What exactly should be async?
Consider the following code from the SSGS:
using (var response = request.GetResponse() as HttpWebResponse)
{
if (response == null) return geocodingResult;
var jsonSerializer = new DataContractJsonSerializer(typeof(CandidateAddress[]));
var stream = response.GetResponseStream();
if (stream == null) return geocodingResult;
var objResponse = jsonSerializer.ReadObject(stream);
jsonResponse = objResponse as CandidateAddress[];
}
if (jsonResponse == null || jsonResponse.Length == 0) //<-- Executes too soon
{
geocodingResult.IsVerified = false;
return geocodingResult;
}
geocodingResult.IsVerified = true;
The fragment above is what I wish to convert to async as the portion of code immediately following the using statement executes too soon. But that leads to questions:
When I do convert to async, what should be asynchronous exactly?
Just the method in which the call to SSGS is located?
Should the PS itself make an async call into the SSGS?
Does the PC need to be async as well?
Should the call from the PC into the PS be async?
Thank you.

The most important thing that should be asynchronous in your case is the HTTP call to the remote service. From there on, everything that is calling this service should of course be asynchronous.
At the end of the day you should have an asynchronous controller action in your outermost layer (the Web API):
public async Task<HttpResponseMessage> Get()
{
var request = WebRequest.Create("http://google.com");
using (var response = await request.GetResponseAsync())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
string responseContent = reader.ReadToEnd();
return Request.CreateResponse(HttpStatusCode.OK, responseContent);
}
}
So as you can see from this controller action I have used an asynchronous Httprequest. So in order to achieve that in your current design you need to make everything asynchronous and expose async methods in your service layer as well so that you can async/await on the result that's gonna get you the JSON from the service layer.
In my example I have simply returned a string from the remote HTTP call but you could easily adapt your service layer to return complex objects as well:
public async Task<MyModel> GetMyModelfromRemoteServiceCall();
{
var request = WebRequest.Create("http://google.com");
using (var response = await request.GetResponseAsync())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
string responseContent = reader.ReadToEnd();
return JsonConvert.DeserializeObject<MyModel>(responseContent);
}
}
You could then trivially easy await on this service layer method from your async Web API controller action.

Http POST using Task.FromAsync with async request and async response

I have to send an async POST to a web service and I want to send/retrieve the data asynchronously.
I also need to send the request but only wait a maximum of 1500 ms waiting for a response. If I don't get a response, the program should continue on (this is a service making an external web service call). I want to offload these service calls to IOCP's instead of blocking for a long time and waiting for them to return. I only want to block for 1500 ms total.
Here's what I have so far:
var httpRequest = (HttpWebRequest)WebRequest.Create(#"urltoPostTo");
httpRequest.Method = "POST";
byte[] data = Encoding.ASCII.GetBytes("test-post");
httpRequest.ContentLength = data.Length;
var asyncTask = Task.Factory.FromAsync<Stream>(httpRequest.BeginGetRequestStream, httpRequest.EndGetRequestStream, httpRequest)
.ContinueWith(response =>
{
var localStream = response.Result;
if (localStream != null)
{
localStream.Write(data, 0, data.Length);
localStream.Close();
}
}); //how do I do the continuation for BeginGetResponse and EndGetResponse from here?
I have a couple of requirements that unfortunately I can't change.
I am using Visual Studio 2010 targeting 4.0
I cannot use the Async BCL
I would like to try and use Task.FromAsync

This has already been answered in Implementing extension method WebRequest.GetResponseAsync with support for CancellationToken with a GetResponseAsync extension method that properly handles timeouts and CancellationTokens. When Request.Timeout expires, the method calls Request.Abort before returning the Task in its cancelled state.
The reason for such involved coding is that it's the client's (yours) responsibility to properly handle timeouts, so you can't depend on FromAsync to handle the timeout expiration. Perhaps this is the reason why FromAsync doesn't accept a cancellation token.
Another option is to avoid cancelling the request itself and cancel the continuation. You can use the ContinueWith overload that accepts a CancellationToken and call CancellationTokenSource.CancelAfter to set the cancellation timeout.
This would allow your code to ignore the results and keep running but it wouldn't break the connection to the server and wouldn't stop the background thread from processing any potentially expensive results.
You could write something like this:
var tcs=new CancellationTokenSource();
var asyncTask = Task.Factory.FromAsync<Stream>(httpRequest.BeginGetRequestStream, httpRequest.EndGetRequestStream, httpRequest)
.ContinueWith(response =>
{...},
cts.Token);
cts.CancelAfter(1500);
Note that the call to CancelAfter is done after starting the asynchronous task.
I would prefer Reed Copsey's extension method in a busy site because a high number of cancelled but outstanding requests can easily exhaust the thread pool, consume a lot of memory without a reason and consume potentially expensive connections to external systems.

why not to work with HttpClient?
webApiHttpClient.PostAsJsonAsync(GetFullAPI("api/Subscribe"), obj)
.ContinueWith(res => _logger.InfoFormat("Subscribe result: {0}", res.Result.StatusCode));

Try this method helper.
public static Task<string> Post(string url, Encoding encoding, string content)
{
var httpRequest = (HttpWebRequest)WebRequest.Create(url);
httpRequest.Method = "POST";
byte[] data = encoding.GetBytes(content);
httpRequest.ContentLength = data.Length;
TaskCompletionSource<string> result = new TaskCompletionSource<string>();
Task.Factory.FromAsync<Stream>(httpRequest.BeginGetRequestStream, httpRequest.EndGetRequestStream, httpRequest)
.ContinueWith(requestStreamTask =>
{
try
{
using (var localStream = requestStreamTask.Result)
{
localStream.Write(data, 0, data.Length);
localStream.Flush();
}
Task.Factory.FromAsync<WebResponse>(httpRequest.BeginGetResponse, httpRequest.EndGetResponse, httpRequest)
.ContinueWith(responseTask =>
{
try
{
using (var webResponse = responseTask.Result)
using (var responseStream = webResponse.GetResponseStream())
using (var sr = new StreamReader(responseStream, encoding))
{
result.SetResult(sr.ReadToEnd());
}
}
catch (Exception e)
{
result.SetException(e);
}
}, TaskContinuationOptions.AttachedToParent);
}
catch (Exception e)
{
result.SetException(e);
}
}, TaskContinuationOptions.AttachedToParent);
return result.Task;
}

Flow of execution in Httpwebrequest POST in Windows hone 8

If I have this code in Windows Phone 8, for instance
string __retS = null;
private String postRequest(String url, String postData)
{
byte[]byteData = Encoding.UTF8.GetBytes(postData);
HttpWebRequest request = null;
try
{
Uri uri = new Uri(url);
request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteData.Length;
// start the asynchronous operation
request.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), request);
} // end try
catch (Exception)
{
}
return __retS;
}
I put a breakpoint on this line request.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), request);. I expected that execution will jump to my GetRequestStreamCallback method but it doesn't. It rather goes on to execute the return statement hence a null value is always returned.
Is that how its supposed to go?

Is that how its supposed to go?
Yes. When the work is done it will invoke the callback function you passed. See "Asynchronous Programming Model (APM)". Starting with .Net 4.5 / c# 5.0, you can use async/await which can help to write async codes simpler.
var stream = await request.GetRequestStreamAsync();
//...do some work using that stream

The callback is executed asynchronously, that means the code is continued after the asynchronous method is assigned. (request.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), request);)
When the WebRequest is finished, the GetRequestStreamCallback is executed.
Because the UI thread would be blocked if this request was synchronous the windows phone sdk only serves an asynchronous one.