I have a little service to upload blobs to Azure Storage. I am trying to use it from a WebApi async action, but my AzureFileStorageService says the stream is closed.
I am new to async/await, are there any good resources to help me better understand it?
WebApi Controller
public class ImageController : ApiController
{
private IFileStorageService fileStorageService;
public ImageController(IFileStorageService fileStorageService)
{
this.fileStorageService = fileStorageService;
}
public async Task<IHttpActionResult> Post()
{
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(new HttpResponseMessage(HttpStatusCode.UnsupportedMediaType));
}
await Request.Content.ReadAsMultipartAsync(new MultipartMemoryStreamProvider()).ContinueWith((task) =>
{
foreach (var item in task.Result.Contents)
{
using (var fileStream = item.ReadAsStreamAsync().Result)
{
fileStorageService.Save(#"large/Sam.jpg", fileStream);
}
item.Dispose();
}
});
return Ok();
}
}
AzureFileStorageService
public class AzureFileStorageService : IFileStorageService
{
public async void Save(string path, Stream source)
{
await CloudStorageAccount.Parse(ConfigurationManager.AppSettings["StorageConnectionString"])
.CreateCloudBlobClient()
.GetContainerReference("images")
.GetBlockBlobReference(path)
.UploadFromStreamAsync(source); // source throws a stream is disposed exception
}
}
You have a problem with your Save() method: you're not returning a Task, and so the calling method has no way to wait for it to finish. That would be fine if you just wanted to fire and forget it, but you can't do that because the stream you pass in is going to be disposed as soon as the Save() method returns (thanks to the using statement).
Instead, you're going to have to either return a Task and await in the calling method, or you're going to have to not have the file stream in a using block, and instead let the Save() method dispose of it when its finished.
One way you could re-write your code would be as follows:
(snippet of calling method):
var result = await Request.Content.ReadAsMultipartAsync(new MultipartMemoryStreamProvider());
foreach (var item in result.Contents)
{
using (var fileStream = await item.ReadAsStreamAsync())
{
await fileStorageService.Save(#"large/Sam.jpg", fileStream);
}
item.Dispose();
}
And the Save method:
public async Task Save(string path, Stream source)
{
await CloudStorageAccount.Parse(ConfigurationManager.AppSettings["StorageConnectionString"])
.CreateCloudBlobClient()
.GetContainerReference("images")
.GetBlockBlobReference(path)
.UploadFromStreamAsync(source);
}
Checkout this AzureBlobUpload sample we just released a few weeks ago:
The previous answer is definitely a good fix. This is just a complete end to end official sample (perhaps for other folks to get started on).
https://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/AzureBlobsFileUploadSample/ReadMe.txt
Related
I have a .NET CORE 2 backend. In one of my controller endpoints, I'm creating invitations to be sent out via email. This seems to be a huge bottleneck on the endpoint and after thinking about it, I don't really need to wait for these invitations. If the email fails to send out, I can't really do anything about it anyway.
If I don't do await sendFn() would it essentially be a fire and forget method? I was reading on another stackoverflow thread that I'd have to do sendFn().ContinueWith(t => throw(t)) to be able to catch the exception since it'll be in another thread.
I have similar mailing functions around the code base. They each do slightly different things, but is there a service fn I can do to wrap these to make them fire and forget? I think some places I can just not use await (if that works), but some things alter the database context so if I don't await them I can potentially run into a case where something is accessing the same db context.
[HttpPost]
public async Task<IActionResult> CreateEvent([FromBody] Event val)
{
_ctx.Event.Add(val);
await _ctx.SaveChangesAsync();
await SendInvitations(val); // fn in question
return Ok();
}
public async Task SendInvitation(Event event)
{
forEach (var person in event.people)
{
await _ctx.Invitation.Add(person); // This shouldn't happen while another iteration or some other async code elsewhere is using the db ctx.
_ctx.SaveChangesAsync();
await _mailService.SendMail(person.email,"you have been invited"); // don't really need to await this.
}
}
I'm posting to my server with data about an event. After I create and save the event to the database, I go and create invitations for each person. These invitations are also database items. I then send out an email. I'm mostly worried that if I drop the await, then when I'm creating invitations, it may conflict with db context elsewhere or the next iteration.
To get your code to compile and run I had to make these changes:
public async Task<IActionResult> CreateEvent(Event val)
{
_ctx.Event.Add(val);
await _ctx.SaveChangesAsync();
await SendInvitation(val);
return Ok();
}
public async Task SendInvitation(Event #event)
{
foreach (var person in #event.people)
{
await _ctx.Invitation.Add(person);
await _ctx.SaveChangesAsync();
await _mailService.SendMail(person.email, "you have been invited");
}
}
I also had to write this harness code:
public OK Ok() => new OK();
public class Event
{
public List<Person> people = new List<Person>();
}
public class Person
{
public string email;
}
public interface IActionResult { }
public class OK : IActionResult { }
public class Invitation
{
public Task Add(Person person) => Task.Run(() => { });
}
public static class _ctx
{
public static List<Event> Event = new List<Event>();
public static Invitation Invitation = new Invitation();
public static Task SaveChangesAsync() { return Task.Run(() => { }); }
}
public static class _mailService
{
public static Task SendMail(string email, string message) { return Task.Run(() => { }); }
}
Then I updated SendInvitation like this:
public async Task SendInvitation(Event #event)
{
Thread.Sleep(2000);
foreach (var person in #event.people)
{
await _ctx.Invitation.Add(person);
await _ctx.SaveChangesAsync();
await _mailService.SendMail(person.email, "you have been invited");
}
Console.WriteLine("Done `SendInvitation`.");
}
Now, I can run it all like so:
var e = new Event();
e.people.Add(new Person() { email = "foo#bar.com" });
CreateEvent(e).ContinueWith(t => Console.WriteLine("Done `CreateEvent`."));
Console.WriteLine("Done `Main`.");
That outputs:
Done `Main`.
Then 2 seconds later:
Done `SendInvitation`.
Done `CreateEvent`.
If I simply change CreateEvent to this:
public async Task<IActionResult> CreateEvent(Event val)
{
_ctx.Event.Add(val);
await _ctx.SaveChangesAsync();
Task.Run(() => SendInvitation(val));
return Ok();
}
Then I get this output:
Done `Main`.
Done `CreateEvent`.
Then 2 seconds later:
Done `SendInvitation`.
That seems to be what you want.
The short answer is that you have no guarantees that that the execution of that code will complete.
That's why ASP.NET Core has infrastructure for background work: Implementing background tasks in .NET Core 2.x webapps or microservices with IHostedService and the BackgroundService class
I have a ASP.Net Core 2.0 webapi code that looks like following
public class TestController : Controller
{
private readonly ISampleRepository _sampleRepository = new SampleRepository();
[HttpPost("{id}")]
public async void Post([FromRoute]int id, IEnumerable<IFormFile> files, [FromForm] NotifyViewModel model)
{
// get data from DB using async
var dbData = await _sampleRepository.GetAsync(id);
//check if email alert is enabled
if(dbData.IsEmailEnabled)
{
//create MailMessage
var mailMessage = new MailMessage() {
//options here
};
foreach (var formFile in files)
{
mailMessage.Attachments.Add(new Attachment(formFile.OpenReadStream(), formFile.FileName, formFile.ContentType));
}
// send email
_emailHelper.SendEmail(mailMessage);
}
}
}
public class SampleRepository
{
public async Task<SampleData> GetAsync(int id)
{
//get data using Dapper & return
var result = await con.QuerySingleAsync<SampleData>(sql, new {id = id}, null, null, CommandType.Text);
return result;
}
}
When I post data to this endpoint I get exception saying 'Cannot access disposed object.' (Refer screenshot)
However when I make DB operation synchronous by doing following code changes, somehow it works without any error.
// Removed
// var dbData = await _sampleRepository.GetAsync(id);
// Newly added. Made DB operations synchronous
var dbData = _sampleRepository.GetAsync(id).Result;
Can someone please help me understand this behavior? Why HTTP posted files gets disposed when async operation is involved?
Because your method is async void it immediately returns as soon as it hits the first await. By the time the code accesses request data, the request has already ended.
Change the signature of Post from async void Post to async Task Post as async Task is the asynchronous analogue of the synchronous void.
This is only the idea on what im doing in a window service.
I get the idea from this video to do it in parallel processing.
I have two different method and a model class.
Model Class code:
public class Email(){
public string Recipient { get; set; }
public string Message { get; set; }
}
Methods is something like this:
public void LoadData(){
while(Main.IsProcessRunning){
// 1. Get All Emails
var emails = new dummyRepositories().GetAllEmails(); //This will return List<Emails>.
// 2. Send it
// After sending assume that the data will move to other table so it will not be query again for the next loop.
SendDataParallel(emails);//this will function async? even though the calling method is sync.
// This will continue here or wait until it already send?
// If it will continue here even though it will not send already
// So there's a chance to get the email again for the next loop and send it again?
}
}
//This will send email at parallel
public async void SendDataParallel(IList<Email> emails){
var allTasks = emails.Select(SendDataAsync);
await TaskEx.WhenAll(allTasks);
}
//Assume this code will send email asynchronously. (this will not send email, for sample only)
public async void SendDataAsync(Email email){
using (var client = new HttpClient())
{
client.PostAsync(email);
}
}
I only want to get all queued emails then send it in parallel then wait until it already send.
I'm avoiding using foreach on every email that I get.
Lets start at the bottom:
You dispose your client before you actually finish receiving the HttpResponseMessage asynchronously. You'll need to make your method async Task and await inside:
public async Task SendDataAsync(Email email)
{
using (var client = new HttpClient())
{
var response = await client.PostAsync(email);
}
}
Currently, your SendDataParallel doesn't compile. Again, it needs to return a Task:
public Task SendEmailsAsync(IList<Email> emails)
{
var emailTasks = emails.Select(SendDataAsync);
return Task.WhenAll(allTasks);
}
At the top, you'll need to await on SendEmailsAsync:
public async Task LoadDataAsync()
{
while (Main.IsProcessRunning)
{
var emails = new dummyRepositories().GetAllEmails();
await SendEmailsAsync(emails);
}
}
Edit:
If you're running this inside a windows service, you can offload it to Task.Run and use the async keyword:
var controller = new Controller();
_processThread = Task.Run(async () => await controller.LoadDataAsync());
Doesn't your compiler highlight your code with errors?
If you mark your method as async while it doesn't return any value, you should set your return type as Task, not void:
public async Task SendDataParallel(IList<Email> emails){
var allTasks = emails.Select(SendDataAsync);
await Task.WhenAll(allTasks);
}
Your second method also shoud return a Task, otherwise what you want to (a)wait in the first method?
public async Task SendDataAsync(Email email){
using (var client = new HttpClient())
{
return client.PostAsync(email);
}
}
Now you can Select all your SendDataAsync tasks in SendDataParallel and .Wait() it's task in LoadData in synchronious mode:
public void LoadData(){
while(Main.IsProcessRunning){
var emails = new dummyRepositories().GetAllEmails(); //This will return List<Emails>.
SendDataParallel(emails).Wait();
}
}
More information you can find reading answers in other SO questions and docs on MSDN:
Can somebody please explain async / await?
Brief explanation of Async/Await in .Net 4.5
how to and when use async and await
Asynchronous Programming with Async and Await
And as you used LINQ's Select() which is based on foreach cycle next article also could be useful:
Nested task inside loop
I have the following code
static void Main(string[] args)
{
string url = "http://www.google.com";
Console.WriteLine(GetUrl(url).Result); // throws TaskCanceledException
Console.WriteLine(GetUrl2(url).Result);
}
public static Task<string> GetUrl(string url)
{
using (var client = new HttpClient())
{
return client.GetStringAsync(url);
}
}
public static Task<string> GetUrl2(string url)
{
using (var client = new WebClient())
{
return client.DownloadStringTaskAsync(url);
}
}
I'm trying to get the string of an url, the problem is GetUrl method (uses HttpClient's GetStringAsync) throws an TaskCacelledException, but GetUrl2 method (uses WebClient's DownloadStringTaskAsync) runs correctly. Is this caused due to using statement? What am I missing?
Edit. In this example I'm calling Result on the task because this is a console application, I know that it is best to await the result in a event handler for example.
Is this caused due to using statement?
Yes. In both code examples, you're disposing the underlying client before the operation completes. Both code examples should be changed as such:
public static async Task<string> GetUrlAsync(string url)
{
using (var client = new HttpClient())
{
return await client.GetStringAsync(url);
}
}
public static async Task<string> GetUrl2Async(string url)
{
using (var client = new WebClient())
{
return await client.DownloadStringTaskAsync(url);
}
}
The behavior of asynchronous downloads when their underlying clients are disposed is undocumented. It's best not to dispose the clients until your code is done using them.
I'm trying to start async task (on .NET 4.5) which downloads content of web page, but somehow this task never finishes.
My PageDownloader class:
using System.Net;
using System.Text;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using System;
namespace ParserConsole.WebClient
{
public class PageDownloader
{
private System.Net.Http.HttpClient _client;
public PageDownloader()
: this(Encoding.UTF8) { }
private Encoding _encoding;
public PageDownloader(Encoding encoding)
{
_encoding = encoding;
_client = new HttpClient() { Timeout = TimeSpan.FromSeconds(10)};
}
private HttpRequestMessage _request;
private HttpResponseMessage _response;
private string _responseString;
public string GetPageData(string link)
{
_request = new HttpRequestMessage(HttpMethod.Get, link);
_request.Headers.Add("User-Agent", "Chrome/21.0.1180.89");
_request.Headers.Add("Accept", "text/html");
GetResponse().Wait();
GetStringFromResponse().Wait();
return _responseString;
}
private async Task<HttpResponseMessage> GetResponse() {
return _response = await _client.GetAsync(_request.RequestUri);
}
private async Task<string> GetStringFromResponse() {
return _responseString = await _response.Content.ReadAsStringAsync();
}
}
}
I start downloading page by calling
new PageDownloader().GetPageData(url);
When I'm trying to debug the code, everything is fine till GetResponse().Wait(). But somehow GetResponse() task never finishes - breakpoint on the next line is never reached. I get no exceptions, application continues running. Any suggestions?
This is a standard deadlock condition you get when you start an async operation and then block on the returned task.
Here is a blog post discussion the topic.
Basically, the await call ensures that the continuation it wires up of the task will run in the context you were originally in (which is very helpful) but because you are calling Wait in that same context it's blocking, so the continuation never runs, and that continuation needs to run for the wait to end. Classic deadlock.
As for the fix; usually it means you just shouldn't be doing a blocking wait on the async operation; it's contrary to the design of the whole system. You should, "async all the way up". In this case it would mean that GetPageData should return a Task<string> rather than a string, and rather than waiting on the other operations that return a task you should await on them.
Now, having said that, there are ways of doing a blocking wait on the async operations without deadlocking. While it can be done, it honestly defeats the purpose of using async/await in the first place. The primary advantage of using that system is that the main context isn't blocked; when you block on it that entire advantage goes away, and you might as well just use blocking code all the way through. async/await is really more of an all-or-nothing paradigm.
Here is how I would structure that class:
public class PageDownloader
{
private System.Net.Http.HttpClient _client;
private Encoding _encoding;
public PageDownloader()
: this(Encoding.UTF8) { }
public PageDownloader(Encoding encoding)
{
_encoding = encoding;
_client = new HttpClient() { Timeout = TimeSpan.FromSeconds(10) };
}
public async Task<string> GetPageData(string link)
{
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, link);
request.Headers.Add("User-Agent", "Chrome/21.0.1180.89");
request.Headers.Add("Accept", "text/html");
HttpResponseMessage response = await _client.GetAsync(request.RequestUri);
return await response.Content.ReadAsStringAsync(); ;
}
}
Why not just do this if you want to have a function like that.
public string GetPageData(string link)
{
_request = new HttpRequestMessage(HttpMethod.Get, link);
_request.Headers.Add("User-Agent", "Chrome/21.0.1180.89");
_request.Headers.Add("Accept", "text/html");
var readTask = _client.GetStringAsync(link);
readTask.Wait();
return readTask.Result;
}
It would be better to return the Task all the way back and handle it with async/await in the calling code.
public Task<string> GetPageData(string link)
{
_request = new HttpRequestMessage(HttpMethod.Get, link);
_request.Headers.Add("User-Agent", "Chrome/21.0.1180.89");
_request.Headers.Add("Accept", "text/html");
return _client.GetStringAsync(link);
}