I have a method that uploads one file to server. Right now is working besides any bad coding on the method (Im new to Task library).
Here is the code that uploads a file to server:
private async void UploadDocument()
{
var someTask = await Task.Run<bool>(() =>
{
// open input stream
using (System.IO.FileStream stream = new System.IO.FileStream(_cloudDocuments[0].FullName, System.IO.FileMode.Open, System.IO.FileAccess.Read))
{
using (StreamWithProgress uploadStreamWithProgress = new StreamWithProgress(stream))
{
uploadStreamWithProgress.ProgressChanged += uploadStreamWithProgress_ProgressChanged;
// start service client
SiiaSoft.Data.FieTransferWCF ws = new Data.FieTransferWCF();
// upload file
ws.UploadFile(_cloudDocuments[0].FileName, (long)_cloudDocuments[0].Size, uploadStreamWithProgress);
// close service client
ws.Close();
}
}
return true;
});
}
Then I have a ListBox where I can drag & drop multiple files so what I want to do is to do a FOR LOOP within the ListBox files and then call UploadDocument(); but I want to first upload 1st file in the listBox then when completed continue with the second file and so on...
Any clue on the best way to do it?
Thanks a lot.
You should make your UploadDocument return Task. Then you can await the task in a loop. For example:
private async Task UploadAllDocuments()
{
string[] documents = ...; // Fetch the document names
foreach (string document in documents)
{
await UploadDocument(document);
}
}
private async Task UploadDocument(string document)
{
// Code as before, but use document instead of _cloudDocuments[0]
}
In fact, your UploadDocument can be made simpler anyway:
private Task UploadDocument()
{
return Task.Run<bool>(() =>
{
// Code as before
});
}
Wrapping that in an async method isn't particularly useful.
(You may well want to change the type to not be string - it's not clear what _cloudDocuments is.)
In general, you should always make an async method return Task or Task<T> unless you have to make it return void to comply with an event-handling pattern.
Related
I have a Net 6 Console app where I use several BlockingCollections to process files that are dropped in a folder. I watch the folder using Net's FileWatcher().
In the Created event, I use a Channel to handle the processing, which is done in two phases, and after each phase the result item is moved to a BlockingCollection, that will then be consumed by the next phase.
Program.cs
public static async Task Main(string[] args)
{
BlockingCollection<FileMetadata> _fileMetaDataQueue = new BlockingCollection<FileMetadata>()
var channel = Channel.CreateUnbounded<FileSystemEventArgs>();
// Start a task to monitor the channel and process notifications
var notificationProcessor = Task.Run(() => ProcessNotifications(channel, _fileMetaDataQueue));
Task fileCopyingTask = Task.Run(() => fileCopyThread.Start()); //injected using DI
Task processMovedFile = Task.Run(() => ProcessDestinationThread.Start()); //injected using DI
Task retryOnErrorTask = Task.Run(() => RetryOnErrorThread.Start()); //injected using DI
using var watcher = new FileSystemWatcher(sourceFolder); //C:\temp
// other fw related config
watcher.Created += (sender, e) => channel.Writer.WriteAsync(e);
}
private async Task ProcessNotifications(Channel<FileSystemEventArgs> channel, BlockingCollection<FileMetadata> queue)
{
await foreach (var e in channel.Reader.ReadAllAsync())
{
Thread.Sleep(300); // So the file is released after it is dropped
try
{
// Process the file and add its name and extension to the queue
FileMetaData fileInfo = ExtractFileMetadata(e.FullPath); //processing method
queue.Add(fileInfo);
}
try
{
// logging etc
}
}
}
The BlockingCollection queue is then consumed in the FileCopyThread class, with the Start() method exposed (and called)
FileCopyThread.cs
BlockingCollection<FileMetadata> resultQueue = new();
BlockingCollection<FileMetadata> retryQueue = new();
public async Task Start()
{
await Task.Run(() => {
ProcessQueue();
});
}
private void ProcessQueue()
{
// Since IsCompleted is never set, it will always run
while (!fileMetadataQueue.IsCompleted)
{
// Try to remove an item from the queue
if (fileMetadataQueue.TryTake(out FileMetadata result))
{
// Copy the file to a new location
var newFileLocation = processorOps.MoveFile(result); // move file to other path
// Add the new file location to the result queue
if (newFileLocation != String.Empty)
{
result.newFileLocation = newFileLocation;
resultQueue.Add(result);
}
else {
retryQueue.Add(result);
}
}
}
}
The ProcessDestinationThread and RetryOnErrorThread work in exactly the same way, but do some different processing, and consume the resultQueue and the retryQueue, respectively.
Now when I run this app, it works fine, everything gets processed as expected, but my CPU and power usage is between 85% and 95%, which is huge, IMO, and does so even when it is not processing anything, just sitting idle. I figured this is because all the infinite loops, but how can I remedy this?
Birds eye view: What I would like is that in case the filewatcher.created event is not firing (ie no files are dropped) then the all the queues after it can be running in idle, so to speak. No need for constant checking, then.
I thought about calling CompleteAdding() on the BlockingCollection<T>, but it seems that I cannot reverse that. And the app is supposed to run indefinitely: So if the drop folder is empty, it might be receiving new files at any time.
Is there a way how I can reduce the CPU usage of my application?
Ps. I am aware that this code is not a fully working example. The real code is far more complex than this, and I had to remove a lot of stuff that is distracting. If you think any pieces of relevant code are missing, I can provide them. I hope this code will at least make clear what I am trying to achieve.
private void ProcessQueue()
{
while (!fileMetadataQueue.IsCompleted)
{
if (fileMetadataQueue.TryTake(out FileMetadata result))
{
//...
}
}
}
This pattern for consuming a BlockingCollection<T> is incorrect. It causes a tight loop that burns unproductively a CPU core. The correct pattern is to use the GetConsumingEnumerable method:
private void ProcessQueue()
{
foreach (FileMetadata result in fileMetadataQueue.GetConsumingEnumerable())
{
//...
}
}
I'm pretty new to C#. I'm trying to open a pair of CSV text files using the StorageFile class, but I need to do it inside a class constructor (which cannot be async):
private async Task SetupFileAccess()
{
MainCarDataStorageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///leaf.csv"));
var MainCarDataInputStream = await MainCarDataStorageFile.OpenReadAsync();
var MainCarDataClassicStream = MainCarDataInputStream.AsStreamForRead();
MainCarDataStream = new StreamReader(MainCarDataClassicStream);
await MainCarDataStream.ReadLineAsync(); //clear header line from csv
LookupTableStorageFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Full_Charge_To_Empty.csv"));
var LookupTableInputStream = await LookupTableStorageFile.OpenReadAsync();
var LookupTableClassicStream = LookupTableInputStream.AsStreamForRead();
LookupTableDataStream = new StreamReader(LookupTableClassicStream);
await LookupTableDataStream.ReadLineAsync(); //clear header line from csv
}
Then I call Wait() on that function to block execution.
The problem is when I reach the second OpenReadAsync() function call to open a stream to the second file (Full_Charge_to_Empty.csv), the program just churns forever. No crash, no progress.
It seems I can only access the first file opened. If I comment out the first File access (leaf.csv), I am able to access the second file. If I switch their positions (leaf.csv is opened 2nd), I cannot open leaf.csv.
I feel like there must be something I don't understand about how these file accesses work...is it because my accessing of the first file (leaf.csv) is occupying a resource of some type? Appreciate the help!
The problem is that you're trying to force async functions to complete synchronously inside a constructor; that's not going to end well. The correct approach is to use an async factory method to front-load the work and then return a populated object. For example:
class Test
{
// Make constructor private so no-one can create it
private Test()
{ }
// Simple string properties that can't be async
public string Text1 { get; private set; }
public string Text2 { get; private set; }
// Async factory method that can do async things
public static async Task<Test> CreateAsync()
{
var result = new Test();
result.Text1 = await LoadFile("file1.txt");
result.Text2 = await LoadFile("file2.txt");
return result;
}
// Helper function
static async Task<string> LoadFile(string filename)
{
var file = await Package.Current.InstalledLocation.GetFileAsync(filename);
var size = (await file.GetBasicPropertiesAsync()).Size;
var reader = new DataReader(await file.OpenReadAsync());
reader.UnicodeEncoding = UnicodeEncoding.Utf8;
await reader.LoadAsync((uint)size);
// assumes single-byte UTF codepoints, eg ASCII
return reader.ReadString((uint)size);
}
}
Using it:
var test = await Test.CreateAsync();
await new MessageDialog($"{test.Text1}{Environment.NewLine}{test.Text2}").ShowAsync();
return;
I need to read a numeric value (a version number) from a text file in my resources.
I compare this version number to the version number of an installed component.
If the version number in the resources is higher than the installed version, I copy the new component (a database) from my resources to a local directory where the user can use it.
I need to do this synchronously because my application can't work without the database.
However, I don't see any way to do it synchronously.
MS forces me to do it with an async task like this:
private async Task<string> ResourcesReadTextFile(string uFileName)
{
string sRet = "";
try
{
StorageFile file = await StorageFile.GetFileFromApplicationUriAsync(new Uri(cstAssets + uFileName));
using (var inputStream = await file.OpenReadAsync())
using (var classicStream = inputStream.AsStreamForRead())
using (var streamReader = new StreamReader(classicStream))
{
while (streamReader.Peek() >= 0)
{
sRet = streamReader.ReadLine();
}
}
}
catch (Exception ex)
{
Debug.Assert(false);//check here
}
return sRet;
}
Now I've encountered a situation where the app started before the database was copied over to the local directory as the copying also needs to be done asynchronously, there simply isn't any way to do it synchronous.
There is no such function as StorageFile.Copy().
What I'm therefore using is:
private async void pCopyFromResourcesToLocal(string uFileName)
{
// Cant await inside catch, but this works anyway
try
{
StorageFile storfile = await StorageFile.GetFileFromApplicationUriAsync(new Uri(cstAssets + uFileName));
await storfile.CopyAsync(ApplicationData.Current.LocalFolder);
}
catch (Exception ex)
{
Debug.WriteLine("");
}
}
This drives me crazy.
People write that Async should be embraced and hugged and appreciated, but in my case it causes nothing but trouble.
I don't see any way of making this thing synchronous, and I wonder why MS forces me to do it that way.
Any help very much appreciated.
Thank you.
Code as a screenshot:
Edit: I've added the top methods here:
public static async Task<DB> InitAppDb()
{
IFileHelper helper = DependencyService.Get<IFileHelper>();
string path = await helper.GetFilePathAndCopyFromResourcesIfNotPresent("tablet.db");
return (_dbApp = new DB(path));
}
public async Task CopyDatabaseIfNotExists(string uFileName)
{
IsolatedStorageFile nExpectedFolder = IsolatedStorageFile.GetUserStoreForApplication();
bool bCopyNewDB = false;
Task<bool> datatask = pResourceIsNewer(uFileName);
bCopyNewDB = await datatask;
if (! bCopyNewDB)
{
try
{
await ApplicationData.Current.LocalFolder.GetFileAsync(uFileName); //nExpectedFolder.GetFileAsync(dbPath);/// ApplicationData.Current.LocalFolder.GetFileAsync("preinstalledDB.db");
// No exception means it exists
return;
}
catch (System.IO.FileNotFoundException)
{
// The file obviously doesn't exist
}
}
pCopyFromResourcesToLocal(uFileName);
}
private async Task<bool>pResourceIsNewer(string uPath)
{
string sFileNameAppDBVersion =uPath + ".txt";
if (IsolatedStorageFileExist(sFileNameAppDBVersion))
{
int iAppDBVersionInstalled = Convert.ToInt32(IsolatedStorageReadTextFile(sFileNameAppDBVersion));
Task<string> datatask = ResourcesReadTextFile(sFileNameAppDBVersion);
string s = await datatask;
int iAppDBResources = Convert.ToInt32(s);
bool b = (iAppDBResources > iAppDBVersionInstalled);
return b;
}
else
{
return true;
}
}
All you have to do is:
//private async void pCopyFromResourcesToLocal(string uFileName) { ... }
private async Task pCopyFromResourcesToLocal(string uFileName) { ... }
and then you can await it:
//pCopyFromResourcesToLocal(uFileName);
await pCopyFromResourcesToLocal(uFileName);
and it will all be completed before you call return (_dbApp = new DB(path));
Nothing in this async/await chain can happen out of order.
When you say that your app can't work without the database, remember that using an await keyword does just that, so the following line of code will not execute until after the async call returns. You can structure your code such that it is responsive while you wait for the DB to come back online.
However, you can force your function to be synchronous by using a construct such as:
StorageFile.GetFileFromApplicationUriAsync(new Uri(cstAssets + uFileName)).GetAwaiter().GetResult();
Or, even better, have a look at the JoinableTaskFactory.
Any asynchronous API method can be made synchronous by simply tagging .GetAwaiter().GetResult() on the end, as #pm_2 says.
In the case of Task<T> results, you can simply use .Result, and for Task results, .Wait().
You can either write an asynchronous function to read your file, then wait for its result at the top level, or you can write a synchronous method and wait for the result of every call it makes to async functions.
So Microsoft is not forcing you into anything: it's just providing a simpler API with the lowest common denominator of both asynchronous and synchronous workflows.
I've been reading examples for a long time now, but unfortunately I've been unable to apply the solutions to the code I'm working with. Some quick Facts/Assorted Info:
1) I'm new to C#
2) The code posted below is modified from Amazon Web Services (mostly stock)
3) Purpose of code is to compare server info to offline already downloaded info and create a list of need to download files. This snip is for the list made from the server side, only option with AWS is to call async, but I need this to finish before moving forward.
public void InitiateSearch()
{
UnityInitializer.AttachToGameObject(this.gameObject);
//these are the access key and secret access key for credentials
BasicAWSCredentials credentials = new BasicAWSCredentials("secret key", "very secret key");
AmazonS3Config S3Config = new AmazonS3Config()
{
ServiceURL = ("url"),
RegionEndpoint = RegionEndpoint.blahblah
};
//Setting the client to be used in the call below
AmazonS3Client Client = new AmazonS3Client(credentials, S3Config);
var request = new ListObjectsRequest()
{
BucketName = "thebucket"
};
Client.ListObjectsAsync(request, (responseObject) =>
{
if (responseObject.Exception == null)
{
responseObject.Response.S3Objects.ForEach((o) =>
{
int StartCut = o.Key.IndexOf(SearchType) - 11;
if (SearchType == o.Key.Substring(o.Key.IndexOf(SearchType), SearchType.Length))
{
if (ZipCode == o.Key.Substring(StartCut + 12 + SearchType.Length, 5))
{
AWSFileList.Add(o.Key + ", " + o.LastModified);
}
}
}
);
}
else
{
Debug.Log(responseObject.Exception);
}
});
}
I have no idea how to apply await to the Client.ListObjectsAsync line, I'm hoping you all can give me some guidance and let me keep my hair for a few more years.
You can either mark your method async and await it, or you can call .Wait() or .Result() on the Task you're given back.
I have no idea how to apply await to the Client.ListObjectsAsync line
You probably just put await in front of it:
await Client.ListObjectsAsync(request, (responseObject) => ...
As soon as you do this, Visual Studio will give you an error. Take a good look at the error message, because it tells you exactly what to do next (mark InitiateSearch with async and change its return type to Task):
public async Task InitiateSearchAsync()
(it's also a good idea to add an Async suffix to follow the common pattern).
Next, you'd add an await everywhere that InitiateSearchAsync is called, and so on.
I'm assuming Client.ListObjectsAsync returns a Task object, so a solution for your specific problem would be this:
public async void InitiateSearch()
{
//code
var collection = await Client.ListObjectsAsync(request, (responseObject) =>
{
//code
});
foreach (var item in collection)
{
//do stuff with item
}
}
the variable result will now be filled with the objects. You may want to set the return type of InitiateSearch() to Task, so you can await it too.
await InitiateSearch(); //like this
If this method is an event handler of some sort (like called by the click of a button), then you can keep using void as return type.
A simple introduction from an unpublished part of the documentation for async-await:
Three things are needed to use async-await:
The Task object: This object is returned by a method which is executed asynchronous. It allows you to control the execution of the method.
The await keyword: "Awaits" a Task. Put this keyword before the Task to asynchronously wait for it to finish
The async keyword: All methods which use the await keyword have to be marked as async
A small example which demonstrates the usage of this keywords
public async Task DoStuffAsync()
{
var result = await DownloadFromWebpageAsync(); //calls method and waits till execution finished
var task = WriteTextAsync(#"temp.txt", result); //starts saving the string to a file, continues execution right await
Debug.Write("this is executed parallel with WriteTextAsync!"); //executed parallel with WriteTextAsync!
await task; //wait for WriteTextAsync to finish execution
}
private async Task<string> DownloadFromWebpageAsync()
{
using (var client = new WebClient())
{
return await client.DownloadStringTaskAsync(new Uri("http://stackoverflow.com"));
}
}
private async Task WriteTextAsync(string filePath, string text)
{
byte[] encodedText = Encoding.Unicode.GetBytes(text);
using (FileStream sourceStream = new FileStream(filePath, FileMode.Append))
{
await sourceStream.WriteAsync(encodedText, 0, encodedText.Length);
}
}
Some thing to note:
You can specify a return value from an asynchronous operations with Task. The await keyword waits till the execution of the method finishes, and returns the string.
the Task object contains the status of the execution of the method, it can be used as any other variable.
if an exception is thrown (for example by the WebClient) it bubbles up at the first time the await keyword is used (in this example at the line string result (...))
It is recommended to name methods which return the Task object as MethodNameAsync
For more information about this take a look at http://blog.stephencleary.com/2012/02/async-and-await.html.
We got an existing library where some of the methods needs to be converted to async methods.
However I'm not sure how to do it with the following method (errorhandling has been removed). The purpose of the method is to zip a file and save it to disk. (Note that the zip class doesn't expose any async methods.)
public static bool ZipAndSaveFile(string fileToPack, string archiveName, string outputDirectory)
{
var archiveNameAndPath = Path.Combine(outputDirectory, archiveName);
using (var zip = new ZipFile())
{
zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;
zip.Comment = $"This archive was created at {System.DateTime.UtcNow.ToString("G")} (UTC)";
zip.AddFile(fileToPack);
zip.Save(archiveNameAndPath);
}
return true;
}
An implementation could look like this:
public static async Task<bool> ZipAndSaveFileAsync(string fileToPack, string archiveName, string outputDirectory)
{
var archiveNameAndPath = Path.Combine(outputDirectory, archiveName);
await Task.Run(() =>
{
using (var zip = new ZipFile())
{
zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;
zip.Comment = $"This archive was created at {System.DateTime.UtcNow.ToString("G")} (UTC)";
zip.AddFile(fileToPack);
zip.Save(archiveNameAndPath);
}
});
return true;
}
Which just seems wrong. The client could just call the sync method using Task.Run
Please, anyone got any hints on how to transform it into a async method ?
Which just seems wrong. The client could just call the sync method
using Task.Run
Spot on. By wrapping synchronous code in Task.Run() the library is now using the client's threadpool resources without it being readily apparent. Imagine what could happen to the client's threadpool if all libraries took this approach? Long story short, just expose the synchronous method and let the client decide if it wants to wrap it in Task.Run().
Having said that, if the ZipFile object had async functionality (e.g. had a SaveAsync() method) then you could make the outer method async as well. Here's an example of how that would look:
public static async Task<bool> ZipAndSaveFileAsync(string fileToPack, string archiveName, string outputDirectory)
{
var archiveNameAndPath = Path.Combine(outputDirectory, archiveName);
using (var zip = new ZipFile())
{
// do stuff
await zip.SaveAsync(archiveNameAndPath);
}
return true;
}
As a temporarily solution, I would introduce an extension method:
public static class ZipFileExtensions
{
public static Task SaveAsync(this ZipFile zipFile, string filePath)
{
zipFile.Save(filePath);
return Task.FromResult(true);
}
}
Then the usage would be:
public static async Task<bool> ZipAndSaveFileAsync(string fileToPack, string archiveName, string outputDirectory)
{
var archiveNameAndPath = Path.Combine(outputDirectory, archiveName);
using (var zip = new ZipFile())
{
...
await zip.SaveAsync(archiveNameAndPath).ConfugureAwait(false);
}
return true;
}
Implementing synchronous tasks does not violate anything (talking about Task.FromResult)
Submit a request to https://github.com/jstedfast/Ionic.Zlib asking for an async support in the library due to IO operations
Hope that's done eventually, and then you can upgrade the Ionic.Zlib in your app, delete the ZipFileExtensions, and continue using async version of the Save method (this time built into the library).
Alternatively, you can clone the repo from GitHub, and add SaveAsync by yourself, the submit a pull request back.
It's just not possible to 'convert' a sync method to an async if a library does not support it.
From performance standpoint, this might not be the best solution, but from management point of view, you can decouple stories "Convert everything to async" and "Improve app performance by having Ionic.Zlib async", what makes your backlog more granular.
public static Task<bool> ZipAndSaveFileAsync(string fileToPack, string archiveName, string outputDirectory)
{
return Task.Run(() =>
{
var archiveNameAndPath = Path.Combine(outputDirectory, archiveName);
using (var zip = new ZipFile())
{
zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;
zip.Comment = $"This archive was created at {System.DateTime.UtcNow.ToString("G")} (UTC)";
zip.AddFile(fileToPack);
zip.Save(archiveNameAndPath);
}
return true;
});
}
Then use like this
public async Task MyMethod()
{
bool b = await ZipAndSaveFileAsync();
}
Some of the answers suggest that zipping a file is not a process that you should do asynchronously. I don't agree with this.
I can imagine that zipping files is a process that might take some time. During this time you want to keep your UI responsive or you want to zip several files simultaneously, or you want to upload a zipped file while zipping the next one/
The code you show is the proper way to make your function asynchronous. You question whether it is useful to create such a small method. Why not let the users call Task.Run instead of call your async function?
The reason for this is called information hiding. By creating the async function you're hiding how you zip asynchronously, thus relieving others from knowing how to do this.
Besides, information hiding gives you the freedom to change the internals of the procedure as long as you don't change the pre- and postcondition.
One of the answers said that your function still is not asynchronous. That is not true. Callers of your function may call your async function without awaiting for it. While the task is zipping, the caller may do other things. As soon as it needs the boolean result of the task if can await for the task.
Example of usage:
private async Task DoSomethingSimultaneously()
{
var taskZipFileA = ZipAndSaveFileAsync(fileA, ...)
// while this task is zipping do other things,
// for instance start zipping file B:
var taskZipFileB = ZipAndSaveFileAsync(fileB, ...)
// while both tasks are zipping do other things
// after a while you want to wait until both files are finished:
await Task.WhenAll(new Task[] {taskZipFileA, taskZipFileB});
// after the await, the results are known:
if (taskZipFileA.Result)
{
// process the boolean result of taskZipFile A
}
Note the difference between Task.WaitAll and Task.WhenAll
In async - await you use Task.WhenAll. The return is a Task, so you can
await Task.WhenAll (...)
For proper async-await, all functions that call any async function need to be async themselves and return a Task (instead of void) or Task<TResult> instead of TResult. There is one exception: the event handler may return void.
private async void OnButton1_clicked(object sender, ...)
{
bool zipResult = await SaveAndZipFileAsync(...);
ProcessZipResult(zipResult);
}
Using this method your UI keeps responsive. You don't have to call Task.Run
If you have a non-async function and want to start zipping while doing something else, your non-async function has to call Task.Run. As the function is not async it can't use await. When it needs the result of task.Run it needs to use Task.Wait, or Task.WaitAll
private void NonAsyncZipper()
{
var taskZipFileA = Task.Run ( () => ZipAndSaveFileAsync(...);
// while zipping do other things
// after a while when the result is needed:
taskZipFileA.Wait();
ProcesZipResult(taskZipFileA.Result);
}
If it's possible to get the binary data from your Zip library after the compression, then instead of using this library to save the file, use .NET IO libraries to save it.
EDIT:
There is no point in using async for CPU bound operations (such as compression). In your case, the only benefit you can get from async is when you save the file to the disk. I thought that's what you were asking for.