Develop Reference

Proper way to read files asynchronously

I'm trying to read files asynchronously. I was wondering if this is a proper way to do so. Below is what I have tried so far. Is this correct?
static void Main(string[] args)
{
Task<string> readFileTask = Task.Run(() => ReadFile(#"C:\Users\User\Desktop\Test.txt"));
readFileTask.Wait();
string astr = readFileTask.Result;
Console.WriteLine(astr);
}
static private async Task<string> ReadFile(string filePath)
{
string text = File.ReadAllText(filePath);
return text;
}
Thanks.

System.IO provides File.ReadAllTextAsync method for .Net Standard > 2.1 and .NET Core 2.0.
If you are using C# 7.1 or higher you can use File.ReadAllTextAsync inside Main function directly.
static async Task Main(string[] args)
{
var astr = await File.ReadAllTextAsync(#"C:\Users\User\Desktop\Test.txt");
Console.WriteLine(astr);
}
Unfortunately, If you are not using C# 7.1 or higher then you can't use Async Main. You have to use Task.Run to calll async methods.
static void Main(string[] args)
{
var astr=Task.Run(async () =>
{
return await File.ReadAllTextAsync(#"C:\Users\User\Desktop\Test.txt");
}).GetAwaiter().GetResult();
Console.WriteLine(astr);
}
In case you are using .NET Framework then you have to use FileStream because System.IO not provides File.ReadAllTextAsync method.
private static async Task<string> ReadAllTextAsync(string filePath)
{
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 4096, useAsync: true))
{
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[0x1000];
int numRead;
while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
string text = Encoding.Unicode.GetString(buffer, 0, numRead);
sb.Append(text);
}
return sb.ToString();
}
}

As it has been said - you can use System.File.ReadAllTextAsync. But in case you need System.File.ReadAllTextAsync(string path) analog for .NET Framework, that is functionally closer to its .NET Core counterpart:
using System;
using System.IO;
using System.Text;
using System.Threading.Tasks;
//...
static async Task<string> ReadAllTextAsync(string path)
{
switch (path)
{
case "": throw new ArgumentException("Empty path name is not legal.", nameof(path));
case null: throw new ArgumentNullException(nameof(path));
}
using var sourceStream = new FileStream(path, FileMode.Open,
FileAccess.Read, FileShare.Read,
bufferSize: 4096,
useAsync: true);
using var streamReader = new StreamReader(sourceStream, Encoding.UTF8,
detectEncodingFromByteOrderMarks: true);
// detectEncodingFromByteOrderMarks allows you to handle files with BOM correctly.
// Otherwise you may get chinese characters even when your text does not contain any
return await streamReader.ReadToEndAsync();
}
// ...

How to remove lag during data fetching from large text files from my code in C#?

I have a text file consisting of 21000 lines, i have an attribute and i need to search it in the .txt file and need to return a value from the same too. All code is done and tried async plus new thread but there is a five second lag during the button click . how can i remove the lag.
Tried on new unity and C#
public async void read()
{
string[] lines = await ReadAllLinesAsync("Assets/Blockchain Module/" + File + ".csv");
fields = null;
for (int j = 0; j < lines.Length; j++)
{
fields = lines[j].Split(',');
x[j] = System.Convert.ToDouble(fields[1]);
y[j] = System.Convert.ToDouble(fields[2]);
z[j] = System.Convert.ToDouble(fields[3]);
temp[j] = System.Convert.ToDouble(fields[4]);
}
}
public void Start()
{
Thread thread = new Thread(read);
thread.Start();
//gradient.Evaluate()
//var main = particleSystem.main;
//main.maxParticles = 200;
}
private const FileOptions DefaultOptions = FileOptions.Asynchronous | FileOptions.SequentialScan;
public static Task<string[]> ReadAllLinesAsync(string path) => ReadAllLinesAsync(path, Encoding.UTF8);
public static async Task<string[]> ReadAllLinesAsync(string path, Encoding encoding)
{
var lines = new List<string>();
// Open the FileStream with the same FileMode, FileAccess
// and FileShare as a call to File.OpenText would've done.
using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, DefaultBufferSize, DefaultOptions))
using (var reader = new StreamReader(stream, encoding))
{
string line;
while ((line = await reader.ReadLineAsync()) != null)
{
lines.Add(line);
}
}
return lines.ToArray();
}

Reading and parsing the file apparently costs 5 seconds on your system. I don't think reading it line by line is the fastest approach, but anyway, don't parse the file for each request.
Read it once on application startup, and cache it in an appropriate data type.

In general if your search is line based is better to read line one by one instead to read all the file:
using (StreamReader reader = new StreamReader("filename"))
{
while (true)
{
string line = await reader.ReadLineAsync();
if (line == null)
{
break;
}
//logic here...
}
}

Filestream does not free file for other processes

I have the following code to write a file. The problem is that if I want to overwrite the same file, it is "locked". The file is opened by another process.
using (FileStream fs = new FileStream("C:\\New\\" + fileName, FileMode.Create, FileAccess.ReadWrite))
using (StreamWriter str = new StreamWriter(fs))
{
str.Write(jsonFile);
str.Dispose();
str.Close();
}
I send a json string to an API, which then generates the file. So I guess it might be a problem in IIS.
EDIT:
By research I have still tried the following code, but which leads to the same result
using (FileStream fs = new FileStream("C:\\New\\" + fileName, FileMode.Create, FileAccess.ReadWrite))
{
StreamWriter sw = new StreamWriter(fs);
sw.Write(jsonFile);
fs.Flush();
fs.Close();
}
EDIT 2:
After reading the comments, it probably has nothing to do with the Filestream to itself. Here is more information about my application:
I have a WPF application which sends a post to my API through a ButtonClick. This is triggered as follows:
private async void btnSend_Click(object sender, RoutedEventArgs e)
{
await Seal();
}
The Seal method says the following:
private async System.Threading.Tasks.Task Seal()
{
var result = await RequestManager.DoPost<bool>("FOO", foo);
}
The RequestManager says the following:
public static async Task<R> DoPost<R>(String route, Object payload, String contenttype)
{
var serializer = new JavaScriptSerializer();
route = route.StartsWith("/") ? route : "/" + route;
var content = new StringContent(serializer.Serialize(payload), Encoding.UTF8, contenttype);
var response = await client.PostAsync(RequestManager.API_URL + route, content);
if (response.StatusCode == HttpStatusCode.OK)
{
var result = await response.Content.ReadAsStringAsync();
return (R)serializer.Deserialize(result, typeof(R));
}
else
{
throw new ResponseException(response.StatusCode, response.Content.ReadAsStringAsync().Result);
}
}
I really do not know where my error is, or where the request must be closed.

This error could occur when multiple threads are attempting to write at the same file. Your code above works with a little modification
private static object lk = new object();
lock (lk)
{
using (FileStream fs = new FileStream("C:\\New\\" +fileName,FileMode.Create, FileAccess.ReadWrite))
using (StreamWriter str = new StreamWriter(fs))
{
str.Write(jsonFile);
str.Dispose();
str.Close();
}
}

How to save data from stream to file in WinRT?

I would like to have a method which takes as an input a System.IO.Stream and use it to write data from it to file. So far I have the following:
public async Task SaveStreamToFileX(Stream stream, string filePath, IProgress<long> progress)
{
var folder = await StorageFolder.GetFolderFromPathAsync(Path.GetDirectoryName(filePath));
var fileName = Path.GetFileName(filePath);
StorageFile file = await folder.CreateFileAsync(fileName, CreationCollisionOption.OpenIfExists);
var istream = stream.AsInputStream();
var canRead = stream.CanRead; //this returns true
using (var reader = new DataReader(istream))
{
using (IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter writer = new DataWriter(outputStream))
{
IBuffer buffer;
long readBytes = 0;
const int bufferSize = 8192;
while ((buffer = reader.ReadBuffer(bufferSize)).Length > 0) //exception System.Exception with message: Out of range ...
{
writer.WriteBuffer(buffer);
readBytes += bufferSize;
progress.Report(readBytes);
}
}
}
}
}
}
Problem is, that an exception (Out of range) is thrown when I try to read data in while cycle (first read). Stream should have data. I am not sure if so long code is neccesarry, if somebody has better solution it would be great.
Side note:
If I try await reader.LoadAsync(50) it returns 50. I am not sure what LoadAsync should do. Maybe I have to call it before read to prepare data for the read? I will investigate this further ...
Also, the Stream.CanRead returns true.

The problem was not with converting streams as I orginally thought. It was just lack of the knowlidge how the work with files is done in WinRT (the docs from microsoft are really terrible in my opinion).
In the end with help of my collegue with tried several ways and end up with the following:
public async Task SaveStreamToFile(Stream stream, string filePath, IProgress<long> progress )
{
var folder = await StorageFolder.GetFolderFromPathAsync(Path.GetDirectoryName(filePath));
var fileName = Path.GetFileName(filePath);
StorageFile file = await folder.CreateFileAsync(fileName, CreationCollisionOption.OpenIfExists);
var istream = stream.AsInputStream();
using (var reader = new DataReader(istream))
{
using (IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter writer = new DataWriter(outputStream))
{
long writtenBytes = 0;
const int bufferSize = 8192;
uint loadedBytes = 0;
while ((loadedBytes = (await reader.LoadAsync(bufferSize))) > 0) //!!!
{
IBuffer buffer = reader.ReadBuffer(loadedBytes);
writer.WriteBuffer(buffer);
uint tmpWritten = await writer.StoreAsync(); //!!!
writtenBytes += tmpWritten;
progress.Report(writtenBytes);
}
}
}
}
}
}
I would like to see some simpler implementation, but this works. The problems were that LoadAsync was missing (which seems to be necessary to call) and during write operation the StoreAsync must be called in order to commit the data (flushing was not sufficient).
I hope this help somebody.

I would advise against that kind of code and instead take advantage of Windows Runtime Interop extension methods. That would produce a neater and more readable code, ex :
private async Task CopyToTempFile(Stream stream, string temporaryFileName) {
var file = await ApplicationData.Current.TemporaryFolder.CreateFileAsync(temporaryFileName, CreationCollisionOption.ReplaceExisting);
using (var outputstream = await file.OpenStreamForWriteAsync()) {
await stream.CopyToAsync(outputstream);
}
}

Writing to a file asynchronously

Is there any way to write an asynchronous function that writes to data to a file repeatedly.
I am getting the following error when I write asynchronous function
The process cannot access the file 'c:\Temp\Data.txt' because it is being used by another process
public void GoButton_Click(object sender, System.EventArgs e)
{
IAsyncResult ar = DoSomethingAsync(strURL, strInput);
Session["result"] = ar;
Response.Redirect("wait1.aspx");
}
private IAsyncResult DoSomethingAsync(string strURL, string strInput)
{
DoSomethingDelegate doSomethingDelegate = new DoSomethingDelegate(DoSomething);
IAsyncResult ar = doSomethingDelegate.BeginInvoke(strURL, strInput, new AsyncCallback(MyCallback), null);
return ar;
}
private delegate void DoSomethingDelegate(string strURL, string strInput);
private void MyCallback(IAsyncResult ar)
{
AsyncResult aResult = (AsyncResult)ar;
DoSomethingDelegate doSomethingDelegate = (DoSomethingDelegate)aResult.AsyncDelegate;
doSomethingDelegate.EndInvoke(ar);
}
private void DoSomething(string strURL, string strInput)
{
int i = 0;
for (i = 0; i < 1000; i++)
{
m_streamWriter.BaseStream.Seek(0, SeekOrigin.End);
m_streamWriter.WriteLine("{0} ", MethodCall(strURL, strInput));
m_streamWriter.Flush();
m_streamWriter.Close();
}
}

Well I had the same problem. And solved it now. It is kind of late suggestion but may be help for others.
Include the following using statements in the console examples below.
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading.Tasks;
Use of the FileStream Class
The examples below use the FileStream class, which has an option that causes asynchronous I/O to occur at the operating system level. In many cases, this will avoid blocking a ThreadPool thread. To enable this option, you must specify the useAsync=true or options=FileOptions.Asynchronous argument in the constructor call.
StreamReader and StreamWriter do not have this option if you open them directly by specifying a file path. StreamReader/Writer do have this option if you provide them a Stream that was opened by the FileStream class. Note that asynchrony provides a responsiveness advantage in UI apps even if a thread pool thread is blocked, since the UI thread is not blocked during the wait.
Writing Text
The following example writes text to a file. At each await statement, the method immediately exits. When the file I/O is complete, the method resumes at the statement following the await statement. Note that the async modifier is in the definition of methods that use the await statement.
static void Main(string[] args)
{
ProcessWrite().Wait();
Console.Write("Done ");
Console.ReadKey();
}
static Task ProcessWrite()
{
string filePath = #"c:\temp2\temp2.txt";
string text = "Hello World\r\n";
return WriteTextAsync(filePath, text);
}
static async Task WriteTextAsync(string filePath, string text)
{
byte[] encodedText = Encoding.Unicode.GetBytes(text);
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Append, FileAccess.Write, FileShare.None,
bufferSize: 4096, useAsync: true))
{
await sourceStream.WriteAsync(encodedText, 0, encodedText.Length);
};
}
Reading Text
The following example reads text from a file. The text is buffered and, in this case, placed into a StringBuilder. Unlike in the previous example, the evaluation of the await produces a value. The ReadAsync method returns a Task, so the evaluation of the await produces an Int32 value (numRead) that is returned after the operation completes..
static void Main(string[] args)
{
ProcessRead().Wait();
Console.Write("Done ");
Console.ReadKey();
}
static async Task ProcessRead()
{
string filePath = #"c:\temp2\temp2.txt";
if (File.Exists(filePath) == false)
{
Console.WriteLine("file not found: " + filePath);
}
else {
try {
string text = await ReadTextAsync(filePath);
Console.WriteLine(text);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
static async Task<string> ReadTextAsync(string filePath)
{
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 4096, useAsync: true))
{
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[0x1000];
int numRead;
while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
string text = Encoding.Unicode.GetString(buffer, 0, numRead);
sb.Append(text);
}
return sb.ToString();
}
}
Original source was here but unfortunately the link seems dead now.
New source can be found here.
Hope that helps...

Example of a helper method to handle async writing to a file.
public async Task FileWriteAsync(string filePath, string messaage, bool append = true)
{
using (FileStream stream = new FileStream(filePath, append ? FileMode.Append : FileMode.Create, FileAccess.Write, FileShare.None, 4096, true))
using (StreamWriter sw = new StreamWriter(stream))
{
await sw.WriteLineAsync(messaage);
}
}

Writing asynchronously to the file will not solve this issue. You'll need to wait for the file to be available.

If you use a simple StreamWriter, you could replace it with a simple class. No need for async/await. This is an example of writing a text file.
using System;
using System.Collections.Concurrent;
using System.IO;
using System.Threading.Tasks;
class LogWriter : IDisposable
{
private BlockingCollection<string> blockingCollection = new BlockingCollection<string>();
private StreamWriter log = null;
bool run = true;
Task task = null;
public LogWriter(string logFilePath)
{
log = new StreamWriter(logFilePath);
task = Task.Run(() =>
{
while (run)
{
log.WriteLine(blockingCollection.Take());
}
});
}
public void WriteLine(string value)
{
blockingCollection.Add(value);
}
public void Dispose()
{
run = false;
task.Dispose();
log.Close();
log.Dispose();
}
}
To use it, do just like you would do with a StreamWriter:
using (var log = new LogWriter(logFileName))
{
log.WriteLine("Hello world");
// Code here that should not be blocked by writing to the file
}

Simple and straightforward solution:
using var file = new StreamWriter(path);
await file.WriteAsync(content);

The accepted answer has the common async pitfall - the buffers are not flushed async-ly. Check this out: https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#always-call-flushasync-on-streamwriters-or-streams-before-calling-dispose
Either use the new await using or flush the buffer manually before disposing
await using (var file = new StreamWriter(path)) //mind the "await using"
{
await file.WriteAsync(content);
}
or
using (var streamWriter = new StreamWriter(context.Response.Body))
{
await streamWriter.WriteAsync("Hello World");
await streamWriter.FlushAsync();
}

Ultimately it depends why you're trying to do it.
If you aren't going to be writing too much data to the file, you can constantly open and close it.
Alternatively, if you know when you want the file open and when you want it closed, you can open it when it's needed, then keep it open for writing until the point you know it's no longer needed.