Develop Reference

Filestream does not free file for other processes

I have the following code to write a file. The problem is that if I want to overwrite the same file, it is "locked". The file is opened by another process.
using (FileStream fs = new FileStream("C:\\New\\" + fileName, FileMode.Create, FileAccess.ReadWrite))
using (StreamWriter str = new StreamWriter(fs))
{
str.Write(jsonFile);
str.Dispose();
str.Close();
}
I send a json string to an API, which then generates the file. So I guess it might be a problem in IIS.
EDIT:
By research I have still tried the following code, but which leads to the same result
using (FileStream fs = new FileStream("C:\\New\\" + fileName, FileMode.Create, FileAccess.ReadWrite))
{
StreamWriter sw = new StreamWriter(fs);
sw.Write(jsonFile);
fs.Flush();
fs.Close();
}
EDIT 2:
After reading the comments, it probably has nothing to do with the Filestream to itself. Here is more information about my application:
I have a WPF application which sends a post to my API through a ButtonClick. This is triggered as follows:
private async void btnSend_Click(object sender, RoutedEventArgs e)
{
await Seal();
}
The Seal method says the following:
private async System.Threading.Tasks.Task Seal()
{
var result = await RequestManager.DoPost<bool>("FOO", foo);
}
The RequestManager says the following:
public static async Task<R> DoPost<R>(String route, Object payload, String contenttype)
{
var serializer = new JavaScriptSerializer();
route = route.StartsWith("/") ? route : "/" + route;
var content = new StringContent(serializer.Serialize(payload), Encoding.UTF8, contenttype);
var response = await client.PostAsync(RequestManager.API_URL + route, content);
if (response.StatusCode == HttpStatusCode.OK)
{
var result = await response.Content.ReadAsStringAsync();
return (R)serializer.Deserialize(result, typeof(R));
}
else
{
throw new ResponseException(response.StatusCode, response.Content.ReadAsStringAsync().Result);
}
}
I really do not know where my error is, or where the request must be closed.

This error could occur when multiple threads are attempting to write at the same file. Your code above works with a little modification
private static object lk = new object();
lock (lk)
{
using (FileStream fs = new FileStream("C:\\New\\" +fileName,FileMode.Create, FileAccess.ReadWrite))
using (StreamWriter str = new StreamWriter(fs))
{
str.Write(jsonFile);
str.Dispose();
str.Close();
}
}

C# Decompress .GZip to file

I have this Code
using System.IO;
using System.IO.Compression;
...
UnGzip2File("input.gz","output.xls");
Which run this procedure, it runs without error but after it, the input.gz is empty and created output.xls is also empty. At the start input.gz had 12MB. What am i doing wrong ? Or have you better/functional solution ?
public static void UnGzip2File(string inputPath, string outputPath)
{
FileStream inputFileStream = new FileStream(inputPath, FileMode.Create);
FileStream outputFileStream = new FileStream(outputPath, FileMode.Create);
using (GZipStream gzipStream = new GZipStream(inputFileStream, CompressionMode.Decompress))
{
byte[] bytes = new byte[4096];
int n;
// To be sure the whole file is correctly read,
// you should call FileStream.Read method in a loop,
// even if in the most cases the whole file is read in a single call of FileStream.Read method.
while ((n = gzipStream.Read(bytes, 0, bytes.Length)) != 0)
{
outputFileStream.Write(bytes, 0, n);
}
}
outputFileStream.Dispose();
inputFileStream.Dispose();
}

Opening the FileStream with the FileMode.Create will overwrite the existing file as documented here. This will cause the file to be empty when you try to decompress it, which in turn leads to an empty output-file.
Below is a working code sample, note that it is async, this can be changed by leaving out async/await and changing the call to the regular CopyTo-method and changing the return type to void.
public static async Task DecompressGZip(string inputPath, string outputPath)
{
using (var input = File.OpenRead(inputPath))
using (var output = File.OpenWrite(outputPath))
using (var gz = new GZipStream(input, CompressionMode.Decompress))
{
await gz.CopyToAsync(output);
}
}

Is there an async alternative to the file operations in .Net?

I am using c# with .Net 4.5. So I can use the async/await feature of .Net 4.5/ However I don't find the async methods for File.Copy, File.Move, Directory.GetFiles etc in System.IO.

There are many ways to do that.
Look to this example I have copied for you from msdn.
You can do that by using FileStream and CopyToAsync:
using System;
using System.Threading.Tasks;
using System.Windows;
using System.IO;
namespace WpfApplication
{
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
}
private async void Button_Click(object sender, RoutedEventArgs e)
{
string StartDirectory = #"c:\Users\exampleuser\start";
string EndDirectory = #"c:\Users\exampleuser\end";
foreach (string filename in Directory.EnumerateFiles(StartDirectory))
{
using (FileStream SourceStream = File.Open(filename, FileMode.Open))
{
using (FileStream DestinationStream = File.Create(EndDirectory + filename.Substring(filename.LastIndexOf('\\'))))
{
await SourceStream.CopyToAsync(DestinationStream);
}
}
}
}
}
}
Or by using StreamReader and StreamWriter objects to read and write the contents of a file asynchronously.
private async void Button_Click(object sender, RoutedEventArgs e)
{
string UserDirectory = #"c:\Users\exampleuser\";
using (StreamReader SourceReader = File.OpenText(UserDirectory + "BigFile.txt"))
{
using (StreamWriter DestinationWriter = File.CreateText(UserDirectory + "CopiedFile.txt"))
{
await CopyFilesAsync(SourceReader, DestinationWriter);
}
}
}
public async Task CopyFilesAsync(StreamReader Source, StreamWriter Destination)
{
char[] buffer = new char[0x1000];
int numRead;
while ((numRead = await Source.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
await Destination.WriteAsync(buffer, 0, numRead);
}
}
More info:
http://msdn.microsoft.com/en-us/library/kztecsys(v=vs.110).aspx

No, there are not async equivalents for Copy/Move/etc. You can build them yourself if necessary.
GetFiles is a bit different. There is an EnumerateFiles which is more responsive (but not actually async).

For copying you could make use of Stream.CopyToAsync method. Rest all you can wrap calls to corresponding methods in Task.Run

How to Async Files.ReadAllLines and await for results?

I have the following code,
private void button1_Click(object sender, RoutedEventArgs e)
{
button1.IsEnabled = false;
var s = File.ReadAllLines("Words.txt").ToList(); // my WPF app hangs here
// do something with s
button1.IsEnabled = true;
}
Words.txt has a ton of words which i read into the s variable, I am trying to make use of async and await keywords in C# 5 using Async CTP Library so the WPF app doesn't hang. So far I have the following code,
private async void button1_Click(object sender, RoutedEventArgs e)
{
button1.IsEnabled = false;
Task<string[]> ws = Task.Factory.FromAsync<string[]>(
// What do i have here? there are so many overloads
); // is this the right way to do?
var s = await File.ReadAllLines("Words.txt").ToList(); // what more do i do here apart from having the await keyword?
// do something with s
button1.IsEnabled = true;
}
The goal is to read the file in async rather than sync, to avoid freezing of WPF app.
Any help is appreciated, Thanks!

UPDATE: Async versions of File.ReadAll[Lines|Bytes|Text], File.AppendAll[Lines|Text] and File.WriteAll[Lines|Bytes|Text] have now been merged into .NET Core and shipped with .NET Core 2.0. They are also included in .NET Standard 2.1.
Using Task.Run, which essentially is a wrapper for Task.Factory.StartNew, for asynchronous wrappers is a code smell.
If you don't want to waste a CPU thread by using a blocking function, you should await a truly asynchronous IO method, StreamReader.ReadToEndAsync, like this:
using (var reader = File.OpenText("Words.txt"))
{
var fileText = await reader.ReadToEndAsync();
// Do something with fileText...
}
This will get the whole file as a string instead of a List<string>. If you need lines instead, you could easily split the string afterwards, like this:
using (var reader = File.OpenText("Words.txt"))
{
var fileText = await reader.ReadToEndAsync();
return fileText.Split(new[] { Environment.NewLine }, StringSplitOptions.None);
}
EDIT: Here are some methods to achieve the same code as File.ReadAllLines, but in a truly asynchronous manner. The code is based on the implementation of File.ReadAllLines itself:
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading.Tasks;
public static class FileEx
{
/// <summary>
/// This is the same default buffer size as
/// <see cref="StreamReader"/> and <see cref="FileStream"/>.
/// </summary>
private const int DefaultBufferSize = 4096;
/// <summary>
/// Indicates that
/// 1. The file is to be used for asynchronous reading.
/// 2. The file is to be accessed sequentially from beginning to end.
/// </summary>
private const FileOptions DefaultOptions = FileOptions.Asynchronous | FileOptions.SequentialScan;
public static Task<string[]> ReadAllLinesAsync(string path)
{
return ReadAllLinesAsync(path, Encoding.UTF8);
}
public static async Task<string[]> ReadAllLinesAsync(string path, Encoding encoding)
{
var lines = new List<string>();
// Open the FileStream with the same FileMode, FileAccess
// and FileShare as a call to File.OpenText would've done.
using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, DefaultBufferSize, DefaultOptions))
using (var reader = new StreamReader(stream, encoding))
{
string line;
while ((line = await reader.ReadLineAsync()) != null)
{
lines.Add(line);
}
}
return lines.ToArray();
}
}

Here are the helper methods I've created for a NetStandart 2.0 class library, that was used both in NetCore 3.1 and NetFramework 4.7.2 projects.
These implementations have matched exactly the names and signatures of the net core 3.1 / net standard 2.1 File class methods, so you only need to put them in any public class. (FileHelper for example...):
Also, this should be most efficient and similar to the source code of .net implementation.
private const int DefaultBufferSize = 4096;
// File accessed asynchronous reading and sequentially from beginning to end.
private const FileOptions DefaultOptions = FileOptions.Asynchronous | FileOptions.SequentialScan;
public static async Task WriteAllTextAsync(string filePath, string text)
{
byte[] encodedText = Encoding.Unicode.GetBytes(text);
using FileStream sourceStream = new FileStream(filePath, FileMode.Append, FileAccess.Write, FileShare.None,
DefaultBufferSize, true);
await sourceStream.WriteAsync(encodedText, 0, encodedText.Length);
}
public static async Task<IEnumerable<string>> ReadAllLinesAsync(string filePath)
{
var lines = new List<string>();
using var sourceStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
DefaultBufferSize, DefaultOptions);
using var reader = new StreamReader(sourceStream, Encoding.Unicode);
string line;
while ((line = await reader.ReadLineAsync()) != null) lines.Add(line);
return lines;
}
public static async Task<string> ReadAllTextAsync(string filePath)
{
using var sourceStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
DefaultBufferSize, DefaultOptions);
using var reader = new StreamReader(sourceStream, Encoding.Unicode);
return await reader.ReadToEndAsync();
}
Edit
Apparently that the StreamReader "async" methods block the current thread for a considerable amount of time before returning an incomplete Task.
(Even the netcore 3.1 File.ReadAllLinesAsyn,File.ReadAllTextAsync currently aren't seems to be fully async. as you can check in source code, they are based on the StreamReader "async" methods).
So, I'm sharing an implementation that seems like the most efficient way currently. \
It's better than options like to run the sync methods in Task.Run(()=>File.ReadAllLines(...)), since its a very bad practice to wrap your sync code with Task.Run and expect this to be full async flow. \ Actually, it breaks the internal queues mechanism of the real asynchronous dotnet structure.
public static async Task<string> ReadAllTextAsync(string filePath)
{
using (var sourceStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read,
DefaultBufferSize, DefaultOptions))
{
var sb = new StringBuilder();
var buffer = new byte[0x1000];
var numRead = 0;
while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
sb.Append(Encoding.Unicode.GetString(buffer, 0, numRead));
return sb.ToString();
}
}
Testing Time
Here is my test and its output that displays clearly that the actual run is async:
var stopwatch = Stopwatch.StartNew();
var fileTask = FileHelper.ReadAllTextAsync("48MB_file.txt");
var duration1 = stopwatch.ElapsedMilliseconds;
var isCompleted = fileTask.IsCompleted;
stopwatch.Restart();
await fileTask;
var duration2 = stopwatch.ElapsedMilliseconds;
Console.WriteLine($"Creation took: {duration1:#,0} ms, Task.IsCompleted: {isCompleted}");
Console.WriteLine($"Calling await took: {duration2:#,0} ms, Task.IsCompleted: {fileTask.IsCompleted}");
Creation took: 43 ms, Task.IsCompleted: False
Calling await took: 508 ms, Task.IsCompleted: True
You can find more in the comments, and in this question: File.ReadAllLinesAsync() blocks the UI thread

private async Task<string> readFile(string sourceFilePath)
{
using (var fileStream = new FileStream(sourceFilePath, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite))
{
using (var streamReader = new StreamReader(fileStream))
{
string data = await streamReader.ReadToEndAsync().ConfigureAwait(false);
streamReader.Close();
fileStream.Close();
return data;
}
}
}

Try this:
private async void button1_Click(object sender, RoutedEventArgs e)
{
button1.IsEnabled = false;
try
{
var s = await Task.Run(() => File.ReadAllLines("Words.txt").ToList());
// do something with s
}
finally
{
button1.IsEnabled = true;
}
}
Edit:
You don't need the try-finally for this to work. It's really only the one line that you need to change. To explain how it works: This spawns another thread (actually gets one from the thread pool) and gets that thread to read the file. When the file is finished reading then the remainder of the button1_Click method is called (from the GUI thread) with the result. Note that this is probably not the most efficient solution, but it is probably the simplest change to your code which doesn't block the the GUI.

I also encountered a problem described in your question. I've solved it just simplier that in previous answers:
string[] values;
StorageFolder folder = ApplicationData.Current.LocalFolder; // Put your location here.
IList<string> lines = await FileIO.ReadLinesAsync(await folder.GetFileAsync("Words.txt"););
lines.CopyTo(values, 0);

Writing to a file asynchronously

Is there any way to write an asynchronous function that writes to data to a file repeatedly.
I am getting the following error when I write asynchronous function
The process cannot access the file 'c:\Temp\Data.txt' because it is being used by another process
public void GoButton_Click(object sender, System.EventArgs e)
{
IAsyncResult ar = DoSomethingAsync(strURL, strInput);
Session["result"] = ar;
Response.Redirect("wait1.aspx");
}
private IAsyncResult DoSomethingAsync(string strURL, string strInput)
{
DoSomethingDelegate doSomethingDelegate = new DoSomethingDelegate(DoSomething);
IAsyncResult ar = doSomethingDelegate.BeginInvoke(strURL, strInput, new AsyncCallback(MyCallback), null);
return ar;
}
private delegate void DoSomethingDelegate(string strURL, string strInput);
private void MyCallback(IAsyncResult ar)
{
AsyncResult aResult = (AsyncResult)ar;
DoSomethingDelegate doSomethingDelegate = (DoSomethingDelegate)aResult.AsyncDelegate;
doSomethingDelegate.EndInvoke(ar);
}
private void DoSomething(string strURL, string strInput)
{
int i = 0;
for (i = 0; i < 1000; i++)
{
m_streamWriter.BaseStream.Seek(0, SeekOrigin.End);
m_streamWriter.WriteLine("{0} ", MethodCall(strURL, strInput));
m_streamWriter.Flush();
m_streamWriter.Close();
}
}

Well I had the same problem. And solved it now. It is kind of late suggestion but may be help for others.
Include the following using statements in the console examples below.
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading.Tasks;
Use of the FileStream Class
The examples below use the FileStream class, which has an option that causes asynchronous I/O to occur at the operating system level. In many cases, this will avoid blocking a ThreadPool thread. To enable this option, you must specify the useAsync=true or options=FileOptions.Asynchronous argument in the constructor call.
StreamReader and StreamWriter do not have this option if you open them directly by specifying a file path. StreamReader/Writer do have this option if you provide them a Stream that was opened by the FileStream class. Note that asynchrony provides a responsiveness advantage in UI apps even if a thread pool thread is blocked, since the UI thread is not blocked during the wait.
Writing Text
The following example writes text to a file. At each await statement, the method immediately exits. When the file I/O is complete, the method resumes at the statement following the await statement. Note that the async modifier is in the definition of methods that use the await statement.
static void Main(string[] args)
{
ProcessWrite().Wait();
Console.Write("Done ");
Console.ReadKey();
}
static Task ProcessWrite()
{
string filePath = #"c:\temp2\temp2.txt";
string text = "Hello World\r\n";
return WriteTextAsync(filePath, text);
}
static async Task WriteTextAsync(string filePath, string text)
{
byte[] encodedText = Encoding.Unicode.GetBytes(text);
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Append, FileAccess.Write, FileShare.None,
bufferSize: 4096, useAsync: true))
{
await sourceStream.WriteAsync(encodedText, 0, encodedText.Length);
};
}
Reading Text
The following example reads text from a file. The text is buffered and, in this case, placed into a StringBuilder. Unlike in the previous example, the evaluation of the await produces a value. The ReadAsync method returns a Task, so the evaluation of the await produces an Int32 value (numRead) that is returned after the operation completes..
static void Main(string[] args)
{
ProcessRead().Wait();
Console.Write("Done ");
Console.ReadKey();
}
static async Task ProcessRead()
{
string filePath = #"c:\temp2\temp2.txt";
if (File.Exists(filePath) == false)
{
Console.WriteLine("file not found: " + filePath);
}
else {
try {
string text = await ReadTextAsync(filePath);
Console.WriteLine(text);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
static async Task<string> ReadTextAsync(string filePath)
{
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 4096, useAsync: true))
{
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[0x1000];
int numRead;
while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
string text = Encoding.Unicode.GetString(buffer, 0, numRead);
sb.Append(text);
}
return sb.ToString();
}
}
Original source was here but unfortunately the link seems dead now.
New source can be found here.
Hope that helps...

Example of a helper method to handle async writing to a file.
public async Task FileWriteAsync(string filePath, string messaage, bool append = true)
{
using (FileStream stream = new FileStream(filePath, append ? FileMode.Append : FileMode.Create, FileAccess.Write, FileShare.None, 4096, true))
using (StreamWriter sw = new StreamWriter(stream))
{
await sw.WriteLineAsync(messaage);
}
}

Writing asynchronously to the file will not solve this issue. You'll need to wait for the file to be available.

If you use a simple StreamWriter, you could replace it with a simple class. No need for async/await. This is an example of writing a text file.
using System;
using System.Collections.Concurrent;
using System.IO;
using System.Threading.Tasks;
class LogWriter : IDisposable
{
private BlockingCollection<string> blockingCollection = new BlockingCollection<string>();
private StreamWriter log = null;
bool run = true;
Task task = null;
public LogWriter(string logFilePath)
{
log = new StreamWriter(logFilePath);
task = Task.Run(() =>
{
while (run)
{
log.WriteLine(blockingCollection.Take());
}
});
}
public void WriteLine(string value)
{
blockingCollection.Add(value);
}
public void Dispose()
{
run = false;
task.Dispose();
log.Close();
log.Dispose();
}
}
To use it, do just like you would do with a StreamWriter:
using (var log = new LogWriter(logFileName))
{
log.WriteLine("Hello world");
// Code here that should not be blocked by writing to the file
}

Simple and straightforward solution:
using var file = new StreamWriter(path);
await file.WriteAsync(content);

The accepted answer has the common async pitfall - the buffers are not flushed async-ly. Check this out: https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#always-call-flushasync-on-streamwriters-or-streams-before-calling-dispose
Either use the new await using or flush the buffer manually before disposing
await using (var file = new StreamWriter(path)) //mind the "await using"
{
await file.WriteAsync(content);
}
or
using (var streamWriter = new StreamWriter(context.Response.Body))
{
await streamWriter.WriteAsync("Hello World");
await streamWriter.FlushAsync();
}

Ultimately it depends why you're trying to do it.
If you aren't going to be writing too much data to the file, you can constantly open and close it.
Alternatively, if you know when you want the file open and when you want it closed, you can open it when it's needed, then keep it open for writing until the point you know it's no longer needed.