I've had 3 reports now of user's machines crashing while using my software.. the crashes are not related to my program but when they restart the config files my program writes are all corrupt.
There is nothing special to how the files are being written, simply creating a Json representation and dumping it to disk using File.WriteAllText()
// save our contents to the disk
string json = JsonConvert.SerializeObject(objectInfo, Formatting.Indented);
// write the contents
File.WriteAllText(path, json);
I've had a user send me one of the files and the length looks about right (~3kb) but the contents are all 0x00.
According to the post below File.WriteAllText should close the file handle, flushing any unwritten contents to the disk:
In my C# code does the computer wait until output is complete before moving on?
BUT, as pointed out by Alberto in the comments:
System.IO.File.WriteAllText when completes, will flush all the text to
the filesystem cache, then, it will be lazily written to the drive.
So I presume what is happening here is that the file is being cleared and initialized with 0x00 but the data is not yet written when the system crashes.
I was thinking of maybe using some sort of temp file so the process would be like this:
Write new contents to temp file
Delete original file
Rename temp file to original
I don't think that will solve the problem as I presume Windows will just move the file even though the IO is still pending.
Is there any way I can force the machine to dump that data to disk instead of it deciding when to do it or perhaps a better way to update a file?
UPDATE:
Based on suggestions by #usr, #mikez and #llya luzyanin I've created a new WriteAllText function that performs the write using the following logic:
Create a temp file with the new contents using the FileOptions.WriteThrough flag
Writes the data to disk (won't return until the write has completed)
File.Replace to copy the contents of the new temp file to the real file, making a backup
With that logic, if the final file fails to load, my code an check for a backup file and load that instead
Here is the code:
public static void WriteAllTextWithBackup(string path, string contents)
{
// generate a temp filename
var tempPath = Path.GetTempFileName();
// create the backup name
var backup = path + ".backup";
// delete any existing backups
if (File.Exists(backup))
File.Delete(backup);
// get the bytes
var data = Encoding.UTF8.GetBytes(contents);
// write the data to a temp file
using (var tempFile = File.Create(tempPath, 4096, FileOptions.WriteThrough))
tempFile.Write(data, 0, data.Length);
// replace the contents
File.Replace(tempPath, path, backup);
}
You can use FileStream.Flush to force the data to disk. Write to a temp file and use File.Replace to atomically replace the target file.
I believe this is guaranteed to work. File systems give weak guarantees. These guarantees are hardly ever documented and they are complex.
Alternatively, you can use Transactional NTFS if available. It is available for .NET.
FileOptions.WriteThrough can replace Flush but you still need the temp file if your data can exceed a single cluster in size.
Related
I am trying to copy data of one file and pasting it into the same file which means overwriting it. I learn that overwrite the file data using file.copy() cannot be possible. but from the one of the post says that if I use File.Copy(Source file, destination file, true) than overwrite is possible. I tried to use but it didn't work. if anyone have solution for that than it will be big help.
public void EditRole(string oldRole, string newRole)
{
if (File.Exists(roleXMLLoc))
{
XDocument doc = XDocument.Load(roleXMLLoc);
var edit = doc.Element("Roles").Elements("Role").Where(x => x.Value == oldRole).SingleOrDefault();
edit.Value = newRole;
doc.Save(roleXMLLoc);
if (File.Exists(userPermissionLoc))
{
File.Copy(userPermissionLoc, userPermissionLoc, true); SaveData();
}
}
}
To read or write files you should open a Stream (https://learn.microsoft.com/en-us/dotnet/standard/io/). If you need to read and write in the same time, then you should use 2 separate streams for reading and writing into 2 separate files. Operating system will protect file from being opened by 2nd stream, otherwise you would fall into problems of concurrency: you read the inactual data that is already being changed by writing stream.
The logic can be:
Read file and close Reading stream. Then open file for writing and do what you need.
OR
Open Reading stream for fileToRead.xml and open Writing stream for temporary fileToWrite.xml. After you done, you can close both streams, remove the source file and rename the new file.
I have a function that checks every file in a directory and writes a list to the console. The problem is, I don't want it to include files that are currently being copied to the directory, I only want it to show the files that are complete. How do I do that? Here is my code:
foreach (string file in Directory.EnumerateFiles("C:\folder"))
{
Console.WriteLine(file);
}
There's really no way to tell "being copied" vs "locked for writing by something". Relevant: How to check for file lock? and Can I simply 'read' a file that is in use?
If you want to simply display a list of files that are not open for writing, you can do that by attempting to open them:
foreach (string file in Directory.EnumerateFiles("C:\folder"))
{
try {
using (var file = file.Open(file, FileMode.Open, FileAccess.Read, FileShare.ReadWrite) {
Console.WriteLine(file);
}
} catch {
// file is in use
continue;
}
}
However -- lots of caveats.
Immediately after displaying the filename (end of the using block) the file could be opened by something else
The process writing the file may have used FileShare.Read which means the call will succeed, despite it being written to.
I'm not sure what exactly you're up to here, but it sounds like two processes sharing a queue directory: one writing, one reading/processing. The biggest challenge is that writing a file takes time, and so your "reading" process ends up picking it up and trying to read it before the whole file is there, which will fail in some way depending on the sharing mode, how your apps are written, etc.
A common pattern to deal with this situation is to use an atomic file operation like Move:
Do the (slow) write/copy operation to a temporary directory that's on the same file system (very important) as the queue directory
Once complete, do a Move from the temporary directory to the queue directory.
Since move is atomic, the file will either not be there, or it will be 100% there -- there is no opportunity for the "reading" process to ever see the file while it's partially there.
Note that if you do the move across file systems, it will act the same as a copy.
There's no "current files being copied" list stored anywhere in Windows/.NET/whatever. Probably the most you could do is attempt to open each file for append and see if you get an exception. Depending on the size and location of your directory, and on the security setup, that may not be an option.
There isn't a clean way to do this, but this... works...
foreach (var file in new DirectoryInfo(#"C:\Folder").GetFiles())
{
try
{
file.OpenRead();
}
catch
{
continue;
}
Console.WriteLine(file.Name);
}
I have a problem aith multithreading when copying and accessing files.
I have a service, that downloads and unpacks a Zip archive, then it copies a file from unzipped folder to the right location:
//Download, and uzip archive...
//Copy a needed file to its right location
File.Copy(fileName, fileDestination);
Then I start a separate thread, that needs to access the copied files:
TheadPool.QueueUserWorkItem(s => processCopiedFile(fileDestination));
Here's the code fragment from ProcessCopiedFile:
private void ProcessCopiedFile(string filePath)
{
...
//Load the file, previously copied here
var xml = XDocument.Load(filePath);
...
//Do other work...
}
The XDoument.Load call fails with exception:
The process cannot access the file <FileName> because it is used by another process.
Seems like File.Copy keeps the result file locked. When do all work synchronuously, it works without errors.
Have you any thoughts?
Thx.
File.Copy does not keep anything open or locked, it is an atomic operation which requires some time, depending of course on Disk/Network I/O and file size.
Of course while moving from sync to async you should make sure you do not access the destination file while the copy is still in progress.
Copy the file with a stream to avoid windows lock from File.Copy
using(var s = new MemoryStream(File.ReadAllBytes(filePath))
{
using(var fs = new FileStream(newLocation, FileMode.Create))
{
s.WriteTo(fs);
}
}
I use to store document/file in byte[] in database, and I want user can view/run that file from my application.
You need to know the file extension for the file you're writing, so the OS can run the default program based on the extension. The code would be something like this:
byte[] bytes = GetYourBytesFromDataBase();
string extension = GetYourFileExtension(); //.doc for example
string path = Path.GetTempFileName() + extension;
try
{
using(BinaryWriter writer = new BinaryWriter(File.Open(path, FileMode.Create)))
{
writer.Write(yourBytes);
}
// open it with default application based in the
// file extension
Process p = System.Diagnostics.Process.Start(path);
p.Wait();
}
finally
{
//clean the tmp file
File.Delete(path);
}
You will need to store the file extension in the database too. If you don't have the file extension the problem becomes very difficult as you cannot rely on the operating system to work out which program to launch to handle the file.
You can use the following pattern:
Load data from database and save to file using the original file extension.
Start a new System.Diagnostics.Process that points to the saved file path.
As you have saved the file with the original file extension, the OS will look for a program that is registered for the extension to open the file.
As chibacity and Daniel suggest, storing the file extension in the db, and agreed -- storing the file extension, or at least some indicator that tells you the file type, is a good idea.
If these files are of a format of your own creation then you might also want to store information about which version of the file format the data is stored in. During development file formats are prone to changing, and if you don't remember which version you used to store the data then you have a hard job recovering the information.
The same problems are faced in object persistence generally.
I have a string with a C# program that I want to write to a file and always overwrite the existing content. If the file isn't there, the program should create a new file instead of throwing an exception.
System.IO.File.WriteAllText (#"D:\path.txt", contents);
If the file exists, this overwrites it.
If the file does not exist, this creates it.
Please make sure you have appropriate privileges to write at the location, otherwise you will get an exception.
Use the File.WriteAllText method. It creates the file if it doesn't exist and overwrites it if it exists.
Generally, FileMode.Create is what you're looking for.
Use the file mode enum to change the File.Open behavior. This works for binary content as well as text.
Since FileMode.Open and FileMode.OpenOrCreate load the existing content to the file stream, if you want to replace the file completely you need to first clear the existing content, if any, before writing to the stream. FileMode.Truncate performs this step automatically
// OriginalFile:
oooooooooooooooooooooooooooooo
// NewFile:
----------------
// Write to file stream with FileMode.Open:
----------------oooooooooooooo
var exists = File.Exists(path);
var fileMode = exists
? FileMode.Truncate // overwrites all of the content of an existing file
: FileMode.CreateNew // creates a new file
using (var destinationStream = File.Open(path, fileMode)
{
await newContentStream.CopyToAsync(destinationStream);
}
FileMode Enum
If your code doesn't require the file to be truncated first, you can use the FileMode.OpenOrCreate to open the filestream, which will create the file if it doesn't exist or open it if it does. You can use the stream to point at the front and start overwriting the existing file?
I'm assuming your using a streams here, there are other ways to write a file.