Develop Reference

Atomically check if a file exists and open it

I am trying to open a FileStream only if the file exists, and do something else otherwise (not create it, so FileMode.OpenOrCreate is not applicable).
However, simply checking File.Exists before creating the FileStream will not prevent race conditions as the file could be deleted before the FileStream has a chance to be created, in which case a FileNotFoundException would be thrown.
Is there a way to achieve this "natively", without resorting to the following try catch wrapper:
/// <returns>false if the file does not exists, true otherwise.</returns>
public static bool TryOpenFileStreamIfExists(string filePath, FileAccess fileAccess, FileShare fileShare, out FileStream fs, FileOptions fileOptions = FileOptions.None) {
try {
if (!File.Exists(filePath)) {
fs = null;
return false;
}
fs = new FileStream(filePath, FileMode.Open, fileAccess, fileShare, short.MaxValue, fileOptions);
return true;
}
catch (FileNotFoundException) {
fs = null;
return false;
}
}

You could use P/Invoke to call the Windows API's CreateFile() function to open the file. This returns a null handle if the file can't be opened (although you'll have to call GetLastError() to determine exactly why the file could not be opened).
Make sure you use a P/Invoke declaration for CreateFile() which returns a SafeHandle, such as:
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
static extern SafeFileHandle CreateFile
(
string lpFileName,
[MarshalAs(UnmanagedType.U4)] FileAccess dwDesiredAccess,
[MarshalAs(UnmanagedType.U4)] FileShare dwShareMode,
IntPtr lpSecurityAttributes,
[MarshalAs(UnmanagedType.U4)] FileMode dwCreationDisposition,
[MarshalAs(UnmanagedType.U4)] FileAttributes dwFlagsAndAttributes,
IntPtr hTemplateFile
);
If you do that then you can pass the handle to the overload of the FileStream() constructor which accepts a SafeHandle.
That's about as "native" as you're going to get...
However, I recommend that you just catch the exception.

Unmanaged file access (WriteFile) slower than managed (FileStream)

I have an application that heavily reads and write to files (a custom format), I was told to improve performance by using direct unmanaged code.
Before attempting in the real application I made a small tests just to see how the performance gains would be, but for my surprise, the unmanaged versions seems to be like 8x slower than using simply filestream.
Here is the managed function:
private int length = 100000;
private TimeSpan tspan;
private void UsingManagedFileHandle()
{
DateTime initialTime = DateTime.Now;
using (FileStream fileStream = new FileStream("data2.txt", FileMode.Create, FileAccess.ReadWrite))
{
string line = "ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890123";
byte[] bytes = Encoding.Unicode.GetBytes(line);
for (int i = 0; i < length; i++)
{
fileStream.Write(bytes, 0, bytes.Length);
}
fileStream.Close();
}
this.tspan = DateTime.Now.Subtract(initialTime);
label2.Text = "" + this.tspan.TotalMilliseconds + " Milliseconds";
}
Here is the unmanaged way:
public void UsingAnUnmanagedFileHandle()
{
DateTime initialTime;
IntPtr hFile;
hFile = IntPtr.Zero;
hFile = FileInteropFunctions.CreateFile("data1.txt",
FileInteropFunctions.GENERIC_WRITE | FileInteropFunctions.GENERIC_READ,
FileInteropFunctions.FILE_SHARE_WRITE,
IntPtr.Zero,
FileInteropFunctions.CREATE_ALWAYS,
FileInteropFunctions.FILE_ATTRIBUTE_NORMAL,
0);
uint lpNumberOfBytesWritten = 0;
initialTime = DateTime.Now;
if (hFile.ToInt64() > 0)
{
string line = "ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890123";
byte[] bytes = Encoding.Unicode.GetBytes(line);
uint bytesLen = (uint)bytes.Length;
for (int i = 0; i < length; i++)
{
FileInteropFunctions.WriteFile(hFile,
bytes,
bytesLen,
out lpNumberOfBytesWritten,
IntPtr.Zero);
}
FileInteropFunctions.CloseHandle(hFile);
this.tspan = DateTime.Now.Subtract(initialTime);
label1.Text = "" + this.tspan.TotalMilliseconds + " Milliseconds";
}
else
label1.Text = "Error";
}
[DllImport("kernel32.dll", SetLastError = true)]
public static extern bool CloseHandle(IntPtr hObject);
[DllImport("kernel32.dll", SetLastError = true)]
public static extern unsafe IntPtr CreateFile(
String lpFileName, // Filename
uint dwDesiredAccess, // Access mode
uint dwShareMode, // Share mode
IntPtr attr, // Security Descriptor
uint dwCreationDisposition, // How to create
uint dwFlagsAndAttributes, // File attributes
uint hTemplateFile); // Handle to template file
[DllImport("kernel32.dll")]
public static extern unsafe int WriteFile(IntPtr hFile,
// byte[] lpBuffer,
[MarshalAs(UnmanagedType.LPArray)] byte[] lpBuffer, // also tried this.
uint nNumberOfBytesToWrite,
out uint lpNumberOfBytesWritten,
IntPtr lpOverlapped);
The iteration using FileStream takes about 70 ms in my computer.
The one using WriteFile takes about 550ms.
I tested several times and with several amount of iterations and the difference in performance is consistent.
I have no idea why the unmanaged code is being slower then the managed code.
EDIT
Thank you very much for your explanations, guys . I thought there was something "magical" undergoing FileStream and you have explained it so well.
So, I know now there's no easy path to gain performance in this part, and I would like to ask you for opinion for other simple ways to gain speed. The file is random access in the real application, and size could range from 1MB to 1GB.

Your unmanaged calls write the data to disk as soon as possible while FileStream is buffered (ie does most operations in-memory and should call the underlying unmanaged calls much less often)
There are constructors on FileStream that let you control the buffer size if you want to tweak performance further.

Well, FileStream is jut a wrapper around CreateFile/WriteFile. It's written by bunch of smart guys. So I see no logical explanation at all why you assume that your one should be faster :P.
As already stated, FileStream probably does extra-buffering before calling WriteFile() thus minimizing unmanaged method calls. And this is important - only make unmanaged calls when they are necessary. They cost. Buffer sizes are usually multiple of disk sector size. You can experiment with different sizes, though this is OS dependent, and most likely will yield other results on other computers.
But it's also important to know that WriteFile() does internal buffering too. It's not like you call WriteFile() and bam it's written to file. It will be flushed to HDD once it's time.
I think there is unnecessary byte[] marshaling going on. Eg when you call WriteFile(), system makes copy of your buffer. It should be avoidable by unsafe() keyword and little bit of hacking.
There is also FILE_FLAG_SEQUENTIAL_SCAN that can't be accessed through FileStream(afaik) and it should let system know that you're gonna do file writes/reads only sequentially. This might give some performance boost theoretically.

The difference is because the calls to WriteFile are synchronous while the writes to the FileStream are not.
By default CreateFile will create a synchronous file handle, so the calls to WriteFile do not return until the data is written. If you add FILE_FLAG_OVERLAPPED to the CreateFile call the un-managed implementation will take approximately the same time as the managed.
See the documenation for Synchronous and Asynchronous I/O Handles section of the CreateFile defini

create file with FileStream and apply FileAttributes

Is it possible while creating the file with FileStream also apply FileAttributes at the same time? I would like to create file for stream writing with FileAttributes.Temporary file attribute.

You can use FileOptions.DeleteOnClose as one of parameters. File will be automatically removed after you finish your operations and dispose a stream.

Ya, surely you can apply FileAttributes also by using File.SetAttributes Method

Why do you need to do it all at once?
Just create the file (using File.Create or, if its a temporary file, use GetTempFileName.)
Set the attributes on the newly created file
Open the file using whatever method suits you

You can do this if you use the Win32 CreateFile method
uint readAccess = 0x00000001;
uint writeAccess = 0x00000002;
uint readShare = 0x00000001;
uint createAlways = 2;
uint tempAttribute = 0x100;
uint deleteOnClose = 0x04000000;
new FileStream(new SafeFileHandle(NativeMethods.CreateFile("filename",
readAccess | writeAccess,
readShare,
IntPtr.Zero,
createAlways,
tempAttribute | deleteOnClose,
IntPtr.Zero),
true),
FileAccess.ReadWrite, 4096, true);
private static class NativeMethods
{
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
internal static extern IntPtr CreateFile(string name, uint accessMode, uint shareMode, IntPtr security, uint createMode, uint flags, IntPtr template);
}
For more information, see the MSDN documentation of CreateFile

IOException "Cyclic Rendundancy Check" FileStream.CopyTo using C#.Net

I have some C# code that uses FileStreams to open a PhysicalDrive and take an image of the whole disk, but s consistently throwing an IOException with the message "Data error (cyclic redundancy check)." After copying about 121MB of a 128MB disk.
using Microsoft.Win32.SafeHandles;
using System.IO;
using System;
[DllImport("kernel32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall, SetLastError = true)]
protected static extern SafeFileHandle CreateFile(
string lpFileName,
uint dwDesiredAccess,
uint dwShareMode,
IntPtr SecurityAttributes,
uint dwCreationDisposition,
uint dwFlagsAndAttributes,
IntPtr hTemplateFile
);
public void MakeImage()
{
SafeFileHandle TheDevice = null;
try
{
TheDevice = CreateFile(#"\\.\PHYSICALDRIVE1", (uint)FileAccess.Read, (uint)0, IntPtr.Zero, (uint)FileMode.Open, (uint)FILE_ATTRIBUTE_SYSTEM, IntPtr.Zero);
if (TheDevice.IsInvalid) { throw new IOException("Unable to access drive. Win32 Error Code " + Marshal.GetLastWin32Error()); }
using (FileStream Dest = System.IO.File.Open("output.bin", FileMode.Create))
{
using (FileStream Src = new FileStream(TheDevice, FileAccess.Read))
{
Src.CopyTo(Dest);
Src.Close();
}
Dest.Close();
}
}
catch(Exception Ex)
{
//Here is where i am getting the IOException
//Handle error..
}
finally
{
if (TheDevice != null)
{
if (!TheDevice.IsClosed)
TheDevice.Close();
TheDevice.Dispose();
}
}
}
I have run Scan Disk on the Drive in question and there doesn't appear to be anything wrong with it. If I change the first Param of CreateFile to just read a partition (not what I want to do), then the image is created fine.
This is a follow up to Windows C# implementation of linux dd command that I've been trying to do.
UPDATE:
Further investigations show that the error has something do with not being able to get or know the Src.Length property. I changed my code to copy byte by byte and keep a count, it errors after 126959616 bytes, which is 1 byte more than the total size of the image produced by dd.

The cyclic redundancy check, or "CRC" error indicates that the disk is corrupt or damaged in some way probably a bad spot on the disk. If you're seeing it when trying to copy a file it could be that the bad spot may be in the file itself.
Try copying the file to some other disk and see if that works.

The cause of the IOException was reading past the end of the Stream. For some reason when opening a Physical Disk as a FileSteam I am unable to get the Length of the stream or otherwise find out its size. The solution is to use the System.Management namespace objects to find out the length of the Device file in bytes and then use Read to copy the correct amount of data from the Stream.
I think the reason this would happen after 121MB is due to something along the way not using 1024B to a KB and instead using 1000B to a KB, so that was a bit of a red herring on my part.
I still don't understand why it would throw a CRC error and not a "Sector Not Found" or "Read Past the End of the Stream" type message.

low level C++ style i/o in C# for reading FAT32

I am working on reading the FAT32 entry of the hard disk and so far have been successful in reading the entries by making use of the CreateFile, ReadFile, and SetFilePointer APIs. Here is my code (written in C#) so far.
---The DLL IMPORTS-----
[DllImport("kernel32.dll", SetLastError = true)]
static extern IntPtr CreateFile(string lpFileName, Int32 dwDesiredAccess,
Int32 dwShareMode, Int32 lpSecurityAttributes, Int32 dwCreationDisposition,
Int32 dwFlagsAndAttributes, IntPtr hTemplateFile);
[DllImport("kernel32.dll")]
static extern bool ReadFile(IntPtr hFile, byte[] lpBuffer,
uint nNumberOfBytesToRead, out uint lpNumberOfBytesRead, uint lpOverlapped);
[DllImport("kernel32.dll")]
extern static int SetFilePointer(IntPtr hFile, int lDistanceToMove, int lpDistanceToMoveHigh, uint dwMoveMethod);
[DllImport("kernel32.dll")]
extern static Boolean CloseHandle(IntPtr hObject);
------CODE----Will Work in any .NET Application---------
int ret, nread;
IntPtr handle;
int s = 512;
byte[] readbuffer = new byte[512];
IntPtr ptr = CreateFile(#"\\.\F:", -1073741824, 3, 0, 3, 128, IntPtr.Zero);
if (ptr != System.IntPtr.Zero)
{
int i = 100;
int ret = SetFilePointer(ptr, 0, 0, 0);
ret = SetFilePointer(ptr, 4194304, 0, 1);
while (true)
{
byte[] inp = new byte[512];
uint read = 0;
if (ret != -1)
{
ReadFile(ptr, inp, 512, out read, 0);
for (int k = 0; k < 16; k++)
{
string s = ASCIIEncoding.ASCII.GetString(inp, k*32, 11);
if (inp[k*32] == 0xE5)
{
MessageBox.Show(s);
}
}
//ret = SetFilePointer(ptr, 512, 0, 1);
}
}
}
The code above reads the F:\ drive and for trial purposes I have made it to read the first File Directory Cluster and query through each file entry and display the file name if it has been deleted.
However I want to make it to a full-blown application, for which I will have to frequently use the byte array and map it to the specified data structures according to the FAT32 Specification.
How can I efficiently use the byte array into which I am reading the data? I have tried the same code using filestream and binaryreader and it works, however now suppose I have a C Structure something like
struct bios_data
{
byte id[3];
char name[11];
byte sectorpercluster[2];
...
}
I want to have a similar data structure in C# and when I read data to a byte array I want to map it to the structure. I tried many options but didn't get a complete solution. I tried making a class and do serialization too but that also didn't work. I have around 3 more structures like theese which I will be using as I read the data from the FAT entry. How can I best achieve the desired results?

If you want to read binary data directly into structs, C-style, this article may interest you. He wrote an unmanaged wrapper around the C stdio functions and interoperates with it. I have tried it - it does work quite well. It is nice to read directly into a struct in C#, and it is fast. You can just do:
unsafe
{
fmp3.Read<MyStruct>(&myStructVar);
}

I gave an answer on how to convert between byte arrays and structs in this question.