Develop Reference

Write to file and execute it atomically

Let's say I have contents of an executable (or a bat script, doesn't matter) in memory and want to run it as a new process. That is easy.
File.WriteAllBytes(filePath, contents);
// gap
Process.Start(filePath)
But I want to make sure that the executed file is not tampered by any other process. And there is a gap between file creation and execution. It gives a chance to tamper the file with the right tools.
So, instead of File.WriteAllBytes, I went with opening a FileStream FileShare.Read and keeping it open until the execution has finished.
using(var fileStream = new FileStream(filePath, FileMode.CreateNew, FileAccess.Write, FileShare.Read))
{
Process.Start(filePath)
}
But this doesn't work. Process.Start fails with:
System.ComponentModel.Win32Exception (32): The process cannot access the file because it is being used by another process.
This question and its answer explains why I think. In a nutshell, Process.Start will attempt to open the file with FileShare.Read and fail because the open FileStream already has Write Access, hence failing the FileShare.Read attempt of the process.
Is there a way to do this cleanly?
A workaround I can think of is to save the file, close it, open a new FileStream with FileShare.Read and FileAccess.Read, make sure the content is still the same before executing it. But that's not pretty.

What you are describing is a classic case of Time of check to time of use vulnerability.
Any solution that involves checking something and then executing it, and where those two operations are not atomic, will still leave you vulnerable. For example:
make sure the content is still the same before executing it. But that's not pretty
There's still a (smaller) gap (timing window) between the "make sure the content is still the same" and "executing it".
In 2004, an impossibility result was published,showing that there was no portable, deterministic technique for avoiding TOCT-TOU race conditions
- https://web.cecs.pdx.edu/~markem/CS333/handouts/tocttou.pdf
You can do a couple of things to mitigate it:
Don't use files! You say you have some code in memory that you need to execute: can you execute it yourself in the same process?
Reduce the window of time.
Make the file name random and hard to predict for other processes.
Run your program as a separate user where there's less likelyhood an attacker (or malicious program) is running, and restrict the file read/write to the new user only.

FileStream.close() does not free file for other processes

I have Following Code in a Page_Load called function. When the Page is loaded the first time after starting Visual Studio, everything works out fine.
But any other opening call to the File after that returns IOException: "File is in use by another process", even when directly opening the File in VisualStudio Solution this Error is returned(of course not as Exception)
FileStream mailinglist_FileStream = new FileStream(#"\foobarFile.txt", FileMode.Open);
PeekingStreamReader mailinglist_Reader = new PeekingStreamReader(mailinglist_FileStream);
//Do some stuff with the file
mailinglist_FileStream.Close();
mailinglist_Reader.Close();
mailinglist_Reader.Dispose();
mailinglist_FileStream.Dispose();
Why is the file still locked? and why does fully restarting Visual Studio reset the File?
when checking file-Properties it says:
Build Action: Content
Copy to output directory: do not Copy
I am only reading this File. can i do something similiar to adLockOptimistic, so that multiple processes can access the File?

Why is the file still locked? and why does fully restarting Visual
Studio reset the File? when checking file-Properties it says [...]
I don't know why the file is still locked: probably because your code fails before the stream is closed/disposed.
About "why fully restarting Visual Studio [...]": because you may be using IIS Express or ASP.NET Dev Server whose are closed when you close the IDE, so locks on files are released since the process holding the locks is no longer running.
And about "why is the file still locked?[...]" it could be because the file stream isn't closed because sometimes the thread may not end successfully and the locks aren't released.
As other answer said, check how using block may avoid that IDisposable objects wouldn't be disposed:
// FileShare.ReadWrite will allow other processes
// to read and write the target file even if other processes
// are working with the same file
using var mailinglist_FileStream = new FileStream(#"\foobarFile.txt", FileMode.Open, FileShare.ReadWrite);
using var mailinglist_Reader = new PeekingStreamReader(mailinglist_FileStream);
// Do your stuff. Using blocks will call Dispose() for
// you even if something goes wrong, as it's equal to a try/finally!
I am only reading this File. can i do something similiar to
adLockOptimistic, so that multiple processes can access the File?
Yes, take a look at File.Open method and FileShare enumeration:
File.Open: http://msdn.microsoft.com/en-us/library/y973b725.aspx
FileShare enum: http://msdn.microsoft.com/en-us/library/system.io.fileshare.aspx

Learn to use using:
using (FileStream fileStream = File.Open(#"C:\somefile", FileMode.Open, FileAccess.Read))
{
...
}
The using construct ensures that the file will be closed when you leave the block even if an exception is thrown.
Your problem might not be here, but somewhere else in your code. You'll have to go through all your code and look for places where you have opened files but not put it inside a using statement.

An old question but unfortunately the given answers can be not applicable to the question.
The problem specifically in Windows lies in two aspects of Windows behavior:
a) when the handle to the file, opened for writing, is closed, the Microsoft Antimalware Service opens the file to check the newly written data for malware;
b) the OS itself keeps the file opened for some time after all handles to it are closed. This time can be from seconds to many minutes depending on the nature of the file and other factors.
We saw this problem many times in our products and had to provide special support for this case - our kernel-mode attempts to close the file as soon as the last handle to it is closed.

Try using using blocks, it may not fix your lock problem, but it is better form for disposable objects.
using (FileStream mailinglist_FileStream = new FileStream(#"\foobarFile.txt", FileMode.Open))
{
using (PeekingStreamReader mailinglist_Reader = new PeekingStreamReader(mailinglist_FileStream))
{
...
}
}
Also, try closing mailinglist_Reader before mailinglist_FileStream.

C# service won't write to XML

I've just written a test service in C# 2010. In the OnStart method i'm opening an XML document in the root of C: drive, parsing it and writing out to another XML document also in the root of C: drive.
When I install the service and start it, im told it stopped again automatically. The output XML file is created but is empty. I'm running the service as Local System account.
Can anyone tell me why no content is being written?
Thanks,
EDIT (to include code for OnStart)...
protected override void OnStart(string[] args)
{
String win32ClassName = "";
String nodeSubkeyName = "";
List<String> propertyList = new List<String>();
List<String> propertyListQuery = new List<String>();
XmlTextReader reader = new XmlTextReader("C:\\hwin.xml");
XmlTextWriter writer = new XmlTextWriter("C:\\hwout.xml", null);
writer.WriteStartDocument();
writer.WriteComment("Asset hardware Inventory for " + System.Environment.MachineName);
writer.WriteStartElement("hardware");
The above code doesn't even write the starting element for the output XML file, but it does create the empty file so suspect it has rights to do that. Perhaps reading the input file IS the issue? I've little to no idea how the Local Service account works!

I'd suspect that it's not flushing the stream it's writing to. Stream output (including file output) is normally buffered because it gives better performance to batch up a bunch of bytes to write to the file rather than do them one by one. (The benefit increases the larger buffer you use until either 4 or 8k, at which point the cost of the memory used outweighs the gain. 4 and 8 both tend to be better than something in between because they also hit memory page sizes well, in any case the default buffers for most framework-supplied streams is 4k).
Anyway, if that is the problem, then you need to flush the stream. This is always done when you close the writer, and closing the writer is always done when you Dispose() it. You should always dispose anything that implements IDisposable as soon as possible anyway as a matter of good practice (always assume something bad at least could happen if you don't, even in those cases where you know a given Dispose() is currently implemented as a no-op). Most of the time, this is most easily done with a using block:
using(XmlTextWriter writer = new XmlTextWriter("C:\\hwout.xml", null))
{
//code that uses writer here
}
Even if my suspicion is wrong, it's well worth getting into the habit of doing this.

Probably an unhandled exception....
Use Try/Catch, write exception to system.diagnostics.trace.writeline and use DebugView(http://technet.microsoft.com/en-us/sysinternals/bb896647) to read the output.

Check the error log in event viewer, its most likely you don't have permission to read the file from c:\
You can make the service run under an account which has permission to read\write to c:\
You should use a try and catch around your xml write code block, and then log the error to debug this further.

It is hard to say without code snippet,but if service doesn't start definitely OnStart throw exception, put Thread.Sleep in OnStart method, attach to process with visual studio and debug

Have you tried executing your code as a console application rather than a service?
Often run-time issues with services are due to privileges. For the writing part, this is not the case here, because your code got to create the output XML file. Therefore, your service has write rights to the right directory.
But it may be that it does not have read rights on the input file, or there just may be an exception happening during execution.
My advice to you:
test your code as a Console app rather than a service, and test it from within Visual Studio
try/catch and log any exceptions
check the Event Viewer to see if windows has logged any details about the problem with your service

C# file handles released too late when opening files on a remote host (via symlink)

My program basically looks like this:
using(XmlReader reader = XmlReader.Create(File.Open(path, FileMode.Open, FileAccess.Reader, FileShare.Read))
{
//a lot of reader.Read() to read through the file
}
The program reads a large amount of small files this way (typically 5k per second) and on local machines this works properly. However I'm now trying to feed it input data from a remote machine. I've created an NTFS symlinked folder to the remote host on my local drive. So when my machine accesses C:\MyFolder it transparently accesses \NetworkComputer\MySharedFolder. So far so good!
Howevermy program crashes with a "The system cannot open the file." IOException. This seems to map to "ERROR_TOO_MANY_OPEN_FILES". As far as I know I only have one file open at a given time, so the exception doesn't make sense.
After some more debugging I found out the following (using the process explorer).
When running locally the process starts with 400 handles (probably because of .NET) then when I press the start button the number of handles increases to 600 quickly, then it decreases to 400, then rises to 600, etc... This is probably the Garbage Collector kicking in, keeping the number of file handles down. When the process is done processing the files the number of file handles returns nicely to 400.
When reading files from the remote machine I see in the local process monitor the exact same behavior. However in the process monitor on the remote machine I see the number of file handles rise with about 1800, never decreasing, until the program crashes. The file handle number increase on the remote machine directly corresponds with the number of processed files.
TLDR:
So this is my question: How can I avoid having dangling file handles on a remote machine when accessing it trough a symlink?
Edit:
Sometimes another exception shows up (IOException) "Not enough server storage is available to process this command". I'm not sure how to counter this, but I found some extra info, still not sure what this means http://support.microsoft.com/kb/285089 http://support.microsoft.com/kb/106167
Edit2:
One solution that seems to help is to slow down reading from the remote machine, by throttling to 2000 files per minute resolves the problem, but this is not really a working solution since then the job takes about 10x longer then it has to take.
I've also found some articles that indicate that increasing IRPSstacksize could do the trick, however since the computer is not directly under my control this might not be possible.

Does XmlReader automatically close/dispose the passed stream? (ie, The FileStream returned from File.Open.)
I don't think it does, which would explain why you're seeing the number of handles climbing high and then dropping on your local machine. The handles aren't being released until the finalisers are triggered by the GC, rather than when each stream is disposed.
Try this and see if this makes any difference:
using (var fs = File.Open(path, FileMode.Open, FileAccess.Reader, FileShare.Read))
using (var reader = XmlReader.Create(fs))
{
// a lot of reader.Read() to read through the file
}

Sounds like finalizer releases the inner stream instead of you dispose it explicitly.
Try this:
using(FileStream steram = File.Open(path, FileMode.Open, FileAccess.Reader, FileShare.Read))
using (XmlReader reader = XmlReader.Create(steram))
{
//a lot of reader.Read() to read through the file
}

I am not sure whether this behaviour is due accessing a remote machine or due to accessing via symlink...
Perhaps you can just check the behaviour when accessing the remote machine without a symlink - through UNC path and/or drive letter (smb mount)...
I can't be a 100% sure but I think the symlink is part of the problem...

Proper methods to have multiple processes reading the same file?

I have many processes reading a file stored on a network share. Originally I was only able to have one process read the file, all the others would throw exceptions. I implemented the following code to deal with that:
using (StreamReader fileStreamReader = new StreamReader(File.Open(path, FileMode.Open, FileAccess.Read, FileShare.Read)))
{
content = fileStreamReader.ReadToEnd();
}
This let multiple processes read the same file, however it still seems to have issues, because sometimes multiple processes still can't access the file. Yet I can go back later when the file isn't in use and open it just fine. Right now I have some retry behavior with random delays implemented that so far, seem to help. It seems a little quirky to me to do it this way, so what would be a better method?
This is the weird part, the exception I'm getting is not from file IO at all, it's from a library called CommStudio. In short, I dump the file to a string, i modify it slightly, dump it into a memory stream, and ship it off over ymodem on rs232. The exception is telling me the remote system has canceled. The device getting the data reports that there was a transmission error, which usually means that an incomplete/empty file was received.
Normally I would blame the library on this, but it works flawlessly at desk-testing and when there is only one process accessing the file. The only thing that really seems to be consistent is that it is likely to fail when multiple processes are accessing a file.

had a similar problem but not allot of time to find an ideal solution. I created a webservice and stuck the file local to the webservice app.. then created a simple one liner GET API which was called over the office intranet.. thus ensureing only the calling application edited the log file.. messy but functional.

I have had a similar problem in the past. Try changing how you access the file to something like this.
//Use FileInfo to get around OS locking of the file
FileInfo fileInfo = new FileInfo(path);
//I actually wanted unblocked read write access so change your access and share appropriately
using (FileStream fs = fileInfo.Open(FileMode.Open, FileAccess.Write, FileShare.ReadWrite))
{
//I'm using CopyTo but use whatever method matches your need
fileInfo.CopyTo(Path.Combine(destination, fileName), false);
}