Let's say I have contents of an executable (or a bat script, doesn't matter) in memory and want to run it as a new process. That is easy.
File.WriteAllBytes(filePath, contents);
// gap
Process.Start(filePath)
But I want to make sure that the executed file is not tampered by any other process. And there is a gap between file creation and execution. It gives a chance to tamper the file with the right tools.
So, instead of File.WriteAllBytes, I went with opening a FileStream FileShare.Read and keeping it open until the execution has finished.
using(var fileStream = new FileStream(filePath, FileMode.CreateNew, FileAccess.Write, FileShare.Read))
{
Process.Start(filePath)
}
But this doesn't work. Process.Start fails with:
System.ComponentModel.Win32Exception (32): The process cannot access the file because it is being used by another process.
This question and its answer explains why I think. In a nutshell, Process.Start will attempt to open the file with FileShare.Read and fail because the open FileStream already has Write Access, hence failing the FileShare.Read attempt of the process.
Is there a way to do this cleanly?
A workaround I can think of is to save the file, close it, open a new FileStream with FileShare.Read and FileAccess.Read, make sure the content is still the same before executing it. But that's not pretty.
What you are describing is a classic case of Time of check to time of use vulnerability.
Any solution that involves checking something and then executing it, and where those two operations are not atomic, will still leave you vulnerable. For example:
make sure the content is still the same before executing it. But that's not pretty
There's still a (smaller) gap (timing window) between the "make sure the content is still the same" and "executing it".
In 2004, an impossibility result was published,showing that there was no portable, deterministic technique for avoiding TOCT-TOU race conditions
- https://web.cecs.pdx.edu/~markem/CS333/handouts/tocttou.pdf
You can do a couple of things to mitigate it:
Don't use files! You say you have some code in memory that you need to execute: can you execute it yourself in the same process?
Reduce the window of time.
Make the file name random and hard to predict for other processes.
Run your program as a separate user where there's less likelyhood an attacker (or malicious program) is running, and restrict the file read/write to the new user only.
Related
I have Following Code in a Page_Load called function. When the Page is loaded the first time after starting Visual Studio, everything works out fine.
But any other opening call to the File after that returns IOException: "File is in use by another process", even when directly opening the File in VisualStudio Solution this Error is returned(of course not as Exception)
FileStream mailinglist_FileStream = new FileStream(#"\foobarFile.txt", FileMode.Open);
PeekingStreamReader mailinglist_Reader = new PeekingStreamReader(mailinglist_FileStream);
//Do some stuff with the file
mailinglist_FileStream.Close();
mailinglist_Reader.Close();
mailinglist_Reader.Dispose();
mailinglist_FileStream.Dispose();
Why is the file still locked? and why does fully restarting Visual Studio reset the File?
when checking file-Properties it says:
Build Action: Content
Copy to output directory: do not Copy
I am only reading this File. can i do something similiar to adLockOptimistic, so that multiple processes can access the File?
Why is the file still locked? and why does fully restarting Visual
Studio reset the File? when checking file-Properties it says [...]
I don't know why the file is still locked: probably because your code fails before the stream is closed/disposed.
About "why fully restarting Visual Studio [...]": because you may be using IIS Express or ASP.NET Dev Server whose are closed when you close the IDE, so locks on files are released since the process holding the locks is no longer running.
And about "why is the file still locked?[...]" it could be because the file stream isn't closed because sometimes the thread may not end successfully and the locks aren't released.
As other answer said, check how using block may avoid that IDisposable objects wouldn't be disposed:
// FileShare.ReadWrite will allow other processes
// to read and write the target file even if other processes
// are working with the same file
using var mailinglist_FileStream = new FileStream(#"\foobarFile.txt", FileMode.Open, FileShare.ReadWrite);
using var mailinglist_Reader = new PeekingStreamReader(mailinglist_FileStream);
// Do your stuff. Using blocks will call Dispose() for
// you even if something goes wrong, as it's equal to a try/finally!
I am only reading this File. can i do something similiar to
adLockOptimistic, so that multiple processes can access the File?
Yes, take a look at File.Open method and FileShare enumeration:
File.Open: http://msdn.microsoft.com/en-us/library/y973b725.aspx
FileShare enum: http://msdn.microsoft.com/en-us/library/system.io.fileshare.aspx
Learn to use using:
using (FileStream fileStream = File.Open(#"C:\somefile", FileMode.Open, FileAccess.Read))
{
...
}
The using construct ensures that the file will be closed when you leave the block even if an exception is thrown.
Your problem might not be here, but somewhere else in your code. You'll have to go through all your code and look for places where you have opened files but not put it inside a using statement.
An old question but unfortunately the given answers can be not applicable to the question.
The problem specifically in Windows lies in two aspects of Windows behavior:
a) when the handle to the file, opened for writing, is closed, the Microsoft Antimalware Service opens the file to check the newly written data for malware;
b) the OS itself keeps the file opened for some time after all handles to it are closed. This time can be from seconds to many minutes depending on the nature of the file and other factors.
We saw this problem many times in our products and had to provide special support for this case - our kernel-mode attempts to close the file as soon as the last handle to it is closed.
Try using using blocks, it may not fix your lock problem, but it is better form for disposable objects.
using (FileStream mailinglist_FileStream = new FileStream(#"\foobarFile.txt", FileMode.Open))
{
using (PeekingStreamReader mailinglist_Reader = new PeekingStreamReader(mailinglist_FileStream))
{
...
}
}
Also, try closing mailinglist_Reader before mailinglist_FileStream.
I need to read a text based log file to check for certain contents (the completion of a backup job). Obviously, the file is written to when the job completes.
My question is, how can I (or how SHOULD I write the code to) read the file, taking into account the file may be locked, or locked by my process when it needs to be read, without causing any reliability concerns.
Assuming the writing process has at least specified System.IO.FileShare.Read when opening the file, you should be able to read the text file while it is still being written to.
In addition to the answer by #BrokenGlass:
Only open the file for reading. If you try to open it for Read/Write access, it's more likely (almost certain) to fail - you may not be able to open it, and/or you may stop the other process being able to write to it.
Close the file when you aren't reading it to minimise the chance that you might cause problems for any other processes.
If the writing process denies read access while it is writing to the file, you may have to write some form of "retry loop", which allows your application to wait (keep retrying) until the file becomes readable. Just try to open the file (and catch errors) - if it fails, Sleep() for a bit and then try again. (However, if you're monitoring a log file, you will probbably want to keep checking it for more data anyway)
When a file is being written to, it is locked for all other processes that try to open the file in Write-mode. Read-mode will always be available.
However, if your writing process saves changes while you have already opened the file in your reading process, the changes will not be reflected there until you refresh (Close-Open) the file again.
My program basically looks like this:
using(XmlReader reader = XmlReader.Create(File.Open(path, FileMode.Open, FileAccess.Reader, FileShare.Read))
{
//a lot of reader.Read() to read through the file
}
The program reads a large amount of small files this way (typically 5k per second) and on local machines this works properly. However I'm now trying to feed it input data from a remote machine. I've created an NTFS symlinked folder to the remote host on my local drive. So when my machine accesses C:\MyFolder it transparently accesses \NetworkComputer\MySharedFolder. So far so good!
Howevermy program crashes with a "The system cannot open the file." IOException. This seems to map to "ERROR_TOO_MANY_OPEN_FILES". As far as I know I only have one file open at a given time, so the exception doesn't make sense.
After some more debugging I found out the following (using the process explorer).
When running locally the process starts with 400 handles (probably because of .NET) then when I press the start button the number of handles increases to 600 quickly, then it decreases to 400, then rises to 600, etc... This is probably the Garbage Collector kicking in, keeping the number of file handles down. When the process is done processing the files the number of file handles returns nicely to 400.
When reading files from the remote machine I see in the local process monitor the exact same behavior. However in the process monitor on the remote machine I see the number of file handles rise with about 1800, never decreasing, until the program crashes. The file handle number increase on the remote machine directly corresponds with the number of processed files.
TLDR:
So this is my question: How can I avoid having dangling file handles on a remote machine when accessing it trough a symlink?
Edit:
Sometimes another exception shows up (IOException) "Not enough server storage is available to process this command". I'm not sure how to counter this, but I found some extra info, still not sure what this means http://support.microsoft.com/kb/285089 http://support.microsoft.com/kb/106167
Edit2:
One solution that seems to help is to slow down reading from the remote machine, by throttling to 2000 files per minute resolves the problem, but this is not really a working solution since then the job takes about 10x longer then it has to take.
I've also found some articles that indicate that increasing IRPSstacksize could do the trick, however since the computer is not directly under my control this might not be possible.
Does XmlReader automatically close/dispose the passed stream? (ie, The FileStream returned from File.Open.)
I don't think it does, which would explain why you're seeing the number of handles climbing high and then dropping on your local machine. The handles aren't being released until the finalisers are triggered by the GC, rather than when each stream is disposed.
Try this and see if this makes any difference:
using (var fs = File.Open(path, FileMode.Open, FileAccess.Reader, FileShare.Read))
using (var reader = XmlReader.Create(fs))
{
// a lot of reader.Read() to read through the file
}
Sounds like finalizer releases the inner stream instead of you dispose it explicitly.
Try this:
using(FileStream steram = File.Open(path, FileMode.Open, FileAccess.Reader, FileShare.Read))
using (XmlReader reader = XmlReader.Create(steram))
{
//a lot of reader.Read() to read through the file
}
I am not sure whether this behaviour is due accessing a remote machine or due to accessing via symlink...
Perhaps you can just check the behaviour when accessing the remote machine without a symlink - through UNC path and/or drive letter (smb mount)...
I can't be a 100% sure but I think the symlink is part of the problem...
Ok guys, this one is a tough one.
The scenario:
I have multiple services running on multiple machines
Each service has multiple threads, and each thread writes a file on a FILER - the shared storage used by my machines (using a share such as \\filername\foo\bar)
The FILER machine is a NetApp machine
Both the FILER and the machines running the services are using SMB2 (http://en.wikipedia.org/wiki/Server_Message_Block)
The instruction used to write the file is as simple as the one listed below in [THE CODE]
[THE CODE]
using (StreamWriter outfile = new StreamWriter(pathToTheFile, false))
{
outfile.Write(stringToWriteInTheFile);
}
[/THE CODE]
The problem:
Sometimes the service remains "stuck" on this instruction. The error given is:
The process cannot access the file '\\filername\foo\bar\myfile.txt' because it is being used by another process.
After some of these errors, the service refuses to release the lock on the file. What happens then?
You can delete the file, but the file is IMMEDIATELY recreated. Like if a sort-of permanent Stream is alive and keeps writing the file indefinitely.
You can stop the service: it's stuck, and won't be stopped, so I forced a Thread.Abort (yeah, I know but practice, but what else?) after 2 minutes.
So, the service is now stopped, but the machine retains an handle to the file and you CANNOT kill the process keeping the handle alive except by rebooting the machine. . .
I don't know what to do right now, I think I tried everything.
Considerations:
Previously, the FILER and the machines were using SMB1, and this problem never arised. So I guess something fishy happens in the background, but I can't understand what...
I changed recently the code used to write the file, in a desperate attempt to "delegate" everything to .net. Now it's:
File.WriteAllText(pathToTheFile, stringToWriteInTheFile);
but my gut feeling is that, under the wraps, .net is doing the exact same thing - the change is quite recent though, so I can't still say if the "fix" is working or not.
EDIT (as per Vash comment): Usually the file is different, but it can happen (and it actually happens) sometimes that multiple threads are trying to write the same file, however :( - doing the File.WriteAllText shouldn't take care of concurrency issues?
Try explicitly opening a FileStream in "exclusive" mode, ie
using (var fs = new FileStream("path",
FileMode.Open, FileAccess.ReadWrite,
FileShare.None))
{
using (var sw = new StreamWriter(fs))
{
...
Of course your code will have to anticipate that the file might be locked when it goes to write it and react appropriately. That part is left as an exercise for the reader :-)
Disclaimer: I have used this in a multi-threaded environment, but I can't guarantee it will work over Samba.
I have many processes reading a file stored on a network share. Originally I was only able to have one process read the file, all the others would throw exceptions. I implemented the following code to deal with that:
using (StreamReader fileStreamReader = new StreamReader(File.Open(path, FileMode.Open, FileAccess.Read, FileShare.Read)))
{
content = fileStreamReader.ReadToEnd();
}
This let multiple processes read the same file, however it still seems to have issues, because sometimes multiple processes still can't access the file. Yet I can go back later when the file isn't in use and open it just fine. Right now I have some retry behavior with random delays implemented that so far, seem to help. It seems a little quirky to me to do it this way, so what would be a better method?
This is the weird part, the exception I'm getting is not from file IO at all, it's from a library called CommStudio. In short, I dump the file to a string, i modify it slightly, dump it into a memory stream, and ship it off over ymodem on rs232. The exception is telling me the remote system has canceled. The device getting the data reports that there was a transmission error, which usually means that an incomplete/empty file was received.
Normally I would blame the library on this, but it works flawlessly at desk-testing and when there is only one process accessing the file. The only thing that really seems to be consistent is that it is likely to fail when multiple processes are accessing a file.
had a similar problem but not allot of time to find an ideal solution. I created a webservice and stuck the file local to the webservice app.. then created a simple one liner GET API which was called over the office intranet.. thus ensureing only the calling application edited the log file.. messy but functional.
I have had a similar problem in the past. Try changing how you access the file to something like this.
//Use FileInfo to get around OS locking of the file
FileInfo fileInfo = new FileInfo(path);
//I actually wanted unblocked read write access so change your access and share appropriately
using (FileStream fs = fileInfo.Open(FileMode.Open, FileAccess.Write, FileShare.ReadWrite))
{
//I'm using CopyTo but use whatever method matches your need
fileInfo.CopyTo(Path.Combine(destination, fileName), false);
}