Capturing standard out from tail -f "follow" - c#

I am trying to capture the output from tail in follow mode, where it outputs the text as it detects changes in the file length - particularly useful for following log files as lines are added. For some reason, my call to StandardOutput.Read() is blocking until tail.exe exits completely.
Relevant code sample:
var p = new Process() {
StartInfo = new ProcessStartInfo("tail.exe") {
UseShellExecute = false,
RedirectStandardOutput = true,
Arguments = "-f c:\\test.log"
}
};
p.Start();
// the following thread blocks until the process exits
Task.Factory.StartNew(() => p.StandardOutput.Read());
// main thread wait until child process exits
p.WaitForExit();
I have also tried using the support for the OutputDataReceived event handler which exhibits the same blocking behavior:
p.OutputDataReceived += (proc, data) => {
if (data != null && data.Data != null) {
Console.WriteLine(data.Data);
}
};
p.BeginOutputReadLine();
I do have a little bit more code around the call to StandardOutput.Read(), but this simplifies the example and still exhibits the undesirable blocking behavior. Is there something else I can do to allow my code to react to the availability of data in the StandardOutput stream prior to the child application exiting?
Is this just perhaps a quirk of how tail.exe runs? I am using version 2.0 compiled as part of the UnxUtils package.
Update: this does appear to be at least partially related to quirks in tail.exe. I grabbed the binary from the GnuWin32 project as part of the CoreUtils package and the version bumped up to 5.3.0. If I use the -f option to follow without retries, I get the dreaded "bad file descriptor" issue on STDERR (easy to ignore) and the process terminates immediately. If I use the -F option to include retries it seems to work properly after the bad file descriptor message has come by and it attempts to open the file a second time.
Is there perhaps a more recent win32 build from the coreutils git repository I could try?

I know it is not exatly what you are asking but as James says in the comments, you could do the equivalent functionality directly in c# to save you having to launch another process.
One way you can do it is like this:
using System;
using System.IO;
using System.Text;
using System.Threading;
public class FollowingTail : IDisposable
{
private readonly Stream _fileStream;
private readonly Timer _timer;
public FollowingTail(FileInfo file,
Encoding encoding,
Action<string> fileChanged)
{
_fileStream = new FileStream(file.FullName,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite);
_timer = new Timer(o => CheckForUpdate(encoding, fileChanged),
null,
0,
500);
}
private void CheckForUpdate(Encoding encoding,
Action<string> fileChanged)
{
// Read the tail of the file off
var tail = new StringBuilder();
int read;
var b = new byte[1024];
while ((read = _fileStream.Read(b, 0, b.Length)) > 0)
{
tail.Append(encoding.GetString(b, 0, read));
}
// If we have anything notify the fileChanged callback
// If we do not, make sure we are at the end
if (tail.Length > 0)
{
fileChanged(tail.ToString());
}
else
{
_fileStream.Seek(0, SeekOrigin.End);
}
}
// Not the best implementation if IDisposable but you get the idea
// See http://msdn.microsoft.com/en-us/library/ms244737(v=vs.80).aspx
// for how to do it properly
public void Dispose()
{
_timer.Dispose();
_fileStream.Dispose();
}
}
Then to call for example:
new FollowingTail(new FileInfo(#"C:\test.log"),
Encoding.ASCII,
s =>
{
// Do something with the new stuff here, e.g. print it
Console.Write(s);
});

Related

How to keep a session going with .net ssh sshclient

SshClient client;
var sethost = client.RunCommand("export DOCKER_HOST=:2375");
var infohost = client.RunCommand("docker info");
var ps = client.RunCommand("docker ps");
I am using the above commands with ssh client for .Net but it does not keep the state from the first command that does an export in the next two commands.
If I ssh in manually and do the 3 commands it works as I expects, but above c# code seem to forget about the export in the followup commands.
I assume you are using the SSH.NET library. In the following solution I used the stable version (2013.4.7) which is also available as NuGet package.
There are two possible solutions to get this to work. The first one is to combine the commands you want to run in one single RunCommand. A RunCommand starts a new shell each time its invoked and with that it forgets any environment settings you have executed. By chaining the commands with the && or || operator you achieve that all commands can be executed in the same Shell.
static void SolutionOne(ConnectionInfo connection)
{
using (SshClient client = new SshClient(connection))
{
client.ErrorOccurred += (e, s) =>
{
Debug.WriteLine(s.Exception);
};
client.Connect();
// each RunCommand starts its own/new Shell...
var sethost = client.RunCommand("export DOCKER_HOST=:2375");
// ... nothing is kept...
var infohost = client.RunCommand("export -p");
if (!infohost.Result.Contains("DOCKER_HOST"))
{
Console.WriteLine("sorry, a new shell was started for this command");
}
// ... across run commands
var ps = client.RunCommand("echo has value: $DOCKER");
Console.WriteLine(ps.Result);
Console.WriteLine("");
Console.WriteLine("Chain the linux commands...");
// chain all commands with &&
var concatAll = client.RunCommand("export DOCKER_HOST=:2375 && export -p | grep DO && echo has value: $DOCKER_HOST");
// you should see DOCKER_HOST on the last line!
Console.WriteLine("> see has value: 2375 at the end?");
Console.WriteLine(concatAll.Result);
}
}
The second solution is to obtain a ShellStream from the SshClient. That gives you the opportunity to read and write bytes very much similar like you would do a file.
After we have a connection and stream established we Read everything there is to read. Once the stream is empty we send our first command by Writing to the stream. Then we wait a bit for the stream to get data to read and then proceed in reading from the stream again until the stream is empty. That is when we can start the next command, rinse and repeat until all commands are done.
static void SolutionTwo(ConnectionInfo connection)
{
using (SshClient client = new SshClient(connection))
{
client.ErrorOccurred += (e, s) =>
{
Debug.WriteLine(s.Exception);
};
client.Connect();
using (var stream = client.CreateShellStream("dumb", 512, 96, 800, 600, 8191))
{
stream.ErrorOccurred += (e, s) =>
{
Debug.WriteLine(s.Exception);
};
// read welcome message
Console.WriteLine(ReadFromStream(stream));
// first command and read its output
WriteToStream(stream, "uname\n");
Console.WriteLine(ReadFromStream(stream));
// second and output
WriteToStream(stream, "df\n");
Console.WriteLine(ReadFromStream(stream));
// third and output
WriteToStream(stream, "ps aux\n");
Console.WriteLine(ReadFromStream(stream));
}
Console.ForegroundColor = ConsoleColor.Green;
Console.WriteLine(" >>>>>we are all done");
Console.ForegroundColor = ConsoleColor.White;
}
}
static void WriteToStream(ShellStream stream, string command)
{
stream.Write(command);
}
static StringBuilder ReadFromStream(ShellStream stream)
{
var result = new StringBuilder();
// there seems to be a timing/concurrency issue
// which I only could fix with using this
// useless Sleep calls
Thread.Sleep(500);
// wait for the Stream to have data
while (stream.Length == 0)
{
// yes, I know ...
Thread.Sleep(500);
}
// let's read!
string line;
while ((line = stream.ReadLine()) != null)
{
result.AppendLine(line);
}
return result;
}
As you noticed in ReadFromStream method there is some dubious hacking. I had a huge struggle to read reliable from the stream. In my initial attempts I used the DataReceived event but no matter how I tried it either did work for a few lines or even chars or it didn't work at all. In the end I fell back to a nasty Thread.Sleep call that worked reliable all the time. Maybe it is a bug in the library or something in my local setup but I gave up.

Why does StandardOutput.Read() never return? (deadlock?)

Using C#, I want to automate a third-party Windows command-line program. Usually, it is an interactive console, you send commands, it may prompt for details, send back a result and display a prompt to ask for more commands. Typically:
c:\>console_access.exe
Prompt> version
2.03g.2321
Prompt>
I used .NET classes Process and ProcessStartInfo along with redirections of stdin/stdout/stderr.
public ConsoleAccess()
{
if (!File.Exists(consoleAccessPath)) throw new FileNotFoundException(consoleAccessPath + " not found");
myProcess = new Process();
ProcessStartInfo myProcessStartInfo = new ProcessStartInfo(consoleAccessPath, ""); // even "2>&1" as argument does not work; my code still hangs
myProcessStartInfo.CreateNoWindow = true;
myProcessStartInfo.UseShellExecute = false;
myProcessStartInfo.RedirectStandardOutput = true;
myProcessStartInfo.RedirectStandardError = true;
myProcessStartInfo.RedirectStandardInput = true;
//myProcessStartInfo.ErrorDialog = true; // I tried, to no avail.
myProcess.StartInfo = myProcessStartInfo;
outputQueue = new ConcurrentQueue<string>(); // thread-safe queue
errorQueue = new ConcurrentQueue<string>();
myProcess.Start();
myStandardOutput = myProcess.StandardOutput;
myStandardError = myProcess.StandardError;
myStandardInput = myProcess.StandardInput;
stdOutPumper = new Thread(new ThreadStart(PumpStdOutLoop));
stdOutPumper.Start();
stdErrPumper = new Thread(new ThreadStart(PumpStdErrLoop));
stdErrPumper.Start();
string empty = getResponse(); // check for prompt
string version = getVersion(); // one simple command
}
// [...]
private void PumpStdErrLoop()
{
while (true)
{
string message = myStandardError.ReadLine();
errorQueue.Enqueue(message);
}
}
private void PumpStdOutLoop()
{
while (true)
{
bool done = false;
string buffer = "";
//int blocksize = 1024;
string prompt = "Prompt> ";
while (!done)
{
//char[] intermediaire = new char[blocksize];
//int res = myStandardOutput.Read(intermediaire, 0, blocksize);
//buffer += new string(intermediaire).Substring(0, res);
byte b = (byte)myStandardOutput.Read(); // I go byte per byte, just in case the char[] above is the source of the problem. To no avail.
buffer += (char)b;
done = buffer.EndsWith(prompt);
}
buffer = buffer.Substring(0, buffer.Length - prompt.Length);
outputQueue.Enqueue(buffer);
}
}
Since this program returns "Prompt> " (important : without "\n" at the end) when it's waiting for commands, I can't use myProcess.BeginOutputReadLine();
However, I have to use threads because I must listen stdout AND stderr at the same time.
This is why I used threads and thread-safe queues for a class producer/consumer pattern.
"You can use asynchronous read operations to avoid these dependencies and their deadlock potential. Alternately, you can avoid the deadlock condition by creating two threads and reading the output of each stream on a separate thread." source: http://msdn.microsoft.com/en-us/library/system.diagnostics.process.standardoutput%28v=vs.100%29.aspx
With this design, all sequences like
* cmd -> result with no err (something on stdout, nothing on stderr)
* cmd -> error (something on stderr, nothing on stdout)
works as expected. no problem.
cmd -> result with warning (something on both stderr and stdout)
should work (I'm trying to reproduce this scenario)
however, for one command in particular -- a command that prompts for a password during its execution -- does not work:
main thread principal loops forever on
if (errorQueue.Count == 0 && outputQueue.Count == 0) { System.Threading.Thread.Sleep(500); }
thread pumping stdout waits forever on
byte b = (byte)myStandardOutput.Read();
thread pumping stdout waits a line forever on
string message = myStandardError.ReadLine();
What I don't get is why byte b = (byte)myStandardOutput.Read(); does not pump the message "password:". Nothing happens. I never get the first 'p'.
I feel I hit a deadlock scenario, but I do not understand why.
What's wrong?
(I don't think it is very relevant but I tried the above on .NET 4.0 with MS Visual Studio 2010 on Windows 7 32-bit.)
This is a very common failure mode for these kind of interactive console mode programs. The C runtime library automatically switches the stderr and stdout streams to buffered mode when it detects that output is being redirected. Important to improve throughput. So output goes into that buffer instead of getting directly written to the console. Getting your program to see the output requires the buffer to be flushed.
There are three scenarios where the buffer gets flushed. A flush occurs when the buffer is full, typically around 2 kilobytes. Or when the program writes a line terminator (\n). Or when the program explicitly calls fflush(). The first two scenarios do not occur, not enough output and the program isn't using \n. Which points at the problem, the original programmer forgot to call fflush(). Forgetting this is very common, the programmer simply never intended the program to be used other than in an interactive way.
Nothing can do about it, you'll need to ask the owner or author of the program to add fflush(). Maybe you can limp along by just assuming that the prompt is being written.

File is being used by another process

I am developing a c# application, backend as sqlite.In my application i have an option for clean databse.It means the curren .db file will delete using File.Delete method and again it create empty databse using File.create method.Now let me explain the problem.
To perform cleandatabse task, i have to stop all the process which is running ,after doing that if i click on clean database it is throwing an error that file cannot delete, it is being used by another process.i am able to stop all the thread which is running.
Somehow i am able to find which process is blocikng the file ,
foreach (var process in Process.GetProcesses()) {
var files = GetFilesLockedBy(process);
if (files.Contains(filePath))
{
procs.Add(process);
Console.WriteLine(process.ProcessName);
process.Kill();
File.Delete(filePath);
}
}
But in the above code i used process.Kill, which close the window form which i am running.
without using kill, i tried close and dispose which doesn't work for me.
Can you please help me to release the file from the process without closing the application and then yo delete the db file.
Thank you in advance
Best regards
Sangita.
You should make sure you close every stream you open it:
using (Stream str = File.Create("C:\\h.txt"))
{
// your code here
} // the stream will be automatically closed here
if you don't put this using statement, it will cause you a lot of bugs, even if you close it manually str.Close();
Streamss are disposable types, you must manage their lifetime manually, either by that using syntax, e.g.:
using (StreamReader f = new ...) {
}
... or by doing it more verbosely (this syntax is required if you allocate and delete the Stream in different code-blocks/functions):
try {
StreamReader f = new ...;
...
} finally {
if (null != f) f.Dispose();
}
... or by making the holding class an IDisposable by itself. See also What Your Mother Never Told You About Resource Deallocation.
Interestingly, this seems to be a practical incarnation of one of those https://stackoverflow.com/questions/2245196/c-urban-myths/2245382#2245382 :
0) In C++, you must mess around with pointers, that's old and dangerous, use C#
Gee, #include boost/shared_ptr> or one of the like. Actually, it is often easier to produce mess in your sowonderful C#:
static void Main () {
foo();
bar();
}
static void foo () {
var f = new StreamWriter ("hello.txt");
f.Write ("hello world");
}
static void bar () {
var f = new StreamReader ("hello.txt");
Console.WriteLine (f.ReadToEnd ());
}
"Unhandled IOException: The process cannot access the file 'hello.txt' because it is being used by another process."
Those claims, btw, are often made by those who happen to never have heard of RAII, and about how far you can get without even smart-pointers.
Not sure but you may be calling Kill on the current process.
EDIT : call Delte after the loop.
Try this :
int currentProcessId = Process.GetCurrentProcess().Id;
foreach (var process in Process.GetProcesses()) {
if (process.Id != currentProcessId)
{
var files = GetFilesLockedBy(process);
if (files.Contains(filePath))
{
procs.Add(process);
Console.WriteLine(process.ProcessName);
process.Kill();
}
}
}
File.Delete(filePath);
Moreover Close doesn't terminate the process, you have to call CloseMainWindow or Kill.

Capture output of process synchronously (i.e. "when it happens")

I am trying to start a process and capture the output, have come a far way, but am not quite at the solution I'd want.
Specifically, I am trying to reset the IIS on my development machine from a small utility application that I am writing. I have come to the conclusion, by experimenting, that the safe way to do this is by running iisreset.exe in a child process.
If you run iisreset.exe on a command prompt, you get feedback during the process. Running iisreset takes several seconds, and several lines of feedback is generated, with pauses in between.
I'd like to capture this feedback and present it in my Windows Forms application (in a ListBox), and I have succeeded with that. My remaining concern is that I dont get it until the child process finishes. I'd like to get the output from the child process, line by line, immediately when the lines are created.
I have tried to do my homework, reading/testing things from e.g. these:
How to spawn a process and capture its STDOUT in .NET?
Capturing console output from a .NET application (C#)
http://www.aspcode.net/ProcessStart-and-redirect-standard-output.aspx
and several more with similar content. Most (all?) get the output asynchronously (e.g. with Process.ReadToEnd()). I want the output synchonously, which acording to the MSDN documentation involves establishing an event handler etc and I've tried that. It works, but the event handler does not get called until the process exits. I get the output from iisreset.exe, but not until it has finished.
To rule out the possibility that this has something to do with iisreset.exe in particular, I wrote a small console application that generates some output, pausing in between:
namespace OutputGenerator
{
class Program
{
static void Main(string[] args)
{
System.Console.WriteLine("OutputGenerator starting and pausing for 10 seconds..");
System.Threading.Thread.Sleep(10000);
System.Console.WriteLine("Pausing for another 10 seconds..");
System.Threading.Thread.Sleep(10000);
System.Console.WriteLine("Exiting!");
}
}
}
Testing with this it turns out that I get captured data diretly when I want. So, to some extent it seems that the way iisreset.exe outputs the data come into play here.
Here is the code of the program (a Windows Forms application) that does the capture:
using System;
using System.Windows.Forms;
using System.Diagnostics;
namespace OutputCapturer
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void btnRun_Click(object sender, EventArgs e)
{
// Running this will show all output after the process has exited
//String path = #"C:\Windows\system32\iisreset.exe";
// Running this will show all output "when it happens"
String path = #"C:\OutputGenerator.exe";
var p = new Process();
p.StartInfo.FileName = path;
p.StartInfo.UseShellExecute = false; // ShellExecute = true not allowed when output is redirected..
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.CreateNoWindow = true;
p.OutputDataReceived += OutputDataReceived;
p.Start();
p.BeginOutputReadLine();
}
private delegate void OutputDataToTextboxDelegate(String s);
void OutputDataToTextbox(String s)
{
tbxOutput.Text += s + Environment.NewLine;
tbxOutput.Refresh();
}
private void OutputDataReceived(object sender, DataReceivedEventArgs e)
{
if (e.Data != null && e.Data.ToString() != "")
{
// Must run the update of the textbox in the same thread that created it..
tbxOutput.Invoke(
new OutputDataToTextboxDelegate(OutputDataToTextbox),
DateTime.Now.ToString() + ": " + e.Data.ToString()
);
}
}
}
}
Thinking it was an EOL-encoding problem (the output of iisreset.exe apearing as one line to my app)), I ran a debug session. Nope. The event handler for StandardOutput gets called several times (one time for each output line from iisreset.exe), buth these calls come in one burst after the process exits.
I would LOVE if I could get the output from iisreset.exe "when it happens" so that I can show it as a progress indication.
I've seen one other thread with the same/similar problem, Asynchronous capture from a process output not working properly , but w/o a solution.
I'm sort of stumped.
To do autoflushing of printfs / stdouts
C equivalent of autoflush (flush stdout after each write)?
This saved my ass...
It seems that sixlettervariables is correct, and that this has something to do with iisreset.exe isn't flushing it's buffers for each line. (I still wonder what makes it work on a plain command line - i.e. what does cmd.exe do?)
Anyhow.. I tried what apacay suggested, and wrote this:
private void btnRun_Click(object sender, EventArgs e)
{
// Running this will show the output after the process has finished
//String path = #"C:\Windows\system32\iisreset.exe";
// Running this will show all output "when it happens"
String path = #"C:\OutputGenerator.exe";
var p = new Process();
p.StartInfo.FileName = path;
p.StartInfo.UseShellExecute = false; // ShellExecute = true not allowed when output is redirected..
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.CreateNoWindow = true;
p.Start();
StreamReader sr = p.StandardOutput;
while (!sr.EndOfStream)
{
String s = sr.ReadLine();
if (s != "")
{
tbxOutput.Text += DateTime.Now.ToString() + ": " + s + Environment.NewLine;
}
tbxOutput.Refresh();
}
}
Notice that I am timestamping when I get each line. For my OutputGenerator I get this:
2011-07-06 17:49:11: OutputGenerator starting and pausing for 10 seconds..
2011-07-06 17:49:21: Pausing for another 10 seconds..
2011-07-06 17:49:31: Exiting!
And for iisreset.exe I get this:
2011-07-06 17:57:11: Attempting stop...
2011-07-06 17:57:11: Internet services successfully stopped
2011-07-06 17:57:11: Attempting start...
2011-07-06 17:57:11: Internet services successfully restarted
Running iisreset.exe on the command line, those lines come with pauses in between, over a span of perhaps 10 seconds.
The case seems more or less closed now. Not that I am all that satisfied, but I'm at roads end it seems. I'll reluctantly live with it..
To summarise: In the general case, it is quite possible to capture output synchronously with when it is generated. This thread presents code for two ways to do that - by establishing an event handler, and by "polling" the stream. In my specific case there is something with how iisreset.exe generates output that prevents this.
Thanks to those who participated and contributed!
Well.... you could kick it old-school. Output can be redirected to the input of another program using old-school DOS commands (foo.exe | bar.exe). Write a program that reads from standard in, and you'll get it every time the stream flushes.
Edit
You could also redirect the ouput to a named pipe and read from that. That would also be "as it happens".
Well, I tried a helper class that I know works: http://csharptest.net/browse/src/Library/Processes/ProcessRunner.cs
ProcessRunner runner = new ProcessRunner("iisreset.exe");
runner.OutputReceived += OutputDataReceived;
runner.Start("/RESTART", "/STATUS");
However, this still doesn't solve the problem with this specific executable. It seems that iisreset was written in such a way that this is not possible. Even running the following from the command line:
iisreset.exe /RESTART /STATUS > temp.txt
Still nothing is written to the text file 'temp.txt' until after all services have been restarted.
As for your example code, I would recommend reading a post I wrote some time ago: How to use System.Diagnostics.Process correctly. Specifically you are not reading the std::err stream or redirecting and closing the std::in stream. This can cause very undesirable results in your program. You can look at the example wrapper class linked above for how to do it with the output events, or if you want to directly read the streams you need to use two of your own threads.
static void Main()
{
ProcessStartInfo psi = new ProcessStartInfo(#"C:\Windows\system32\iisreset.exe", "/RESTART /STATUS");
psi.CreateNoWindow = true;
psi.UseShellExecute = false;
psi.RedirectStandardError = true;
psi.RedirectStandardOutput = true;
psi.RedirectStandardInput = true;
ManualResetEvent output_complete = new ManualResetEvent(false);
ManualResetEvent error_complete = new ManualResetEvent(false);
Process p = Process.Start(psi);
new ReadOutput(p.StandardOutput, output_complete);
new ReadOutput(p.StandardError, error_complete);
p.StandardInput.Close();
p.WaitForExit();
output_complete.WaitOne();
error_complete.WaitOne();
}
private class ReadOutput
{
private StreamReader _reader;
private ManualResetEvent _complete;
public ReadOutput(StreamReader reader, ManualResetEvent complete)
{
_reader = reader;
_complete = complete;
Thread t = new Thread(new ThreadStart(ReadAll));
t.Start();
}
void ReadAll()
{
int ch;
while(-1 != (ch = _reader.Read()))
{
Console.Write((char) ch);
}
_complete.Set();
}
}
I wrote this just to see if anything was coming through. Still got nothing until the end, so I think your just SOL on getting asynchronous output from iisreset.
I've had that problem and had to solve it when my logs where too long to read in a single readtoend.
This is what I've done to solve it. It's been doing Ok so far.
myProcess.StartInfo.FileName = path;
myProcess.StartInfo.Arguments = args;
myProcess.StartInfo.UseShellExecute = false;
myProcess.StartInfo.ErrorDialog = false;
myProcess.StartInfo.CreateNoWindow = true;
myProcess.StartInfo.RedirectStandardError = true;
myProcess.StartInfo.RedirectStandardInput = (stdIn != null);
myProcess.StartInfo.RedirectStandardOutput = true;
myProcess.Start();
int index;
OpenLogFile(myLog); //LOGGGGGGGGGGGGG
if (myProcess.StartInfo.RedirectStandardInput)
{
StreamWriter sw = myProcess.StandardInput;
sw.Write(stdIn + Convert.ToChar(26));
}
StreamReader sr = myProcess.StandardOutput;
/*stdOut = new ArrayLi
*/
while (!sr.EndOfStream)
{ //LOGGGGGGGGGGGGG
Log(sr.ReadLine(), true);
}
Here's OpenLogFile
private void OpenLogFile(string fileName)
{
if (file == StreamWriter.Null)
{
file = new StreamWriter(fileName, true);
file.AutoFlush = true;
}
}
Of course that Log is a function that does something elsewhere. But the solution to you question lies here:
while (!sr.EndOfStream)
{ //LOGGGGGGGGGGGGG
Log(sr.ReadLine(), true);
}
while stream reader is still reading, you can be writing it down as the log comes out.
For my specific situation, the solution is what Mr Moses suggested in a comment above, i.e. run iisreset /stop followed by iisreset /start.
I need a proper answer, rather than a comment, in order to mark it as my "accepted answer", so this answer is more of administrativa than a new contribution. The cred should go to Mr Moses.. :-)

Detecting a File Delete on an Open File

I am opening a file with read access and allowing subsequent read|write|delete file share access to the file (tailing the file). If the file is deleted during processing is there a way to detect that the file is pending delete (see Files section http://msdn.microsoft.com/en-us/library/aa363858(v=VS.85).aspx)? If some outside process (the owning process) has issued a delete, I want to close my handle as soon as possible to allow the file deletion so as not to interfere with any logic in the owning process.
I'm in C# and see no method of detecting the pending delete. The file was opened using a FileStream object. Is there some method for detecting the delete in C# or in some other windows function?
You can use the Windows API function GetFileInformationByHandleEx to detect a pending delete on a file you have open. The second argument is an enumeration value which lets you specify what kind of information the function should return. The FileStandardInfo (1) value will cause it to return the FILE_STANDARD_INFO structure, which includes a DeletePending boolean.
Here is a demonstration utility:
using System;
using System.Text;
using System.IO;
using System.Runtime.InteropServices;
using System.Threading;
internal static class Native
{
[DllImport("kernel32.dll", SetLastError = true)]
public extern static bool GetFileInformationByHandleEx(IntPtr hFile,
int FileInformationClass,
IntPtr lpFileInformation,
uint dwBufferSize);
public struct FILE_STANDARD_INFO
{
public long AllocationSize;
public long EndOfFile;
public uint NumberOfLinks;
public byte DeletePending;
public byte Directory;
}
public const int FileStandardInfo = 1;
}
internal static class Program
{
public static bool IsDeletePending(FileStream fs)
{
IntPtr buf = Marshal.AllocHGlobal(4096);
try
{
IntPtr handle = fs.SafeFileHandle.DangerousGetHandle();
if (!Native.GetFileInformationByHandleEx(handle,
Native.FileStandardInfo,
buf,
4096))
{
Exception ex = new Exception("GetFileInformationByHandleEx() failed");
ex.Data["error"] = Marshal.GetLastWin32Error();
throw ex;
}
else
{
Native.FILE_STANDARD_INFO info = Marshal.PtrToStructure<Native.FILE_STANDARD_INFO>(buf);
return info.DeletePending != 0;
}
}
finally
{
Marshal.FreeHGlobal(buf);
}
}
public static int Main(string[] args)
{
TimeSpan MAX_WAIT_TIME = TimeSpan.FromSeconds(10);
if (args.Length == 0)
{
args = new string[] { "deleteme.txt" };
}
for (int i = 0; i < args.Length; ++i)
{
string filename = args[i];
FileStream fs = null;
try
{
fs = File.Open(filename,
FileMode.CreateNew,
FileAccess.Write,
FileShare.ReadWrite | FileShare.Delete);
byte[] buf = new byte[4096];
UTF8Encoding utf8 = new UTF8Encoding(false);
string text = "hello world!\r\n";
int written = utf8.GetBytes(text, 0, text.Length, buf, 0);
fs.Write(buf, 0, written);
fs.Flush();
Console.WriteLine("{0}: created and wrote line", filename);
DateTime t0 = DateTime.UtcNow;
for (;;)
{
Thread.Sleep(16);
if (IsDeletePending(fs))
{
Console.WriteLine("{0}: detected pending delete", filename);
break;
}
if (DateTime.UtcNow - t0 > MAX_WAIT_TIME)
{
Console.WriteLine("{0}: timeout reached with no delete", filename);
break;
}
}
}
catch (Exception ex)
{
Console.WriteLine("{0}: {1}", filename, ex.Message);
}
finally
{
if (fs != null)
{
Console.WriteLine("{0}: closing", filename);
fs.Dispose();
}
}
}
return 0;
}
}
I would use a different signaling mechanism. (I am making the assumption all file access is within your control and not from a closed external program, mainly due to the flags being employed.)
The only "solution" within those bounds I can think of is a poll on file-access and check the exception (if any) you get back. Perhaps there is something much more tricky (at a lower-level than the win32 file API?!?), but this is already going down the "uhg path" :-)
FileSystemWatcher would probably be the closest thing, but it can't detect a "pending" delete; when the file IS deleted, an event will be raised on FileSystemWatcher, and you can attach a handler that will gracefully interrupt your file processing. If the lock (or lack of one) you acquire in opening the file makes it possible for the file to be deleted at all, simply closing your read-only FileStream when that happens should not affect the file system.
The basic steps of a file watcher are to create one, passing an instance of a FileInfo object to the constructor. FileInfos can be created inexpensively by just instantiating one, passing it the path and filename of the file as a string. Then, set its NotifyFilter to the type(s) of file system modifications you want to watch for on this file. Finally, attach your process's event handler to the OnDeleted event. This event handler can probably be as simple as setting a bit flag somewhere that your main process can read, and closing the FileStream. You'll then get an exception on your next attempt to work with the stream; catch it, read the flag, and if it's set just gracefully stop doing file stuff. You can also put the file processing in a seperate worker thread, and the event handler can just tell the thread to die in some graceful method.
If the file is small enough, your application could process a copy of the file, rather than the file itself. Also, if your application needs to know whether the owning process deleted the original file, set up a FileSystemWatcher (FSW) on the file. When the file disappears, the FSW could set a flag to interrupt processing:
private bool _fileExists = true;
public void Process(string pathToOriginalFile, string pathToCopy)
{
File.Copy(pathToOriginalFile, pathToCopy);
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = pathToOriginalFile;
watcher.Deleted += new FileSystemEventHandler(OnFileDeleted);
bool doneProcessing = false;
watcher.EnableRaisingEvents = true;
while(_fileExists && !doneProcessing)
{
// process the copy here
}
...
}
private void OnFileDeleted(object source, FileSystemEventArgs e)
{
_fileExists = false;
}
No, there's no clean way to do this. If you were concerned about other processes opening and/or modifying the file, then oplocks could help you. But if you're just looking for notification of when the delete disposition gets set to deleted, there isn't a straightforward way to do this (sans building a file system filter, hooking the APIs, etc. all of which spooky for an application do be doing w/o very good reason).

Categories

Resources