Is there a good way to do this code, (to wait until this file is unlocked) when I get:
The process cannot access the file because it is being used by another process
I'm working on a web app so I can have concurrent access to this file from several different applications.
bool isLock = false;
do
{
try
{
File.WriteAllText(path, value, Encoding.Unicode);
isLock = false;
}
catch
{
Thread.Sleep(30);
isLock = true;
}
}while (isLock);
Your lock variable is of no use when multiple applications are in the scenario.
Rather test if you can File.OpenWrite and then write to the file.
If you cannot access the file loop and wait or write to a temporary file and start another thread looping and waiting until the temporary file can be merged.
Maybe even better would be to store immediatley to temp and let a watchdog write to your storage file.
public void SomeWhereInYourWebServerApplication
{
FileSystemWatcher fsw = new FileSystemWatcher("tempfolder");
fsw.Created += fsw_Created;
// save new entries to tempfolder
}
void fsw_Created(object sender, FileSystemEventArgs e)
{
foreach (string file in Directory.GetFiles("tempfolder"))
{
try
{
string fileText = File.ReadAllText(file);
File.AppendAllText("myStorage.txt", fileText);
File.Delete(file);
}
catch
{
// for me it's ok when we try to append the file the next time
// something new is coming
}
}
}
Nice and simple as i think.
Don't forget to do proper exception handling when files are involved.
If you absolutely must do it this way, and you have no control over the apps that are using the file, then your code is almost there.
It just needs to be made slightly more robust:
public static bool TryWriteText(string path, string text, TimeSpan timeout)
{
Contract.Requires(path != null); // Or replace with: if (path == null) throw new ArgumentNullException("path");
Contract.Requires(text != null); // Or replace with: if (text == null) throw new ArgumentNullException("text");
Stopwatch stopwatch = Stopwatch.StartNew();
while (stopwatch.Elapsed < timeout)
{
try
{
File.WriteAllText(path, text);
return true;
}
catch (IOException){} // Ignore IOExceptions until we time out, then return false.
Thread.Sleep(100); // 100ms is rather short - it could easily be 1000 I think.
} // Perhaps this should be passed in as a parameter.
return false;
}
Alternative version that rethrows the last IOException on timeout (this is arguably better since you don't hide all the exceptions):
public static void TryWriteText(string path, string text, TimeSpan timeout)
{
Contract.Requires(path != null); // Or replace with: if (path == null) throw new ArgumentNullException("path");
Contract.Requires(text != null); // Or replace with: if (text == null) throw new ArgumentNullException("text");
Stopwatch stopwatch = Stopwatch.StartNew();
while (true)
{
try
{
File.WriteAllText(path, text);
}
catch (IOException)
{
if (stopwatch.Elapsed > timeout)
throw;
}
Thread.Sleep(100);
}
}
But you should only use such code as a last resort.
Related
I have this part:
public string SendResponse(HttpListenerRequest request)
{
string result = "";
string key = request.QueryString.GetKey(0);
if (key == "cmd")
{
if (request.QueryString[0] == "uploadstatus")
{
switch (Youtube_Uploader.uploadstatus)
{
case "uploading file":
return "uploading " + Youtube_Uploader.fileuploadpercentages;
case "status":
return Youtube_Uploader.fileuploadpercentages.ToString();
case "file uploaded successfully":
Youtube_Uploader.uploadstatus = "";
return "upload completed," + Youtube_Uploader.fileuploadpercentages + ","
+ Youtube_Uploader.time;
default:
return "upload unknown state";
}
}
if (request.QueryString[0] == "nothing")
{
return "Connection Success";
}
if (request.QueryString[0] == "start")
{
StartRecrod();
result = "Recording started";
}
if (request.QueryString[0] == "stop")
{
dirchanged = false;
StartRecrod();
result = "Recording stopped and preparing the file to be shared on youtube";
string fileforupload = await WatchDirectory();
await WaitForUnlockedFile(fileforupload);
using (StreamWriter w = new StreamWriter(userVideosDirectory + "\\UploadedVideoFiles.txt", true))
{
w.WriteLine(fileforupload);
}
uploadedFilesList.Add(fileforupload);
Youtube_Uploader youtubeupload = new Youtube_Uploader(uploadedFilesList[0]);
}
}
else
{
result = "Nothing have been done";
}
return result;
}
Then i have the WatchDirectory method:
FileSystemWatcher watcher;
private void WatchDirectory()
{
watcher = new FileSystemWatcher();
watcher.Path = userVideosDirectory;
watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size;
watcher.Filter = "*.mp4";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
}
The OnChanged event:
private void OnChanged(object source, FileSystemEventArgs e)
{
var info = new FileInfo(e.FullPath);
fileforupload = info.FullName;
while(IsFileLocked(fileforupload) == true)
{
System.Threading.Thread.Sleep(100);
}
}
Then the IsFileLocked method:
public bool IsFileLocked(string filename)
{
bool Locked = false;
try
{
FileStream fs =
File.Open(filename, FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None);
fs.Close();
}
catch (IOException ex)
{
Locked = true;
}
return Locked;
}
Now the order should be that first it will go to the WatchDirectory and then in the while loop will check if the file is locked/busy once the file is not locked/busy any more continue with the rest of the code the StreamWriter the uploadedFilesList.Add and the Youtube_Uploader...
First i'm not sure if it's the right way to use the While loop with the 100.
And second how do i make that it will first finish the file lock checking before continue ? Now what it does is getting to the WatchDirectory then making the StreamWriter... Not the order i want it to be.
The biggest problem with your code is that it doesn't wait in any useful place, and it does wait in the last place you want:
You call WatchDirectory() and then immediately move on to creating your writer. There's nothing in the WatchDirectory() method that would delay its return, so of course you move on to the next statement before anything's happened.
In the OnChanged() method, you poll for the file lock status. But this method is an event handler for the FileSystemWatcher event, and will be called in a context where you really don't/shouldn't be delaying the thread.
I would change your code to take advantage of the async pattern, not only to fix the problems above, but also to provide asynchronous operation, i.e. to prevent this logic from holding up the rest of your program while it's waiting for something interesting to happen in the watched directory.
Here are new versions of your methods that I think would be better:
private async Task<string> WatchDirectoryAsync()
{
using (FileSystemWatcher watcher = new FileSystemWatcher())
{
TaskCompletionSource<string> tcs = new TaskCompletionSource<string>();
watcher.Path = userVideosDirectory;
watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size;
watcher.Filter = "*.mp4";
watcher.Changed += (sender, e) => tcs.SetResult(e.FullPath);
watcher.EnableRaisingEvents = true;
return await tcs.Task;
}
}
// You can get rid of the OnChanged() method altogether
private async Task WaitForUnlockedFileAsync(string fileName)
{
while (true)
{
try
{
using (IDisposable stream = File.Open(fileName, FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None))
{ /* on success, immediately dispose object */ }
break;
}
catch (IOException)
{
// ignore exception
// NOTE: for best results, consider checking the hresult value of
// the exception, to ensure that you are only ignoring the access violation
// exception you're expecting, rather than other exceptions, like
// FileNotFoundException, etc. which could result in a hung process
}
// You might want to consider a longer delay...maybe on the order of
// a second or two at least.
await Task.Delay(100);
}
}
Which you can then use like this:
if (request.QueryString[0] == "stop")
{
dirchanged = false;
StartRecrod();
result = "Recording stopped and preparing the file to be shared on youtube";
string fileforupload = await WatchDirectoryAsync();
await WaitForUnlockedFileAsync(fileforupload);
using (StreamWriter w = new StreamWriter(userVideosDirectory + "\\UploadedVideoFiles.txt",true))
{
w.WriteLine(fileforupload);
}
uploadedFilesList.Add(fileforupload);
Youtube_Uploader youtubeupload = new Youtube_Uploader(uploadedFilesList[0]);
}
Naturally, to use await in the above, the code will need to be contained in an async method. Without a good, minimal, complete code example showing the entire context, it's impossible to say how exactly you'd incorporate that into the program as a whole. But there is lots of advice on Stack Overflow and elsewhere on that topic. The basic idea is that, typically, the calling methods would all have to be turned into async methods, up to the top of the stack where the call chain starts (which is often some kind of event handler, invoked when the user performs some kind of input).
In some cases, you can just call the async method and ignore the returned Task object reference (not ideal), or defer handling of the return value (better). You'll have to decide based on your own scenario what works best in your case.
EDIT:
If you cannot or will not change the original calling method to be an async method, it is possible to perform these operations synchronously. They can be implemented themselves as synchronous methods as follows:
private string WatchDirectory()
{
using (FileSystemWatcher watcher = new FileSystemWatcher())
{
TaskCompletionSource<string> tcs = new TaskCompletionSource<string>();
watcher.Path = userVideosDirectory;
watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size;
watcher.Filter = "*.mp4";
watcher.Changed += (sender, e) => tcs.SetResult(e.FullPath);
watcher.EnableRaisingEvents = true;
return tcs.Task.Result;
}
}
// You can get rid of the OnChanged() method altogether
private void WaitForUnlockedFile(string fileName)
{
while (true)
{
try
{
using (IDisposable stream = File.Open(fileName, FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None))
{ /* on success, immediately dispose object */ }
break;
}
catch (IOException)
{
// ignore exception
// NOTE: for best results, consider checking the hresult value of
// the exception, to ensure that you are only ignoring the access violation
// exception you're expecting, rather than other exceptions, like
// FileNotFoundException, etc. which could result in a hung process
}
// You might want to consider a longer delay...maybe on the order of
// a second or two at least.
Thread.Sleep(100);
}
}
Or you could simply consume the asynchronously implemented operations synchronously:
if (request.QueryString[0] == "stop")
{
dirchanged = false;
StartRecrod();
result = "Recording stopped and preparing the file to be shared on youtube";
string fileforupload = WatchDirectoryAsync().Result;
WaitForUnlockedFileAsync(fileforupload).Wait();
using (StreamWriter w = new StreamWriter(userVideosDirectory + "\\UploadedVideoFiles.txt",true))
{
w.WriteLine(fileforupload);
}
uploadedFilesList.Add(fileforupload);
Youtube_Uploader youtubeupload = new Youtube_Uploader(uploadedFilesList[0]);
}
You might choose the latter if, for example, you intend to convert your other code to async eventually but for some reason just can't right now.
Note that I don't advise this sort of approach. These operations are inherently asynchronous; i.e. they depend on and wait for some external activity that is itself not happening synchronously. And so in the long run, your program will work much better if it itself is not made to stop its progress while waiting for this external operation.
When the code is working so laggy it would be very good so that it is not laggy when it works.
How the code works:
It searches the computer for a file is then when it find it to change the file but if the file is running line will loop until it manages to do its job.
Main class
public Form1(string[] Args)
{
InitializeComponent();
backgroundWorker1.RunWorkerAsync();
}
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
Thread.Sleep(1000); // One second.Thread.Sleep(1000); // One second.
MessageBox.Show("Testing");
Fille mc = new Fille();
mc.Search();
}
Fille clss
private static ArrayList list2 = new ArrayList();
private static ArrayList listRemove = new ArrayList();
public void Search()
{
try
{
foreach (string file in Directory.EnumerateFiles(#"C:\Users\user\Downloads\MCFILE\trrtrt\", "*.exe", SearchOption.AllDirectories))
{
// Display file path.
if (SHA1Hash.GetSHA1Hash(file) == "1233456") // fake SHA1Hash
{
try
{
COPYWithReplace(#"C:\Users\user\Downloads\MCFILE\Fake2\Test.exe", file);
}
catch (IOException)
{
// log errors
if (list2.Count == 0)
{
list2.Add(file);
Thread thread = new Thread(new ThreadStart(Test2));
thread.Start();
}
else
{
Thread thread = new Thread(new ThreadStart(Test2));
thread.Abort();
list2.Add(file);
thread.Join();
}
}
}
}
}
catch (Exception ex)
{
// log errors
}
}
private void Test2()
{
if (list2.Count == 0)
{
}
else
{
foreach (string _item in list2)
{
try
{
//Run
COPYWithReplace(#"C:\Users\user\Downloads\MCFILE\Fake2\Test.exe", _item);
listRemove.Add(_item);
}
catch (IOException)
{
//error
}
}
foreach (var Remove in listRemove)
{
list2.Remove(Remove);
}
listRemove.Clear();
if (list2.Count == 0)
{
}
else
{
Thread thread = new Thread(new ThreadStart(Test2));
thread.Start();
}
}
}
I made a new thread because I found the problem. But now it's just that it lags.
I suspect the reason it's "lagging" is because you have the system in a very convoluted but rather processor intensive and I/O intensive loop. If a file fails the first test, your code starts a thread that tries it again. And if that fails then you start another thread to try it again, lather, rinse, repeat.
That's going to absolutely kill performance. You're basically doing this:
while (forever)
{
if I can overwrite the file
{
break;
}
}
Except if you have multiple files that you're trying to write, then you're doing that loop for every file. Concurrently. And you're not just using a loop. Instead, you're starting and stopping threads like nobody's business.
Yeah, that's going to slow down your computer.
A more reasonable way to do this would be with a thread to do the first check, and a timer that will limit how often you do the other checks. Communication is with a simple queue, because only one thread will be accessing it at a time.
Here's what I would recommend:
private static Queue<string> filesToCheck = new Queue<string>();
private System.Timers.Timer copyTimer;
public void Search()
{
try
{
foreach (string file in Directory.EnumerateFiles(#"C:\Users\user\Downloads\MCFILE\trrtrt\", "*.exe", SearchOption.AllDirectories))
{
// Display file path.
if (SHA1Hash.GetSHA1Hash(file) == "1233456") // fake SHA1Hash
{
if (!TryToCopy(file)) // See function below
{
filesToCheck.Enqueue(file);
}
}
}
// Checked all the files once.
// If there are any in the queue, start the timer.
if (filesToCheck.Count > 0)
{
copyTimer = new System.Timers.Timer(CopyTimerProc, null, 1000, Timeout.Infinite);
}
}
catch (Exception)
{
// do your error handling
}
}
private void CopyTimerProc(object state)
{
string filename = filesToCheck.Dequeue();
if (TryToCopy(filename))
{
// success. If the queue is empty, kill the timer.
if (filesToCheck.Count == 0)
{
copyTimer.Dispose();
}
}
else
{
// File still locked.
// Put it back on the queue and reset the timer.
filesToCheck.Enqueue(filename);
copyTimer.Change(1000, 0);
}
}
private bool TryToCopy(string filename)
{
try
{
COPYWithReplace(#"C:\Users\user\Downloads\MCFILE\Fake2\Test.exe", filename);
return true;
}
catch (IOException)
{
// log error
return false;
}
}
The timer is a one-shot that is reset after each time it ticks. The reason I did it this way is to prevent another tick coming along while the previous tick is still processing. After all, it takes time to copy a file.
There's no reason to do this with a bunch of threads. The file system can only do one thing at a time, anyway, and it's not like an extra second or two while you wait for a file to become available is going to hurt anything.
This is really short question. I don't understand try-catch mechanism completely.
This is my current code:
public static void WriteText(string filename, string text)
{
try
{
System.IO.StreamWriter file = new System.IO.StreamWriter(filename);
file.Write(text);
file.Close();
}
catch(Exception exc)
{
MessageBox.Show("File is probably locked by another process.");
}
}
Background:
Im writing application that shares configuration files with another application.
I need some dialog messagebox with "retry" and "abort" buttons, when that file is used by other application. When that message will appear - I will close that other application and I will try to rewrite that file again by pressing "Retry" button.
Whatr we have is using a counter for re-tries and possibly a thread sleep.
So something like
int tries = 0;
bool completed = false;
while (!completed)
{
try
{
System.IO.StreamWriter file = new System.IO.StreamWriter(filename);
file.Write(text);
file.Close();
completed = true;
}
catch(Exception exc)
{
tries++;
//You could possibly put a thread sleep here
if (tries == 5)
throw;
}
}
Even though there's a good answer already I'll submit one that's more tuned towards the OP's question (let the user decide instead of using a counter).
public static void WriteText(string filename, string text)
{
bool retry = true;
while (retry)
{
try
{
System.IO.StreamWriter file = new System.IO.StreamWriter(filename);
file.Write(text);
file.Close();
retry=false;
}
catch(Exception exc)
{
MessageBox.Show("File is probably locked by another process.");
// change your message box to have a yes or no choice
// yes doesn't nothing, no sets retry to false
}
}
}
If you need more info on how to implement the messagebox check out the following links;
http://msdn.microsoft.com/en-us/library/0x49kd7z.aspx
MessageBox Buttons?
I would do it like that:
public static void WriteText(string filename, string text, int numberOfTry = 3, Exception ex = null)
{
if (numberOfTry <= 0)
throw new Exception("File Canot be copied", ex);
try
{
var file = new System.IO.StreamWriter(filename);
file.Write(text);
file.Close();
}
catch (Exception exc)
{
WriteText(filename,text,--numberOfTry,ex);
}
}
I like it more like this (example tries to save a RichTextBox on close and allows retrying save or aborting close):
protected override void OnClosing(CancelEventArgs e)
{
if (richTextBox_Query.Modified)
{
DialogResult result;
do
try
{
richTextBox_Query.SaveFile(
Path.ChangeExtension(Application.ExecutablePath, "sql"),
RichTextBoxStreamType.UnicodePlainText);
result = DialogResult.OK;
richTextBox_Query.Modified = false;
}
catch (Exception ex)
{
result = MessageBox.Show(ex.ToString(), "Exception while saving sql query",
MessageBoxButtons.AbortRetryIgnore);
e.Cancel = result == DialogResult.Abort;
}
while (result == DialogResult.Retry);
}
base.OnClosing(e);
}
I need to create a C# Console Application that will parse the file from SFTP directory when the new file created. For that I implemented FileSystemWatcher with FileCreated event which enqueue the new file path and create a new thread and parse the file.
I read in the blogs that some times FileSystemWatcher may fail to detect new files, for that I implemented Timer which will fire every 1 hr and if the FileSystemWatcher thread is in waitsleep state then will read the IMCOMING SFTP folder and parse the file.
Below is the code i written for FileSystemWatcher and Timer, but its not working properly and I think filesystemwatcher is not in Multithreading. Please help me to get right solution.
MAIN
static void Main(string[] args)
{
try
{
string path = incomingFilePath;
if (Directory.Exists(path))
{
#region Initiate Timer
Thread t = new Thread(new ParameterizedThreadStart(ThreadLoop));
t.Start((Action)fileProcessor.StartTimer);
#endregion
#region FileSystemWatcher
watcher = new FileSystemWatcher { Path = path, Filter = "*.CUST", IncludeSubdirectories = true };
watcher.Created += new
FileSystemEventHandler(watcher_FileCreated);
watcher.Error += new
ErrorEventHandler(watcher_OnError);
watcher.EnableRaisingEvents = true;
#endregion
}
}
catch (Exception Err)
{
}
}
FILESYSTEMWATCHER CODE:
private static void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
if (e.FullPath.ToUpper().Contains("INCOMING"].ToString()))
{
fileProcessor.EnqueueFile(e.FullPath);
lock (lockObject)
{
files.Enqueue(path);
}
if (FileWacherThread == null || shouldStop)
{
FileWacherThread = new Thread(new ThreadStart(Work));
FileWacherThread.Start();
}
// If the thread is waiting then start it
else if (FileWacherThread.ThreadState == ThreadState.WaitSleepJoin)
{
waitHandle.Set();
}
}
}
private void Work()
{
while (!shouldStop)
{
string path = String.Empty;
lock (lockObject)
{
if (files.Count > 0)
{
path = files.Dequeue();
}
}
if (!String.IsNullOrEmpty(path))
{
// Process the file
ParseFile(path);
}
else
{
// If no files are left to process then wait
waitHandle.WaitOne();
}
}
}
TIMER CODE
public void StartTimer()
{
lock (lockObject)
{
if (FileWacherThread == null || FileWacherThread.ThreadState == ThreadState.WaitSleepJoin)
{
if (files.Count == 0)
{
IEnumerable<string> result = new List<string>(Directory.GetFiles(incomingFilePath, "*.CUST", SearchOption.AllDirectories)).Where(s => s.Contains(incomingFilePrefix));
foreach (string path in result)
{
ParseFile(path);
}
}
}
}
}
Things to check...
is waitHandle an AutoResetEvent or a ManualResetEvent? (from the way that you are using it, it should be an AutoResetEvent
If shouldStop is true, is FileWacherThread(sic) set to null when Work() exits...
How are you protecting access to FileWacherThread? if it is accessed from multiple threads (to check its state, assign etc, then it too should be protected with a lock).
You shouldn't worry about the state of the FileWacherThread when you set the event. If you want to signal to that thread, just set it, (i.e. build your multithreaded code such that the publisher doesn't know/care about the current state of the subscriber).
Currently there are states that your FileWacherThread can be in where it isn't waiting but it might still need to be signaled. If you always set the event, the worst that can happen is that it loops one time unnecessarily.
I found a post talking about handling concurrent file access with StreamWriter.
The problem is that the answers do not manage the scenario where the file is being accessed but multiple processes.
Let's tell it shortly :
I have multiple applications
I need a centralised logging system in dataBase
If database fail, I need a fallback on a file system log
There is a known concurrency scenario, where multiple applications (processes) will try to write in that file.
This can be managed by re-attempt the writing after a short delay.
But I don't want ot reattempt if it's a security error or filename syntax error.
The code is here :
// true if an access error occured
bool accessError = false;
// number fo writing attemps
int attempts = 0;
do
{
try
{
// open the file
using (StreamWriter file = new StreamWriter(filename, true))
{
// write the line
file.WriteLine(log);
// success
result = true;
}
}
/////////////// access errors ///////////////
catch (ArgumentException)
{
accessError = true;
}
catch (DirectoryNotFoundException)
{
accessError = true;
}
catch (PathTooLongException)
{
accessError = true;
}
catch (SecurityException)
{
accessError = true;
}
/////////////// concurrent writing errors ///////////////
catch (Exception)
{
// WHAT EXCEPTION SHOULD I CATCH HERE ?
// sleep before retrying
Thread.Sleep(ConcurrentWriteDelay);
}
finally
{
attempts++;
}
// while the number of attemps has not been reached
} while ((attempts < ConcurrentWriteAttempts)
// while we have no access error
&& !accessError
// while the log is not written
&& !result);
My only question is the type of exception that will be raised in the case of concurrency writting. I already know things can be done differently. Let me add a few considerations :
No, I don't want to use NLog in that scenario
Yes I handle concurrency with IOC + Mutex for the in-process concurrency
Yes I really want all log to be written in the same file
It will be an IOException with text:
"The process cannot access the file '{0}' because it is being used by another process."
This is a simplistic approach:
static bool LogError(string filename, string log)
{
const int MAX_RETRY = 10;
const int DELAY_MS = 1000; // 1 second
bool result = false;
int retry = 0;
bool keepRetry = true;
while (keepRetry && !result && retry < MAX_RETRY )
{
try
{
using (StreamWriter file = new StreamWriter(filename, true))
{
// write the line
file.WriteLine(log);
// success
result = true;
}
}
catch (IOException ioException)
{
Thread.Sleep(DELAY_MS);
retry++;
}
catch (Exception e)
{
keepRetry = false;
}
}
return result;
}