Develop Reference

Co-routine killed after yield return UnityWebRequest.SendWebRequest

In my application several co-routines are downloading files in parallel, and there is a chance that they can try do download the same file to the same path - resulting into a collision and an access exception. I've tried solving this by using a static collection private static List<string> filesBeingDownloadedNow = new List<string>(); which holds filePath for files which are currently being downloaded.
It looks like this:
private static List<string> filesBeingDownloadedNow = new List<string>();
private IEnumerator DoDownloadCoroutine(string url, string filePath)
{
filesBeingDownloadedNow.Add(filePath);
var webRequest = new UnityWebRequest(url)
{
method = UnityWebRequest.kHttpVerbGET,
downloadHandler = new DownloadHandlerFile(filePath)
{
removeFileOnAbort = true
}
};
yield return webRequest.SendWebRequest();
if (webRequest.isNetworkError || webRequest.isHttpError)
{
// do something if the file failed to download
}
else
{
// do something with the downloaded file
}
filesBeingDownloadedNow.Remove(filePath);
}
and elsewhere:
if (filesBeingDownloadedNow.Contains(filePath))
{
// start another coroutine which simply waits until
// filesBeingDownloadedNow doesn't contain filePath anymore
// and then works with the downloaded file
// as if this coroutine downloaded it itself
}
else
{
StartCoroutine(DoDownloadCoroutine(url, filePath));
}
Now, this works perfectly fine, but there is a rare case where the gameObject on which this coroutine runs can be destroyed before the file download coroutine finishes. So, yield return webRequest.SendWebRequest(); will be the last thing ever called here, and neither success nor failure can be checked, and filesBeingDownloadedNow.Remove(filePath); is never reached.
I tested this, and it seems that UnityWebRequest.SendWebRequest still downloads the file in full - however, it's unclear when it will end or if the file gets downloaded successfully at all. It simply "gets lost in the ether".
I could simply trying doing exception handling around File.Open or whatever other things I'd like to do with the downloaded file (and actually should do that in any case). However, UnityWebRequest doesn't allow handling exceptions since it's called with yield return.
Even if before calling webRequest.SendWebRequest(); I check if the file already exists with if (File.Existst(filePath)), that doesn't help if 2 different UnityWebRequests try downloading the same file at almost the same time. It can be that the 1st UnityWebRequest has only just connected to the server and hasn't created a file at filePath yet, and the 2nd coroutine will see that the file doesn't exist and assume that no one is downloading it right now, and still attempt a download with webRequest.SendWebRequest();, resulting in a collision and an exception which I can't handle.
So basically, there is a combination of factors:
Co-routines can be silently killed at any moment without anything like "finally" to do clean up or anything before it's killed
UnityWebRequest.SendWebRequest doesn't allow handling exceptions (?)
Delay before the downloaded file actually appears at the location makes it impossible to check if a file is already being downloaded
Frankly, I also really dislike the static list here, but it seems that since files are a global resource anyway, making it non-static solves nothing. UnityWebRequest.SendWebRequest will keep downloading the file on its own no matter what I do with that list.

You never waited for the download handler to finish, it will result in weird behaviour sometimes if you try to use it before it finished, better yield null until the handler isDone flag is true, that is it finishes handling the downloaded data, depending on its type.
and YES, the request finishes downloading but sometimes the handler doesn't finish its 'handling', the same instant the download is done.
UnityWebRequest webRequest = UnityWebRequest.Get(url);
webRequest.downloadHandler = new DownloadHandlerFile(filePath) { removeFileOnAbort = true };
// 1) Wait until the download is done.
yield return webRequest.SendWebRequest();
if (webRequest.isNetworkError || webRequest.isHttpError)
{
// do something if the file failed to download
}
else
{
// 2) Wait until the handler itself finishes.
while (!webRequest.downloadHandler.isDone)
yield return null;
// Now you can safely do something with the downloaded file
}

Handling concurrent reading/writing of a file

I have a web service that is generating random errors and I think I've found the fault.
Basically, it is reading a config file as follows, then loaded into an XmlDocument:
var config = File.ReadAllText(filename);
xmlDoc.Load(config);
However, at a later time (maybe a second later) one of the config values is updated and the file saved
xmlDoc.Save(filename);
So I'm now experiencing more errors (unfortunately the original developer added an empty try block so I can't check just now) during the first READ operation and I think it's because it's trying to read the file just as another process spawned from IIS is at the .Save part. I don't know how File.ReadAllText works and whether it will fail on a write locked file.
What's the best solution to handle this to ensure reading will always work? the value being written is just a counter and if it fails it is ignored as it's not that important but would prefer it was written. I guess I could put it into a separate config file and live with the error but I'd rather it was just one file.
Thanks.

You can use a lock to make sure that a read is completed before a write and vice verser. As in:
using System;
using System.Threading;
class Program
{
static readonly object _fileAccess = new object();
static void Write()
{
// Obtain lock and write
lock (_fileAccess)
{
// Write data to filename
xmlDoc.Save(filename);
}
}
static void Read()
{
// Obtain lock and read
lock (_fileAccess)
{
// Read some data from filename
xmlDoc.load(filename);
}
}
static void Main()
{
ThreadStart writeT = new ThreadStart(Write);
new Thread(writeT).Start();
ThreadStart readT = new ThreadStart(Read);
new Thread(readT).Start();
}
}
With the lock, the Read() must wait for the Write() to complete and Write() must wait for Read() to complete.

To answer your question about how File.ReadAllText() works, looking at the source, it uses a StreamReader internally which in turn uses a FileStream opened with FileAccess.Read and FileShare.Read, so that would prevent any other process from writing to the file (e.g. your XmlDocument.Save()) until the ReadAllText completed.
Meanwhile, your XmlDocument.Save() eventually uses FileStream opened with
FileAccess.Write and FileShare.Read, so it would allow the File.ReadAllText() as long as the Save started before the ReadAllText.
References: https://referencesource.microsoft.com/#mscorlib/system/io/streamreader.cs,a820588d8233a829
https://referencesource.microsoft.com/#System.Xml/System/Xml/Dom/XmlDocument.cs,1db4dba15523d588

Windows: Delete EXE after execution

I am working on an application in C# which does the following things:
Write an EXE to disk
Execute the EXE through Process.Start()
I am now trying to ensure that the EXE will be deleted once it is closed.
The easiest way to do so is to set the FileOptions.DeleteOnClose parameter when creating the EXE using File.Create.
However, this means that the EXE can not be executed while is in use. Once the file handle is closed, the EXE is immediately deleted before it can be executed.
Is there any way to retain a "weak reference" to the EXE in my application which does not lock the file and allows it to be executed? Alternatively, is there any way to unlock the EXE for execution with the file handle still open? Are there any other obvious solutions I am missing?
CLARIFICATION A: I am aware of other methods to delete files in use which will delete the file eventually (e.g. upon reboot). I am however looking for a method to delete the file immediately once it starts executing which is handled by the OS (e.g. when running a batch that first executes the file and then deletes it, the file would remain on disk if the batch job is terminated).
CLARIFICATION B: To explain the bigger picture: The application receives and decrypts an executable file. After decryption, the file should be executed. However, I want to make sure the decrypted version of the EXE does not stay on disk. Ideally, I also want to prevent users from copying the decrypted EXE. However, since the decryption application runs as the same user, this will be impossible to achieve in a truly secure fashion as both have the same privileges on the system.

You could use Process.WaitForExit:
var process = Process.Start(processPath);
process.WaitForExit();
// File.Delete(processPath); // Not strong enough (thanks to Binary Worrier)
DeleteOrDie(processPath); // Will attempts anything to delete the file.
But it gives the possibility to copy the exe from where you writed it.
A good solution is to run it from memory.
If your target exe is a CLR program, you can use the Assembly.Load function:
// read the file and put data in bin
...
Assembly a = Assembly.Load(bin);
MethodInfo method = a.EntryPoint;
if (method == null) throw new NoEntryPointException();
object o = a.CreateInstance(method.Name);
method.Invoke(o, null);
More details here.
If you want to load/execute any exe in memory, you could use the Nebbett’s Shuttle approach but you will need to code it in C/C++ and make a call to it from C#.
Also it looks like Microsoft doesn't like it (security issues) and I don't think you can achieve it from C# only. Good anti-virus will probably detect it.

In a not very good way but a way that can give you what you want, I suggest this solution:
(I use a Console Application with some input arguments for this solution)
[1: ] Write a function to check opened processes:
/// <summary>
/// Check application is running by the name of its process ("processName").
/// </summary>
static bool IsProcessOpen(string processName)
{
foreach (Processing.Process clsProcess in Processing.Process.GetProcesses())
{
if (clsProcess.ProcessName.ToUpper().Contains(processName.ToUpper()))
return true;
}
return false;
}
[2: ] Define some variables:
static bool IamRunning = true;
static int checkDuration = 1; // in seconds
[3: ] Use a Thread for run a loop for checking:
Thread t = new Thread(delegate() {
DateTime lastCheck = DateTime.MinValue;
while (IamRunning)
{
var now = DateTime.Now;
int dd = (now.Hour - lastCheck.Hour) * 3600 + (now.Minute - lastCheck.Minute) * 60 + now.Second - lastCheck.Second;
if (dd >= checkDuration)
if (!IsProcessOpen("ProcessName"))
{
delApplication(); // You have a function to delete ...
break;
}
}
});
t.SetApartmentState(ApartmentState.STA);
t.Start();
[4: ] Use a loop at the end of the program:
while (t.ThreadState == ThreadState.Running)
{
// just wait.
}
Note: This solution by Console Application in my low computer have 50% usage of CPU.

detecting that a file is currently being written to

(I know It's a common problem but I couldn't find an exact answer)
I need to write a windows service that monitors a directory, and upon the arrival of a file, opens it, parses the text, does something with it and moves it to another directory afterwards. I used IsFileLocked method mentioned in this post to find out if a file is still been written. My problem is that I don't know how much it takes for another party to complete writing into the file. I could wait a few seconds before opening the file but this is not a perfect solution since I don't know in which rate is the file written to and a few seconds may not suffice.
here's my code:
while (true)
{
var d = new DirectoryInfo(path);
var files = d.GetFiles("*.txt").OrderBy(f => f);
foreach (var file in files)
{
if (!IsFileLocked(file))
{
//process file
}
else
{
//???
}
}
}

I think you might use a FileSystemWatcher (more info about it here: http://msdn.microsoft.com/it-it/library/system.io.filesystemwatcher(v=vs.110).aspx ).
Specificially you could hook to the OnChanged event and after it raises you can check IsFileLocked to verify if it's still being written or not.
This strategy should avoid you to actively wait through polling.

one process waiting for another process's output via the file system

I have a process A that reads in some data produced by some other process B. The data is 'exchanged' via the file system. To ensure that the file exists, process A currently checks for the file's existence like this:
while (!File.Exists(FileLocation))
{
Thread.Sleep(100);
}
This only seems to work 99 percent of the time. The other 1 percent of the time, process A establishes that the file exists but process B has not written everything yet (i.e. some data is missing).
Is there another simpler way to make the above situation more bullet proofed? Thanks.

Is there another simpler way to make the above situation more bullet proofed?
You could use a Mutex for reliable inter-process synchronization. Another possibility is to use a FileSystemWatcher.

After determining that the file exists, you can try opening the file for exclusive access, which will fail if another process still has the file open:
try
{
File.Open("foo",FileMode.Open,FileAccess.Read,FileShare.None);
}
catch(IOException ex)
{
// go back to
}

Given that you say that you can change both processes' code, you can use an EventWaitHandle to communicate between the processes.
In your program that creates the file, in the Main() method you can create an EventWaitHandle and keep it around until the end of the program. You'll need to pass the EventWaitHandle object around in your program so that it is available to the bit of code that creates the file (or provide some method that the file-creating code can call to set the event).
using (EventWaitHandle readySignaller = new EventWaitHandle(false, EventResetMode.ManualReset, "MySignalName"))
{
// Rest of program goes here...
// When your program creates the file, do this:
readySignaller.Set();
}
Then have some code like this in the program that's waiting for the file:
// Returns true if the wait was successful.
// Once this has returned true, it will return false until the file is created again.
public static bool WaitForFileToBeCreated(int timeoutMilliseconds) // Pass Timeout.Infinite to wait infinitely.
{
using (EventWaitHandle readySignaller = new EventWaitHandle(false, EventResetMode.ManualReset, "MySignalName"))
{
bool result = readySignaller.WaitOne(timeoutMilliseconds);
if (result)
{
readySignaller.Reset();
}
return result;
}
}
NOTE: If we successfully wait note that I am resetting the signal and it will remain reset until the other process sets it again. You can handle the logic differently if you need to; this is just an example.
Essentially what we are (logically) doing here is sharing a bool between two processes. You have to be careful about the order in which you set and reset that shared bool.

Try the FileSystemWatcher.
Listens to the file system change notifications and raises events when
a directory, or file in a directory, changes.