How to use Thread Pool and Mutex in c#? - c#

I try to learn how to use Thread Pool and Mutex, as a practice I'm trying to make an application that copy files from one path in the computer to another path in the computer.
To do this application I used Thread Pool (so a few copies could happen simultaneously):
object[] paths = new object [2]; // The source path and the destination path
string[] files[] = System.IO.Directory.GetFiles(sourceFolderPath); //string sourceFolderPath = Folder path that contains the files
foreach(string s in files)
{
paths[0] = s; // The file source - s = the file name with the file path;
paths[1] = s.Replace(sourceFolderPath, destFolderPath); // Replaces between the source folder and the destination folder but keeps the file name
ThreadPool.QueueUserWorkItem(new waitCallback(CopyFIle), paths);
}
So far, the application sends each one of the files to the function that copy the file from the source folder to the destination folder.
The CopyFile function looks like this:
static void CopyFiles(object paths)
{
object[] temp = paths as object[]; // The casting to object array
string sourcePath = temp[0].ToString();
string destPath = temp[1].ToString();
System.IO.File.Copy(filePath, destPath, true); // The copy from the source to the dest
}
The weird thing is that when I run the application it throws an exception: "The process cannot access to the file 'C:..........' because it is being used by another process".
When I try to find out the mistake and I run the application step by step the application run properly and the whole files are copied from the source folder to the destination folder.
That case made me think that maybe the fact that I'm using ThreadPool made a two threads or more to open the same file (the thing that should not happen because I used foreach and each one of the files paths is transferred as a parameter only once).
To solve this I tried to use Mutex and now the CopyFile function looks like this:
static Mutex mutex = new Mutex();
static void CopyFiles(object paths)
{
Mutex.WaitOne(); //Waits until the critical section is free from other threads
try
{
object[] temp = paths as object[]; // The casting to object array
string sourcePath = temp[0].ToString();
string destPath = temp[1].ToString();
System.IO.File.Copy(filePath, destPath, true); // The copy from the source to the dest
}
Finally
{
Mutex.ReleaseMutex(); //Release the critical area
}
}
Now the application should wait until the critical area is free and just then try to copy the file so the exception: "The process cannot access to the file 'C:..........' because it is being used by another process" should not appear.
As I thought, the exception did not appear but the application copied only one file from the source folder to the destination folder and not all of the files.
When I tried to run this application step by step the same weird thing happened and everything went properly, all of the files were copied to the destination folder.
Why this is happen? And how can I solve this problem so all the files will be copied to the destination folder in a normal application running and not step by step?

Your problem isn't the ThreadPool. What goes wrong is the argument.
In your first code-snippet, you fill an object-array with two argument-values and pass that to the Queue-method. What happens here is, that you use always the same array. So in the first iteration of your foreach-loop, you write two values into the array, pass it. Eventually, the method gets executed be the ThreadPool, using that object-array. In the same time, in the second iteration of the foreach-loop, you write again into that exact array and pass it again to the ThreadPool. That means, two (or more) Threads are starting to work on that array.
You don't know when the CopyFiles-method is active, so you can't tell when the array was unboxed and is ready for re-usage. That could be achieved using a mutual exclusion (the easiest way in C# is using the lock-keyword), but that's not the way you should use here.
The better way would be to create new object-arrays each iteration of the foreach-loop. Simply change the code to:
string[] files[] = System.IO.Directory.GetFiles(sourceFolderPath); //string sourceFolderPath = Folder path that contains the files
foreach(string s in files)
{
object[] paths = new object [2]; // The source path and the destination path
paths[0] = s; // The file source - s = the file name with the file path;
paths[1] = s.Replace(sourceFolderPath, destFolderPath); // Replaces between the source folder and the destination folder but keeps the file name
ThreadPool.QueueUserWorkItem(new waitCallback(CopyFIle), paths);
}

Related

Move Directory Into Current Directory C#

I want to do something simple. Take the contents of folder A and move the files and folders into a folder within folder A. Then make folder A hidden.
The code below is getting an exception of hiddenTarget is not found.
Directory.Create(hiddenTarget) is not helping
There must be a simple way to do this. Currently I am attempting to make a temp directory. Put all files from the current directory in it. Then Move the temp directory to the current directory. Then make the current directory hidden.
Here is the code in issue..
string tempFolder = Path.Combine(Environment.ExpandEnvironmentVariables("%TEMP%"), "tempTarget");
//Directory.CreateDirectory(tempFolder);
Directory.Move(currentTarget, tempFolder);
string hiddenTarget = Path.Combine(currentTarget, #".bak");
//Directory.CreateDirectory(hiddenTarget);
Directory.Move(tempFolder, hiddenTarget);
DirectoryInfo di = new DirectoryInfo(currentTarget);
di.Attributes = FileAttributes.Directory | FileAttributes.Hidden;
So you have two issues here first is that your hidden target cannot start with a '.' because as pointed out in the comments that is illegal in NTFS.
As you can see File Explorer doesn't like this syntax.
As pointed out by #Dour High Arch here is the link to the relevant information: Naming Files, Paths, and Namespaces
Your next issue is that the move is destroying the original directory structure. Therefore, your steps need to be as follows:
1) Move to a temporary directory to avoid any issues with the two processes (file system & your process) fighting for access.
2) Due to the fact that the Directory.Move in step #1 destroyed the original source directory. Recreate the destroyed source folder.
3) Then move into the desired nested folder. This move operation will automatically create the desired sub-directory. Step #2 is required because for whatever reason I'm still looking into Directory.Move cannot automatically create the structure without the source directory already existing.
string currentTarget = #"C:\A";
string hiddenTarget = #"C:\A\Subfolder";
string tempTarget = #"C:\Temp";
Directory.Move(currentTarget, tempTarget);
Directory.CreateDirectory(currentTarget);
Directory.Move(tempTarget, hiddenTarget);
DirectoryInfo di = new DirectoryInfo(currentTarget);
di.Attributes = FileAttributes.Directory | FileAttributes.Hidden;
Update
From an engineering perspective you really should be performing copies if you really care about the data being moved. It may be slower, but will help prevent any horrible things from happening to the data. You should check whether or not these directories exist first before creating them. Exception handling within this example is at a minimum as well. What I'm really trying to stress here is handle the data that is being moved with care!
static void Main(string[] args)
{
string sourceDir = #"C:\Src";
string tempDir = #"C:\Temp";
string destDir = Path.Combine(sourceDir, "Dest");
// Could optionally check to verify that the temp directory already exists here and destroy it if it does.
// Alternatively, pick a really unique name for the temp directory by using a GUID, Thread Id or something of that nature.
// That way you can be sure it does not already exist.
// Copy to temp, then destroy source files.
CopyDirectory(sourceDir, tempDir);
Directory.Delete(sourceDir, true);
// Copy to dest
CopyDirectory(tempDir, destDir);
// Hide the source directory.
DirectoryInfo di = new DirectoryInfo(sourceDir);
di.Attributes = FileAttributes.Directory | FileAttributes.Hidden;
// Clean up the temp directory that way copies of the files aren't sitting around.
// NOTE: Be sure to do this last as if something goes wrong with the move the temp directory will still exist.
Directory.Delete(tempDir, true);
}
/// <summary>
/// Recursively copies all subdirectories.
/// </summary>
/// <param name="sourceDir">The source directory from which to copy.</param>
/// <param name="destDir">The destination directory to copy content to.</param>
static void CopyDirectory(string sourceDir, string destDir)
{
var sourceDirInfo = new DirectoryInfo(sourceDir);
if (!sourceDirInfo.Exists)
{
throw new DirectoryNotFoundException($"Source directory does not exist or could not be found: '{sourceDir}'");
}
// If the destination directory doesn't exist, create it.
if (!Directory.Exists(destDir))
{
Directory.CreateDirectory(destDir);
}
// Get the files in the directory and copy them to the new location.
FileInfo[] files = sourceDirInfo.GetFiles();
foreach (FileInfo file in files)
{
string tempPath = Path.Combine(destDir, file.Name);
file.CopyTo(tempPath, false);
}
// Copy subdirectories
DirectoryInfo[] subDirs = sourceDirInfo.GetDirectories();
foreach (DirectoryInfo subdir in subDirs)
{
string tempPath = Path.Combine(destDir, subdir.Name);
CopyDirectory(subdir.FullName, tempPath);
}
}
Here are a few more links regarding the above.
MSDN - How to: Copy Directories
Stack Overflow - How to copy the entire contents of directory in C#?

jpegoptim on ASP.Net - "error opening temporary file"

I suspect I'm failing to understand where jpegoptim tries to write its temp files.
I have IIS 7.5 running an ASP.Net 4 AppDomain. In it I have a process that optimizes JPEGs with jpegoptim like so:
FileHelper.Copy(existingPath, optimizerPath);
var jpegOptimResult = await ImageHelper.JpegOptim(optimizerPath, 30);
Running locally I get an optimized image. Running on the above server I get:
D:\www\hplusf.com\b\pc\test.jpg 4096x2990 24bit N Adobe [OK] jpegoptim: error opening temporary file.
I can show the code for FileHelper.Copy(), but it's basically just File.Copy() that overwrites if the file already exists.
Here's ImageHelper.JpegOptim:
public static async Task<string> JpegOptim(string path, int quality)
{
string jpegOptimPath = Path.GetDirectoryName(new Uri(Assembly
.GetExecutingAssembly().CodeBase).LocalPath)
+ #"\Lib\jpegoptim.exe";
var jpegOptimResult = await ProcessRunner.O.RunProcess(
jpegOptimPath,
"-m" + quality + " -o -p --strip-all --all-normal \"" + path + "\"",
false, true
);
return jpegOptimResult;
}
jpegOptimResult is what you're seeing there as the error message it's producing. And here's ProcessRunner.RunProcess:
public async Task<string> RunProcess(string command, string args,
bool window, bool captureOutput)
{
var processInfo = new ProcessStartInfo(command, args);
if (!window)
makeWindowless(processInfo);
string output = null;
if (captureOutput)
output = await runAndCapture(processInfo);
else
runDontCapture(processInfo);
return output;
}
protected void makeWindowless(ProcessStartInfo processInfo)
{
processInfo.CreateNoWindow = true;
processInfo.WindowStyle = ProcessWindowStyle.Hidden;
}
protected async Task<string> runAndCapture(ProcessStartInfo processInfo)
{
processInfo.UseShellExecute = false;
processInfo.RedirectStandardOutput = true;
processInfo.RedirectStandardError = true;
var process = Process.Start(processInfo);
var output = process.StandardOutput;
var error = process.StandardError;
while (!process.HasExited)
{
await Task.Delay(100);
}
string s = output.ReadToEnd();
s += '\n' + error.ReadToEnd();
return s;
}
So:
jpegOptim runs properly on my local machine, and optimizes the file, so it's not how I'm calling jpegOptim.
The Copy operation succeeds without Exception, so it's not a Permissions issue with the ASP.Net user reading/writing from that directory
jpegOptim just optimizes and overwrites the file, so if it is in fact running under the same ASP.Net user, it should have no problem writing this file, but...
It's unclear where jpegOptim attempts to write its temp file, so perhaps the underlying issue is where this temporary file is being written.
However, judging by the Windows source:
http://sourceforge.net/p/jpegoptim/code/HEAD/tree/jpegoptim-1.3.0/trunk/jpegoptim.c
jpegOptim's "temporary file" appears to just be the destination file when used with the above options. Relevant lines of jpegOptim source:
int dest = 0;
int main(int argc, char **argv)
{
...
There's some code here looking for the -d argument that sets dest=1 - meaning here dest remains 0. It then hits an if branch, and the else clause, for dest == 0, does this:
if (!splitdir(argv[i],tmpdir,sizeof(tmpdir)))
fatal("splitdir() failed!");
strncpy(newname,argv[i],sizeof(newname));
That's copying the directory name portion of the input image filename to the variable tmpdir - so like C:\Blah\18.jpg would assign tmpdir="C:\Blah\". Then it dumps the entire input image filename to newname, meaning it's just going to overwrite it in place.
At this point in the code the variables it's using should be:
dest=0
argv[i]=D:\www\hplusf.com\b\pc\test.jpg
tmpdir=D:\www\hplusf.com\b\pc\
newname=D:\www\hplusf.com\b\pc\test.jpg
It then in fact opens the file, and there's an opportunity to error out there, suggesting jpegoptim is successfully opening the file. It also decompresses the file further confirming it's successfully opening it.
The specific error message I'm seeing occurs in these lines - I'll confess I don't know if MKSTEMPS is set or not for a default build (which I'm using):
snprintf(tmpfilename,sizeof(tmpfilename),
"%sjpegoptim-%d-%d.XXXXXX.tmp", tmpdir, (int)getuid(), (int)getpid());
#ifdef HAVE_MKSTEMPS
if ((tmpfd = mkstemps(tmpfilename,4)) < 0)
fatal("error creating temp file: mkstemps() failed");
if ((outfile=fdopen(tmpfd,"wb"))==NULL)
#else
tmpfd=0;
if ((outfile=fopen(tmpfilename,"wb"))==NULL)
#endif
fatal("error opening temporary file");
So snprintf is like C# String.Format(), which should produce a path like:
D:\www\hplusf.com\b\pc\jpegoptim-1-2.XXXXXX.tmp
Judging by what I can find it's likely MKSTEMPS is not defined meaning fopen is being called with "wb" meaning it's writing a binary file, and it's returning null meaning it failed to open, and out comes the error message.
So - possible causes:
Bad path in tmpdir It's possible I'm following the C++ poorly (likely), but, from the looks of it it should be identical to the source path of the image. But perhaps it's mangled for tmpdir, by jpegoptim? The input path is clearly clean because jpegoptim actually emits it cleanly in the error message.
Permissions issue Seems fairly unlikely. The ASP.Net user this is running under can clearly read and write because it copies to the dir before jpegoptim fires, and the only user on the machine with any permissions to this dir is that user, so jpegoptim should have failed prior to this point if it were permissions. It could be attempting to access a different dir, but that would really be the Bad tmpdir scenario.
Something else I've not thought of.
Ideas?
Note: This question is similar:
Using jpegtran, jpegoptim, or other jpeg optimization/compression in C#
However, that question is asking about a shared env on GoDaddy, causing answers to spiral around the likelihood he can't spin up processes. We have full control over our server, and as should be clear from the above, the jpegoptim Process is definitely starting successfully, so it's a different scenario.
As it turns out my reading of jpegoptim was incorrect. The tmpdir it uses is where the executable's Working Directory points to, not where the input images are, and not where the executable sits. So, the solution was 2-fold:
Give the exe permissions to write to its own directory* (but deny it access to modify itself)
Modify ProcessRunner to run processes in-place - set the Working Directory to where the exe resides.
The second modification looks like this:
var processInfo = new ProcessStartInfo(command, args);
// Ensure the exe runs in the path where it sits, rather than somewhere
// less safe like the website root
processInfo.WorkingDirectory = (new FileInfo(command)).DirectoryName;
*Note: I happen to have jpegoptim.exe isolated on the server to its own dir to limit risk. If you had it someplace more global like Program Files, you definitely should not do this - instead set the Working Directory as above, but to someplace isolated/safe like a tmp dir or even better a scratch disk. If you've got the RAM for it a RAMdrive would be fastest.
**Second Note: Because of how hard drives and jpegoptim work if the tmp location is not the same disk as the ultimate destination of the output there is a potential, partial race condition introduced between jpegoptim and other code you might be using that depends on its outputs. In particular if you use the same disk, when jpegoptim is done the output JPEG is complete - the OS changes the entry in its file table but the data for the image on the hard drive has already been written to completion. When tmp and destination are separate disks, jpegoptim finishes by telling the OS to move from the tmpdir to the output dir. That's a data move that finishes sometime after jpegoptim is done running. If your waiting code is fast enough it will start its work with an incomplete JPEG.

Reset Application setting in wp7

I have created an app that initially creates a database and saves some data in it.
Now I want to delete this database and its files when the user clicks on the reset button but I am getting an error – 'this is use in another process'. I want it to delete and recreate the database when click on the reset button. Any ideas?
The most frequent cause of this is ude to the thread unsafe nature of interacting with isolated storage on Windows Phone. Regardless of how you're implementing the database (be it in a file, or series of files), you're interacting with the isolated storage on some level.
I highly encourage you to read, and make sure you understand this overview of isolated storage before going too far.
You're remark:
This is in use in another process
makes me think you're using a third party library to do your database stuff. This exception/error is being thrown when the library itsself is unable to access isolated storage. Without knowing exactly how you're implementing the database, it's hard to be exactly speak to your situation.
You never "recreate IsolatedStorage", Isolated Storage is a term used to define the collection of disk space your application has access to. Much like a folder, this disk space has a root, and contains only files that you create.
In order to avoid thread exceptions when accessing Isolated Storage, make sure you use the using keyword in C# like so:
namespace IsolatedStorageExample
{
public class ISOAccess
{
// This example method will read a file inside your Isolated Storage.
public static String ReadFile(string filename)
{
string fileContents = "";
// Ideally, you should enclose this entire next section in a try/catch block since
// if there is anything wrong with below, it will crash your app.
//
// This line returns the "handle" to your Isolated Storage. The phone considers the
// entire isolated storage folder as a single "file", which is why it can be a
// little bit of a confusing name.
using(IsolatedStorageFile file = IsolatedStorageFile.GetUserStoreForAppliaction())
{
// If the file does not exist, return an empty string
if(file.Exists(filename))
{
// Obtain a stream to the file
using(IsolatedStorageFileStream stream = File.OpenFile(filename, FileMode.Open)
{
// Open a stream reader to actually read the file.
using(StreamReader reader = new StreamReader(stream))
{
fileContents = reader.ReadToEnd();
}
}
}
}
return fileContents;
}
}
}
That should help with your problem of thread safety. To be more specifically helpful toward what you want to do, take a look at the following methods (you can add this to the above class):
// BE VERY CAREFUL, running this method will delete *all* the files in isolated storage... ALL OF THEM
public static void ClearAllIsolatedStorage()
{
// get the handle to isolated storage
using(IsolatedStorageFile file = IsolatedStorageFile.GetUserStoreForApplication())
{
// Get a list of all the folders in the root directory
Queue<String> rootFolders = new Queue<String>(file.GetDirectoryNames());
// For each folder...
while(0 != rootFolders.Count)
{
string folderName = rootFolders.Dequeue();
// First, recursively delete all the files and folders inside the given folder.
// This is required, because you cannot delete a non-empty directory
DeleteFilesInFolderRecursively(file, folderName);
// Now that all of it's contents have been deleted, you can delete the directory
// itsself.
file.DeleteDirectory(rootFolders.Dequeue());
}
// And now we delete all the files in the root directory
Queue<String> rootFiles = new Queue<String>(file.GetFileNames());
while(0 != rootFiles.Count)
file.DeleteFile(rootFiles.Dequeue());
}
}
private static void DeleteFilesInFolderRecursively(IsolatedStorageFile iso, string directory)
{
// get the folders that are inside this folder
Queue<string> enclosedDirectories = new Queue<string>(iso.GetDirectoryNames(directory));
// loop through all the folders inside this folder, and recurse on all of them
while(0 != enclosedDirectories.Count)
{
string nextFolderPath = Path.Combine(directory, enclosedDirectories.Dequeue());
DeleteFilesInFolderRecursively(nextFolderPath);
}
// This string will allow you to see all the files in this folder.
string fileSearch = Path.Combine(directory, "*");
// Getting the files in this folder
Queue<string> filesInDirectory = iso.GetFileNames(fileSearch);
// Finally, deleting all the files in this folder
while(0 != filesInDirectory.Count)
{
iso.DeleteFile(filesInDirectory.Dequeue());
}
}
Another thing I highly recommend is implementing the class that accesses IsolatedStorage using a "Multithreaded Singleton Pattern" as described here.
Hope that's helpful. Code is provided "as-is", I have not compiled it, but the general concepts are all there, so if there's something amiss, read the MSDN docs to see where I goofed. But I assure you, most of this is copied from functional code of mine, so it should work properly with very little fanagaling.

Unlock file after program executing

When i started my c# program i can not delete executing file.
How can i just unlock my assembly after executing and give to user possibility to delete my file? Maybe copy assembly to other place and then execute it? But i think it's not better way.
I think you'll have to copy your assembly to a temporary location and relaunch it. Something like this (haven't tested it):
public class Program {
static void Main() {
if (Array.IndexOf(Environment.GetCommandLineArgs(), "/unlocked") == -1) {
// We have not been launched by ourself! Copy the assembly to a temp location
// so the user can delete our original location.
string temp_location = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), Path.GetFileName(Application.ExecutablePath));
File.Copy(Application.ExecutablePath, temp_location);
Process proc = new Process();
proc.StartInfo.FileName = temp_location;
proc.StartInfo.Arguments = "/unlocked";
proc.Start();
// no more work in main, so application closes...
} else {
// else initialize application as normal...
}
}
}
You cannot delete your file because the file itself is the backing store for the virtual memory pages that hold (in memory) the program code.
When a page of your code is paged out from memory by the OS, it does not go to the pagefile, but it is simply discarded and (when needed) loaded back from the file.
I suppose you could force the program to be pagefile backed (iirc, programs executed from a removable media like a cdrom are pagefile backed), but probably the best solution is just to copy the assembly to another location, as you suggested.

Access a file created by a separate thread

I have a problem aith multithreading when copying and accessing files.
I have a service, that downloads and unpacks a Zip archive, then it copies a file from unzipped folder to the right location:
//Download, and uzip archive...
//Copy a needed file to its right location
File.Copy(fileName, fileDestination);
Then I start a separate thread, that needs to access the copied files:
TheadPool.QueueUserWorkItem(s => processCopiedFile(fileDestination));
Here's the code fragment from ProcessCopiedFile:
private void ProcessCopiedFile(string filePath)
{
...
//Load the file, previously copied here
var xml = XDocument.Load(filePath);
...
//Do other work...
}
The XDoument.Load call fails with exception:
The process cannot access the file <FileName> because it is used by another process.
Seems like File.Copy keeps the result file locked. When do all work synchronuously, it works without errors.
Have you any thoughts?
Thx.
File.Copy does not keep anything open or locked, it is an atomic operation which requires some time, depending of course on Disk/Network I/O and file size.
Of course while moving from sync to async you should make sure you do not access the destination file while the copy is still in progress.
Copy the file with a stream to avoid windows lock from File.Copy
using(var s = new MemoryStream(File.ReadAllBytes(filePath))
{
using(var fs = new FileStream(newLocation, FileMode.Create))
{
s.WriteTo(fs);
}
}

Categories

Resources