I am writting a client-server laucher application.
An administrator will, from the server side, select executable files ('.exes') from a list and add these to a short-list of the apps that standard users can run on the client.
To complile this list, my client app will reculsively search though all the folders in the system for exes and send this list over to the server via wcf.
To save search time, and keep the list short, I would like to avoid searching through folders that are not LIKELY to contain '.exes' that human users are intended to run directly.
Examples (i think) :
%windir%\WinSxS - Windows Side-by-Side - used to store versions of Windows components that are built to reduce configuration problems with Dynamic Link Libraries.
%windir%\installer - used to store installation information for installed programs
C:\MSOCache - MS Office local install source
Most hidden folders
What other folders should I avoid searching through and what are they likely to contain?
I am interested in WinXP/WinVista/Win7.
EDIT:
Search time is not the most important factor.
It is very important not to exclude exes that the user may need to run AND to exclude exes like:
c:\Windows\winsxs\x86_microsoft-windows-x..rtificateenrollment_31bf3856ad364e35_6.1.7600.20520_none_f43289dd08ebec20\CertEnrollCtrl.exe
that were never meant to be directly launched by the user.
Since any heuristic approach dealing in "likely" will need the ability to add further items to catch cases deemed "unlikely" but which were actually important, you can't be "wrong" as such, just not "perfect".
With this in mind, I would take the opposite approach, and concentrate on those that are likely to have executables.
Recurse through the directories contained in the directory %ProgramFiles%
points to.
Look in all other directories mentioned by the %Path% system variable (semicolon-delimited) but do not recurse it.
Look for shortcuts on desktops, start menus and quicklaunch folders.
As a rule, one would expect the former to find executables used by people selecting executables from the start menu and icons, and for the second to find executables used from the command-line and by the system. Between the two you'll find the large majority of executables on a system with relatively little wasted directory examinations.
How often are you going to run this program? It doesn't seem like it would take very long at all to just scan the whole drive for all the executables. I have a one-gigabyte drive on my development machine and "dir /b/s *.exe" only takes a minute or two. And it's very likely that my development machine has a whole lot more files on it than the typical user's computer.
Come to think of it, your client program could execute that command, capture the output, and send it to the server. Perhaps after a little pre-processing.
My point is that it shouldn't matter if the process were to take a full five minutes, if it's done only once (or very rarely) on each machine. The benefit is that you won't miss any executables that way.
Related
Please help me to understand which one is better for logging performancewise.
logging in sql vs files vs aws which is faster in c# Applications
If I understand it correctly, you want to log useful information from your application (C#) somewhere to be able to refer to it (presumably when something goes wrong or to extract information for analytics).
Rule of thumb, in interprocess communications, maximum time is spent on sending data over network. If you apply this knowledge, you will be able to order your choices (and other options) from performance point of view.
As an indication order in terms of performance for few cases will be
Log file on the same drive as your program and being written from within the same process
Log file on a mounted drive on the same machine that runs your program and being written from within the same process
Log written in a database that resides on the same machine (localhost) as program
Log written in a database that resides on a different machine but in a local network
Log written on AWS which obviously will not be within your local network.
...
This said there are other considerations as well. For example a DB in local high bandwidth network on a powerful machine may write faster than a low configuration machine (e.g. ordinary laptop) having DB and program. Similarly, use of Direct Connect or fibre line between AWS and local network boosts the performance many many folds.
Thus, the answer is not straight forward, lot many factors contribute to change the order. Safest bet is to use log files on the same machine. You can always run a separate process to read asynchronously from the file and write wherever you wish.
It is entirely possible I'm going about this entirely the wrong way, but here is what I'm doing:
I have a device that is communicating with a DLL over a COM port. This works fine for a single program, however I need multiple programs to be running, each updating with the state of the device.
Since I can't work out how to share access to a COM port, my solution is that each DLL checks for the existance of a timestamped file. If the file is there, the DLL goes into 'slave' mode and just reads the state of the device from the file. If the file doesn't exist or is over 30ms old, the DLL appoints itself 'master', claims the COM port, and writes the file itself.
(The device can also be sent instructions, so the master will have to handle collecting and sending the slaves' requests somehow, but I can deal with that later. Also I might want the DLLs to be aware of each other, so if one is causing problems then the user can be told about it - again, will get to this later.)
My immediate problem now is, where to store this file/possible collection of files? It must:
be somewhere that doesn't force the programs using the DLLs to have admin privileges or anything else that means they can't "just run".
be always the same place, not somewhere that might change based on some outside factor. Googling has shown me things like, "Environment.SpecialFolder.CommonDocuments" (in C#) but if that location is sometimes C:\ProgramData and other times C:\Users\Public then we'll get two parallel masters, only one of which can claim the COM.
work on as many Windowses as possible (all the way back to XP if I can get away with it) because no one wants to maintain more versions than necessary.
preferably, be somewhere non-technical users won't see it. Quite apart from confusing/scaring them, it just looks unprofessional, doesn't it?
Two sidenotes:
I looked at using the Registry and learned there's a time cost to reading it. The DLLs need to be reading with a period of maybe 10ms, so I assume the Registry is a bad idea.
After we get this working on Windows, we need to turn around and address Android, OSX, Linux, Mac, etc, etc, so I'm trying to bear that in mind when deciding how we structure everything.
EDIT:
I should add for context that we're releasing the DLL and letting other devs create apps that use it. We want them to work as frictionlessly as possible, without for example requiring the user install anything first, or the dev needing to jump through a bunch of hoops to be sure it will work.
I have to design a backup algorithm for some files used by a Windows Service and I already have some ideas, but I would like to hear the opinion of the wiser ones, in order to try and improve what I have in mind.
The software that I am dealing with follows a client-server architecture.
On the server side, we have a Windows Service that performs some tasks such as monitoring folders, etc, and it has several xml configuration files (around 10). These are the files that I want to backup.
On the client side, the user has a graphical interface that allows him to modify these configuration files, although this shouldn't happen very often. Communication with the server are made using WCF.
So the config files might be modified remotely by the user, but the administrator might also modify them manually on the server (the windows service monitors these changes).
And for the moment, this is what I have in mind for the backup algorithm (quite simple though):
When - backups will be performed in two situations:
Periodically: a parallel thread on the server application will perform a copy of the configuration files every XXXX months/weeks/whatever (configurable parameter). This is, it does not perform the backup each time the files are modified by user action, but only when the client app is launched.
Every time the user launches the client: every time the server detects that a user has launched the application, the server side will perform a backup.
How:
There will be a folder named Backup on the Program Data folder of the Windows Service. There, each time a backup is performed, a sub-folder named BackupYYYYMMDDHHmm will be created, containing all the concerned files.
Maintenance: Backup folders won't be kept forever. Periodically, all of those older than XXXX weeks/months/year (configurable parameter) will be deleted. Alternatively, I might only maintain N backup sub-folders (configurable parameter). I still haven't chosen an option, but I think I'll go for the first one.
So, this is it. Comments are very welcome. Thanks!!
I think your design is viable. just a few comments:
do you need to back up to a separate place other than the server? I don't feel it's safe to back up important data on same server, and I would rather back them up to a separate disk (perhaps a network location)
you need to implement the monitoring/backup/retention/etc. by yourself, and it sounds complicated - how long do you wish to spend on this?
Personally i would use some simple trick to achieve the backup, for example, since the data are plain text files (xml format) and light, I might simply back them up to some source control system: make the folder a checkout of SVN (or some other means) and create a simple script that detects/checks in changes to SVN, and schedule the script to be executed once a few hours (or more often up to your needs, or can be triggered by your service/app on demand) - this way it eliminates the unnecessary copy of data (as it checks in changes only), and it's much more trackable as svn provides all the history.
hope above can help a bit...
As far as you know, are there any problems in running a C# application from a shared exe file? This is a request from a customer asking their 20 clients to run the same exe file on shared path.
First tests didn't show problems, but don't know on long terms. I personally don't like this, don't think that framework was developed with this in mind, but they do for a quick upgrade of the exe file when needed.
Any point to discourage this?
Thanks
Sav
The first consideration is deployment concerns. Prior to .NET 3.5 SP1, this was not allowed by default because the shipped security policy treated network locations in a less trusted way. .NET 3.5 SP1 and later, this is no longer the case. You could, of course, use caspol to modify this security policy to allow this, if you are working with versions of the framework prior to that. Additionally, some more recent versions of Windows may have additional security policies outside of .NET that can prevent execution from remote locations.
The second consideration is making sure the application is designed in a way that it is aware of its environment, not assuming the environment is relative to the local machine when it is expected to be so (which could affect resolution of external resources and, depending on the situation, could result in resource contention or users overwriting each other's data).
The third is availability. What if the server hosting that executable becomes unavailable (is powered off by accident, crashes, experiences networking issues, is renamed, etc.)? Is that acceptable? How large is the executable? If it is large, that can increase network traffic and at any rate result in the executable being slow to start as it is invoked over the network.
I suppose for trivial applications, these issues may be negligible. However, there are lots of ways of installing applications on client computers in a way that they are installed and updated quickly and easily, such as ClickOnce deployment.
We currently run software designed in house. This runs off a central SQL database. Each computer is set up with a batch program which runs through Windows Start Up and downloads the current program files from the central server. The .exe is therefore run off the individuals computer and not off the server. This has been found, in our case at least, to be the most efficient method.
For the game Minecraft, the general approach when running the server application is to run it in a RAMDisk, as it is uses hundreds of tiny files for world generation, and the I/O speeds are the major bottleneck.
In a recent attempt, I tried to use Dokan/ImDisk to create a RAMDisk programatically for the Server Application. Dokan was considerably slower than the average Hard-Drive, and I was unable to get ImDisk to function properly. Since these are the only 2 Filesystem Drivers I know of that have a .NET API, i'm looking into alternatives now.
It was mentioned to me previously to try Memory-Mapped Files. My approach currently is to Create RAMDisk, Create Symbolic Link between Data Folder for Game Server and the RAMDisk, then launch the Game Server process.
Can Memory-Mapped Files function the same way, I.E. creating a virtual drive which I can create a symbolic link to? Such as G:\Data_Files\?
Are there any other alternatives to Dokan/ImDisk with a .NET API/Bindings floating around?
After looking at a bunch of solutions and doing a few benchmarks, we couldn't pass up RAMDisk from DataRam. We kicked around a bunch of the Windows driver stuff and some other freebie solutions and ultimately couldn't justify the expense compared to the tiny price tag of a commercial solution.
There are several approaches that depend on specifics of your task.
If you need to work with file system (i.e. via filesystem API functions and classes), and you want it fast, then (as I suggested in reply to your previous question) you'd need to create a RAMDisk driver. Windows Driver Kit includes a sample driver, which (coincidence?) has the name "RamDisk". Driver development, though, is tricky, and if something goes wrong with the sample or you need to extend it, you would need to dig deep into kernel-mode development (or hire someone to do the job). Why kernel mode? Cause as you could see with Dokan, switching back to user mode to store the data causes major slowdown.
If all you need is a handy management of bunch of files in memory using Stream class (with possibility to flush all of this to the disk), then you can make use of one of virtual file systems. Our SolFS (Application Edition) is one of such products that you can use (I can also remember CodeBase File System, but they don't seem to provide an evaluation version). SolFS seems to fit your task nicely so if you think so too, you can contact me privately (see my profile) for assistance.
To answer your questions:
No, memory-mapped files (MMF) are literally files on the disk (including a virtual disk if you have one), which can be accessed not via filesystem API but directly using in-memory operations. MMFs tend to be faster for most file operations, that's why they are frequently mentioned.
Our Callback File System or CallbackDisk products (see virtual storage line) are an alternative, however, as I mentioned in the first paragraph, they won't solve your problem due to user-mode context switch.
Update:
I see no obstacles for the driver to have a copy in memory and perform writes to disk asynchronously when needed. But this will require modifying sample RAMDisk driver (and this involves quite a lot of kernel-mode programming).
With SolFS or other virtual file system you can have a copy of the storage on the disk as well. In case of virtual file system it might appear that working with container file on the disk will give you satisfactory results (as virtual file system usually has a memory cache) and you won't need to keep in-memory copy at all.