I have massive directories, and I would like to read all the files as fast as I can. I mean, not DirectoryInfo.GetFiles fast, but 'get-clusters-from-disk-low-level' fast.
Of course, .NET 2.0, c#
Similar question was here, but this approach wasn't any good:
C# Directory listing massive directory
Someone suggested pInvoke on FindFirst/FindNext. Anybody tried that and is able to share results?
For a "normal" approach, basically everything boils down to FindFirstFile/FindNextFile, you don't really get much faster than that... and that isn't super-turbo-fast.
If you really need speed, look into reading the MFT manually - but know that this requires admin privileges, and is prone to break whenever NTFS gets updated (and, oh yeah, won't work for non-NTFS filesystems). You might want to have a look at this code which has USN and MFT stuff.
However, perhaps there's a different solution. If your app is running constantly and needs to pick up changes, you can start off by doing one slow FindFirstFile/FindNextFile pass, and then use directory change notification support to be informed of updates... that works for limited users, and doesn't depend on filesystem structures.
For the best performance, it is possible to P/Invoke NtQueryDirectoryFile, documented as ZwQueryDirectoryFile.
(That short of accessing the disk directly and reading the raw file system structures directly, which usually is not practical.)
Try using something like this DirectoryManager and refine it by your needs. Works faster than the .NET Framework GetDirectories() or GetFiles() because we ommitted there cross-platform checkings and adaptations.
Related
I just found out that even though I obfuscate my application it can still be cracked using code injection. This pretty much makes C# completely useless for me. Is there really no way to protect or make the process very difficult to do? Do 'crackers' have to know any class names etc. to do this? Thanks!
UPDATE: I'm trying to protect a binary (exe) not code.
I've read somewhere on here from several users that whatever the code is written in, it can be decompiled. Think of it like this, how else would the system know how to run the code? It is the same concept.
Usually though, most developers(that I know of) don't worry so much about this. Of course, this is a good reason not to have any sensitive details within your code but rather somewhere else.
If you're worried about protected industry secrets, it's impossible to send a computer the right instructions, but expect those instructions cannot be read by a capable person. If your goal is to stop somebody from modifying your EXE, why not simply sign your executable?
http://blogs.msdn.com/b/shawnfa/archive/2007/01/10/combining-strong-names-with-authenticode.aspx
Any of the .NET languages, and Java can be de-compiled. This includes .Exe and .dll files. What you will need to do is get an tool that obfuscates the code.
I've listed one below for you so you can get an idea:
http://www.red-gate.com/products/dotnet-development/smartassembly/
Are you sure your code is worth the trouble for someone to do that? Very very few applications are.
I want to run a thread that checks the memory image of the current executable, for protection reasons. Any ideas how to do CRC on the current memory executable (WinAPI or .NET way)? My app is written in .NET.
Signing your assemblies will give you as good verification as you can get with relation to verify CRC of .Net assembly (see Rodrigo's answer).
If you are worried that someone will patch assembly at runtime you probably worried too much. It requires better understanding of runtime to in memory patch IL for a method that is already JIT'ed compared to simply disassembling your .Net code and fixing it up (including removal of your CRC checks).
If you doing it more for fun than you shoud be able to find base address where assembly is loaded and compute CRC of some sort... or see if pages are marked as modified...
I think that's going to be quite difficult in .NET. When an executable is loaded, it can potentially be split up and loaded into several different regions in memory. You'll need to acquaint yourself with the Window's Executable format:
http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx
as well as the Windows executable loading process.
You'll might also want to concern yourself with depenency dlls as well. You'll be making so many native calls, that you might want to consider doing this in C.
Not much of an answer, I'm afraid.
Any runtime check you do will have the following drawbacks:
False positives. Because this is .NET, you cannot assume the runtime doesn't modify your in-memory code. You may detect a hack where there is none.
Any run-time check you make will be no more secure than the code you are trying to protect. This includes any runtime mechanism you create in your app such as periodic CRC checks, sentinel processes, or even checking with a server where the request can be faked.
You will decrease performance in your legitimate application, where the pirated version will run better without all these checks
You will do nothing to solve patching your EXE.
I understand that you are just trying to make it as hard as possible, even though it's not 100% uncrackable. But the solutions you propose (and likely any solution you can implement yourself) will do extremely little to thwart any average cracker.
Because this is such a demanded feature though, I would look for 3rd party solutions where they have put forth the effort for a sophisticated solution which can be updated as cracking techniques evolve. I cannot recommend any personally though.
I am not aware of a way to do this in .NET.
If you are interested in protecting you executables, you can generate a new key with sn and add it to AssemblyInfo.cs, so that if the application is modified at least it will not run.
Jon Skeet's Miscellaneous Utility Library contains a method to compute the Adler32 checksum on a stream. Its usage would be:
MiscUtil.Checksum.Adler32.ComputeChecksum(stream);
As for creating a memorystream out of the assembly that is currently running... I don't know if that is even possible (or advisable).
I have several projects that require me to monitor files, and then edit them as they are getting written to disk. I have a feeling that what I am looking for is operationally the same as how anti-virus tools operate. Let me give more details:
1) I need to trap all files saved by Office application, and then add specific company tags to the headers/footers of each document as they are getting written to disk.
2) I need to know immediately when an editable file (of pretty much any type) is written to disk, so that I can undertake some scanning operations to check if files content meets certain company policies.
In short, you can see that I need to process any user files as they are being written to disk.
Here is my problem. I want to use C# for this task, but I am not sure if it has the ability to meet my requirements. Everything I have seen on the net is geared towards lower-level C programming, which I specifically want to avoid due to time constraints for this project. Anyone aware of how to easily do this task in C#? Is it even feasible (ie too high-level a language, too slow a language etc.)?
Performance won't be the issue. I guess I'd question the entire process- it sounds like a recipe for disaster. You can easily hack something together in C# using a FileSystemWatcher in a matter of minutes, but it will be fraught with issues. AV software is bad enough about locking files and screwing up various software, and it's not even trying to modify the file. How do you know when the other app is "done" writing the file? What do you do when you've got the file locked and something else breaks because it can't get access?
Have you looked at the FileSystemWatcher?
C# can easily do this. Look at the FileSystemWatcher class (http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx).
Is there a SDK that can be used in managed code to shred files securely?
EDIT: This is the only link i could find in google that helps me
EDIT: Either SDK or some kind of COM based component.
This code from codeproject may be a good starting point.
Eraser has been around for years, you could call out to it by using System.Diagnostics.Process, or at least review the algorithm there.
Take a look at Windows.WinAny.Helper at the CodePlex. It has SecureDelete extension which allows you to shredd files with different algorithms like Gutmann, DoD-7, DoD-3, Random or Quick.
Technology has changed in the past few years so when I happened to see this answer (why wasn't an answer accepted again?) I wanted to provide an update for others with similar questions.
Please note that shredding is very much filesystem and media dependent. Attempting to "shred" a file on a log based filesystem or a filesystem stored on smart (write leveling) flash isn't going to get you very far. You would have to, at a minimum, write enough data to complete fill the device to hope that the old data might be overwritten one time.
More likely you would have to write several smaller files and when you get FS full, delete one and then keep writing a new one, to ensure that all reserved space has been overwritten as well. Then you will probably be fairly safe. Probably.
I say probably because the storage media/FS could decide that a block was failing (or used too much relatively) and map it away substituting some other part of the disk instead. This is a per-block thing of course, so any much larger file is unlikely to be reconstructed.
i am creating my own CMS frame work, because many of the clients i have, the have same requirements, like news module, newsletter module, etc.
now i am doing it fine, the only thing that is bothering me, is if a client wants to move from my server he would ask me to gibe him his files, and of course if i do so the new person who will take it he will see all my code, use it and benefit from i, and this is so bad for me that i spend all this time on creating my system and any one can easily see the code, plus he will see all the logic for my system, and he can easily know how other clients of mine sites are working, and that is a threat to me, finally i am using third party controls that i have paid for their license, and i don't want him to take it on a golden plate.
now what is the best way to solve this ? i thought it is encrypting, but how can i do that and how efficient is it ?
-should i merge all my CS files and Dlls in bin folder to one Dll and encrypt it, and how can i do that ?
i totally appreciate all the help on this matter as it is really crucial for me.
you should read this
Best .NET obfuscation tools/strategy
How effective is obfuscation?
In my experience, this is rarely worth the effort. Lots of companies who provide libraries like this don't bother obfuscating their code (Telerik, etc).
Especially considering what you are writing (CMSes are everywhere), you'd likely see more benefit from your time spent implementing features that put your product/implementation in a competitive advantage and make companies see that the software you are capable of writing has value, rather than the code itself.
In the end, you want to ensure you are a key factor in making software work for a company, not the DLLs you give them.
You'll need to precompile your site and obfuscate dlls.
Visual Studio has something like Dotfuscator Community Edition shipped with it. You could give it a try.
Of course, HTML output, CSS declarations, database structure and stored procedures code cannot be encrypted.
You can however try to compress CSS which will also reduce its readbility by humans.
Check here: The best approach to scramble CSS definitions to a human-unreadable state throughout an ASP.NET application
One other idea would be to use a frame in your HTML and put the most of the site pages inside of it. This way, it will not be visible when doing "View source".
Or just state it clearly that you offer whatever you're doing as a service and do not provide source codes of your work. I somehow doubt salesforce would be willing to give their sources to anyone who asks.