I have an application that stores an XML Encrypted Document, each time the application wants to access the XML file it have to (decrypt-->read-->write-->encrypt). if a sudden shutdown or restart even a end task from task manager will result on either corrupted data in the XML or a un-encrypted XML file, so in the next run it will produce an error.
What i though about is to store the whole content of the XML in a the database and check if there are any errors then replace the old content.
Any ideas, tips, or thoughts much appreciated.
You should do the decryption/encryption in memory. Never, ever store unencrypted data on disk. Load the encrypted data, decrypt it in memory. When saving, first encrypt in memory, then store.
Replacing the entire content of a file in a way that is safe even in the event of sudden shutdowns is hard. A workaround is to create a new file under a temporary name and when it is written completely to disk, delete the old file and rename the new one. There are other implications of this though, such as security rights specific to the file being lost.
A better, but more advanced option, is to move everything into a database with transaction support.
Related
When reading and following log files on a system where logrotate is installed, it happens at certain times that the existing log file is renamed and a new file with the same name is created. The application will then write new log entries into the new file. When I'm still reading the old file and waiting for new data to be appended there, I'll have to know when the file with that name was replaced so that I can stop reading at the end of the file and restart reading the new file.
My log reader is written in C# (.NET Core 3.1) and will run on Linux. I can use native functions through the Mono.Posix.NETStandard package. But I'm not sure how to do that properly.
Should I fetch the inode number from the file name before I start reading? Or should I compare on other data like the size or time? What's the most robust approach to detecting when the file I've currently opened for reading is replaced?
The solution should ideally still work if the log file was replaced exactly at or around the moment when I opened the file for reading, and also if the old file was very small and the new file will be big from the start (because suddenly a lot happens). I couldn't find any information about this topic at all.
I've got a program that will be running over a long duration (hours), and regularly writing output to a text file.
I'm looking to use a TextWriter implementation to write to the file, and I am concerned that keeping the file locked open during the entire length of operations may be problematic.
First question: Will there be performance problems (or other kinds) for keeping a stream open to a file for an extended duration?
If so, will a StreamWriter (opened with File Name constructor) manage opening and closing the file on a regular buffered basis for me, or will it hold the file open for the duration of its existence?
Lastly, is there a built in option for handling these more long-duration writes? Or will I need a custom Writer/Stream implementation?
Personally I would use one of the File.Appendxxx routines, which open the file, append the data and then close it again.
If I'm writing at such a rate that the cost of all this opening and closing is too high, then I add some kind of memory-based queue and flush it periodically.
If you're doing text-based logging, you might look at one of the umpteen logging frameworks for .NET, which can do this sort of stuff for you, along with things like rotating filenames, etc.
StreamWriter / FileStream, etc, will generally hold the file open until you dispose them.
Having a file opened indefinitely for it to be written to is not advised because this can cause issues when you need to back-up the file, edit the file, read the file, or even if the system crashes while it is in the process of writing to the file inevitably corrupting the data.
Will's answer does provide a solution of opening and appending the file when needed. The point here is if the file is not being written to 24/7 then there is no reason why you shouldn't close it.
To directly answer your question in the sense of the file needing to be open 24/7. I would use asynchronous methods to write to the file using tasks. This way you can then invoke another task to write a back-up of the file, say everyday. The back-up file will allow you to view the information of the production file.
In my C# windows application (using visual C# 2012 if you need to know) I need to add things to listboxes and save the items so when the program closes they stay there next time it opens. I would prefer not using the settings to store the information.
Some more things I need for this. Saving them to a text file can't happen either. It needs to be saved in a way that it can not be edited outside of the program.
Sorry but its not possible. Anything that you save can be modified by another program.
You could sign the saved data and then detect that some other has modified the data, but you cannot prevent other programs from read/changing the data.
EDIT
DPAPI can be used to encrypt/decrypt the data. See Really simple encryption with C# and SymmetricAlgorithm
Set up SQL on a web server.
Save the info to your web server.
They can't modify data on a remote server, so I think a database is the way to go.
If you are the only person using that program... I mean... if you are the only one that knows what the program is doing, you could go for saving data in binary format into binary files. I doubt that someone looking at your files will say "Oh damn there is a .bin file in this directory! I have to find a way to discover how to read it and what's it's content!".
As long as you keep your program and its sources protected... I doubt someone will get something out of that content. You transform all your data into a Byte[], you compress it, you encrypt it and you write in a file. Bye bye.
Is it possible to create a file on a disk which is full??
Does creation of the file take any space??
Basically I am seeing a case where C# has created but failed to write anything whhich I think points to a full disk.
Does anyone know whether creating a file on a full disk will fail or not??
This wa done using c# o Windw xSerevr- The log file was also written to the same drive
Creating (empty) files should still be possible in most cases. The MFT is a separate part of the volume which won't get used for file data.
It should even be possible to store small amounts of data without needing more than the file entry in the MFT. NTFS can store streams as "resident data" in the stream descriptor which doesn't need any additional space, but only works for very small files.
I think your issue is another problem, though. It may be that you have permissions to create a file but not to write anything to it. You might want to check the ACLs of the location where you're trying to write.
I have a database that is locking mdb's and such that I'd like to backup. However the tool (I have the source) I am using opens the file before backing it up and finds that it is locked.
Is there a way I can open it for read-only purposes?
For reference the tool uses C# and .NET 2.0 (but can be updated to 3.5).
The reason your tool locks the file is to prevent changes to the file as it is being backed up. For example, if you begun your backup, but halfway in the DBMS (i.e. SQL Server) decided to make a change to a file, then your backup would be corrupt.
I recommend you use the tools that are provided with your database solutions to perform a backup. The other option is to stop the database before backing it up.
If the DBMS is holding a write lock on the file, and you read it, you're risking the DBMS writing the file as you're reading it. Depending on what part was written, you could end up with a corrupt backup of the file. You're best off reading the file only if the DBMS isn't writing to the file or letting the DBMS handle its own backups.
This is similar to this question:
Opening a file's Shadow Copy if the current copy is in use
It depends on how the database is opening the MDB file. If it's not allowing read sharing then you're out of luck unless you are able to open the shadow copy. There's a discussion on how to do this here:
How do I copy a file or folder that is locked under windows programmatically?