So the basic question is in encrypting files in resource constrained devices.
I have used a rather dangerous approach to use two FileStreams, where
FileStream 1 is reading from the file and copying it to a byte array
The contents of the byte array is encrypted.
FileStream 2, writes the bytes back to the same file.
This works fine but has a great chance of messing up a file if the encryption stops halfway etc.
So the normal approach is to write to a temp file, and then move it to the original location and replacing the original file.
However the problem is in mobile phones where resources (especially storage) are very limited, creating lets say another 200MB or 300MB file maybe impossible.
So what approaches are there to handle this problem in Mobile Devies? Do I have to gamble between space and messing up the file?
One way to make the process a little safer, could be to:
FileStream 1 is reading from the file and copying it to a byte array
The bytes you read, is written to a small "scratch" file the same size as your buffer, along with position of last block succesfully read.
The contents of the byte array is encrypted.
FileStream 2, writes the bytes back to the same file.
If the process is interrupted, check in the scratch file to see where your last position was. Then you can re-start the process from there, and still be able to encrypt the whole file. (And if you wanted to get the original file back, you would encrypt the remaining blocks, then decrypt it).
Of course, this process only works if you are using an encryption algorithm, that relies on the result of the preceding blocks when encrypting the current block. Depending on your choice of algorithm, you might need to store a little bit more.
First of all, you can always check if there is enough space to write your array to a tmp file.
Next, the problem you ask is not a real problem since if you're encrypting, you have read the complete file to array. Once encryption is finished, you can be sure that the byte array is encrypted. If this was not the case, the function would throw an exception. So, in step 3, when you write to file, you can overwrite it.
edit
I now realize that you encrypt and write to file partially since otherwise it wouldn't fit into ram. Is that correct?
Do I have to gamble between space and messing up the file?
Basically, Yes.
If space-constraints force you to convert (encrypt) in-place, there is no rollback option.
The next problem is Size. If your conversion (can) increase the size of the data, you have very limited room to maneuver. If ResultSize > (InputSize + Buffer) Then you're not going to succeed.
In the case of encryption, you can use a CompressStream in front of the CryptoStream, but you won't be able to predict if it's going to work.
In short, on a Mobile device you have reached a limit. You will have to mandate an extra Memory device.
Related
I'm trying to work with real time media input, and for that am trying to understand how can methods that record to streams (such as StartRecordToStreamAsync) work, since they don't know the length of the input at the start, while file types like WAV need that information before the actual data.
To be clear, I can understand how the methods that save to file work - they, presumably, just fill in that information later. But when feeding a stream - how do they solve that?
Only they know exactly how they solve that, but a reasonable assumption is that they write the audio data to the output stream leaving the size blank, and when the recording is done, they seek to the beginning of the file, and update the size.
If you read the documentation carefully you will note that it says:
Starts recording to a random-access stream
And that's almost certainly because they need to be able to seek to the start to update the header.
For example, I recorded a video using my camera and saved it as my_vacation.mp4 which size is 50MB. I opened the video file and an encrypted file called secret_message.dat using Visual Studio, by using File.ReadAllBytes() in C#, concatenated both arrays of bytes, and then saved it as my_vacation_2.mp4.
The program I created for testing purpose is able to save the byte index where the hidden file begin and I want to use it as key to extract that hidden file later.
Now I can play that video file normally, without any error. Total file size is 65MB. Suppose no one could access the original file, of course no one would know that the last 15MB part of that video file is actually another file, right?
What might be the flaw of this technique? Is this also a valid steganography technique?
Is this a valid steganography technique?
Yes, it is. The definition of steganography is hiding information in another medium without someone suspecting its presence or existence. Just because it may be a bad approach doesn't change its intentions at all. If anything, a multitude of papers on steganography mention this technique in their introduction section as an example of how steganography can be applied.
What might be the flaw of this technique?
There are mainly 2 flaws: it is trivial to detect and is absolutely fragile to modification attacks.
Many formats encode their data either by a header which says in advance how many bytes to read before the end of file, or by putting an end-of-file marker, which means to keep on reading data until the marker is encountered. By attaching your data after that, you ensure they won't be read by the appropriate format decoder. This can fool your 11-year old cousin who knows nothing about that sort of stuff, but anyone mildly experienced can load the file and count how many bytes were read. If there are unaccounted bytes in the physical file, that will instantly raise red flags.
Even worse, it's trivial to fully extract your secret. You may argue it's encrypted, but remember, the aim of steganography is to not raise any suspicion. Most steganalysis approaches put a statistical number to it, e.g., 60% there is a message hidden in X medium. A few others can go a bit further and guess the approximate length of the embedded secret. In comparison, you're already caught red-handed.
Talking about length, a file of X bitrate/compression and Y duration approximately results to a file of size Z. Even an unsavvy one will know what's up when the size is 30% larger than expected.
Now, imagine your file is communicated through an insecure channel where a warden inspects its contents and if he suspects foul play, he can modify the file so that the recipient doesn't get the message. In this case, it's as simple as loading the file and resaving it. In fact, your method is so fragile it can be destroyed by even the most unintentional of attacks. By just uploading your track to a site for playback, it can unwittingly reencode it for higher compression, just because it makes sense.
Suppose no one could access the original file, of course no one would know that the last 15MB part of that video file is actually another file, right?
No. Your secret file is encrypted, so that probably rules out any headers showing up in hex editor, but there is a problem - MP4 container format and its structure is well known.
You can extract all video/audio tracks and what you are left with is some metadata and your secret message, so it will be obvious that it's not supposed to be there.
It is a valid technique, just not a very effective one.
For example, if I want to delete 10 bytes in the middle of file or add 10 bytes to the middle, are there anyway to do it without total file rewrite?
I'd like a solution that does this task the fastest.
I use C#, but the solution maybe be also in C or C++.
You have to copy the entire file, omitting or inserting the relevant bytes. This is an OS constraint; because of the way files are laid out on disk, it simply isn't possible to support this type of operation (at least with simple file systems like those used by Unix or Windows).
Try something that uses MapViewOfFile. There are probably C# bindings.
You can then treat the file like memory which should be fast.
if you want to do at same place delete and add ==> modify/update
open file in r+ mode this will give you change to read and write both operation can perform.
then
move the file stream to where you want to modify and add that many bytes there
you need not to delete 10 bytes. need to write 10 bytes. this will overwrite ten bytes from the current location. nothing but delete old ten bytes and add new ten bytes.
if you want to delete and don't want to add any thing ,then copy to new file except that ten bytes.
if you just want to add , move the stream position to end , write ten bytes.
you need fseek()
In C#, I have a ZIP file that I want to corrupt by XORing or Nulling its bytes.
(by Nulling I mean make all the bytes in the file zeros)
XORing its bytes requires me to first, read the bytes to a byte array, XOR the bytes in the array with some value, then write the bytes back to the file.
Now, if I XOR/Null All (or half) of the file's bytes, it gets corrupted, but if Just
XOR/Null some of the bytes, say the first few bytes (or any few number of bytes in any position of the file) it doesn't get corrupted, and by that I mean that i can still access the file as if nothing really happend.
Same thing happened with mp3 files.
Why isn't the file getting corrupted ?
and is there a "FAST" way that i could corrupt a file with ?
the problem is that the zip file that I'm dealing with is big,
so XORing/Nulling even half of its bytes will take a couple of secs.
Thank You So Much In Advance .. :)
Just read all files completely and you probaly will get reading errors.
But of course, if you want to keep something 'secret', use encryption.
A zip contains a small header, a directory structure (a the end) and in between the individual files. See Wikipedia for details.
Corrupting the first bytes is sure to corrupt the file but it is also very easily repaired. The reader won't be able to find the directory block at the end.
Damaging the last block has the same effect: the reader will give up immediately but it is repairable.
Changing a byte in the middle will corrupt 1 file. The CRC will fail.
It depends on the file format you are trying to "corrupt". It also depends on what portion of the file you are trying to modify. Lastly, it depends how you are verifying if it is corrupted. Most file formats have some type of error detection.
The other thing working against you is that the zip file format uses a CRC algorithm for corruption. In addition, there are two copies of the directory structure, so you need to corrupt both.
I would suggest you corrupt the directory structure at the end and then modify some of the bytes in the front.
I could just lock the zip entries with a pass, but I don't want anybody to even open it up and see what's in it
That makes it sound as if you're looking for a method of secure deletion. If you simply didn't want someone to read the file, delete it. Otherwise, unless you do something extreme like go over it a dozen times with different values or apply some complex algorithm over it a hundred times, there are still going to be ways to read the data, even if the format is 'corrupt'.
On the other hand, breaking a file simply to stop someone else accessing it conventionally just seems overkill. If it's a zip, you can read it in (there are plenty of questions here for handling archive files), encrypt it with a password and then write it back out. If it's a different type of file, there are literally a million different questions and solutions for encrypting, hiding or otherwise preventing access to data. Breaking a file isn't something you should being going out of your way to do, unless this is to help test some sort of un-zip-corrputing-program or something similar, but your comments imply this is to prevent access. Perhaps a bit more background on why you want to do this could help us provide a better answer?
I have an implementation of a custom DataObject (Virtual File) see here. I have drag and drop functionality in a control view (drag and drop a file OUT of a control view without having a temp local file).
This works fine with smaller files but as soon as the file is larger than say 12-15megs it says not enough memory available. seems like the memory stream is out of memory.
what can i do about this? can i somehow split a larger byte[] into several memoryStreams and reassemble those to a single file?
Any help would be highly appreciated.
can i somehow split a larger byte[]
into several momoryStreams and
reassemble those to a single file?
Yes.
When I had to deal with a similar situation I built my own stream that internally used byte arrays of 4mb. This "paging" means it never has to allocate ONE LARGE BYTE ARRAY, which is what memory stream does. So, dump memory stream, build your own stream based on another internal storage mechanism.