I will like to use a memory mapped file to virtualize opening a file on windows when that file is realy on the internet.
So I create the memory mapped file as:
// data that we write to the file. we will get this a tcp
var data = System.Text.Encoding.UTF8.GetBytes("Hello World");
var fileStream = new FileStream("SomeFile.txt", FileMode.Create);
using (MemoryMappedFile memoryMapped = MemoryMappedFile.CreateFromFile(fileStream, "MapName", 1024,
MemoryMappedFileAccess.ReadWrite, new MemoryMappedFileSecurity(), HandleInheritability.Inheritable, true))
{
var viewStream = memoryMapped.CreateViewStream();
viewStream.Write(data, 0, data.Length); // write hello world
}
And I can read from it on windows but not save it:
Note how I was able to open the file (meanwhile the data was on memory and not the hard disk) but the moment I tried saving changes I was not able. So my question is: How could I enable saving changes to that file and be just changing the content in memory of the memory mapped file without actually trying to save anything to disk.
You need to specify the sharing mode when creating the file stream.
var fileStream =
new FileStream("SomeFile.txt", FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite);
Also, you need to dispose of your FileStream when done, e.g. with a using statement.
UPDATE
It worked just fine for me. Using Notepad I had to manually re-open the file, but I could update it while Notepad had it open (Notepad just did not check for external modifications).
Side note: The code writes a bunch of NUL (0x00) bytes to the end of the file. You'll probably want to look into that.
Here's the exact code I used (note the local path to C:\Temp. Change if needed):
static private void WriteMMF()
{
// data that we write to the file. we will get this a tcp
var data = System.Text.Encoding.UTF8.GetBytes("Hello World 2");
using (var fileStream = new FileStream(#"C:\Temp\SomeFile.txt", FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite))
using (MemoryMappedFile memoryMapped = MemoryMappedFile.CreateFromFile(fileStream, "MapName", 1024,
MemoryMappedFileAccess.ReadWrite, new MemoryMappedFileSecurity(), HandleInheritability.Inheritable, true))
{
var viewStream = memoryMapped.CreateViewStream();
viewStream.Write(data, 0, data.Length); // write hello world
}
}
static void Main(string[] args)
{
Console.WriteLine("Writing MMF");
WriteMMF();
Console.WriteLine("Done. Press a key.");
var ch = Console.ReadKey();
return;
}
Related
I learned some basics about MemoryMappedFile and I saw that there is an enum member called MemoryMappedFileAccess.ReadExcute.
I thought it means when you open an exe file, it (my programme) execute it (.exe file) and read the bytes that inside it (.exe file) but when i execute the program it throw me an error:
Acceess to the path is denied [UnauthorizedAccessException]
mycode:
static void Main(string[] args)
{
FileStream fs = new FileStream("programe.exe", FileMode.OpenOrCreate, FileAccess.ReadWrite,
FileShare.ReadWrite);
MemoryMappedFile memory = MemoryMappedFile.CreateFromFile(fs, "mapname", 0,
MemoryMappedFileAccess.ReadExecute,null,0,false);
MemoryMappedViewAccessor mmr = memory.CreateViewAccessor(0, fs.Length, MemoryMappedFileAccess.Read);
Console.ReadKey();
}
Can any one explain it to me ?
Plenty of things to try here:
Try opening with Read to see if this fixes your problem
Try adding the full file path (in case it's not opening)
Add the length of the file when you open it
Check three and one below:
static void Main(string[] args)
{
// Try with Read here and Read on your create view to see if anything changes
FileStream fs = new FileStream("programe.exe", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Read);
// Set the length of the file here.
MemoryMappedFile memory = MemoryMappedFile.CreateFromFile(fs, "mapname", fs.Length, MemoryMappedFileAccess.ReadExecute,null,0,false);
MemoryMappedViewAccessor mmr = memory.CreateViewAccessor(0, fs.Length, MemoryMappedFileAccess.Read);
Console.ReadKey();
}
Then check that your file "programe.exe", has the appropriate rights in your filesystem
Based on the documentation the ReadExecute is just a Read access right for executable files:
ReadExecute 4
Read access to the file that can store and run executable code.
Can I get a GZipStream for a file on disk without writing the entire compressed content to temporary storage? I'm currently using a temporary file on disk in order to avoid possible memory exhaustion using MemoryStream on very large files (this is working fine).
public void UploadFile(string filename)
{
using (var temporaryFileStream = File.Open("tempfile.tmp", FileMode.CreateNew, FileAccess.ReadWrite))
{
using (var fileStream = File.OpenRead(filename))
using (var compressedStream = new GZipStream(temporaryFileStream, CompressionMode.Compress, true))
{
fileStream.CopyTo(compressedStream);
}
temporaryFileStream.Position = 0;
Uploader.Upload(temporaryFileStream);
}
}
What I'd like to do is eliminate the temporary storage by creating GZipStream, and have it read from the original file only as the Uploader class requests bytes from it. Is such a thing possible? How might such an implementation be structured?
Note that Upload is a static method with signature static void Upload(Stream stream).
Edit: The full code is here if it's useful. I hope I've included all the relevant context in my sample above however.
Yes, this is possible, but not easily with any of the standard .NET stream classes. When I needed to do something like this, I created a new type of stream.
It's basically a circular buffer that allows one producer (writer) and one consumer (reader). It's pretty easy to use. Let me whip up an example. In the meantime, you can adapt the example in the article.
Later: Here's an example that should come close to what you're asking for.
using (var pcStream = new ProducerConsumerStream(BufferSize))
{
// start upload in a thread
var uploadThread = new Thread(UploadThreadProc(pcStream));
uploadThread.Start();
// Open the input file and attach the gzip stream to the pcStream
using (var inputFile = File.OpenRead("inputFilename"))
{
// create gzip stream
using (var gz = new GZipStream(pcStream, CompressionMode.Compress, true))
{
var bytesRead = 0;
var buff = new byte[65536]; // 64K buffer
while ((bytesRead = inputFile.Read(buff, 0, buff.Length)) != 0)
{
gz.Write(buff, 0, bytesRead);
}
}
}
// The entire file has been compressed and copied to the buffer.
// Mark the stream as "input complete".
pcStream.CompleteAdding();
// wait for the upload thread to complete.
uploadThread.Join();
// It's very important that you don't close the pcStream before
// the uploader is done!
}
The upload thread should be pretty simple:
void UploadThreadProc(object state)
{
var pcStream = (ProducerConsumerStream)state;
Uploader.Upload(pcStream);
}
You could, of course, put the producer on a background thread and have the upload be done on the main thread. Or have them both on background threads. I'm not familiar with the semantics of your uploader, so I'll leave that decision to you.
I don't understand what I'm doing wrong here. I generate couple of memory streams and in debug-mode I see that they are populated. But when I try to copy MemoryStream to FileStream in order to save the file fileStream is not populated and file is 0bytes long (empty).
Here is my code
if (file.ContentLength > 0)
{
var bytes = ImageUploader.FilestreamToBytes(file); // bytes is populated
using (var inStream = new MemoryStream(bytes)) // inStream is populated
{
using (var outStream = new MemoryStream())
{
using (var imageFactory = new ImageFactory())
{
imageFactory.Load(inStream)
.Resize(new Size(320, 0))
.Format(ImageFormat.Jpeg)
.Quality(70)
.Save(outStream);
}
// outStream is populated here
var fileName = "test.jpg";
using (var fileStream = new FileStream(Server.MapPath("~/content/u/") + fileName, FileMode.CreateNew, FileAccess.ReadWrite))
{
outStream.CopyTo(fileStream); // fileStream is not populated
}
}
}
}
You need to reset the position of the stream before copying.
outStream.Position = 0;
outStream.CopyTo(fileStream);
You used the outStream when saving the file using the imageFactory. That function populated the outStream. While populating the outStream the position is set to the end of the populated area. That is so that when you keep on writing bytes to the steam, it doesn't override existing bytes. But then to read it (for copy purposes) you need to set the position to the start so you can start reading at the start.
If your objective is simply to dump the memory stream to a physical file (e.g. to look at the contents) - it can be done in one move:
System.IO.File.WriteAllBytes(#"C:\\filename", memoryStream.ToArray());
No need to set the stream position first either, since the .ToArray() operation explicitly ignores that, as per #BaconBits comment below https://learn.microsoft.com/en-us/dotnet/api/system.io.memorystream.toarray?view=netframework-4.7.2.
Another alternative to CopyTo is WriteTo.
Advantage:
No need to reset Position.
Usage:
outStream.WriteTo(fileStream);
Function Description:
Writes the entire contents of this memory stream to another stream.
I have the following code for copying file:
var copedFile = ConfigurationManager.AppSettings["PathToFirebirdDB"] + ".001";
using (var inputFile = new FileStream( ConfigurationManager.AppSettings["PathToFirebirdDB"],
FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (var outputFile = new FileStream(copedFile, FileMode.Create))
{
var buffer = new byte[0x10000];
int bytes;
while ((bytes = inputFile.Read(buffer, 0, buffer.Length)) > 0)
{
outputFile.Write(buffer, 0, bytes);
}
}
}
This code works fine only one time. The next time I get the folowing message:
The process cannot access the file 'D:\Programs\IBExpert\db.fdb.001' because it is being used by another process. System.IO.IOException: The process cannot access the file 'D:\Programs\IBExpert\db.fdb.001' because it is being used by another process.
Why? There are using block.
If you try to reopen the file just after closing it, there is a chance the file is still considered open by the system because it actually is.
A typical reason is that a virus scanner is keeping the file open to ensure it is not infected, this happens in the background and might continue running after you have closed the file yourself.
Probably because you are not closing the files.
BTW why don't you just use File.Copy?
I'm writing code that deals with a file that uses hashes. I need to read a chunk, then hash it, then write it, then read another chunk, etc.
In other words, I need to do a lot of reading and writing. I'm sure this is really simple, but I just wanted to run it by the pros...
Is it possible, and acceptable to do something like:
BinaryReader br = new BinaryReader (File.OpenRead(path));
BinaryWriter bw = new BinaryWriter (File.OpenWrite(path));
br.dostuff();
bw.dostuff();
I remember running into some sort of conflicting file streams error when experimenting with opening and writing to files, and I'm not sure what I had done to get it. Is it two file streams that's the issue? Can I have one stream to read from and write to?
This is perfecty possible and desired, A technicality, if your write method doesn't change the length of the file and is always behind the reader, this should not give any problems. In fact, from an API point of view, this is desirable since this allows the user to control where to read from and where to write to. (It's a recommended specification to write to a different file, in case any bad things happen during the encryption process, your input file wont be messed up).
Something like:
protected void Encrypt(Stream input, Stream output)
{
byte[] buffer = new byte[2048];
while (true)
{
// read
int current = input.Read(buffer, 0, buffer.Length);
if (current == 0)
break;
// encrypt
PerformActualEncryption(buffer, 0, current);
// write
output.Write(buffer, 0, current);
}
}
public void Main()
{
using (Stream inputStream = File.Open("file.dat", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (Stream outputStream = File.Open("file.dat", FileMode.Open, FileAccess.Write, FileShare.ReadWrite))
{
Encrypt(inputStream, outputStream);
}
}
Now since you're using an encryption, i would even recommend to perform the actual encryption in another specialized stream. This cleans the code up nicely.
class MySpecialHashingStream : Stream
{
...
}
protected void Encrypt(Stream input, Stream output)
{
Stream encryptedOutput = new MySpecialHashingStream(output);
input.CopyTo(encryptedOutput);
}