Develop Reference

Disposable that depends on other Disposables

I have an application that processes file streams based on a list of strings, and the string can either be a file on disk, or a file inside a Zip file. To clean up the code, I'd like to refactor out the process of opening the file.
I've created a method that returns a Stream of the file contents, but because the stream depends on the ZipFile IDisposable, by the time I read the stream, the ZipFile is disposed an throws an exception.
void Main()
{
using (var stream = OpenFileForImport("zipfile.zip;insidefile.txt"))
new StreamReader(stream).ReadToEnd(); // Exception
using (var stream = OpenFileForImport("outside.txt"))
new StreamReader(stream).ReadToEnd(); // Works
}
public static Stream OpenFileForImport(string filePath)
{
var path = Path.Combine(basefolder, filePath);
if (path.Contains(";"))
{
var parts = path.Split(';');
var zipPath = parts[0];
//Error checking logic to ensure zip file exists and is valid...
using (var zip = ZipFile.OpenRead(zipPath))
using (var entry = zip.GetEntry(parts[1]))
{
//Error checking logic to ensure inside file exists within zip file.
return entry.Open();
}
}
var file = new FileInfo(path);
if (file != null)
return file.OpenRead();
return null;
}
I could remove the using clause from the zip and entry declarations, but I doubt they'd ever get disposed. Is there an appropriate pattern to return a disposable, when it depends on other disposables?

Don't return the stream directly, instead return a disposable object which can provide the stream you want to dispose, but that cleans up that stream and the other dependant resources when it is disposed of:
public class NameToBeDetermined : IDisposable
{
private ZipFile zip;
public Stream Stream { get; }
public NameToBeDetermined(ZipFile zip, Stream stream)
{
this.zip = zip;
Stream = stream;
}
public void Dispose()
{
zip.Dispose();
Stream.Dispose();
}
}
Then return that, rather than the stream itself. If it's worth spending the time, you could turn that wrapper into a Stream itself, that just forwards all Stream methods into the composed stream, but that does the extra work when disposing. Whether it's worth the time to create that more involved wrapper rather than having a caller access a Stream property is up to you.

You likely should copy the file from the ZipEntry into a MemoryStream so that you have a copy to work with.
//Error checking logic to ensure zip file exists and is valid...
using (var zip = ZipFile.OpenRead(zipPath))
using (var entry = zip.GetEntry(parts[1]))
{
//Error checking logic to ensure inside file exists within zip file.
MemoryStream stream = new MemoryStream();
entry.Open().CopyTo(stream);
stream.Seek(0, SeekOrigin.Begin);
return stream;
}

Remove temporary file after returning in raw format in WCF

I have a WCF service which works in raw format, using streams:
[ServiceContract]
public interface IEncryptingService
{
[WebInvoke(UriTemplate = "/")]
[OperationContract]
Stream SignDocument(Stream requestStream);
}
public class EncryptingService : IEncryptingService
{
public Stream SignDocument(Stream requestStream)
{
string originalFileName = Path.GetTempFileName();
string signedFileName = Path.GetTempFileName();
using (var originalFileStream = File.Open(originalFileName, FileMode.Create, FileAccess.Write))
{
requestStream.CopyTo(originalFileStream);
}
XmlDocumentSigner.SignFile(originalFileName, signedFileName);
return File.Open(signedFileName, FileMode.Open, FileAccess.Read);
}
}
Now, how can I remove this file after WCF ends returning the file?
I have tried to use finally block, but it gets called right after return, and throws exception, since the file is still used by a process.
Of course, these is a workaround like a background worker waiting for a file to be available for deletion, but, in my opinion, it is not like how web-services should be implemented.

I haven't tried it but you could open the file and write that stream to another stream. Something like:
MemoryStream ms = new MemoryStream();
using (FileStream fs = File.OpenRead(signedFileName))
{
//Read from fs and write to ms
}
Then all you will need to do is call delete on the file and return ms:
File.Delete(signedFileName);
return ms;

The solution came to mind inexcusably quickly, and it is absolutely logical: I can simply read file contents to memory and remove the file.
public Stream SignDocument(Stream requestStream)
{
string originalFileName = Path.GetTempFileName();
string signedFileName = Path.GetTempFileName();
using (var originalFileStream = File.Open(originalFileName, FileMode.Create, FileAccess.Write))
{
requestStream.CopyTo(originalFileStream);
}
XmlDocumentSigner.SignFile(originalFileName, signedFileName);
byte[] signedFileBytes = File.ReadAllBytes(signedFileName);
File.Delete(signedFileName);
return new MemoryStream(signedFileBytes);
}
Note that using statement makes this code fail as well:
using (var ms = new MemoryStream(signedFileBytes))
{
return ms;
}

Can I get a GZipStream for a file without writing to intermediate temporary storage?

Can I get a GZipStream for a file on disk without writing the entire compressed content to temporary storage? I'm currently using a temporary file on disk in order to avoid possible memory exhaustion using MemoryStream on very large files (this is working fine).
public void UploadFile(string filename)
{
using (var temporaryFileStream = File.Open("tempfile.tmp", FileMode.CreateNew, FileAccess.ReadWrite))
{
using (var fileStream = File.OpenRead(filename))
using (var compressedStream = new GZipStream(temporaryFileStream, CompressionMode.Compress, true))
{
fileStream.CopyTo(compressedStream);
}
temporaryFileStream.Position = 0;
Uploader.Upload(temporaryFileStream);
}
}
What I'd like to do is eliminate the temporary storage by creating GZipStream, and have it read from the original file only as the Uploader class requests bytes from it. Is such a thing possible? How might such an implementation be structured?
Note that Upload is a static method with signature static void Upload(Stream stream).
Edit: The full code is here if it's useful. I hope I've included all the relevant context in my sample above however.

Yes, this is possible, but not easily with any of the standard .NET stream classes. When I needed to do something like this, I created a new type of stream.
It's basically a circular buffer that allows one producer (writer) and one consumer (reader). It's pretty easy to use. Let me whip up an example. In the meantime, you can adapt the example in the article.
Later: Here's an example that should come close to what you're asking for.
using (var pcStream = new ProducerConsumerStream(BufferSize))
{
// start upload in a thread
var uploadThread = new Thread(UploadThreadProc(pcStream));
uploadThread.Start();
// Open the input file and attach the gzip stream to the pcStream
using (var inputFile = File.OpenRead("inputFilename"))
{
// create gzip stream
using (var gz = new GZipStream(pcStream, CompressionMode.Compress, true))
{
var bytesRead = 0;
var buff = new byte[65536]; // 64K buffer
while ((bytesRead = inputFile.Read(buff, 0, buff.Length)) != 0)
{
gz.Write(buff, 0, bytesRead);
}
}
}
// The entire file has been compressed and copied to the buffer.
// Mark the stream as "input complete".
pcStream.CompleteAdding();
// wait for the upload thread to complete.
uploadThread.Join();
// It's very important that you don't close the pcStream before
// the uploader is done!
}
The upload thread should be pretty simple:
void UploadThreadProc(object state)
{
var pcStream = (ProducerConsumerStream)state;
Uploader.Upload(pcStream);
}
You could, of course, put the producer on a background thread and have the upload be done on the main thread. Or have them both on background threads. I'm not familiar with the semantics of your uploader, so I'll leave that decision to you.

Difficulty with IsolatedStorageFile

I'm building a Windows Phone 7 app in Silverlight. I'm having difficulty using IsolatedStorageFile.
The following method is supposed to write some data to a file:
private static void writeToFile(IList<Story> stories)
{
IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication();
using (IsolatedStorageFileStream stream = storage.OpenFile(STORIES_FILE, FileMode.Append))
{
using (StreamWriter writer = new StreamWriter(stream))
{
StringBuilder toJson = new StringBuilder();
IList<StoryJson> storyJsons = (from story in stories
where !storageStories.Contains(story)
select story.ToStoryJson()).ToList();
writer.Write(JsonConvert.SerializeObject(storyJsons));
}
}
#if DEBUG
StreamReader reader = new StreamReader(storage.OpenFile(STORIES_FILE, FileMode.Open));
string contents = reader.ReadToEnd();
#endif
}
The DEBUG at the end is for me to check that the data is actually being written. I have verified that it is. This method is called 6+ times. Each time, more data is appended.
However, when I go to read the data, the only JSON I get back is that which I wrote in one call of writeToFile(). Here is my method to read:
private static IList<Story> storageStories;
private static IList<Story> readFromStorage()
{
if (storageStories != null)
{
return storageStories;
}
IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication();
if (! storage.FileExists(STORIES_FILE))
{
storage.CreateFile(STORIES_FILE);
storageStories = new List<Story>();
return storageStories;
}
string contents;
using (IsolatedStorageFileStream stream = storage.OpenFile(STORIES_FILE, FileMode.OpenOrCreate))
{
using (StreamReader reader = new StreamReader(stream))
{
contents = reader.ReadToEnd();
}
}
JsonSerializer serializer = new JsonSerializer();
storageStories = JArray.Parse(contents).Select(storyData => storyOfJson(serializer, storyData)).ToList();
return storageStories;
}
What could I be doing wrong here? Am I writing to the file incorrectly? I'm pretty sure that the only data that is able to be read back is from the first write.
Update: I added two Flush() calls, but it crashes:
private static void writeToFile(IList<Story> stories)
{
IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication();
using (IsolatedStorageFileStream stream = storage.OpenFile(STORIES_FILE, FileMode.Append))
{
using (StreamWriter writer = new StreamWriter(stream))
{
StringBuilder toJson = new StringBuilder();
IList<StoryJson> storyJsons = (from story in stories
where !storageStories.Contains(story)
select story.ToStoryJson()).ToList();
writer.Write(JsonConvert.SerializeObject(storyJsons));
writer.Flush();
}
// FAILS
// "Cannot access a closed file." {System.ObjectDisposedException}
stream.Flush();
}
}
If I comment out the stream.Flush() but leave writer.Flush(), I have the same problem.
Update 2: I added some print statements. Looks like everything is getting serialized:
Serializing for VID 43
Serializing for VID 17
Serializing for VID 6
Serializing for VID 33
Serializing for VID 4
Serializing for VID 5
Serializing for VID 3
But only the first set is actually being read back:
Deserializing stories with vid: 43
I have run the test a few more times. I'm pretty sure that only the first item is ever being read back.

At first glance it sounds like your stream data is not being flushed to disk.
You are probably thinking that the using block will perform a flush when it Disposes the stream. However I have found that is not always the case and sometimes it is best to force a Flush() at the end.
I remember recently in a codebase that we received from a Microsoft team to port to WP7 that they were forcing a Flush. I questioned it initially, thinking that the Dispose should handle that, however as it was working and we were on a short deadline I did not investigate it further.
Give it go, see what happens... :)

Have you tried explicitly calling
writer.Close()
rather than relying on writer.Dispose()

Create Zip archive from multiple in memory files in C#

Is there a way to create a Zip archive that contains multiple files, when the files are currently in memory? The files I want to save are really just text only and are stored in a string class in my application. But I would like to save multiple files in a single self-contained archive. They can all be in the root of the archive.
It would be nice to be able to do this using SharpZipLib.

Use ZipEntry and PutNextEntry() for this. The following shows how to do it for a file, but for an in-memory object just use a MemoryStream
FileStream fZip = File.Create(compressedOutputFile);
ZipOutputStream zipOStream = new ZipOutputStream(fZip);
foreach (FileInfo fi in allfiles)
{
ZipEntry entry = new ZipEntry((fi.Name));
zipOStream.PutNextEntry(entry);
FileStream fs = File.OpenRead(fi.FullName);
try
{
byte[] transferBuffer[1024];
do
{
bytesRead = fs.Read(transferBuffer, 0, transferBuffer.Length);
zipOStream.Write(transferBuffer, 0, bytesRead);
}
while (bytesRead > 0);
}
finally
{
fs.Close();
}
}
zipOStream.Finish();
zipOStream.Close();

Using SharpZipLib for this seems pretty complicated. This is so much easier in DotNetZip. In v1.9, the code looks like this:
using (ZipFile zip = new ZipFile())
{
zip.AddEntry("Readme.txt", stringContent1);
zip.AddEntry("readings/Data.csv", stringContent2);
zip.AddEntry("readings/Index.xml", stringContent3);
zip.Save("Archive1.zip");
}
The code above assumes stringContent{1,2,3} contains the data to be stored in the files (or entries) in the zip archive. The first entry is "Readme.txt" and it is stored in the top level "Directory" in the zip archive. The next two entries are stored in the "readings" directory in the zip archive.
The strings are encoded in the default encoding. There is an overload of AddEntry(), not shown here, that allows you to explicitly specify the encoding to use.
If you have the content in a stream or byte array, not a string, there are overloads for AddEntry() that accept those types. There are also overloads that accept a Write delegate, a method of yours that is invoked to write data into the zip. This works for easily saving a DataSet into a zip file, for example.
DotNetZip is free and open source.

This function should create a byte array from a stream of data: I've created a simple interface for handling files for simplicity
public interface IHasDocumentProperties
{
byte[] Content { get; set; }
string Name { get; set; }
}
public void CreateZipFileContent(string filePath, IEnumerable<IHasDocumentProperties> fileInfos)
{
using (var memoryStream = new MemoryStream())
{
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
foreach(var fileInfo in fileInfos)
{
var entry = zipArchive.CreateEntry(fileInfo.Name);
using (var entryStream = entry.Open())
{
entryStream.Write(fileInfo.Content, 0, fileInfo.Content.Length);
}
}
}
using (var fileStream = new FileStream(filePath, FileMode.OpenOrCreate, System.IO.FileAccess.Write))
{
memoryStream.Position = 0;
memoryStream.CopyTo(fileStream);
}
}
}

Yes, you can use SharpZipLib to do this - when you need to supply a stream to write to, use a MemoryStream.

I come across this problem, using the MSDN example I created this class:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO.Packaging;
using System.IO;
public class ZipSticle
{
Package package;
public ZipSticle(Stream s)
{
package = ZipPackage.Open(s, FileMode.Create);
}
public void Add(Stream stream, string Name)
{
Uri partUriDocument = PackUriHelper.CreatePartUri(new Uri(Name, UriKind.Relative));
PackagePart packagePartDocument = package.CreatePart(partUriDocument, "");
CopyStream(stream, packagePartDocument.GetStream());
stream.Close();
}
private static void CopyStream(Stream source, Stream target)
{
const int bufSize = 0x1000;
byte[] buf = new byte[bufSize];
int bytesRead = 0;
while ((bytesRead = source.Read(buf, 0, bufSize)) > 0)
target.Write(buf, 0, bytesRead);
}
public void Close()
{
package.Close();
}
}
You can then use it like this:
FileStream str = File.Open("MyAwesomeZip.zip", FileMode.Create);
ZipSticle zip = new ZipSticle(str);
zip.Add(File.OpenRead("C:/Users/C0BRA/SimpleFile.txt"), "Some directory/SimpleFile.txt");
zip.Add(File.OpenRead("C:/Users/C0BRA/Hurp.derp"), "hurp.Derp");
zip.Close();
str.Close();
You can pass a MemoryStream (or any Stream) to ZipSticle.Add such as:
FileStream str = File.Open("MyAwesomeZip.zip", FileMode.Create);
ZipSticle zip = new ZipSticle(str);
byte[] fileinmem = new byte[1000];
// Do stuff to FileInMemory
MemoryStream memstr = new MemoryStream(fileinmem);
zip.Add(memstr, "Some directory/SimpleFile.txt");
memstr.Close();
zip.Close();
str.Close();

Note this answer is outdated; since .Net 4.5, the ZipArchive class allows zipping files in-memory. See johnny 5's answer below for how to use it.
You could also do it a bit differently, using a Serializable object to store all strings
[Serializable]
public class MyStrings {
public string Foo { get; set; }
public string Bar { get; set; }
}
Then, you could serialize it into a stream to save it.
To save on space you could use GZipStream (From System.IO.Compression) to compress it. (note: GZip is stream compression, not an archive of multiple files).
That is, of course if what you need is actually to save data, and not zip a few files in a specific format for other software.
Also, this would allow you to save many more types of data except strings.

I was utilizing Cheeso's answer by adding MemoryStreams as the source of the different Excel files. When I downloaded the zip, the files had nothing in them. This could be the way we were getting around trying to create and download a file over AJAX.
To get the contents of the different Excel files to be included in the Zip, I had to add each of the files as a byte[].
using (var memoryStream = new MemoryStream())
using (var zip = new ZipFile())
{
zip.AddEntry("Excel File 1.xlsx", excelFileStream1.ToArray());
zip.AddEntry("Excel File 2.xlsx", excelFileStream2.ToArray());
// Keep the file off of disk, and in memory.
zip.Save(memoryStream);
}

Use a StringReader to read from your string objects and expose them as Stream s.
That should make it easy to feed them to your zip-building code.