I have some code where I'm trying to read a line within a Mainframe file before I download it from the host. I create an instance of the Stream class as an object called reader, retrieve the FTP data stream from the host and place it into the Stream object, and then create a copy of the original Stream object and its data into another Stream object called readerCopy. My issue, I think, is that when I pass readerCopy into a method that retrieves some data from the Stream(RetrieveDateFromFile), that the resources for both readerCopy and reader are disposed of after the method ends. So when my calling method tries to use reader later on it throws the following:
System.ObjectDisposedException: 'Cannot access a disposed object. Object name: 'System.Net.Sockets.NetworkStream''
I thought that encapsulating all of the Stream objects in using statements would make it so that the resources wouldn't be disposed of until the end of those statements are reached but it seems like they might be disposed of sooner.
What am I missing?
Calling method:
public void FtpFile()
{
// Gets the FTP data stream and stores it into reader, creates a new Stream object called readercopy.
using (Stream reader = request.GetResponse().GetResponseStream(), readerCopy = new MemoryStream())
{
if (reader != null)
{
reader.CopyTo(readerCopy);// Copies the original Stream to readerCopy.
readerCopy.Position = 0; //Sets the position to be beginning of the Stream.
SMDPPITrigger trigger = new SMDPPITrigger(); //Custom class
using (StreamReader fileReader = new StreamReader(readerCopy))
{
using (FileStream fileStream = new FileStream(ftpFileDestination, FileMode.Create))
{
if (trigger.CheckIfExists(RetrieveDateFromFile(fileReader)) == false)
while (true)
{
bytesRead = reader.Read(buffer, 0, buffer.Length); //<--- error occurs here.
if (bytesRead == 0)
break;
fileStream.Write(buffer, 0, bytesRead);
}
}
}
}
}
}
Method to retrieve data from stream:
public DateTime RetrieveDateFromFile(StreamReader mainframeFile)
{
string lineParsed = "";
// StreamReader fileReader = new StreamReader(mainframeFile);
for (int i = 0; i <= 2; i++)
switch (i)
{
case 2:
string line = mainframeFile.ReadLine();
if (line != null)
{
lineParsed = line.Substring(124);
break;
}
else
{
break;
}
default:
{
mainframeFile.ReadLine();
break;
}
}
return DateTime.Parse(lineParsed);
}
I am suspecting on below two lines of code causing problem.
reader.CopyTo(readerCopy);// Copies the original Stream to readerCopy.
readerCopy.Position = 0; //Sets the position to be beginning of the Stream.
Can you try specifying length of stream you want to copy into readerCopy?
e.g
reader.CopyTo(readerCopy,124);
Closing the loop on my question. It turns out the issue and resolution is exactly what #KlausGütter had said. Using reader.CopyTo(readerCopy) was setting my position to the end of the stream and my error message wasn't that it had been disposed but instead that there was nothing left to read. Using the stream that I copied over to, readerCopy, solved my issue since the stream is seekable.
I've been needing to convert my driver to bytes, so I can load it without downloading anything.
Here is what I've tried.
class Program
{
public byte[] StreamFile(string filename)
{
FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
// Create a byte array of file stream length
byte[] ImageData = new byte[fs.Length];
//Read block of bytes from stream into the byte array
fs.Read(ImageData, 0, System.Convert.ToInt32(fs.Length));
byte[] bytes = System.IO.File.ReadAllBytes(filename);
//Close the File Stream
fs.Close();
return ImageData; //return the byte data
}
static void Main(string[] args)
{
StreamFile(#"");
}
}
I get an error in my Main,
An object is required for the non-static void field, method, or property "Program.StreamFile(string)"
Does anyone know why this happens?
Besides it is not clear to me what you would like to achieve, your code does not compile because you need StreamFile to be static, since you are calling it from a static method.
So this fixes the syntax error
class Program
{
// !!!!!! add this
public static byte[] StreamFile(string filename)
{
FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
// Create a byte array of file stream length
byte[] ImageData = new byte[fs.Length];
//Read block of bytes from stream into the byte array
fs.Read(ImageData, 0, System.Convert.ToInt32(fs.Length));
byte[] bytes = System.IO.File.ReadAllBytes(filename);
//Close the File Stream
fs.Close();
return ImageData; //return the byte data
}
static void Main(string[] args)
{
StreamFile(#"");
}
}
You can rewrite your code and make it more robust in this way (.net core)
class Program
{
public static byte[] StreamFile(string filename)
{
return System.IO.File.ReadAllBytes(filename);
}
static void Main(string[] args)
{
StreamFile(#"");
}
}
EDITED the below code shows how to use using, but requires little more knowledge about how to handle the buffers (I just placed an exception as this is out of scope now)
class Program
{
public static byte[] StreamFile(string filename)
{
byte[] data;
// let the stream be managed on its own
using (var fs = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.Read))
{
// If you want to ask for the length of the file, be sure nobody is changing it over time. See the FileShare.Read above
data = new byte[fs.Length];
if (data.Length != fs.Read(data, 0, data.Length))
throw new InvalidOperationException("Something went wrong.");
}
return data;
}
static void Main(string[] args)
{
StreamFile(#"");
}
}
Please note that the using construct automatically closes/disposes the resource you are using when it goes out of scope1. This way you cannot make mistakes and it is clear what is your intent.
You must lock the file (in your case, just asking nobody can change it via FileShare.Read) otherwise you may have a race condition on the file since you might read an inconsistent data.
1 They must implement the IDisposable interface.
the error you get is because of calling a non static method from inside of the static main method,
may be you should change your question title.
add static keyword to StreamFile method.
My program is using excessive memory. I am trying to read the first 512 bytes of a program and store them in memory. I believe it should only use 512 bytes of memory, but for some reason, it is using 1GB.
BinaryReader reader;
byte[] buffer = new byte[0];
foreach (IStorageDevice device in Devices)
{
reader = new BinaryReader(File.Open(device.Location, FileMode.Open));
buffer = reader.ReadBytes(512);
reader.Close();
reader.Dispose();
}
There was only one StorageDevice in the test I did, so it is only loading one file.
I can't seem to find the reason why it is using so much memory. Any help would be greatly appreciated.
Devices is a List of IStorageDevices. A storage device is just a class with a string object which is the path to the file that is read (at the moment it is a .bin file on my desktop)
public class ROM : IStorageDevice
{
public string Location { get; set; }
public ROM(string Location)
{
this.Location = Location;
}
}
You need to dispose of your resources. That's what using does. The stream will be disposed when the using block is exited.
Try something like this:
byte[] buffer = new byte[512];
foreach (IStorageDevice device in Devices)
{
using (var stream = File.OpenRead(device.Location))
{
// Read 512 bytes into buffer if possible.
var readCount = stream.Read(buffer, 0, 512);
StoreData(buffer, readCount); // A method you write to store the data
}
}
I have a text file. Several processes can simultaneously try to read and edit this file. I have a problem with FileStream.Unlock() method:
using System;
using System.IO;
using System.Text;
static class Program
{
static void Main()
{
var fileName = #"c:\temp\data.txt";
// Content of the 'c:\temp\data.txt' file:
// Hello!
// The magic number is 000. :)))
// Good luck...
using (var stream = new FileStream(fileName, FileMode.Open,
FileAccess.ReadWrite, FileShare.ReadWrite))
{
using(var reader = new StreamReader(stream))
{
var value = 0;
Console.Write("New value [0-999]: ");
while(int.TryParse(Console.ReadLine(), out value))
{
var prevPosition = stream.Position;
stream.Position = 28;
var data = Encoding.UTF8.GetBytes(value.ToString());
try
{
stream.Lock(stream.Position, data.LongLength);
Console.WriteLine("Data locked. Press any key for continuation...");
Console.ReadKey();
stream.Write(data, 0, data.Length);
stream.Flush();
// I get the Exception here: The segment already unlocked.
stream.Unlock(stream.Position, data.LongLength);
}
catch(Exception ex)
{
Console.WriteLine("Error: {0}", ex.Message);
}
stream.Position = prevPosition;
Console.Write("New value: ");
}
}
}
}
}
Why my stream is unlocked before I do it myself?
The reason is stream.Position is advanced after you locked the file (because you write to it), and you use stream.Position (which is now different) to unlock a file. In result - you are trying to unlock not the same range you locked. Instead, save stream.Position:
var position = stream.Position; // < save
stream.Lock(position, data.LongLength);
Console.WriteLine("Data locked. Press any key for continuation...");
stream.Write(data, 0, data.Length); // < this changes stream.Position, breaking your old logic
stream.Flush();
// I get the Exception here:
// The blocking of the segment already taken off.
stream.Unlock(position, data.LongLength); // < now you unlock the same range
Not sure, but maybe when you Write, the Stream.Position changes.
I've been trying to get the "new" ZipArchive included in .NET 4.5 (System.IO.Compression.ZipArchive) to work in a ASP.NET site. But it seems like it doesn't like writing to the stream of HttpContext.Response.OutputStream.
My following code example will throw
System.NotSupportedException: Specified method is not supported
as soon as a write is attempted on the stream.
The CanWrite property on the stream returns true.
If I exchange the OutputStream with a filestream, pointing to a local directory, it works. What gives?
ZipArchive archive = new ZipArchive(HttpContext.Response.OutputStream, ZipArchiveMode.Create, false);
ZipArchiveEntry entry = archive.CreateEntry("filename");
using (StreamWriter writer = new StreamWriter(entry.Open()))
{
writer.WriteLine("Information about this package.");
writer.WriteLine("========================");
}
Stacktrace:
[NotSupportedException: Specified method is not supported.]
System.Web.HttpResponseStream.get_Position() +29
System.IO.Compression.ZipArchiveEntry.WriteLocalFileHeader(Boolean isEmptyFile) +389
System.IO.Compression.DirectToArchiveWriterStream.Write(Byte[] buffer, Int32 offset, Int32 count) +94
System.IO.Compression.WrappedStream.Write(Byte[] buffer, Int32 offset, Int32 count) +41
Note: This has been fixed in .Net Core 2.0. I'm not sure what is the status of the fix for .Net Framework.
Calbertoferreira's answer has some useful information, but the conclusion is mostly wrong. To create an archive, you don't need seek, but you do need to be able to read the Position.
According to the documentation, reading Position should be supported only for seekable streams, but ZipArchive seems to require this even from non-seekable streams, which is a bug.
So, all you need to do to support writing ZIP files directly to OutputStream is to wrap it in a custom Stream that supports getting Position. Something like:
class PositionWrapperStream : Stream
{
private readonly Stream wrapped;
private long pos = 0;
public PositionWrapperStream(Stream wrapped)
{
this.wrapped = wrapped;
}
public override bool CanSeek { get { return false; } }
public override bool CanWrite { get { return true; } }
public override long Position
{
get { return pos; }
set { throw new NotSupportedException(); }
}
public override void Write(byte[] buffer, int offset, int count)
{
pos += count;
wrapped.Write(buffer, offset, count);
}
public override void Flush()
{
wrapped.Flush();
}
protected override void Dispose(bool disposing)
{
wrapped.Dispose();
base.Dispose(disposing);
}
// all the other required methods can throw NotSupportedException
}
Using this, the following code will write a ZIP archive into OutputStream:
using (var outputStream = new PositionWrapperStream(Response.OutputStream))
using (var archive = new ZipArchive(outputStream, ZipArchiveMode.Create, false))
{
var entry = archive.CreateEntry("filename");
using (var writer = new StreamWriter(entry.Open()))
{
writer.WriteLine("Information about this package.");
writer.WriteLine("========================");
}
}
If you compare your code adaptation with the version presented in MSDN page you'll see that the ZipArchiveMode.Create is never used, what is used is ZipArchiveMode.Update.
Despite that, the main problem is the OutputStream that doesn't support Read and Seek which is need by the ZipArchive in Update Mode:
When you set the mode to Update, the underlying file or stream must
support reading, writing, and seeking. The content of the entire
archive is held in memory, and no data is written to the underlying
file or stream until the archive is disposed.
Source: MSDN
You weren't getting any exceptions with the create mode because it only needs to write:
When you set the mode to Create, the underlying file or stream must support writing, but does not have to support seeking. Each entry in the archive can be opened only once for writing. If you create a single entry, the data is written to the underlying stream or file as soon as it is available. If you create multiple entries, such as by calling the CreateFromDirectory method, the data is written to the underlying stream or file after all the entries are created.
Source: MSDN
I believe you can't create a zip file directly in the OutputStream since it's a network stream and seek is not supported:
Streams can support seeking. Seeking refers to querying and modifying the current position within a stream. Seek capability depends on the kind of backing store a stream has. For example, network streams have no unified concept of a current position, and therefore typically do not support seeking.
An alternative could be writing to a memory stream, then use the OutputStream.Write method to send the zip file.
MemoryStream ZipInMemory = new MemoryStream();
using (ZipArchive UpdateArchive = new ZipArchive(ZipInMemory, ZipArchiveMode.Update))
{
ZipArchiveEntry Zipentry = UpdateArchive.CreateEntry("filename.txt");
foreach (ZipArchiveEntry entry in UpdateArchive.Entries)
{
using (StreamWriter writer = new StreamWriter(entry.Open()))
{
writer.WriteLine("Information about this package.");
writer.WriteLine("========================");
}
}
}
byte[] buffer = ZipInMemory.GetBuffer();
Response.AppendHeader("content-disposition", "attachment; filename=Zip_" + DateTime.Now.ToString() + ".zip");
Response.AppendHeader("content-length", buffer.Length.ToString());
Response.ContentType = "application/x-compressed";
Response.OutputStream.Write(buffer, 0, buffer.Length);
EDIT: With feedback from comments and further reading, you could be creating large Zip files, so the memory stream could cause you problems.
In this case i suggest you create the zip file on the web server then output the file using Response.WriteFile .
A refinement to svick's answer of 2nd February 2014. I found that it was necessary to implement some more methods and properties of the Stream abstract class and to declare the pos member as long. After that it worked like a charm. I haven't extensively tested this class, but it works for the purposes of returning a ZipArchive in the HttpResponse. I assume I've implemented Seek and Read correctly, but they may need some tweaking.
class PositionWrapperStream : Stream
{
private readonly Stream wrapped;
private long pos = 0;
public PositionWrapperStream(Stream wrapped)
{
this.wrapped = wrapped;
}
public override bool CanSeek
{
get { return false; }
}
public override bool CanWrite
{
get { return true; }
}
public override long Position
{
get { return pos; }
set { throw new NotSupportedException(); }
}
public override bool CanRead
{
get { return wrapped.CanRead; }
}
public override long Length
{
get { return wrapped.Length; }
}
public override void Write(byte[] buffer, int offset, int count)
{
pos += count;
wrapped.Write(buffer, offset, count);
}
public override void Flush()
{
wrapped.Flush();
}
protected override void Dispose(bool disposing)
{
wrapped.Dispose();
base.Dispose(disposing);
}
public override long Seek(long offset, SeekOrigin origin)
{
switch (origin)
{
case SeekOrigin.Begin:
pos = 0;
break;
case SeekOrigin.End:
pos = Length - 1;
break;
}
pos += offset;
return wrapped.Seek(offset, origin);
}
public override void SetLength(long value)
{
wrapped.SetLength(value);
}
public override int Read(byte[] buffer, int offset, int count)
{
pos += offset;
int result = wrapped.Read(buffer, offset, count);
pos += count;
return result;
}
}
An simplified version of svick's answer for zipping a server-side file and sending it via the OutputStream:
using (var outputStream = new PositionWrapperStream(Response.OutputStream))
using (var archive = new ZipArchive(outputStream, ZipArchiveMode.Create, false))
{
var entry = archive.CreateEntryFromFile(fullPathOfFileOnDisk, fileNameAppearingInZipArchive);
}
(In case this seems obvious, it wasn't to me!)
Presumably this is not an MVC app, where you could easily just use the FileStreamResult class.
I'm using this currently with ZipArchive created using a MemoryStream, so I know it works.
With that in mind, have a look at the FileStreamResult.WriteFile() method:
protected override void WriteFile(HttpResponseBase response)
{
// grab chunks of data and write to the output stream
Stream outputStream = response.OutputStream;
using (FileStream)
{
byte[] buffer = newbyte[_bufferSize];
while (true)
{
int bytesRead = FileStream.Read(buffer, 0, _bufferSize);
if (bytesRead == 0)
{
// no more data
break;
}
outputStream.Write(buffer, 0, bytesRead);
}
}
}
(Entire FileStreamResult on CodePlex)
Here is how I'm generating and returning the ZipArchive.
You should have no issues replacing the FSR with the guts of the WriteFile method from above, where FileStream becomes resultStream from the code below:
var resultStream = new MemoryStream();
using (var zipArchive = new ZipArchive(resultStream, ZipArchiveMode.Create, true))
{
foreach (var doc in req)
{
var fileName = string.Format("Install.Rollback.{0}.v{1}.docx", doc.AppName, doc.Version);
var xmlData = doc.GetXDocument();
var fileStream = WriteWord.BuildFile(templatePath, xmlData);
var docZipEntry = zipArchive.CreateEntry(fileName, CompressionLevel.Optimal);
using (var entryStream = docZipEntry.Open())
{
fileStream.CopyTo(entryStream);
}
}
}
resultStream.Position = 0;
// add the Response Header for downloading the file
var cd = new ContentDisposition
{
FileName = string.Format(
"{0}.{1}.{2}.{3}.Install.Rollback.Documents.zip",
DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day, (long)DateTime.Now.TimeOfDay.TotalSeconds),
// always prompt the user for downloading, set to true if you want
// the browser to try to show the file inline
Inline = false,
};
Response.AppendHeader("Content-Disposition", cd.ToString());
// stuff the zip package into a FileStreamResult
var fsr = new FileStreamResult(resultStream, MediaTypeNames.Application.Zip);
return fsr;
Finally, if you will be writing large streams (or a larger number of them at any given time), then you may want to consider using anonymous pipes to write the data to the output stream immediately after you write it to the underlying stream in the zip file. Because you will be holding all the file contents in memory on the server. The end of this answer to a similar question has a nice explanation of how to do that.