Develop Reference

Avoiding nested using blocks when outer is used only to construct inner

Often, the IDisposable object of a using block is itself constructed from another IDisposable object, e.g.
using (FileStream stream = File.Open(path, FileMode.Open))
using (MyObject obj = new MyObject(stream))
{
// do something with obj
}
Unfortunately the code above keeps the file stream open until the MyObject object is disposed.
To dispose of the file stream as soon as the MyObject constructor has completed, I could instead write:
MyObject CreateMyObject(string path)
{
using (FileStream stream = File.Open(path, FileMode.Open))
{
return new MyObject(stream);
}
}
using (MyObject obj = CreateMyObject(path))
{
// do something with obj
}
But I don't like the verbosity of this solution. I tried replacing CreateMyObject() with a lambda but I failed to find a legal syntax. Is there a way to do this without calling a custom creator function?
Edit: Bearing in mind some of the comments, I should point out that I'm trying to avoid try...finally - kind of the main reason for a using block in the first place.
Additional clarification: The MyObject object is constructed from information in the stream - i.e. its constructor reads the content of the stream in its entirity. No other method in MyObject references the stream. The content of the stream could come from anywhere - a file, a resource, an Internet socket, etc.

You could invoke some magic like so:
TResult CreateUsingDisposable<TDisposable, TResult>(TDisposable disposable, Func<TDisposable, TResult> getResult)
where TDisposable : IDisposable
{
using (disposable)
{
return getResult(disposable);
}
}
using (var obj = CreateUsingDisposable(new FileStream(path, FileMode.Open), stream => new MyObject(stream)))
{
}
But why? There's a super easy to read no-nonsense way of doing that:
MyObject obj;
using (var stream = new FileStream(path, FileMode.Open))
{
obj = new MyObject(stream);
}
using (obj)
{
}

While I don't see a way to avoid the creator function, you can make it generic enough to define once and use for any class:
static T WithStream<T>(string path, Func<FileStream, T> getter)
{
using (FileStream stream = File.Open(path, FileMode.Open))
{
return getter(stream);
}
}
class MyObject : IDisposable
{
public MyObject (Stream stream){ /* Work with stream */}
public void Dispose(){}
}
static void Main()
{
using (MyObject obj = WithStream("path", fs => new MyObject(fs)))
{
// do something with obj
}
}

How to stop StreamWriter to not to create file if nothing to Write?

Stream Writer creates a file even if there is nothing to write if Stream Writer was used in using statement, the Reason is using block calls dispose method at the end of block.
Code is
using (StreamWriter sw = new StreamWriter(#"d:\abc.txt"))
{
}
I wan't Not to create the blank file if there is nothing to write. The reason I am looking for is We are writing logs conditionally.

You could write something like:
Lazy<StreamWriter> sw = new Lazy<StreamWriter>(() => new StreamWriter(#"d:\abc.txt"));
try
{
if (something)
{
sw.Value.WriteLine("Foo");
}
}
finally
{
if (sw.IsValueCreated)
{
sw.Value.Dispose();
}
}
using Lazy<T> to encapsulate the StreamWriter.
You could even encapsulate Lazy<T> to "handle" IDisposable:
public sealed class DisposableLazy<T> : Lazy<T>, IDisposable where T : IDisposable
{
public DisposableLazy(Func<T> valueFactory) : base(valueFactory)
{
}
// No unmanaged resources in this class, and it is sealed.
// No finalizer needed. See http://stackoverflow.com/a/3882819/613130
public void Dispose()
{
if (IsValueCreated)
{
Value.Dispose();
}
}
}
and then
using (DisposableLazy<StreamWriter> sw = new DisposableLazy<StreamWriter>(() => new StreamWriter(#"d:\abc.txt")))
{
if (something)
{
sw.Value.WriteLine("Foo");
}
}

File being used by another process using StreamWriter

My program was practice for me, however, when I try to write all the directories it found, it crashes.
I tried the following:
Having it write to a file stream instead of the file itself
using File.Writealllines using a list<> (this worked, only it did the first five and no more)
FileStream.Write(subdir.ToCharArray())
I cannot see why this wouldn't work, what have I done wrong?
static void Main(string[] args)
{
Method(#"C:\");
}
static void Method(string dir)
{
//crash happens here v
StreamWriter sw = new StreamWriter(#"C:\users\"+Environment.UserName+"\desktop\log.txt",true);
foreach (string subdir in Directory.GetDirectories(dir))
{
try
{
Console.WriteLine(subdir);
sw.Write(subdir);
Method(subdir);
}
catch (UnauthorizedAccessException)
{
Console.WriteLine("Error");
}
}
sw.Close();
}

Its recursive.
Because you're calling Method again here:
Console.WriteLine(subdir);
sw.Write(subdir);
Method(subdir); // BOOM
Your file is already open. You can't open it for writing again.
Open the file in Main once..
static void Main(string[] args) {
using (StreamWriter sw = new StreamWriter(#"C:\users\"+Environment.UserName+"\desktop\log.txt",true)) {
Method(#"C:\", sw);
}
}
Then accept it in your method:
public static void Method(string dir, StreamWriter sw) {
Then when you call it again:
sw.Write(subdir);
Method(subdir, sw); // pass in the streamwriter.
Note though, that you will quickly start chewing up memory. You're recursing through your entire C:\ drive. Maybe test it on a smaller folder?

I am agree with above but in my case solution different a little.
private static object locker = new object();
private static void WriteMessageToFile(string message)
{
string dateStr = DateTime.Now.Date.Day.ToString()+"_"+ DateTime.Now.Date.Month.ToString()+"_"+ DateTime.Now.Date.Year.ToString();
if (!Directory.Exists("Logs"))
{
DirectoryInfo di = Directory.CreateDirectory("Logs");
}
//Guid guidGenerator = Guid.NewGuid();
string filePath = _env.ContentRootPath + "\\Logs\\ProcessLog_" + dateStr + ".txt";
FileInfo fi = new FileInfo(filePath);
lock (locker)
{
using (FileStream file = new FileStream(fi.FullName, FileMode.Append, FileAccess.Write, FileShare.Read))
using (StreamWriter streamWriter = new StreamWriter(file))
{
streamWriter.WriteLine(message);
streamWriter.Close();
}
}
}
Because the following function is called asynchronous and asynchronous in many places in my asp.net core application. In this case, one thread was trying to write a file, another thread wanted to write the same file, and there was an error. As a solution, I tried the above, but it didn't work either because I tried to open a new stream before closing the previous stream. So I decided to write a secure block of code as a solution. In this case, since the other threads could not reach the locked area, they made the write operation by waiting for the previous operation and I was able to write the file without error.
I think; there is another reason code behind, cause i have used Singleton registration on startup. This function's caller classes are isolated from each other. with this reason they didn't know which thread is called the function before. Their lifetime has been finished while. Also FileStream wraps the StreamWriter then it also may work without lock, anyway it is guaranty.
Even Microsoft.Extensions.Logging does not support FileLoger by default, but we can write custom. I share the entire implementation below
public class FileLoger : ILogger
{
public static IHostingEnvironment _env;
private static object locker = new object();
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
var message = string.Format("{0}: {1} - {2}", logLevel.ToString(), eventId.Id, formatter(state, exception));
WriteMessageToFile(message);
}
private static void WriteMessageToFile(string message)
{
string dateStr = DateTime.Now.Date.Day.ToString()+"_"+ DateTime.Now.Date.Month.ToString()+"_"+ DateTime.Now.Date.Year.ToString();
if (!Directory.Exists("Logs"))
{
DirectoryInfo di = Directory.CreateDirectory("Logs");
}
//Guid guidGenerator = Guid.NewGuid();
string filePath = _env.ContentRootPath + "\\Logs\\ProcessLog_" + dateStr + ".txt";
FileInfo fi = new FileInfo(filePath);
lock (locker)
{
using (FileStream file = new FileStream(fi.FullName, FileMode.Append, FileAccess.Write, FileShare.Read))
using (StreamWriter streamWriter = new StreamWriter(file))
{
streamWriter.WriteLine(message);
streamWriter.Close();
}
}
}
public IDisposable BeginScope<TState>(TState state)
{
return null;
}
public bool IsEnabled(LogLevel logLevel)
{
return true;
}
}
public class FileLogProvider : ILoggerProvider
{
public FileLogProvider(IHostingEnvironment env)
{
FileLoger._env = env;
}
public ILogger CreateLogger(string category)
{
return new FileLoger();
}
public void Dispose()
{
}
}

Seems you didn't close your streamwriter before you use it again
public static void Method(string dir)
{
//crash happens here v
StreamWriter sw = new StreamWriter(#"C:\users\"+Environment.UserName+"\desktop\log.txt",true);
foreach (string subdir in Directory.GetDirectories(dir))
{
try
{
Console.WriteLine(subdir);
sw.Write(subdir);
//This line you'll call "Method" again
Method(subdir);
}
catch (UnauthorizedAccessException)
{
Console.WriteLine("Error");
}
}
sw.Close();
}
Also, another suggestion, why don't you use "System.IO.File.AppendAllText(Path,Text)" method? it's easier to use

Why does my non closing stream still close?

I have a non-closing stream class that's wrapped in a using block with a binary reader, but for some reason when the block ends, my non-closing stream still closes.
The stream is defined as:
internal class NonClosingStream : Stream, IDisposable
{
private Stream baseStream;
public NonClosingStream(Stream baseStream)
{
this.baseStream = baseStream;
}
public override bool CanRead{ get { return baseStream.CanRead; } }
public override bool CanSeek{ get { return baseStream.CanSeek; } }
public override bool CanWrite { get { return baseStream.CanWrite; } }
public override void Flush()
{
baseStream.Flush();
}
public override long Length { get { return baseStream.Length; } }
public override long Position
{
get { return baseStream.Position; }
set { baseStream.Position = value; }
}
public override int Read(byte[] buffer, int offset, int count)
{
return baseStream.Read(buffer, offset, count);
}
public override long Seek(long offset, SeekOrigin origin)
{
return baseStream.Seek(offset, origin);
}
public override void SetLength(long value)
{
baseStream.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count)
{
baseStream.Write(buffer, offset, count);
}
public override void Close()
{
// Disconnects from base stream, but does not close it
this.baseStream = null;
}
void IDisposable.Dispose()
{
// Disconnects from base stream, but does not close it
this.baseStream = null;
}
}
and the read block looks like this:
public T Deserialize<T>(Stream stream)
{
using (NonClosingStream nonClosingStream = new NonClosingStream(stream))
using (BinaryReader reader = new BinaryReader(nonClosingStream, Encoding.ASCII, true))
{
// Read the type name, then convert it to an actual type
String typeName = reader.ReadString();
Type graphType = AvailableTypes.GetType(typeName);
// If a deserializer for this type already exists, use it.
if (deserializerFunctions.ContainsKey(graphType))
{
return (T)deserializerFunctions[graphType](reader);
}
// Otherwise, create one and use it
T graph = (T)FormatterServices.GetUninitializedObject(graphType);
typeof(ServiceSerializer).GetMethod("DeserializeObject",
BindingFlags.NonPublic | BindingFlags.Static)
.MakeGenericMethod(graphType)
.Invoke(this, new Object[] { reader, graph });
return graph;
}
}
what am I doing wrong?
Update
So I wrote this little diddy:
static void Main()
{
MemoryStream stream = new MemoryStream();
using (NonClosingStream nonCloser = new NonClosingStream(stream))
using (BinaryWriter writer = new BinaryWriter(nonCloser))
using (BinaryReader reader= new BinaryReader(nonCloser))
{
writer.Write("Lorem ipsum");
stream.Seek(0, SeekOrigin.Begin);
String data = reader.ReadString();
Console.WriteLine(data);
}
stream.Seek(0, SeekOrigin.Begin);
using (NonClosingStream nonCloser = new NonClosingStream(stream))
using (BinaryWriter writer = new BinaryWriter(nonCloser))
using (BinaryReader reader = new BinaryReader(nonCloser))
{
writer.Write("Lorem ipsum");
stream.Seek(0, SeekOrigin.Begin);
String data = reader.ReadString();
Console.WriteLine(data);
}
Console.ReadLine();
}
and it seems to work fine, the stream stays open like it's supposed to. So I guess the consensus is right. Somehow I'm closing the stream elsewhere. When I figure out what I'll post the results. Thanks all.
Update
Gaaaahhh, I figured out the problem. So the way the code works is that while it serializes/deserializes an object, it then builds a customized serializer out of expression trees and then compiles it so that future serializations are more fluid. That means my code is littered with stuff like this:
Action<BinaryReader, Object> assignmentAction = delegate(BinaryReader bReader, Object oGraph)
{
bReader.ReadByte(); // Read the next action
bReader.ReadString(); // Read the field name
bReader.ReadByte(); // Read the field type
// Call the assignment lambda
assignmentLambda(reader, deserializerFunctions[primitiveType], (T)oGraph);
};
did you catch that? No? Neither did I apparently. Let's add some context:
private static void DeserializeObject<T>(BinaryReader reader, T graph)
{
...
Action<BinaryReader, Object> assignmentAction = delegate(BinaryReader bReader, Object oGraph)
{
bReader.ReadByte(); // Read the next action
bReader.ReadString(); // Read the field name
bReader.ReadByte(); // Read the field type
// Call the assignment lambda
assignmentLambda(reader, deserializerFunctions[primitiveType], (T)oGraph);
};
...
}
The lambda is closing over reader from the outside block, instead of using the bReader provided when the cached deserializer runs. Hence, when the deserializer runs, it's using an already discarded Binary reader object rather than the fresh one being provided to it. I guess the issue wasn't that I was closing the stream, but that I was using a disposed reader. At least that explains why it would work once, and then fail the second time, since the second time it relies on the cached deserializer. Oops!
Thanks all.

Since your stream does not create inner stream most likely outer code closes your inner stream. Chances are your code look like:
NonClosingStream nonClosing;
using(var stream = new FileStream(...))
{
nonClosing = new NonClosingStream(stream );
....
}
// inner stream now closed and nonClosing will fail all operations.

void IDisposable.Dispose()
Your class has two Dispose() methods. The one you explicitly implemented. And the one that you inherited from the Stream class. Problem is, BinaryStream doesn't know beans about yours. It only knows about the one that Stream implements. Furthermore, when you use the BinaryStream(Stream) constructor, the BinaryStream object assumes ownership of the passed Stream object. Which means it will dispose that stream when it gets disposed itself. Perhaps you see the problem now, the inherited Dispose() method will be called, not yours. And it closes the base stream.
This is why Stream implements the Dispose pattern. You'll need to make it look like this instead:
internal class NonClosingStream : Stream {
protected override Dispose(bool disposing) {}
}

It depends on whether the stream which is being wrapped by your NonClosingStream class is referenced elsewhere. If not, then the underlying stream will have no references, so at some point afterwards its finalizer will close the stream.

Work with an Amazon S3 response stream after response has been disposed

I'm using the Amazon SDK and I have a method that returns a Stream for an object stored in Amazon S3 service.
It contains something like this:
var request = new GetObjectRequest().WithBucketName(bucketName).WithKey(keyName);
using (var response = client.GetObject(request))
{
return response.ResponseStream;
}
Obviously, when doing this, the stream is not readable from the calling method because the request object has been disposed and when this is done, it closes the stream.
I don't want to download the file to a MemoryStream or a FileStream.
If I don't use the using clause, the garbage collector will dispose the request object at some point so I can't just not use it.
What I'm asking is, is there a way to wrap or copy the Stream into another Stream and then return it without having to download the file?
I'm using .NET 3.5.
Edit: The method is inherited from an abstract class and the caller method doesn't know it is working with Amazon. So it HAS to return a Stream.

You can't work with the stream after it's disposed, but you can postpone disposing the response object until after the response stream has been used. There are three options I can suggest.
Return Response. One option is to return the response object to the caller. The caller can access the contained response stream and then dispose the response when done. This is the easiest change but requires the caller to change as well.
Wrap the stream. Instead of returning the response stream directly, create a new object that extends Stream and wraps the response stream but also has a reference to the response itself. Then when your wrapper is disposed, you can internally dispose the response object. This requires only changing your called code as long as the type being returned is just Stream.
Callback. Don't return anything from the method and use a callback mechanism instead. Accept an Action<Stream> as a parameter and call this callback with the stream. That way the callers code is called and you still have control over disposing the response and stream. This is the safest mechanism since you're not relying on the caller to dispose anything, but requires the most changes.

There is a method OpenStream in TransferUtility class which returns a stream object.
public Stream OpenStream(
String bucketName,
String key
)
I looked through the source code of AWSSDK, and found in the SDK the OpenStream just returns the response stream directly. (It doesn't use "using" statement on response object.)

If your function returns the response object (without using the using statement), and the caller assigns it to a variable, there will still be a reference to the response object. Thus it won't be eligible garbage collected.

Had the same issue (I thought). But have you tried to not use "using". This will not use the stream and send it to the caller who will have the responsibility to dispose it. As simple as that.
var request = new GetObjectRequest { BucketName = containerName, Key = blobName };
GetObjectResponse response = null;
try
{
response = client.GetObject(request));
}
catch (AmazonS3Exception ex)
{
if ((ex.ErrorCode == "NoSuchBucket") || (ex.ErrorCode == "AccessDenied") || (ex.ErrorCode == "InvalidBucketName") || (ex.ErrorCode == "NoSuchKey"))
{
return null;
}
throw;
}
return response.ResponseStream;

For completeness, and since I went with #Samuel option number 2 (Wrap the stream) which #spakinz commented he did too, I include here my implementation of what I called AmazonS3Stream
public class AmazonS3Stream : Stream
{
private Stream stream;
private GetObjectResponse response;
public AmazonS3Stream(GetObjectResponse response)
{
this.stream = response.ResponseStream;
this.response = response;
}
// The whole purpose of this class
protected override void Dispose(bool disposing)
{
// base.Dispose(disposing); // Do we really need this line? Probably not since I tested it and I can see that the stream is disposed when Response.Dispose is called by itself. And that makes sense because we know that this.stream is pointing to response.ResponseStream (that's what we declared in the constructor: this.stream = response.ResponseStream; ) So, what do we expect from response.Dispose() ? Obviously the first thing it would do is call ResponseStream.Dispose()
response.Dispose();
}
public override long Position
{
get { return stream.Position; }
set { stream.Position = Position; }
}
public override long Length
{
get { return stream.Length; }
}
public override bool CanRead
{
get { return stream.CanRead; }
}
public override bool CanSeek
{
get { return stream.CanSeek; }
}
public override bool CanWrite
{
get { return stream.CanWrite; }
}
public override void Flush()
{
stream.Flush();
}
public override void Write(byte[] buffer, int offset, int count)
{
stream.Write(buffer, offset, count);
}
public override void SetLength(long value)
{
stream.SetLength(value);
}
public override long Seek(long offset, SeekOrigin origin)
{
return stream.Seek(offset, origin);
}
public override int Read(byte[] buffer, int offset, int count)
{
return stream.Read(buffer, offset, count);
}
}

To add a code example to #Jun Y.'s answer. This is what I did and it was the cleanest solution by far.
using (var client = new AmazonS3Client(Amazon.RegionEndpoint.USEast2))
{
var transferUtility = new TransferUtility(client);
return await transferUtility.OpenStreamAsync(S3BucketName, key);
}