Develop Reference

How to combine partially pre-cached MemoryStream with FileStream?

For a time-critical media presentation application, it is important that media files be presented right at the instance when the user selects it. Those files reside in a truly humongous directory structure, comprised of thousands of media files.
Clearly, caching the media files in a MemoryStream is the way to go; however, due to the sheer amount of files, it’s not feasible to cache each file entirely. Instead, my idea is to pre-cache a certain buffer of each file, and once the file is presented, play from that cache until the rest of the file is loaded from the hard disk.
What I don’t see is how to “concatenate” both the MemoryStream and the FileStream so as to provide a seamless playback experience. I’m not very strong in data streams (yet), and I see several problems:
How does one keep track of the current read position within the MemoryStream and provide that to the FileStream without the MemoryStream reading more than that?
How does one switch from one stream to the other without having both streams either partially overlap each other or creating a “playback break”?
If using a queue of streams (as suggested in How do I concatenate two System.Io.Stream instances into one?), how can I specify that the second stream has to be ready for read access instantaneously after the first stream is done? Here, in particular, I don’t see how the MemoryStream would help at all, since the FileStream, as second one in the queue, would only begin accessing the hard disk once it’s actually used.
Is it really a feasible approach to have literally hundreds, if not thousands of open streams at once?
Note that I don’t need write access—reading is fully sufficient for the problem at hand. Also, this question is similar to Composite Stream Wrapper providing partial MemoryStream and full original Stream, but the solution provided there is a bug fix for Windows Phone 8 that doesn’t apply in my case.
I’d very much like to widen my rather limited understanding of this, so any help is greatly appreciated.

I would suggest something like the following solution:
Inherit your own CachableFileStream from FileStream
Implement a very simple Cache which uses a data structure you prefer (like a Queue)
Allow Preloading data into the internal cache
Allow Reloading data into the internal cache
Modify the original Read behaviour in a way, that your cache is used
To give you an idea of my idea I would suggest some implementation like the following one:
The usage could be like that:
CachableFileStream cachedStream = new CachableFileStream(...)
{
PreloadSize = 8192,
ReloadSize = 4096,
};
// Force preloading data into the cache
cachedStream.Preload();
...
cachedStream.Read(buffer, 0, buffer.Length);
...
Warning: The code below is neither correctly tested nor ideal - this shall just give you an idea!
The CachableFileStream class:
using System;
using System.IO;
using System.Threading.Tasks;
/// <summary>
/// Represents a filestream with cache.
/// </summary>
public class CachableFileStream : FileStream
{
private Cache<byte> cache;
private int preloadSize;
private int reloadSize;
/// <summary>
/// Gets or sets the amount of bytes to be preloaded.
/// </summary>
public int PreloadSize
{
get
{
return this.preloadSize;
}
set
{
if (value <= 0)
throw new ArgumentOutOfRangeException(nameof(value), "The specified preload size must not be smaller than or equal to zero.");
this.preloadSize = value;
}
}
/// <summary>
/// Gets or sets the amount of bytes to be reloaded.
/// </summary>
public int ReloadSize
{
get
{
return this.reloadSize;
}
set
{
if (value <= 0)
throw new ArgumentOutOfRangeException(nameof(value), "The specified reload size must not be smaller than or equal to zero.");
this.reloadSize = value;
}
}
/// <summary>
/// Initializes a new instance of the <see cref="CachableFileStream"/> class with the specified path and creation mode.
/// </summary>
/// <param name="path">A relative or absolute path for the file that the current CachableFileStream object will encapsulate</param>
/// <param name="mode">A constant that determines how to open or create the file.</param>
/// <exception cref="System.ArgumentException">
/// Path is an empty string (""), contains only white space, or contains one or more invalid characters.
/// -or- path refers to a non-file device, such as "con:", "com1:", "lpt1:", etc. in an NTFS environment.
/// </exception>
/// <exception cref="System.NotSupportedException">
/// Path refers to a non-file device, such as "con:", "com1:", "lpt1:", etc. in a non-NTFS environment.
/// </exception>
/// <exception cref="System.ArgumentNullException">
/// Path is null.
/// </exception>
/// <exception cref="System.Security.SecurityException">
/// The caller does not have the required permission.
/// </exception>
/// <exception cref="System.IO.FileNotFoundException">
/// The file cannot be found, such as when mode is FileMode.Truncate or FileMode.Open, and the file specified by path does not exist.
/// The file must already exist in these modes.
/// </exception>
/// <exception cref="System.IO.IOException">
/// An I/O error, such as specifying FileMode.CreateNew when the file specified by path already exists, occurred.-or-The stream has been closed.
/// </exception>
/// <exception cref="System.IO.DirectoryNotFoundException">
/// The specified path is invalid, such as being on an unmapped drive.
/// </exception>
/// <exception cref="System.IO.PathTooLongException">
/// The specified path, file name, or both exceed the system-defined maximum length.
/// For example, on Windows-based platforms, paths must be less than 248 characters, and file names must be less than 260 characters.
/// </exception>
/// <exception cref="System.ArgumentOutOfRangeException">
/// Mode contains an invalid value
/// </exception>
public CachableFileStream(string path, FileMode mode) : base(path, mode)
{
this.cache = new Cache<byte>();
this.cache.CacheIsRunningLow += CacheIsRunningLow;
}
/// <summary>
/// Reads a block of bytes from the stream and writes the data in a given buffer.
/// </summary>
/// <param name="array">
/// When this method returns, contains the specified byte array with the values between
/// offset and (offset + count - 1) replaced by the bytes read from the current source.
/// </param>
/// <param name="offset">The byte offset in array at which the read bytes will be placed.</param>
/// <param name="count">The maximum number of bytes to read.</param>
/// <returns>
/// The total number of bytes read into the buffer. This might be less than the number
/// of bytes requested if that number of bytes are not currently available, or zero
/// if the end of the stream is reached.
/// </returns>
/// <exception cref="System.ArgumentNullException">
/// Array is null.
/// </exception>
/// <exception cref="System.ArgumentOutOfRangeException">
/// Offset or count is negative.
/// </exception>
/// <exception cref="System.NotSupportedException">
/// The stream does not support reading.
/// </exception>
/// <exception cref="System.IO.IOException">
/// An I/O error occurred.
/// </exception>
/// <exception cref="System.ArgumentException">
/// Offset and count describe an invalid range in array.
/// </exception>
/// <exception cref="System.ObjectDisposedException">
/// Methods were called after the stream was closed.
/// </exception>
public override int Read(byte[] array, int offset, int count)
{
int readBytesFromCache;
for (readBytesFromCache = 0; readBytesFromCache < count; readBytesFromCache++)
{
if (this.cache.Size == 0)
break;
array[offset + readBytesFromCache] = this.cache.Read();
}
if (readBytesFromCache < count)
readBytesFromCache += base.Read(array, offset + readBytesFromCache, count - readBytesFromCache);
return readBytesFromCache;
}
/// <summary>
/// Preload data into the cache.
/// </summary>
public void Preload()
{
this.LoadBytesFromStreamIntoCache(this.PreloadSize);
}
/// <summary>
/// Reload data into the cache.
/// </summary>
public void Reload()
{
this.LoadBytesFromStreamIntoCache(this.ReloadSize);
}
/// <summary>
/// Loads bytes from the stream into the cache.
/// </summary>
/// <param name="count">The number of bytes to read.</param>
private void LoadBytesFromStreamIntoCache(int count)
{
byte[] buffer = new byte[count];
int readBytes = base.Read(buffer, 0, buffer.Length);
this.cache.AddRange(buffer, 0, readBytes);
}
/// <summary>
/// Represents the event handler for the CacheIsRunningLow event.
/// </summary>
/// <param name="sender">The sender of the event.</param>
/// <param name="e">Event arguments.</param>
private void CacheIsRunningLow(object sender, EventArgs e)
{
this.cache.WarnIfRunningLow = false;
new Task(() =>
{
Reload();
this.cache.WarnIfRunningLow = true;
}).Start();
}
}
The Cache class:
using System;
using System.Collections.Concurrent;
/// <summary>
/// Represents a generic cache.
/// </summary>
/// <typeparam name="T">Defines the type of the items in the cache.</typeparam>
public class Cache<T>
{
private ConcurrentQueue<T> queue;
/// <summary>
/// Is executed when the number of items within the cache run below the
/// specified warning limit and WarnIfRunningLow is set.
/// </summary>
public event EventHandler CacheIsRunningLow;
/// <summary>
/// Gets or sets a value indicating whether the CacheIsRunningLow event shall be fired or not.
/// </summary>
public bool WarnIfRunningLow
{
get;
set;
}
/// <summary>
/// Gets or sets a value that represents the lower warning limit.
/// </summary>
public int LowerWarningLimit
{
get;
set;
}
/// <summary>
/// Gets the number of items currently stored in the cache.
/// </summary>
public int Size
{
get;
private set;
}
/// <summary>
/// Initializes a new instance of the <see cref="Cache{T}"/> class.
/// </summary>
public Cache()
{
this.queue = new ConcurrentQueue<T>();
this.Size = 0;
this.LowerWarningLimit = 1024;
this.WarnIfRunningLow = true;
}
/// <summary>
/// Adds an item into the cache.
/// </summary>
/// <param name="item">The item to be added to the cache.</param>
public void Add(T item)
{
this.queue.Enqueue(item);
this.Size++;
}
/// <summary>
/// Adds the items of the specified array to the end of the cache.
/// </summary>
/// <param name="items">The items to be added.</param>
public void AddRange(T[] items)
{
this.AddRange(items, 0, items.Length);
}
/// <summary>
/// Adds the specified count of items of the specified array starting
/// from offset to the end of the cache.
/// </summary>
/// <param name="items">The array that contains the items.</param>
/// <param name="offset">The offset that shall be used.</param>
/// <param name="count">The number of items that shall be added.</param>
public void AddRange(T[] items, int offset, int count)
{
for (int i = offset; i < count; i++)
this.Add(items[i]);
}
/// <summary>
/// Reads one item from the cache.
/// </summary>
/// <returns>The item that has been read from the cache.</returns>
/// <exception cref="System.InvalidOperationException">
/// The cache is empty.
/// </exception>
public T Read()
{
T item;
if (!this.queue.TryDequeue(out item))
throw new InvalidOperationException("The cache is empty.");
this.Size--;
if (this.WarnIfRunningLow &&
this.Size < this.LowerWarningLimit)
{
this.CacheIsRunningLow?.Invoke(this, EventArgs.Empty);
}
return item;
}
/// <summary>
/// Peeks the next item from cache.
/// </summary>
/// <returns>The item that has been read from the cache (without deletion).</returns>
/// <exception cref="System.InvalidOperationException">
/// The cache is empty.
/// </exception>
public T Peek()
{
T item;
if (!this.queue.TryPeek(out item))
throw new InvalidOperationException("The cache is empty.");
return item;
}
}
I hope this helps, have fun ;-)

inheritdoc specific exception using cref

How to use <inheritdoc/> to inherit specific exception from another method. Is it possible to do something like this:
/// <summary>
/// Summary
/// </summary>
/// <inheritdoc cref="Method2(int)" select="exception[#cref='CustomException']" />
public int Method1()
{
return Method2(1);
}
/// <summary>
/// Summary...
/// </summary>
/// <exception cref="ArgumentException">
/// ArgumentException...
/// </exception>
/// <exception cref="CustomException">
/// CustomException...
/// </exception>
public int Method2(int count)
{
// do something
}
When I build documentation and open it, there is no exception for Method1. Currently, my workaround is to add id="CustomException" to Method2 and use <inheritdoc cref="Method2(int)" select="exception[#id='CustomException']" />

Converter file section is null

I'm making a BRRES to BFRES converter. For some reason, the MDL0 section seems to be always null when debugging. Here is the code for the Brres file loader:
/// <summary>
/// Represents a set of properties controlling the load of a <see cref="Brres.BrresFile"/>.
/// </summary>
internal class BrresLoaderContext
{
// ---- CONSTRUCTORS -------------------------------------------------------------------------------------------
/// <summary>
/// Initializes a new instance of the <see cref="BrresLoaderContext"/> class for the given <see cref="BrresFile"/>
/// instance using the given <see cref="BinaryDataReader"/>.
/// </summary>
/// <param name="BrresFile">The <see cref="BrresFile"/> instance to load the data in.</param>
/// <param name="reader">The <see cref="BinaryDataReader"/> to use for reading the data.</param>
internal BrresLoaderContext(BrresFile brresFile, BinaryDataReader reader)
{
BrresFile = brresFile;
Reader = reader;
Warnings = new List<string>();
}
// ---- PROPERTIES ---------------------------------------------------------------------------------------------
/// <summary>
/// Gets the <see cref="BrresFile"/> instance which is loaded to.
/// </summary>
internal BrresFile BrresFile
{
get;
private set;
}
/// <summary>
/// Gets the <see cref="BinaryDataReader"/> which is used to read data from the input stream.
/// </summary>
internal BinaryDataReader Reader
{
get;
private set;
}
/// <summary>
/// Gets or sets the warnings raised after loading the Brres file.
/// </summary>
internal List<string> Warnings
{
get;
private set;
}
}
I mostly didn't update comments.
So why is it null?

C# wlanapi.dll issue receiving the signal strength

I am currently trying to retrieve the signal strength from a wireless access point using the wlanapi.dll WlanQueryInterface call.
Here is my api declaration
[DllImport("wlanapi.dll", SetLastError = true)]
private static extern UInt32 WlanQueryInterface(IntPtr hClientHandle, ref Guid pInterfaceGuid, WLAN_INTF_OPCODE OpCode, IntPtr pReserved, out Int32 pdwDataSize, ref IntPtr ppData, IntPtr pWlanOpCodeValueType);
The problem is when I look at the wlanSignalStrength entry in the WLAN_ASSOCIATION_ATTRIBUTES structure it is set to 5400. Microsoft API documentation says that it should be a value between 0 - 100.
Here is my structure declaration:
/// <summary>
/// Structure contains association attributes for a connection.
/// </summary>
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
private struct WLAN_ASSOCIATION_ATTRIBUTES
{
/// <summary>
/// A DOT11_SSID structure that contains the SSID of the association.
/// </summary>
public DOT11_SSID dot11Ssid;
/// <summary>
/// A DOT11_BSS_TYPE value that specifies whether the network is infrastructure or ad hoc.
/// </summary>
public DOT11_BSS_TYPE dot11BssType;
/// <summary>
/// A DOT11_MAC_ADDRESS that contains the BSSID of the association.
/// </summary>
public DOT11_MAC_ADDRESS dot11Bssid;
/// <summary>
/// A DOT11_PHY_TYPE value that indicates the physical type of the association
/// </summary>
public DOT11_PHY_TYPE dot11PhyType;
/// <summary>
/// The position of the DOT11_PHY_TYPE value in the structure containing the list of PHY types.
/// </summary>
public ulong uDot11PhyIndex;
/// <summary>
/// A percentage value that represents the signal quality of the network.
/// </summary>
public Int32 wlanSignalQuality;
/// <summary>
/// Contains the receiving rate of the association.
/// </summary>
public ulong ulRxRate;
/// <summary>
/// Contains the transmission rate of the association.
/// </summary>
public ulong ulTxRate;
}
Here is my call to WlanQueryInterface:
Int32 iDataSize;
IntPtr ppData = IntPtr.Zero;
WLAN_CONNECTION_ATTRIBUTES wcaAttributes = new WLAN_CONNECTION_ATTRIBUTES();
String[] sReturn = new String[4];
if (WlanQueryInterface(pClientHandle, ref gInterfaceGUID, WLAN_INTF_OPCODE.wlan_intf_opcode_current_connection, IntPtr.Zero, out iDataSize, ref ppData, IntPtr.Zero) == ERROR_SUCCESS)
{
wcaAttributes = (WLAN_CONNECTION_ATTRIBUTES)Marshal.PtrToStructure(ppData, typeof(WLAN_CONNECTION_ATTRIBUTES));
sReturn[0] = wcaAttributes.wlanAssociationAttributes.dot11Ssid.ucSSID;
sReturn[1] = wcaAttributes.strProfileName;
sReturn[2] = String.Format("{0:X2}-{1:X2}-{2:X2}-{3:X2}-{4:X2}-{5:X2}", wcaAttributes.wlanAssociationAttributes.dot11Bssid.bOne, wcaAttributes.wlanAssociationAttributes.dot11Bssid.bTwo, wcaAttributes.wlanAssociationAttributes.dot11Bssid.bThree, wcaAttributes.wlanAssociationAttributes.dot11Bssid.bFour, wcaAttributes.wlanAssociationAttributes.dot11Bssid.bFive, wcaAttributes.wlanAssociationAttributes.dot11Bssid.bSix);
sReturn[3] = wcaAttributes.wlanAssociationAttributes.wlanSignalQuality.ToString(); //This returns 5400 when the actual strength is ~99
WlanFreeMemory(ppData);
return sReturn;
}
else
{
throw new Exception("Unable to get interface connected SSID.");
}
Does anyone see what I am doing wrong? Thanks in advance!

I finally got it fixed. I had to set the LayoutKind for the WLAN_ASSOCIATION_ATTRIBUTES to Explicit and set the charset to Unicode.
[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Unicode)]
private struct WLAN_ASSOCIATION_ATTRIBUTES
{
/// <summary>
/// A DOT11_SSID structure that contains the SSID of the association.
/// </summary>
[FieldOffset(0)]
public DOT11_SSID dot11Ssid;
/// <summary>
/// A DOT11_BSS_TYPE value that specifies whether the network is infrastructure or ad hoc.
/// </summary>
[FieldOffset(0x24)]
public DOT11_BSS_TYPE dot11BssType;
/// <summary>
/// A DOT11_MAC_ADDRESS that contains the BSSID of the association.
/// </summary>
[FieldOffset(40)]
public DOT11_MAC_ADDRESS dot11Bssid;
/// <summary>
/// A DOT11_PHY_TYPE value that indicates the physical type of the association
/// </summary>
[FieldOffset(0x30)]
public DOT11_PHY_TYPE dot11PhyType;
/// <summary>
/// The position of the DOT11_PHY_TYPE value in the structure containing the list of PHY types.
/// </summary>
[FieldOffset(0x34)]
public ulong uDot11PhyIndex;
/// <summary>
/// A percentage value that represents the signal quality of the network.
/// </summary>
[FieldOffset(0x38)]
public Int32 wlanSignalQuality;
/// <summary>
/// Contains the receiving rate of the association.
/// </summary>
[FieldOffset(60)]
public ulong ulRxRate;
/// <summary>
/// Contains the transmission rate of the association.
/// </summary>
[FieldOffset(0x40)]
public ulong ulTxRate;
}

Is there is a fluent approach to handling WinForm event?

I was handling yet another KeyDown event in a user control when I wondered if it existed a library for typing fluent code for handling event like
editor.When(Keys.F).IsDown().With(Keys.Control).Do((sender, e) => ShowFindWindow());
Does that exist?

I decided that this was a challenge and that this could let me learn what Fluent is, so I coded a Fluent keyboard class. I don't think I follow all fluent language structure and rules, but it work.
Helper extension method
/// <summary>
/// Provides a set of static (Shared in Visual Basic) methods for starting a fluent expression on a <see cref="System.Windows.Form.Control"/> object.
/// </summary>
public static class ControlExtensions
{
/// <summary>
/// Starts a fluent expression that occurs when a key is pressed.
/// </summary>
/// <param name="control">The control on which the fluent keyboard expression is occuring.</param>
/// <param name="keys">The key when it will happen.</param>
/// <returns>A <see cref="KeyboardFluent"/> object that makes it possible to write a fluent expression.</returns>
public static KeyboardFluent When(this Control control, Keys keys)
{
return new KeyboardFluent(control).When(keys);
}
}
The fluent class
/// <summary>
/// Represents a fluent expression for handling keyboard event.
/// </summary>
public class KeyboardFluent
{
/// <summary>
/// The control on which the fluent keyboard expression is occuring.
/// </summary>
private Control control;
/// <summary>
/// The KeyDown and KeyUp handler.
/// </summary>
private KeyEventHandler keyHandler;
/// <summary>
/// Stores if the IsDown method was called and that the KeyDown event is registered.
/// </summary>
private bool isDownRegistered = false;
/// <summary>
/// Stores if the IsUp method was called and that the KeyUp event is registered.
/// </summary>
private bool isUpRegistered = false;
/// <summary>
/// The list of keys that will make the actions be executed when they are down or up.
/// </summary>
private List<Keys> triggerKeys;
/// <summary>
/// The modifiers keys that must be down at the same time than the trigger keys for the actions to be executed.
/// </summary>
private Keys modifiers;
/// <summary>
/// The list of actions that will be executed when the trigger keys and modifiers are down or up.
/// </summary>
private List<Action<object, KeyEventArgs>> actions;
/// <summary>
/// Initializes a new instance of the <see cref="KeyboardFluent"/> class.
/// </summary>
/// <param name="control">The control on which the fluent keyboard expression is occuring.</param>
public KeyboardFluent(Control control)
{
this.control = control;
this.triggerKeys = new List<Keys>();
this.actions = new List<Action<object, KeyEventArgs>>();
this.keyHandler = new KeyEventHandler(OnKeyHandler);
}
/// <summary>
/// Handles the KeyDown or KeyUp event on the control.
/// </summary>
/// <param name="sender">The control on which the event is occuring.</param>
/// <param name="e">A <see cref="KeyEventArgs"/> that gives information about the keyboard event.</param>
private void OnKeyHandler(object sender, KeyEventArgs e)
{
if (this.triggerKeys.Contains(e.KeyCode) && e.Modifiers == this.modifiers)
{
this.actions.ForEach(action => action(sender, e));
}
}
/// <summary>
/// Makes the keyboard event occured when a key is pressed down.
/// </summary>
/// <returns>Returns itself to allow a fluent expression structure.</returns>
public KeyboardFluent IsDown()
{
if (!isDownRegistered)
{
this.control.KeyDown += this.keyHandler;
isDownRegistered = true;
}
return this;
}
/// <summary>
/// Makes the keyboard event occured when a key is pressed up.
/// </summary>
/// <returns>Returns itself to allow a fluent expression structure.</returns>
public KeyboardFluent IsUp()
{
if (!isUpRegistered)
{
this.control.KeyUp += this.keyHandler;
isUpRegistered = true;
}
return this;
}
/// <summary>
/// Creates a new trigger on a key.
/// </summary>
/// <param name="key">The key on which the actions will occur.</param>
/// <returns>Returns itself to allow a fluent expression structure.</returns>
public KeyboardFluent When(Keys key)
{
this.triggerKeys.Add(key);
return this;
}
/// <summary>
/// Adds a modifier filter that is checked before the action are executed.
/// </summary>
/// <param name="modifiers">The modifier key.</param>
/// <returns>Returns itself to allow a fluent expression structure.</returns>
public KeyboardFluent With(Keys modifiers)
{
this.modifiers |= modifiers;
return this;
}
/// <summary>
/// Executes the action when the specified keys and modified are either pressed down or up.
/// </summary>
/// <param name="action">The action to be executed.</param>
/// <returns>Returns itself to allow a fluent expression structure.</returns>
public KeyboardFluent Do(Action<object, KeyEventArgs> action)
{
this.actions.Add(action);
return this;
}
}
I can now type
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
this.When(Keys.F).With(Keys.Control).IsDown().Do((sender, e) => MessageBox.Show(e.KeyData.ToString()));
}
}
and it will display the messagebox only when Ctrl+F is down.

I downloaded the Reactive Extension Framework for the .Net Framework 3.5 SP1.
I was able to do the same:
Observable.FromEvent<KeyEventArgs>(this, "KeyDown")
.Where(e => e.EventArgs.KeyCode == Keys.F && e.EventArgs.Modifiers == Keys.Control)
.Subscribe(e => MessageBox.Show(e.EventArgs.KeyData.ToString()));
This is some pretty powerful stuff.

.NET 4 will introduce the reactive framework (IObservable<T>, IObserver<T> and helper extension types) which should offer exactly this.