In C# I want to record an audio stream
I am doing something along the lines of:
HttpWebRequest req;
req = (HttpWebRequest)WebRequest.Create("http://url.com/stream");
Webresponse resp = req.GetResponse();
Stream s = resp.GetResponseStream();
fs = File.Exists(fileName)
? new FileStream(fileName, FileMode.Append)
: new FileStream(fileName, FileMode.Create);
byte[] buffer = new byte[4096];
while (s.CanRead)
{
Array.Clear(buffer, 0, buffer.Length);
total += s.Read(buffer, 0, buffer.Length);
fs.Write(buffer, 0, buffer.Length);
}
and the file size grows but can't be played back by VLC or any other program.
This is not my exact code I do a lot of error checking etc, but this gives the general idea.
Array.Clear(buffer, 0, buffer.Length);
total += s.Read(buffer, 0, buffer.Length);
fs.Write(buffer, 0, buffer.Length);
You do not have to clear the whole array before you read - there's no point in doing this. But you have to check how many bytes you actually read, there's no guarantee the whole array is filled every time (and it probably won't):
int bytesRead = s.Read(buffer, 0, buffer.Length);
fs.Write(buffer, 0, bytesRead);
total+=bytesRead;
Also whether the file plays (even when it is not corrupted anymore once you fix the file writing code) back depends on the format that you are downloading - what codec / file type is it?
THe problem is the streamed bits don't have context. When you stream to an application, there is a tacit agreement that you are dealing with file type X and the streaming program then tries to play the bits.
WHen you stream to a file, you have to add the context. One of the most important bits is the header identifying the type of file and other information.
If you can add the header, you can play the file from the file system. The header will not be part of the stream, as the server and client have agreed on what type fo file it it is already.
If you create a streaming player, you can possibly play back the bits you saved, as you negotiate the type. BUt to have it automagically work from file, you have to add the header.
Trying to save streamed MP3 audio to disk is essentially impossible without a detailed understanding of both the stream format and the file format for MP3. What you're getting from the stream is a series of "windowed" chunks of audio converted to frequency domain; the player receiving the stream converts the chunks back into time-domain audio on the fly and plays them one after the other.
To make an MP3 file, you would have to first write out a header containing the format information and then write each chunk of data. But most likely the format for storing these chunks in a file is different from the way in which they're compacted into a stream.
Sorry, but I would seriously advise you to give this up. One major reason that music services stream instead of offering file downloads is specifically because it's so difficult to save an MP3-type stream to disk (it would be a trivial matter to save an uncompressed audio stream to a WAV file).
Related
I have a large file on disk whose contents I need to store in a SqlServer database as a VARBINARY(MAX) field. By "contents", I mean using something like File.ReadAllBytes(). (I understand the pitfalls of this, but it's the current mode of operations, so I have to deal with it for now.)
I found this answer, which provides a way to stream a large Byte[] by using UPDATE.WRITE:
How to I serialize a large graph of .NET object into a SQL Server BLOB without creating a large buffer?
However, simply reading the contents of a large file into a Byte[] in memory leads to this problem:
OutOfMemoryException when I read 500MB FileStream
I might be overlooking something obvious, but I just can't work out how should I go about getting from a large file on disk, to the resulting storage into the database.
Using some hints I got from these two pages, I have an answer that works:
http://www.syntaxwarriors.com/2013/stream-varbinary-data-to-and-from-mssql-using-csharp/ (this looks a lot like the serialize SO answer, but there's more here...not sure who copied who!).
How do I copy the contents of one stream to another?
Basically, it uses the same methodology as the answer about serializing Blobs, but instead of using BinaryFormatter (a class I'm not fond of anyhow), it creates a FileStream that takes the path to the file, and an extension method to copy that stream into the target stream, or BlobStream, as the example named it.
Here's the extension:
public static class StreamEx
{
public static void CopyTo(this Stream Input, Stream Output)
{
var buffer = new Byte[32768];
Int32 bytesRead;
while ((bytesRead = Input.Read(buffer, 0, buffer.Length)) > 0)
Output.Write(buffer, 0, bytesRead);
}
}
So the trick was to link two streams, copying the data from one to another in chunked fashion, as noted in the comments.
I've been working on a project recently that involves a lot of FileStreaming, something which I've not really touched on before.
To try and better acquaint myself with the principles of such methods, I've written some code that (theoretically) downloads a file from one dir to another, and gone through it step by step, commenting in my understanding of what each step achieves, like so...
Get fileinfo object from DownloadRequest Object
RemoteFileInfo fileInfo = svr.DownloadFile(request);
DownloadFile method in WCF Service
public RemoteFileInfo DownloadFile(DownloadRequest request)
{
RemoteFileInfo result = new RemoteFileInfo(); // create empty fileinfo object
try
{
// set filepath
string filePath = System.IO.Path.Combine(request.FilePath , #"\" , request.FileName);
System.IO.FileInfo fileInfo = new System.IO.FileInfo(filePath); // get fileinfo from path
// check if exists
if (!fileInfo.Exists)
throw new System.IO.FileNotFoundException("File not found",
request.FileName);
// open stream
System.IO.FileStream stream = new System.IO.FileStream(filePath,
System.IO.FileMode.Open, System.IO.FileAccess.Read);
// return result
result.FileName = request.FileName;
result.Length = fileInfo.Length;
result.FileByteStream = stream;
}
catch (Exception ex)
{
// do something
}
return result;
}
Use returned FileStream from fileinfo to read into a new write stream
// set new location for downloaded file
string basePath = System.IO.Path.Combine(#"C:\SST Software\DSC\Compilations\" , compName, #"\");
string serverFileName = System.IO.Path.Combine(basePath, file);
double totalBytesRead = 0.0;
if (!Directory.Exists(basePath))
Directory.CreateDirectory(basePath);
int chunkSize = 2048;
byte[] buffer = new byte[chunkSize];
// create new write file stream
using (System.IO.FileStream writeStream = new System.IO.FileStream(serverFileName, FileMode.OpenOrCreate, FileAccess.ReadWrite))
{
do
{
// read bytes from fileinfo stream
int bytesRead = fileInfo.FileByteStream.Read(buffer, 0, chunkSize);
totalBytesRead += (double)bytesRead;
if (bytesRead == 0) break;
// write bytes to output stream
writeStream.Write(buffer, 0, bytesRead);
} while (true);
// report end
Console.WriteLine(fileInfo.FileName + " has been written to " + basePath + " - Done!");
writeStream.Close();
}
What I was hoping for is any clarification or expansion on what exactly happens when using a FileStream.
I can achieve the download, and now I know what code I need to write in order to perform such a download, but I would like to know more about why it works. I can find no 'beginner-friendly' or step by step explanations on the web.
What is happening here behind the scenes?
A stream is just an abstraction, fundamentally it works like a pointer within a collection of data.
Take the example string of "Hello World!" for example, it is just a collection of characters, which are fundamentally just bytes.
As a stream, it could be represented to have:
A length of 12 (possibly more including termination characters etc)
A position in the stream.
You read a stream by moving the position around and requesting data.
So reading the text above could be (in pseudocode) seen to be like this:
do
get next byte
add gotten byte to collection
while not the end of the stream
the entire data is now in the collection
Streams are really useful when it comes to accessing data from sources such as the file system or remote machines.
Imagine a file that is several gigabytes in size, if the OS loaded all of that into memory any time a program wanted to read it (say a video player), there would be a lot of problems.
Instead, what happens is the program requests access to the file, and the OS returns a stream; the stream tells the program how much data there is, and allows it to access that data.
Depending on implementation, the OS may load a certain amount of data into memory ahead of the program accessing it, this is known as a buffer.
Fundamentally though, the program just requests the next bit of data, and the OS either gets it from the buffer, or from the source (e.g. the file on disk).
The same principle applies to streams between different computers, except requesting the next bit of data may very well involve a trip to the remote machine to request it.
The .NET FileStream class and the Stream base class, all just defer to the windows systems for working with streams in the end, there's nothing particularly special about them, it's just what you can do with the abstraction that makes them so powerful.
Writing to a stream is just the same, but it just puts data into the buffer, ready for the requester to access.
Infinite Data
As a user pointed out, streams can be used for data of indeterminate length.
All stream operations take time, so reading a stream is typically a blocking operation that will wait until data is available.
So you could loop forever while the stream is still open, and just wait for data to come in - an example of this in practice would be a live video broadcast.
I've since located a book - C# 5.0 All-In-One For Dummies - It explains everything about all Stream classes, how they work, which one is most appropriate and more.
Only been reading about 30 minutes, already have such a better understanding. Excellent guide!
I am working on developing an HTTP Server/Client and I can currently send small files over it such as .txt files and other easy to read files that do not require much memory. However when I want to send a larger file say a .exe or large .pdf I get memory errors. This are occurring from the fact that before I try to send or receive a file I have to specify the size of my byte[] buffer. Is there a way to get the size of the buffer while reading it from stream?
I want to do something like this:
//Create the stream.
private Stream dataStream = response.GetResponseStream();
//read bytes from stream into buffer.
byte[] byteArray = new byte[Convert.ToInt32(dataStream.Length)];
dataStream.read(byteArray,0,byteArray.Length);
However when calling "dataStream.Length" it throws the error:
ExceptionError: This stream does not support seek operations.
Can someone offer some advice as to how I can get the length of my byte[] from the stream?
Thanks,
You can use CopyTo method of the stream.
MemoryStream m = new MemoryStream();
dataStream.CopyTo(m);
byte[] byteArray = m.ToArray();
You can also write directly to file
var fs = File.Create("....");
dataStream.CopyTo(fs);
The network layer has no way of knowing how long the response stream is.
However, the server is supposed to tell you how long it is; look in the Content-Length response header.
If that header is missing or incorrect, you're out of luck; you'll need to keep reading until you run out of data.
I get from server images and videos by stream.
Now I'm saving it:
Stream str = client.GetFile(path);
using (var outStream = new FileStream(#"c:\myFile.jpg", FileMode.Create))
{
var buffer = new byte[4096];
int count;
while ((count = str.Read(buffer, 0, buffer.Length)) > 0)
{
outStream.Write(buffer, 0, count);
}
}
I can be jpg, mpg, flv and a lot of other multimedia types (Before I get stream I know what is a extension of this file).
Now I want to not save it , but run direct from stream.
Examples:
I get stream which is mybirthay.avi and I call my method RunFile(stream) and I think this method should works like System.Diagnostics.Process.Start(path), so my stream should be opened by default program in my SO for example allplayer.
I get stream from myfile.jpg and it is opening by irfanview,
etc...
Is it possible ??
If you are set on not saving the stream to the disk, you could use something like Eldos' Callback File System: http://www.eldos.com/cbfs/
Or, use a ramdisk, save your stream there, and shell to that file location.
Windows puts up a wall between processes so they cannot directly access each other's memory without going through privileged debug-like API functions like ReadProcessMemory. This is what keeps the operating system stable and secure. But spells doom for what you are trying to accomplish.
You'll need to use a file. It won't be much slower than direct memory access, the file system cache takes care of that.
I'm trying to stream a pdf file. Most of the files open without any problems but sometimes it fails. When it fails, it also looks like file size is smaller than the original one. For example, I was trying to open a 47K file but when the streamed output to the browser it's only 44.5K. When check the size of the stream (result.FileStream), it's 47K like it supposed to be.
I'm using Stream.Read to output the file to the browser. When I had a problem, I was using buffer size of 10000 bytes. However, when I changed the buffer size from 10000 to 1000 the problem disappear and I was able to the file. I cannot explain why the change in the buffer size makes the streaming behave differently.
Here's the code I'm using result.FileStream is of type Stream:
using (result.FileStream)
{
int length;
const int byteSize = 1000;
var buffer = new byte[byteSize];
while ((length = result.FileStream.Read(buffer, 0, byteSize)) > 0 && Response.IsClientConnected)
{
Response.OutputStream.Write(buffer, 0, length);
Response.Flush();
}
}
Response.Close();
Please enlighten me because I definitely don't understand something.
You're using Response.Close(), which seems to be much more evil then the documentation would make you believe.
http://forums.iis.net/t/1152058.aspx