Been pulling my hair out over what should have been a quick and easy task.
I have a self-hosted WCF service in which I need to implement real-time video transcoding, the transcoding isn't a problem as such, using FFMpeg to a local temp file.
Quick sample of what my code looks like;
public Stream StreamMedia(int a)
{
String input = #"\media\" + a + ".mkv";
String output = #"\temp\transcoded\" + a + DateTime.Now.Ticks.ToString() + ".wmv";
ProcessStartInfo pi = new ProcessStartInfo("ffmpeg.exe");
pi.Arguments = "-i " + input + " -y -ab 64k -vcodec wmv2 -b 800k -mbd 2 -cmp 2 -subcmp 2 -s 320x180 -f asf " + output;
Process p = new Process;
p.StartInfo = pi;
p.Start();
Thread.Sleep(2500);
return new FileStream(output, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
}
The problem I am facing is that the returned Stream only gives me what was written to the file when it is returned - resulting in a rather short video file :)
I've played around with the obvious here, but no matter what I do it will only return what's available there and then.
What I need to happen is for the Stream to be returned with no respect to the actual current lenght of the output file - there is other code involved which makes sure the data is never sent to the client faster than what FFMpeg manages to encode, so basically I just need an open-ended stream.
Any takers?
One solution would be to create your custom Stream class which would wrap around the file from disk; BUT, there's also the concurrency issue, meaning that you need some locking mechanism as for the writing process (video transcoder) to properly share the file with your FileStream.
Is it possible for your transcoder to create multi-volume output? If so, then your lucky and this would work with (almost) no pain at all, just do the streaming of the volume N, then the transcoder writes the volume N + 1, and you'll not have any file access concurrency issues.
happy coding!
- Adrian
The simplest may be to use the Streaming Media service that is built into the operating system. See: http://technet.microsoft.com/en-us/windowsserver/dd448620
The other way to do it would be not to read from the file, but send the stream that is writing to the file, straight out to the client.
What is obvious is, this cannot be done via file system. You need a dynamic solution.
You can do it via your own made media service. In your case it could be a WCF or windows service.
This service should be responsible for both writing to the file (as data receives) and streaming.
Related
I'm currently using the answer to this question to pipe a string of bitmaps into ffmpeg. It requires redirecting StandardInput stream and writing to it.
Is something similar available with NRECO.VideoConverter? Is there a way to get either access to the running process, or access to just the StandardInput base stream?
If you want to provide input data to stdin or read output data from stdout (or both) you may use ConvertLiveMedia method; it has overloads for different usage scenarios:
var videoConv = new FFMpegConverter();
var ffMpegTask = videoConv.ConvertLiveMedia(
"rawvideo",
h264stream,
"h264",
new ConvertSettings() {
CustomInputArgs = String.Format(" -pix_fmt bgr24 -video_size 640x480 -framerate 5 ",
frameBmp.Width, frameBmp.Height)
});
ffMpegTask.Start();
ffMpegTask.Write( bmpBytes ); // call N times for each input frame. Image size should be exactly 640x480
ffMpegTask.Stop();
You can adopt this code snippet for "image2pipe" if needed.
I need to stream the standard output from ffmpeg.exe directly to a web response stream on the fly. I have no problem setting up the command line with Process/RedirectStandardOutput/etc to pipe the output stream.
However, the problem seems to be the output from Process.StandardOutput is not in the correct raw format. Something like this (pseudocode):
var ffpsi = new ProcessStartInfo("ffmpeg.exe", "-i input.mp3 -acodec copy -f mp3 -")
ffpsi.UseShellExecute = false;
ffpsi.RedirectStandardOutput = true;
var ffmpegProcess = new Process();
ffpmpegProcess.StartInfo = ffpsi;
ffmpegProcess.Start();
var outputFile = new FileStream("out.mp3");
ffmpegProcess.StandardOutput.BaseStream.CopyTo(outputFile);
creates a file a little larger than the original and it's clearly not a valid MP3.
I've played with various encodings copying strategies with the base streams and get data from the async callbacks, but nothing seems to work.
Any ideas here? I guess this comes down to how to get the raw binary output from ffmpeg stdout into a .NET stream I can pass to a response stream?
I will like to compress a file before sending it through the network. I think the best approach is 7zip because it is free and open source.
How I use 7zip with .net?
I know that 7zip is free and that they have the source code in c# but for some reason it is very slow on c# so I rather call the dll 7z.dll that comes when installing 7zip for performance reasons. So the way I am able to eassily marshal and call the methods in 7z.dll is with the help of the library called sevenzipsharp . For example adding that dll to my project will enable me to do:
// if you installed 7zip 64bit version then make sure you change plataform target
// like on the picture I showed above!
SevenZip.SevenZipCompressor.SetLibraryPath(#"C:\Program Files\7-Zip\7z.dll");
var stream = System.IO.File.OpenRead(#"SomeFileToCompress.txt");
var outputStream = System.IO.File.Create("Output.7z");
SevenZip.SevenZipCompressor compressor = new SevenZip.SevenZipCompressor();
compressor.CompressionMethod = SevenZip.CompressionMethod.Lzma2;
compressor.CompressionLevel = SevenZip.CompressionLevel.Ultra;
compressor.CompressStream(stream, outputStream);
that's how I use 7zip within c#.
Now my question is:
I will like to send a compressed file over the network. I know I could compress it first then send it. The file is 4GB so I will have to wait a long time for it to compress. I will be wasting a lot of space on hard drive. then I will finally be able to send it. I think that is to complicated. I was wondering how it will be possible to send the file meanwhile it is being compressed.
It seems to be a problem with SevenZipSharp:
Have you considered an alternate library - one that doesn't even require 7-Zip to be installed / available?
From the description posted at http://dotnetzip.codeplex.com/ :
creating zip files from stream content, saving to a stream, extracting
to a stream, reading from a stream
Unlike 7-Zip, DotNetZip is designed to work with C# / .Net.
Plenty of examples - including streaming, are available at http://dotnetzip.codeplex.com/wikipage?title=CS-Examples&referringTitle=Examples .
Another option is to use the 7-Zip Command Line Version (7z.exe), and write to/read from standard in/out. This would allow you to use the 7-Zip file format, while also keeping all of the core work in native code (though there likely won't be much of a significant difference).
Looking back at SevenZipSharp:
Since the 0.29 release, streaming is supported.
Looking at http://sevenzipsharp.codeplex.com/SourceControl/changeset/view/59007#364711 :
it seems you'd want this method:
public void CompressStream(Stream inStream, Stream outStream)
Thank you for considering performance here! I think way too many people would do exactly what you're trying to avoid: compress to a temp file, then do something with the temp file.
CompressStream threw an exception. My code is as follows:
public void TestCompress()
{
string fileToCompress = #"C:\Users\gary\Downloads\BD01.DAT";
byte[] inputBytes = File.ReadAllBytes(fileToCompress);
var inputStream = new MemoryStream(inputBytes);
byte[] zipBytes = new byte[38000000]; // this memory size is large enough.
MemoryStream outStream = new MemoryStream(zipBytes);
string compressorEnginePath = #"C:\Engine\7z.dll";
SevenZipCompressor.SetLibraryPath(compressorEnginePath);
compressor = new SevenZip.SevenZipCompressor();
compressor.CompressionLevel = CompressionLevel.Fast;
compressor.CompressionMethod = CompressionMethod.Lzma2;
compressor.CompressStream(inputStream, outputStream);
inputStream.Close();
outputStream.Close();
The exception messages:
Message: Test method Test7zip.UnitTest1.TestCompress threw exception:
SevenZip.SevenZipException: The execution has failed due to the bug in the SevenZipSharp.
Please report about it to http://sevenzipsharp.codeplex.com/WorkItem/List.aspx, post the release number and attach the archive
I've been trying for a while now to set up an ftp connection with our mainframe via c#. So far I've been able to connect and upload a file to the mainframe, which is great, but the file has the wrong record length and is set up as variable, instead of fixed record length. After quite a bit a research I have come to the same conclusion as most people, that the ftpwebresponse function will not allow "qoute SITE" commands to be issued to the mainframe. If i'm wrong please don't hesitate to correct me, heres the code i'm using:
private void ftpButton_Click(object sender, EventArgs e)
{
string ftphost = "ftpSite";
string user = "myUser";
string pwd= "myPass";
string ftpfileName = "test.file2";
string inputfilePath="d:\\documents and settings\\gheff\\My Documents\\Visual Studio 2005\\Projects\\biWeeklyJob\\BiWeekly\\bin\\Debug\\file2.TXT";
string ftpfullpath = String.Format("ftp://{0}//'{1}'", ftphost,ftpfileName);
try
{
FtpWebRequest ftp = (FtpWebRequest)FtpWebRequest.Create(ftpfullpath);
ftp.Credentials = new NetworkCredential(user,pwd);
ftp.KeepAlive = true;
ftp.UseBinary = false; //Use ascii.
ftp.Method = WebRequestMethods.Ftp.UploadFile;
FileStream fs = File.OpenRead(inputfilePath);
byte[] buffer = new byte[fs.Length];
fs.Read(buffer, 0, buffer.Length);
fs.Close();
Stream ftpstream = ftp.GetRequestStream();
ftpstream.Write(buffer, 0, buffer.Length);
ftpstream.Close();
}
catch (WebException ex)
{
String status = ((FtpWebResponse)ex.Response).StatusDescription;
throw new Exception(status);
}
}
So undeterred, I tried to think of another approach, I have successfully used the command line to upload a file, change the file properties and also changed the filetype so I can send JCL jobs to the mainframe. I am now tyring to implement this into my existing c# application, but proving harder than I thought.
I am by no means an expert in c#, but can use answers to then go and do more research to gain a better understanding.
I have seen this piece of code C# cmd excute commands, but looking around it seems that only one command can be issued from this.
So my question is, I know it took a while to get there, is it possible to run the follow commands on the command prompt without the use of a batch file as i'm looking some feedback as the process runs?
open "cmd.exe"
type "ftp"
type "FTPserver"
Type "USERNAME"
type "PASSWORD"
Then once connection has been established
then run pre-defined commands i.e upload files, upload and run JCL jobs.
I think what i'm looking for is somthing that will write text to cmd.exe but keep the session?
I hope this makes sense.
Any help would be greatly appreciated.
Thanks
So here is code you are asking for nevertheless I doubt that it solve your problem.
var cmd= new Process();
cmd.StartInfo.FileName = "cmd.exe";
cmd.StartInfo.UseShellExecute = false;
cmd.StartInfo.RedirectStandardInput = true;
cmd.Start();
using (var stdin = cmd.StandardInput)
{
stdin.WriteLine("ftp");
stdin.WriteLine("FTPserver");
stdin.WriteLine("USERNAME");
stdin.WriteLine("PASSWORD");
}
cmd.WaitForExit();
cmd.Close();
Try this cmd:
FTP -s:ftpcommandfilename url
sense the follow mode can run one command only, you can write the ftp cmd in a file and use this cmd to execute them.
WriteLine("FTP -s:ftpcommandfilename url");
By the way, I am also finding the multi-ftp-cmd-line-in-code mode, if you have got it, sharing?
I used this CodeProject project a few years ago for a simple application that needed to send some files. The thing that may help you is that it uses a socket to talk to the other server. You can send raw FTP commands which should sidestep the problems you're having with the FtpWebRequest object, without having to "shell out" and use the ftp command line program.
Another approach you could try is to submit a jcl job to allocate the dataset with the record length, block size, et al, that your process needs.
Once that job complets, "put" your data into the dataset you just allocated correctly. The z/OS ftp server should pick up the attributes from your existing dataset.
Have you considered using a commercial FTP client that supports a COM interface and is able to send raw protocol commands? This might not be economical if you are writing software to redistribute but if you are only trying to automate a recurring task for an internal IT project it could definately be worth it in terms of your time as a developer. Robo-FTP might be a good choice for connecting to a mainframe because it has built in ASCII to EBCDIC translation.
I'm upload big files dividing its on chunks(small parts) on my ASMX webservice(asmx doesn't support streaming, I not found another way):
bool UploadChunk(byte[] bytes, string path, string md5)
{
...
using (FileStream fs = new FileStream(tempPath, FileMode.Append) )
{
fs.Write( bytes, 0, bytes.Length );
}
...
return status;
}
but on some files after ~20-50 invokes I catch this error: The process cannot access the file because it is being used by another process.
I suspect that this related with Windows can't realize the file. Any idea to get rid of this boring error?
EDIT
the requests executes sequentially and synchronously
EDIT2
client code looks like:
_service.StartUpload(path);
...
do
{
..
bool status = _service.UploadChunk(buf, path, md5);
if(!status)return Status.Failed;
..
}
while(bytesRead > 0);
_service.CheckFile(path, md5);
Each request is handled independently. The process still accessing the file may be the previous request.
In general, you should use file transfer protocols to transfer files. ASMX is not good for that.
And, I presume you have a good reason to not use WCF?
Use WhoLockMe at the moment the error occurs to check who is using the file. You could put the application into debug mode and hold the break point to do this. In all probability it will be your process.
Also try adding a delay after each transfer (and before the next) to see if it helps. Maybe your transfers are too fast and the stream is still in use or being flushed when the next transfer comes in.
Option 1: Get the requirements changed so you don't have to do this using ASMX. WCF supports a streaming model that I'm about to experiment with, but it should be much more effective for what you want.
Option 2: Look into WSE 3.0. I haven't looked at it much, but I think it extends ASMX web services to support things like DIME and MTOM which are designed for transferring files so that may help.
Option 3: Set the system up so that each call writes a piece of the file into a different filename, then write code to rejoin everything at the end.
use this for creating a file
if you want to append something then add FileMode.Append
var filestreama = new FileStream(name, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);