Develop Reference

C# How to create in-memory video chunks

I need to record screen and on event like button click it should save last 60 seconds of recording.
I know how to capture screenshots but I have problems with converting the images to the video. I want to save the final video file and everything else should be in memory operations.
Currently I save captures as JPEG to memory stream and when the event is fired then I convert the images to the video files. But there is tripple conversion: Bitmap[] -> JPEG[] -> Bitmap[] -> Video. That seems ineffective.
When I googled I found only how to save video file to file system. For example Accord library have VideoFileWritter (example for version 3.8.2 alpha)
Bitmap bitmap; //bitmap object with screenshot
List<byte[]> data = new List<byte[]>();
// fill data with JPEG images (called periodically)
using (var ms = new MemoryStream())
{
bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
data.Add(ms.GetBuffer());
}
// create video file (called on demand)
using (VideoFileWriter videoWriter = new VideoFileWriter())
{
videoWriter.BitRate = videoBitRate;
videoWriter.FrameRate = Settings.CurrentFramesPerSeconds;
videoWriter.Width = 1920;
videoWriter.Height = 1080;
videoWriter.VideoCodec = VideoCodec.H264;
videoWriter.VideoOptions["crf"] = "18"; // visually lossless
videoWriter.VideoOptions["preset"] = "veryfast";
videoWriter.VideoOptions["tune"] = "zerolatency";
videoWriter.VideoOptions["x264opts"] = "no-mbtree:sliced-threads:sync-lookahead=0";
videoWriter.Open(Path.Combine(dirName, "output.avi"));
foreach (var frame in data)
{
using (var bmpStream = new MemoryStream(frame))
using (var img = Image.FromStream(bmpStream))
using (var bmp = new Bitmap(img))
videoWriter.WriteVideoFrame(bmp);
}
}
How to create video chunks in memory? I guess it should be CPU and memory more efficient than my current solution.
Or is there more efficient way to record last few seconds of screen without file system usage?
I don't want to use file system because only ssd is installed in computer.

Image stored in SQL Server database after resize with Imagesharp not showing in html

I have a .NET Core 2.2 web app where I am storing user images in a SQL Server database. When I save the file as is, it displays in the web page with no problems.
After that I wanted to resize the image when the user uploads the file and installed Imagesharp package and used the following code to resize the image. The code runs perfectly but when the use logs in. the image is not displayed.
My code to resize and store the image into the database is:
Image Processing Methods
public Stream ResizeImage(Image<Rgba32> sourceImage, int destHeight, int destWidth)
{
sourceImage.Mutate<Rgba32>(ctx => ctx.Resize(destWidth,destHeight));
Stream outputStream = new MemoryStream();
sourceImage.Save(outputStream, new JpegEncoder());
return outputStream;
/*return sourceImage.
.Crop(new Rectangle(sourceX, sourceY, sourceWidth,
sourceHeight))
.Resize(destinationWidth, destinationHeight);*/
}
My code to store the image:
if (ModelState.IsValid)
{
if (ProfileImage != null)
{
var stream = ProfileImage.OpenReadStream();
Image<Rgba32> theimage = _imageProcessor.GetImageFromStream(stream);
var outstream = _imageProcessor.ResizeImage(theimage, 100, 100);
//Input.Image = commonData.ConvertToBytes(ProfileImage);
Input.Image = commonData.ConvertStreamToBytes(outstream);
}
var user = new ApplicationUser
{
UserName = Input.Email,
Email = Input.Email.ToLower().Trim(),
FirstName = Input.FirstName,
LastName = Input.LastName,
Image = Input.Image,
AccountDisabled = true
};
}
I am trying to display the image in html after converting it to the base64string as follows:
userView.UserImageBytes = new string(Convert.ToBase64String(userView.TheUser.Image));
My HTML markup to display the image:
<img src="data:image/jpg;base64,#Model.UserImageBytes" alt="No Image" />
The above code works fine but the stored image is not displayed.
How to display the image properly. Any help is appreciated.

After this line:
sourceImage.Save(outputStream, new JpegEncoder());
outputStream is has its .Position set after the last written byte. You might, for instance, write more data to that stream after the image. In order for another bit of code to read the image, you need to reposition outputStream before you return it. eg
outputStream.Position = 0;
return outputStream;

Using SharpAvi to save screenshots to an AVI produces 100 frames of blank video

My game takes a screenshot each game loop and stores it memory. The user can then press "print screen" to trigger "SaveScreenshot" (see code below) to store each screenshot as a PNG and also compile them into an AVI using SharpAvi. The saving of images works fine, and a ~2sec AVI is produced, but it doesn't show any video when played. It's just the placeholder VLC Player icon. I think this is very close to working, but I can't determine what's wrong. Please see my code below. If anyone has any ideas, I'd be very appreciative!
private Bitmap GrabScreenshot()
{
try
{
Bitmap bmp = new Bitmap(this.ClientSize.Width, this.ClientSize.Height);
System.Drawing.Imaging.BitmapData data =
bmp.LockBits(this.ClientRectangle, System.Drawing.Imaging.ImageLockMode.WriteOnly,
System.Drawing.Imaging.PixelFormat.Format24bppRgb);
GL.ReadPixels(0, 0, this.ClientSize.Width, this.ClientSize.Height, PixelFormat.Bgr, PixelType.UnsignedByte,
data.Scan0);
bmp.UnlockBits(data);
bmp.RotateFlip(RotateFlipType.RotateNoneFlipY);
return bmp;
} catch(Exception ex)
{
// occasionally getting GDI generic exception when rotating the image... skip that one.
return null;
}
}
private void SaveScreenshots()
{
var directory = "c:\\helioscreenshots\\";
var rootFileName = string.Format("{0}_", DateTime.UtcNow.Ticks);
var writer = new AviWriter(directory + rootFileName + ".avi")
{
FramesPerSecond = 30,
// Emitting AVI v1 index in addition to OpenDML index (AVI v2)
// improves compatibility with some software, including
// standard Windows programs like Media Player and File Explorer
EmitIndex1 = true
};
// returns IAviVideoStream
var aviStream = writer.AddVideoStream();
// set standard VGA resolution
aviStream.Width = this.ClientSize.Width;
aviStream.Height = this.ClientSize.Height;
// class SharpAvi.KnownFourCCs.Codecs contains FOURCCs for several well-known codecs
// Uncompressed is the default value, just set it for clarity
aviStream.Codec = KnownFourCCs.Codecs.Uncompressed;
// Uncompressed format requires to also specify bits per pixel
aviStream.BitsPerPixel = BitsPerPixel.Bpp32;
var index = 0;
while (this.Screenshots.Count > 0)
{
Bitmap screenshot = this.Screenshots.Dequeue();
var screenshotBytes = ImageToBytes(screenshot);
// write data to a frame
aviStream.WriteFrame(true, // is key frame? (many codecs use concept of key frames, for others - all frames are keys)
screenshotBytes, // array with frame data
0, // starting index in the array
screenshotBytes.Length); // length of the data
// save it!
// NOTE: compared jpeg, gif, and png. PNG had smallest file size.
index++;
screenshot.Save(directory + rootFileName + index + ".png", System.Drawing.Imaging.ImageFormat.Png);
}
// save the AVI!
writer.Close();
}
public static byte[] ImageToBytes(Image img)
{
using (var stream = new MemoryStream())
{
img.Save(stream, System.Drawing.Imaging.ImageFormat.Png);
return stream.ToArray();
}
}

From what I see, you're providing the byte-array in png-encoding, yet the stream is configured as KnownFourCCs.Codecs.Uncompressed.
Furthermore, from the manual:
AVI expects uncompressed data in format of standard Windows DIB, that is bottom-up bitmap of the specified bit-depth. For each frame, put its data in byte array and call IAviVideoStream.WriteFrame()
Next, all encoders expect input image data in specific format. It's BGR32 top-down - 32 bits per pixel, blue byte first, alpha byte not used, top line goes first. This is the format you can often get from existing images. [...] So, you simply pass an uncompressed top-down BGR32
I would retrieve the byte-array directly from the Bitmap using LockBits and Marshal.Copy as described in the manual.

A generic error occurred in GDI+ while converting dataimage string to image

I am trying to save image date to physical file
Below is image data which I got from a jpeg image (via some browser response):
data:image/jpeg;base64,/9j/4AAQSkZJRgABAgEASABIAAD/.....blah blah .....//2Q==
Below is the code I am using to save image data string to Image
Image image = LoadImage(dataURL);
image.Save(saveLocation);
image.Dispose();
public Image LoadImage(string imageString)
{
imageString = imageString.Substring(imageString.IndexOf(',') + 1);
byte[] bytes = Convert.FromBase64String(imageString);
Image image;
using (MemoryStream ms = new MemoryStream(bytes))
{
image = Image.FromStream(ms);
}
return image;
}
Getting the following exception at image.Save(saveLocation); line
A generic error occurred in GDI+.
I have no issues when the image source with png images with the same code but with jpeg images no matter what the size of jpeg is I am getting that exception every time.
I am able to save the png images with same code at same location.
Edit: Please note that I am getting just data image string from browser (which I am capturing via clipboard of browser) i.e I am not getting bytes from client.
Is there any limit for the bytes which can be converted to image stream and then save back to physical file?
Is there any other approach to do the same?

There doesn't seem to be any reason to go through Image at all here.
Assuming the posted data is valid, just write the content to a file directly:
SaveImage(dataURL, saveLocation);
public bool SaveImage(string imageString, string location)
{
try {
imageString = imageString.Substring(imageString.IndexOf(',') + 1);
byte[] bytes = Convert.FromBase64String(imageString);
using (FileStream fs = new FileStream(location, FileMode.Create))
{
fs.Write(bytes, 0, bytes.Count);
}
}
catch(Exception)
{
return false;
}
return true;
}

Image/ImageSource/Interop Image to bytearray

I'm developing a control where user can set an image and i want this this to be as user friendly as possible - so support for copy & paste, drag & drop.
I've got this part working using IDataObjects, testing for fileformats of FileDrop, FileContents (eg from outlook), and bitmap eg:
private void GetImageFromIDataObject(IDataObject myIDO)
{
string[] dataformats = myIDO.GetFormats();
Boolean GotImage = false;
foreach (string df in dataformats)
{
if (df == DataFormats.FileDrop)
{
// code here
}
if (df == DataFormats.Bitmap)
{
// Source of my problem here... this gets & displays image but
// how do I then convert from here ?
ImageSource myIS = Utilities.MyImaging.ImageFromClipboardDib();
ImgPerson.Source = myIS;
}
}
}
The ImageFromClipboard code is Thomas Levesque's as referenced in the answer to this SO question wpf InteropBitmap to bitmap
http://www.thomaslevesque.com/2009/02/05/wpf-paste-an-image-from-the-clipboard/
No matter how I get the image onto ImgPerson, this part is working fine; image displays nicely.
When user presses save I need to convert the image to a bytearray and send to a WCF server which will save to server - as in, reconstruct the bytearray into an image and save it in a folder.
For all formats of drag & drop, copy & paste the image is some form of System.Windows.Media.Imaging.BitmapImage.
Except for those involving the clipboard which using Thomas's code becomes System.Windows.Media.Imaging.BitmapFrameDecode.
If I avoid Thomas's code and use:
BitmapSource myBS = Clipboard.GetImage();
ImgPerson.Source = myBS;
I get a System.Windows.Interop.InteropBitmap.
I can't figure out how to work with these; to get them into a bytearray so I can pass to WCF for reconstruction and saving to folder.

Try this piece of code
public byte[] ImageToBytes(BitmapImage imgSource)
{
MemoryStream objMS = new MemoryStream();
PngBitmapEncoder encoder = new PngBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(imgSource));
encoder.Save(objMS);
return objMS.GetBuffer();
}
You can also use JpegBitmapEncoder, BmpBitmapEncoder based on your requirements.
byte[] arr = ImageToBytes(ImgPerson.Source as BitmapImage);

I can't believe I didn't see this SO question but the this is essentially the same as my question:
WPF: System.Windows.Interop.InteropBitmap to System.Drawing.Bitmap
The answer being:
BitmapSource bmpSource = msg.ThumbnailSource as BitmapSource;
MemoryStream ms = new MemoryStream();
BitmapEncoder encoder = new PngBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(bmpSource));
encoder.Save(ms);
ms.Seek(0, SeekOrigin.Begin);
System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap(ms);
So similar in execution to Nitesh's answer but crucially, works with an inter-op bitmap.