Threading slowdown camera operation of live streaming - c#

I am working on one project where I have implemented threading for processing images captured by the camera.
Please find the below code for ProcessFrame() which is called by the timer after some time Interval.
private void ProcessFrame()
{
try
{
Image<Bgr, Byte> ImageFrame;
// Get Image from the camera
ImageFrame = capture.QueryFrame();
// check if imageFrame is null or not
if (ImageFrame == null)
{
// if null then re- initialize the camera
try
{
capture.Dispose();
capture = new Capture(URL);
ImageFrame = capture.QueryFrame();
}
catch (Exception ex)
{
}
}
// resize image to show on Picture control
ImageFrame = ImageFrame.Resize(img.Width, img.Height, Emgu.CV.CvEnum.INTER.CV_INTER_LINEAR);
// show image on picture control
img.Image = ImageFrame;
try
{
#region Making directory and image in this code
string VerifyImageFolderId = VerifyImageFolder + "\\" + "Camera_" + (1) + "\\";
if (!Directory.Exists(VerifyImageFolderId))
Directory.CreateDirectory(VerifyImageFolderId);
string VerifyImageFileName = VerifyImageFolderId + "\\" + nSavedImagesCounter + ".jpg";
img.Image.Save(VerifyImageFileName); // Save Image
nSavedImagesCounter++;
#endregion
#region Starting thread For processing Image
Thread processImage = new Thread(new ThreadStart(() => ProcessImage(VerifyImageFileName)));
processImage.Start();
#endregion
}
catch (Exception ex)
{
Log(ex.Message);
MessageBox.Show(ex.Message);
}
finally
{
GC.Collect();
}
//#endregion
}
catch (NullReferenceException e)
{
Console.Write("Exception:\n" + DateTime.Now.ToString("hhmmss"));
}
}
Here, is the second function ProcessImage(string ImagePath) which is use to perform some file processing operations as follows:
private void ProcessImage(string ImagePath)
{
#region Check for threadSafeFunction
if (this.InvokeRequired)
{
Console.WriteLine("Inside InvokeRequired");
this.Invoke(new MethodInvoker(delegate() { ProcessImage(ImagePath); }));
}
else
{
1. Detect faces in Image
2. Draw Face markers on Image
3. Some database based on result of Face Detection
4. Delete image File
}
}
After adding of the threading working of ProcessFrame() function was slow down.
and I am not able to get the live streaming on display.
Can any one help me on this?
Thanks in advance.

i suggest you use queue for processing images
run this code once
BlockingCollection<string> imageQueue=new BlockingCollection<string>();
new Thread(() =>
{
foreach (string imagePath in imageQueue.GetConsumingEnumerable())
{
ProcessImage(imagePath);
}
}).Start();
and change ProcessFrame Like This
private void ProcessFrame()
{
.....
#region Starting thread For processing Image
imageQueue.Add(VerifyImageFileName);
#endregion
...
}

Related

C# Bitmap LockBits/UnlockBits in multi-thread

I am working on a CCTV project which employs ONVIF. I use a Winform sample, which is provided by "ONVIF Device Manager" project, to obtain video frames from a camera. (You can find it here). I found that the sample put a CopyMemory() block code inside UI thread by using dispatcher.BeginInvoke(). I would slow down main UI thread because this block is repeated to display images in a PictureBox.
void InitPlayback(VideoBuffer videoBuffer, bool isInitial)
{
//....
var renderingTask = Task.Factory.StartNew(delegate
{
var statistics = PlaybackStatistics.Start(Restart, isInitial);
using (videoBuffer.Lock())
{
try
{
//start rendering loop
while (!cancellationToken.IsCancellationRequested)
{
using (var processingEvent = new ManualResetEventSlim(false))
{
var dispOp = disp.BeginInvoke((MethodInvoker)delegate
{
using (Disposable.Create(() => processingEvent.Set()))
{
if (!cancellationToken.IsCancellationRequested)
{
//update statisitc info
statistics.Update(videoBuffer);
//render farme to screen
//DrawFrame(bitmap, videoBuffer, statistics);
DrawFrame(videoBuffer, statistics);
}
}
});
processingEvent.Wait(cancellationToken);
}
cancellationToken.WaitHandle.WaitOne(renderinterval);
}
}
catch (OperationCanceledException error) { } catch (Exception error) { } finally { }
}
}, cancellationToken);
}
[DllImport("kernel32.dll", EntryPoint = "CopyMemory", SetLastError = false)]
public static extern void CopyMemory(IntPtr dest, IntPtr src, int count);
private void DrawFrame(VideoBuffer videoBuffer, PlaybackStatistics statistics)
{
Bitmap bmp = img as Bitmap;
BitmapData bd = null;
try
{
bd = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);//bgra32
using (var md = videoBuffer.Lock())
{
CopyMemory(bd.Scan0, md.value.scan0Ptr, videoBuff.stride * videoBuff.height);
//bitmap.WritePixels(
// new Int32Rect(0, 0, videoBuffer.width, videoBuffer.height),
// md.value.scan0Ptr, videoBuffer.size, videoBuffer.stride,
// 0, 0
//);
}
}
catch (Exception err)
{
//errBox.Text = err.Message;
Debug.Print("DrawFrame:: " + err.Message);
}
finally
{
bmp.UnlockBits(bd);
}
imageBox.Image = bmp;
// var dispOp = disp.BeginInvoke((MethodInvoker)delegate {imageBox.Image = bmp;}); =>>Bitmap is already locked
}
I tried to exclude the CopyMemory() statement outside of UI thread by calling BeginInvoke() after UnlockBits() bitmap. But, an error is raised "Bitmap is already locked". There has one question which was posted, I have followed the answer of that question but another error occurs "Invalid parameter" while redrawing imageBox. I guess if we lock at bitmap lock(bmp) {CopyMemory();...} the imageBox cannot get information of bitmap associated with it.
Any help is highly appreciated.
Update Proposed Solution
private void DrawFrame(PlaybackStatistics statistics)
{
Bitmap bmp = new Bitmap(videoBuff.width, videoBuff.height);//img as Bitmap;
//...
imageBox.Invoke((MethodInvoker)delegate
{
Image bmTemp = imageBox.Image;
imageBox.Image = bmp;
if (bmTemp != null)
{
bmTemp.Dispose();
}
});
}
You get the error "Bitmap is already locked" because of the following line:
Bitmap bmp = img as Bitmap;
It seems that img is declared globally, and it is being used both by your thread, and the UI thread ath the same time. When a Bitmap object is being displayed in UI, it is being Locked by UI thread for painting. The Lock method in your thread conflicts with this operation in UI thread.
To get better performance, I recommend you to generate a Bitmap for each frame you get in your thread. Then BeginInvoke it to display the prepared image. In UI thread you should care for disposing the Bitmap when replacing in PictureBox property.

InvalidOperationException while saving a bitmap and using graphics.copyFromScreen parallel-y

I am writing an application to capture the screen using the CopyFromScreen method, and also want to save the image I capture to send over my local network.
So, I am trying store the captured screen on one bitmap, and save another bitmap, which is the previously captured screen, on two threads.
However, this is throwing an InvalidOperationException, which says object is currently in use elsewhere. The exception is thrown by System.Drawing.dll.
I have tried locking, and am using separate bitmaps for saving and capturing the screen. How do I stop this from happening? Relevant code:
Bitmap ScreenCapture(Rectangle rctBounds)
{
Bitmap resultImage = new Bitmap(rctBounds.Width, rctBounds.Height);
using (Graphics grImage = Graphics.FromImage(resultImage))
{
try
{
grImage.CopyFromScreen(rctBounds.Location, Point.Empty, rctBounds.Size);
}
catch (System.InvalidOperationException)
{
return null;
}
}
return resultImage;
}
void ImageEncode(Bitmap bmpSharedImage)
{
// other encoding tasks
pictureBox1.Image = bmpSharedImage;
try
{
Bitmap temp = (Bitmap)bmpSharedImage.Clone();
temp.Save("peace.jpeg");
}
catch (System.InvalidOperationException)
{
return;
}
}
private void button1_Click(object sender, EventArgs e)
{
timer1.Interval = 30;
timer1.Start();
}
Bitmap newImage = null;
private async void timer1_Tick(object sender, EventArgs e)
{
//take new screenshot while encoding the old screenshot
Task tskCaptureTask = Task.Run(() =>
{
newImage = ScreenCapture(_rctDisplayBounds);
});
Task tskEncodeTask = Task.Run(() =>
{
try
{
ImageEncode((Bitmap)_bmpThreadSharedImage.Clone());
}
catch (InvalidOperationException err)
{
System.Diagnostics.Debug.Write(err.Source);
}
});
await Task.WhenAll(tskCaptureTask, tskEncodeTask);
_bmpThreadSharedImage = newImage;
}
I reproduced your problem in a nutshell by creating a simple winforms project with a single button on it.
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
Task.Run(() => SomeTask());
}
public void SomeTask() //this will result in 'Invalid operation exception.'
{
var myhandle = System.Drawing.Graphics.FromHwnd(Handle);
myhandle.DrawLine(new Pen(Color.Red), 0, 0, 100, 100);
}
}
In order to fix this you need to do the following:
public partial class Form1 : Form
{
private Thread myUIthred;
public Form1()
{
myUIthred = Thread.CurrentThread;
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
Task.Run(() => SomeTask());
}
public void SomeTask() // Works Great.
{
if (Thread.CurrentThread != myUIthred) //Tell the UI thread to invoke me if its not him who is running me.
{
BeginInvoke(new Action(SomeTask));
return;
}
var myhandle = System.Drawing.Graphics.FromHwnd(Handle);
myhandle.DrawLine(new Pen(Color.Red), 0, 0, 100, 100);
}
}
The issue is (as Spektre implied) a result of trying to call a UI method from a non-UI thread. The 'BeginInvoke' is actually `this.BeginInvoke' and 'this' is the form which was created by the UI thread and therefore all works.
I do not code in C# so I may be wrong here but I assume you are using Windows...
Accessing any visual components (like GDI Bitmap or window ...) is not safe outside WndProc function. So if you are using GDI bitmap (bitmap with device context) or rendering/accessing any visual component from your window inside any thread then there is your problem. After that any call to WinAPI in your app can throw an exception (even unrelated to graphics)
So try to move any such code into your WndProc function. In case you do not have access to it use any event of your window (like OnTimer or OnIdle).

Increment ProgressBar within another class

it's my first question I'm asking here, so please be gentle with me ;)
So I've actually got two WinForms in my C# application I'm writing at the moment (I'm quite new to C#).
This window has a button, which saves photos from an usb device you selected before in a list box to another folder.
After clicking on this button my main thread is of course busy with copying, so I decided to create another WinForm which contains my ProgressBar.
Foreach completed copy, I want to increment my ProgressBar accordingly.
So I count the number of copies I have to do and give it the progressbar as maximum. But my problem at the moment is, that I really don't know how to increment the ProgressBar without getting a Thread Unsafe Exception.
Here's my ProgressWindow code:
public partial class ProgressWindow : Form
{
BackgroundWorker updateProgressBarThread = new BackgroundWorker();
private Boolean _isThreadRunning = false;
public Boolean IsThreadRunning
{
get { return _isThreadRunning; }
set { _isThreadRunning = value; }
}
private int _progressbarLength;
public int ProgressbarLength
{
get { return _progressbarLength; }
set { _progressbarLength = value; }
}
private int progress = 1;
public ProgressWindow()
{
Show();
InitializeComponent();
}
private void StartUpdateThread(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = sender as BackgroundWorker;
// Reports progress to the ProgressChangedEvent function. (Thread Safe)
}
private void FinishProgressThread(object sender, RunWorkerCompletedEventArgs e)
{
if (!_isThreadRunning)
{
MessageBox.Show("Erfolgreich kopiert");
Close();
}
}
private void ProgressChangedEvent(object sender, ProgressChangedEventArgs e)
{
this.copyProgressbar.Value = e.ProgressPercentage;
this.progressStatus.Text = e.ProgressPercentage.ToString();
}
public void CallUpdateThread()
{
updateProgressBarThread.WorkerReportsProgress = true;
updateProgressBarThread.DoWork += new DoWorkEventHandler(StartUpdateThread);
updateProgressBarThread.ProgressChanged += new ProgressChangedEventHandler(ProgressChangedEvent);
updateProgressBarThread.RunWorkerCompleted += new RunWorkerCompletedEventHandler(FinishProgressThread);
updateProgressBarThread.RunWorkerAsync();
}
}
I want to increment my ProgressBar with 1 after each succesful copy.
How do I do this from my main thread?
This is the function which actually handles the copy process
private void SaveFile(System.IO.DirectoryInfo root)
{
try
{
IEnumerable<DirectoryInfo> directoriesNames = root.EnumerateDirectories();
// New instance of thread ProgressWindow.
ProgressWindow progress = new ProgressWindow();
progress.CallUpdateThread();
foreach (DirectoryInfo element in directoriesNames)
{
// Query all subdirectories and count everything with the in the configuration made settings.
if (!element.Attributes.ToString().Contains("System"))
{
// Now we insert the configuration we applied.
String fileExtension = null;
if (Properties.Settings.Default._configPhoto)
{
fileExtension = "*.jpg";
}
if (Properties.Settings.Default._configWordDocument)
{
fileExtension = "*.odt";
}
FileInfo[] jpgList = element.GetFiles(fileExtension, SearchOption.AllDirectories);
// set the size of the progress bar
progress.ProgressbarLength = jpgList.Count();
// Now we go through all our results and save them to our backup folder.
foreach (FileInfo tmp in jpgList)
{
string sourceFilePath = tmp.FullName;
string destFilePath = PATHTOBACKUP + "\\" + tmp.Name;
progress.IsThreadRunning = true;
try
{
System.IO.File.Copy(sourceFilePath, destFilePath, true);
}
catch (IOException ioe)
{
MessageBox.Show(ioe.Message);
}
}
}
}
// progress.IsThreadRunning = false;
}
catch (UnauthorizedAccessException e)
{
MessageBox.Show(e.Message);
}
}
It's pretty obvious that I have to do this after this function
System.IO.File.Copy(sourceFilePath, destFilePath, true);
But how do I report this to my ProgressWindow?
I really hope I explained it well enough, not sure if I'm missing something important.
Thanks in advance guys
Here is a compact example of the key components:
Clicking button starts new thread worker
Progress is done by file lengths, not by number of files
BeginInvoke used to update the progress bar (avoid cross Thread exception)
ProgressBar pb = new ProgressBar() { Minimum = 0, Maximum = 100 };
Button btn = new Button();
btn.Click += delegate {
Thread t = new Thread(() => {
DirectoryInfo dir = new DirectoryInfo("C:\\temp\\");
var files = dir.GetFiles("*.txt");
long totalLength = files.Sum(f => f.Length);
long length = 0;
foreach (var f in files) {
length += f.Length;
int percent = (int) Math.Round(100.0 * length / totalLength);
pb.BeginInvoke((Action) delegate {
pb.Value = percent;
});
File.Copy(f.FullName, "...");
}
});
t.IsBackground = true;
t.Start();
};

Emgu CV check web cam connectivity

I am writing a C# web camera application using Emgu CV. I tried to handle when user unplug the web cam during frame capturing in pictureBox.
If the web camera is unplugged, then the application should start scanning for new web cam connectivity every 2 seconds until the pictureBox can be updated again.
The following timer code could not catch anything, the program initially captures frames, I unplug the camera, then plug back, but the camera can not be restart.
private void timer1_Tick(object sender, EventArgs e)
{
if (cap == null)
{
try
{
cap = new Capture(0);
cap.SetCaptureProperty(CAP_PROP.CV_CAP_PROP_FRAME_WIDTH, 320);
cap.SetCaptureProperty(CAP_PROP.CV_CAP_PROP_FRAME_HEIGHT, 240);
Console.WriteLine("Restarting Cam");
}
catch (Exception ee){
Console.WriteLine("null"); cap = null; return;
}
}
else
{
Console.WriteLine("NO null");
}
try
{
Image<Bgr, byte> nextFrame = cap.QueryFrame();
}
catch(Exception ee)
{
Console.WriteLine("Frame Capture fail");
cap.Dispose();
cap = null;
return;
}
using (Image<Bgr, byte> nextFrame = cap.QueryFrame())
{
if (nextFrame != null)
{
Image<Gray, byte> grayframe = nextFrame.Convert<Gray, byte>();
videoBox.Image = nextFrame.ToBitmap();
}
}
}
The program keep printing "No null", 20 second after unplugging the camera, the output console printed out The thread '' (0xb96c) has exited with code 0 (0x0)
You can check connectivity with DirectShow lib. Get an array of all connected cameras first
DirectShowLib.DsDevice[] allCameras = DirectShowLib.DsDevice.GetDevicesOfCat(FilterCategory.VideoInputDevice);
add to you class property for selected camera name, and then you can check if camera is connected
bool isConnected = DirectShowLib.DsDevice.GetDevicesOfCat(FilterCategory.VideoInputDevice).Any(c => c.Name == selectedCameraName);

Record continuously 5min (x-minutes) of video with Microsoft Expression Encoder

My Software is to record continuously 5min of video.
Example: the software should start the recording on program-start and hold continuously 5min of video in buffer. When I stop the recording the last 5min of recording should save to disk
private void CaptureMoni()
{
try
{
Rectangle _screenRectangle = Screen.PrimaryScreen.Bounds;
_screenCaptureJob = new ScreenCaptureJob();
_screenCaptureJob.CaptureRectangle = _screenRectangle;
_screenCaptureJob.ShowFlashingBoundary = true;
_screenCaptureJob.ScreenCaptureVideoProfile.FrameRate = 20;
_screenCaptureJob.CaptureMouseCursor = true;
_screenCaptureJob.OutputScreenCaptureFileName = string.Format(#"C:\test.wmv");
if (File.Exists(_screenCaptureJob.OutputScreenCaptureFileName))
{
File.Delete(_screenCaptureJob.OutputScreenCaptureFileName);
}
_screenCaptureJob.Start();
}
catch(Exception e) { }
}
something like that:
private void SaveRecord(int cntMinutes)
{
try
{
_screenCaptureJob.Stop();
// something like that
_screenCaptureJob.SaveLastXMinutes(cntMinutes);
}
catch(Exception e) { }
}

Categories

Resources