I am working on a CCTV project which employs ONVIF. I use a Winform sample, which is provided by "ONVIF Device Manager" project, to obtain video frames from a camera. (You can find it here). I found that the sample put a CopyMemory() block code inside UI thread by using dispatcher.BeginInvoke(). I would slow down main UI thread because this block is repeated to display images in a PictureBox.
void InitPlayback(VideoBuffer videoBuffer, bool isInitial)
{
//....
var renderingTask = Task.Factory.StartNew(delegate
{
var statistics = PlaybackStatistics.Start(Restart, isInitial);
using (videoBuffer.Lock())
{
try
{
//start rendering loop
while (!cancellationToken.IsCancellationRequested)
{
using (var processingEvent = new ManualResetEventSlim(false))
{
var dispOp = disp.BeginInvoke((MethodInvoker)delegate
{
using (Disposable.Create(() => processingEvent.Set()))
{
if (!cancellationToken.IsCancellationRequested)
{
//update statisitc info
statistics.Update(videoBuffer);
//render farme to screen
//DrawFrame(bitmap, videoBuffer, statistics);
DrawFrame(videoBuffer, statistics);
}
}
});
processingEvent.Wait(cancellationToken);
}
cancellationToken.WaitHandle.WaitOne(renderinterval);
}
}
catch (OperationCanceledException error) { } catch (Exception error) { } finally { }
}
}, cancellationToken);
}
[DllImport("kernel32.dll", EntryPoint = "CopyMemory", SetLastError = false)]
public static extern void CopyMemory(IntPtr dest, IntPtr src, int count);
private void DrawFrame(VideoBuffer videoBuffer, PlaybackStatistics statistics)
{
Bitmap bmp = img as Bitmap;
BitmapData bd = null;
try
{
bd = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb);//bgra32
using (var md = videoBuffer.Lock())
{
CopyMemory(bd.Scan0, md.value.scan0Ptr, videoBuff.stride * videoBuff.height);
//bitmap.WritePixels(
// new Int32Rect(0, 0, videoBuffer.width, videoBuffer.height),
// md.value.scan0Ptr, videoBuffer.size, videoBuffer.stride,
// 0, 0
//);
}
}
catch (Exception err)
{
//errBox.Text = err.Message;
Debug.Print("DrawFrame:: " + err.Message);
}
finally
{
bmp.UnlockBits(bd);
}
imageBox.Image = bmp;
// var dispOp = disp.BeginInvoke((MethodInvoker)delegate {imageBox.Image = bmp;}); =>>Bitmap is already locked
}
I tried to exclude the CopyMemory() statement outside of UI thread by calling BeginInvoke() after UnlockBits() bitmap. But, an error is raised "Bitmap is already locked". There has one question which was posted, I have followed the answer of that question but another error occurs "Invalid parameter" while redrawing imageBox. I guess if we lock at bitmap lock(bmp) {CopyMemory();...} the imageBox cannot get information of bitmap associated with it.
Any help is highly appreciated.
Update Proposed Solution
private void DrawFrame(PlaybackStatistics statistics)
{
Bitmap bmp = new Bitmap(videoBuff.width, videoBuff.height);//img as Bitmap;
//...
imageBox.Invoke((MethodInvoker)delegate
{
Image bmTemp = imageBox.Image;
imageBox.Image = bmp;
if (bmTemp != null)
{
bmTemp.Dispose();
}
});
}
You get the error "Bitmap is already locked" because of the following line:
Bitmap bmp = img as Bitmap;
It seems that img is declared globally, and it is being used both by your thread, and the UI thread ath the same time. When a Bitmap object is being displayed in UI, it is being Locked by UI thread for painting. The Lock method in your thread conflicts with this operation in UI thread.
To get better performance, I recommend you to generate a Bitmap for each frame you get in your thread. Then BeginInvoke it to display the prepared image. In UI thread you should care for disposing the Bitmap when replacing in PictureBox property.
Related
I am writing an application to capture the screen using the CopyFromScreen method, and also want to save the image I capture to send over my local network.
So, I am trying store the captured screen on one bitmap, and save another bitmap, which is the previously captured screen, on two threads.
However, this is throwing an InvalidOperationException, which says object is currently in use elsewhere. The exception is thrown by System.Drawing.dll.
I have tried locking, and am using separate bitmaps for saving and capturing the screen. How do I stop this from happening? Relevant code:
Bitmap ScreenCapture(Rectangle rctBounds)
{
Bitmap resultImage = new Bitmap(rctBounds.Width, rctBounds.Height);
using (Graphics grImage = Graphics.FromImage(resultImage))
{
try
{
grImage.CopyFromScreen(rctBounds.Location, Point.Empty, rctBounds.Size);
}
catch (System.InvalidOperationException)
{
return null;
}
}
return resultImage;
}
void ImageEncode(Bitmap bmpSharedImage)
{
// other encoding tasks
pictureBox1.Image = bmpSharedImage;
try
{
Bitmap temp = (Bitmap)bmpSharedImage.Clone();
temp.Save("peace.jpeg");
}
catch (System.InvalidOperationException)
{
return;
}
}
private void button1_Click(object sender, EventArgs e)
{
timer1.Interval = 30;
timer1.Start();
}
Bitmap newImage = null;
private async void timer1_Tick(object sender, EventArgs e)
{
//take new screenshot while encoding the old screenshot
Task tskCaptureTask = Task.Run(() =>
{
newImage = ScreenCapture(_rctDisplayBounds);
});
Task tskEncodeTask = Task.Run(() =>
{
try
{
ImageEncode((Bitmap)_bmpThreadSharedImage.Clone());
}
catch (InvalidOperationException err)
{
System.Diagnostics.Debug.Write(err.Source);
}
});
await Task.WhenAll(tskCaptureTask, tskEncodeTask);
_bmpThreadSharedImage = newImage;
}
I reproduced your problem in a nutshell by creating a simple winforms project with a single button on it.
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
Task.Run(() => SomeTask());
}
public void SomeTask() //this will result in 'Invalid operation exception.'
{
var myhandle = System.Drawing.Graphics.FromHwnd(Handle);
myhandle.DrawLine(new Pen(Color.Red), 0, 0, 100, 100);
}
}
In order to fix this you need to do the following:
public partial class Form1 : Form
{
private Thread myUIthred;
public Form1()
{
myUIthred = Thread.CurrentThread;
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
Task.Run(() => SomeTask());
}
public void SomeTask() // Works Great.
{
if (Thread.CurrentThread != myUIthred) //Tell the UI thread to invoke me if its not him who is running me.
{
BeginInvoke(new Action(SomeTask));
return;
}
var myhandle = System.Drawing.Graphics.FromHwnd(Handle);
myhandle.DrawLine(new Pen(Color.Red), 0, 0, 100, 100);
}
}
The issue is (as Spektre implied) a result of trying to call a UI method from a non-UI thread. The 'BeginInvoke' is actually `this.BeginInvoke' and 'this' is the form which was created by the UI thread and therefore all works.
I do not code in C# so I may be wrong here but I assume you are using Windows...
Accessing any visual components (like GDI Bitmap or window ...) is not safe outside WndProc function. So if you are using GDI bitmap (bitmap with device context) or rendering/accessing any visual component from your window inside any thread then there is your problem. After that any call to WinAPI in your app can throw an exception (even unrelated to graphics)
So try to move any such code into your WndProc function. In case you do not have access to it use any event of your window (like OnTimer or OnIdle).
I am working on a project which makes drawing.
I don't use axml because I do my drawing in a class called filledpolygon and calling the function in MainActivity. I just want to take screenshot in my project. Is there any basic function, which I can call in onCreate method? So, when the program runs, it will automatically take the screenshot. I found answers except Xamarin platform.
Since Android 28 DrawingCacheEnabled is deprecated and without it we are forcing our view to to redraw on our custom canvas wich can cause artifacts with custom controls and renderers and the screenshot version might be different from what we see on screen.
The legacy code that is still working on simple cases is:
public byte[] CaptureScreenshot()
{
var view=
Xamarin.Essentials.Platform.CurrentActivity.Window.DecorView.RootView;
if (view.Height < 1 || view.Width < 1)
return null;
byte[] buffer = null;
view.DrawingCacheEnabled = true;
using (var screenshot = Bitmap.CreateBitmap(
view.Width,
view.Height,
Bitmap.Config.Argb8888))
{
var canvas = new Canvas(screenshot);
view.Draw(canvas);
using (var stream = new MemoryStream())
{
screenshot.Compress(Bitmap.CompressFormat.Png, 90, stream);
buffer = stream.ToArray();
}
}
view.DrawingCacheEnabled = false;
return buffer;
}
Use legacy method above as follows
if ((int)Android.OS.Build.VERSION.SdkInt < 28)
{
//legacy
}
The DrawingCacheEnabled obsolete warning redirects us to use PixelCopy. This method is acting with a callback so to use it synchronously have made some helpers:
Usage:
public byte[] CaptureScreenshot()
{
using var helper = new ScreenshotHelper(
Xamarin.Essentials.Platform.CurrentActivity.Window.DecorView.RootView,
Xamarin.Essentials.Platform.CurrentActivity);
byte[] buffer = null;
bool wait = true;
Task.Run(async () =>
{
helper.Capture((Bitmap bitmap) =>
{
try
{
if (!helper.Error)
{
using (var stream = new MemoryStream())
{
bitmap.Compress(Bitmap.CompressFormat.Png, 90, stream);
buffer = stream.ToArray();
}
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
finally
{
wait = false;
}
});
}).ConfigureAwait(false);
while (wait)
{
Task.Delay(10).Wait();
}
return buffer;
}
The helper:
public class ScreenshotHelper : Java.Lang.Object, PixelCopy.IOnPixelCopyFinishedListener
{
public void OnPixelCopyFinished(int copyResult)
{
var stop = true;
if (copyResult == (int) PixelCopyResult.Success)
{
Error = false;
//todo CallbackGotScreenshot();
_callback(_bitmap);
}
else
{
Error = true;
}
_callback(_bitmap);
}
public bool Error { get; protected set; }
public ScreenshotHelper(Android.Views.View view, Activity activity)
{
_view = view;
_activity = activity;
_bitmap = Bitmap.CreateBitmap(
_view.Width,
_view.Height,
Bitmap.Config.Argb8888);
}
// Starts a background thread and its {#link Handler}.
private void StartBackgroundThread()
{
_BackgroundThread = new HandlerThread("ScreeshotMakerBackground");
_BackgroundThread.Start();
_BackgroundHandler = new Handler(_BackgroundThread.Looper);
}
// Stops the background thread and its {#link Handler}.
private void StopBackgroundThread()
{
try
{
_BackgroundThread.QuitSafely();
_BackgroundThread.Join();
_BackgroundThread = null;
_BackgroundHandler = null;
}
catch (Exception e)
{
//e.PrintStackTrace();
}
}
public void Capture(Action<Bitmap> callback)
{
//var locationOfViewInWindow = new int[2];
//_view.GetLocationInWindow(locationOfViewInWindow);
_callback = callback;
try
{
StartBackgroundThread();
//todo could create-use background handler
PixelCopy.Request(_activity.Window, _bitmap, this,
_BackgroundHandler);
}
catch (Exception e)
{
Console.WriteLine(e);
}
finally
{
Task.Run(StopBackgroundThread);
}
}
private Android.Views.View _view;
private Activity _activity;
private Bitmap _bitmap;
private HandlerThread _BackgroundThread;
private Handler _BackgroundHandler;
private Action<Bitmap> _callback;
public new void Dispose()
{
_bitmap?.Dispose();
_bitmap= null;
_activity = null;
_view = null;
_callback = null;
base.Dispose();
}
}
In your View you could run the following code which will take a screenshot. I have not tried running it in OnCreate() before so you may need to test that out to make sure the view has been fully rendered.
*Edit: According to this post you may have trouble running this code in OnCreate() so you will need to find a better place. I was unable to figure out what post the user was referring to in the link he posted.
*Edit #2: Just found out that Compress() does not take the quality parameter (which is listed as 0 below) into account since PNG is lossless, but if you change the format to JPEG for example, then you may want to turn up the quality parameter since your image will look like garbage.
public byte[] SaveImage() {
DrawingCacheEnabled = true; //Enable cache for the next method below
Bitmap bitmap = GetDrawingCache(true); //Gets the image from the cache
byte[] bitmapData;
using(MemoryStream stream = new MemoryStream()) {
bitmap.Compress(Bitmap.CompressFormat.Png, 0, stream);
bitmapData = stream.ToArray();
}
return bitmapData;
}
I am working on one project where I have implemented threading for processing images captured by the camera.
Please find the below code for ProcessFrame() which is called by the timer after some time Interval.
private void ProcessFrame()
{
try
{
Image<Bgr, Byte> ImageFrame;
// Get Image from the camera
ImageFrame = capture.QueryFrame();
// check if imageFrame is null or not
if (ImageFrame == null)
{
// if null then re- initialize the camera
try
{
capture.Dispose();
capture = new Capture(URL);
ImageFrame = capture.QueryFrame();
}
catch (Exception ex)
{
}
}
// resize image to show on Picture control
ImageFrame = ImageFrame.Resize(img.Width, img.Height, Emgu.CV.CvEnum.INTER.CV_INTER_LINEAR);
// show image on picture control
img.Image = ImageFrame;
try
{
#region Making directory and image in this code
string VerifyImageFolderId = VerifyImageFolder + "\\" + "Camera_" + (1) + "\\";
if (!Directory.Exists(VerifyImageFolderId))
Directory.CreateDirectory(VerifyImageFolderId);
string VerifyImageFileName = VerifyImageFolderId + "\\" + nSavedImagesCounter + ".jpg";
img.Image.Save(VerifyImageFileName); // Save Image
nSavedImagesCounter++;
#endregion
#region Starting thread For processing Image
Thread processImage = new Thread(new ThreadStart(() => ProcessImage(VerifyImageFileName)));
processImage.Start();
#endregion
}
catch (Exception ex)
{
Log(ex.Message);
MessageBox.Show(ex.Message);
}
finally
{
GC.Collect();
}
//#endregion
}
catch (NullReferenceException e)
{
Console.Write("Exception:\n" + DateTime.Now.ToString("hhmmss"));
}
}
Here, is the second function ProcessImage(string ImagePath) which is use to perform some file processing operations as follows:
private void ProcessImage(string ImagePath)
{
#region Check for threadSafeFunction
if (this.InvokeRequired)
{
Console.WriteLine("Inside InvokeRequired");
this.Invoke(new MethodInvoker(delegate() { ProcessImage(ImagePath); }));
}
else
{
1. Detect faces in Image
2. Draw Face markers on Image
3. Some database based on result of Face Detection
4. Delete image File
}
}
After adding of the threading working of ProcessFrame() function was slow down.
and I am not able to get the live streaming on display.
Can any one help me on this?
Thanks in advance.
i suggest you use queue for processing images
run this code once
BlockingCollection<string> imageQueue=new BlockingCollection<string>();
new Thread(() =>
{
foreach (string imagePath in imageQueue.GetConsumingEnumerable())
{
ProcessImage(imagePath);
}
}).Start();
and change ProcessFrame Like This
private void ProcessFrame()
{
.....
#region Starting thread For processing Image
imageQueue.Add(VerifyImageFileName);
#endregion
...
}
How to call method sendResponse to cross threaded call?
void watcher_Created(object sender, FileSystemEventArgs e) {
System.Diagnostics.Debug.WriteLine(e.FullPath);
sendResponse(e.FullPath); //this method must causes cross threaded call
}
I was trying :
Deployment.Current.Dispatcher.BeginInvoke(() => {
});
or Application.Current.Dispatcher.Invoke
But there is no Current .
How to deal with that?
I tried also:
if (this.InvokeRequired) {...}
But there is no InvokeRequired.
EDIT I get unsupported exception when I create any object like Bitmap in sendResponse method. This method is called after firing event. So I thought it is crossthreading.
void watcher_Created(object sender, FileSystemEventArgs e) {
System.Diagnostics.Debug.WriteLine(e.FullPath);
sendResponse(e.FullPath);
}
private void sendResponse(string path) {
try {
BitmapImage bmi = new BitmapImage(new Uri(#path, UriKind.Relative));
byte[] data;
JpegBitmapEncoder encoder = new JpegBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(bmi));
using (MemoryStream ms = new MemoryStream()) {
encoder.Save(ms);
data = ms.ToArray();
}
clientStream.Write(data, 0, data.Length);
} catch (Exception e) { System.Diagnostics.Debug.WriteLine(e); }
}
EDIT2 Client is widnows phone.. this(client) stopped worked after using code from the Nikita's answer(server stopped trhowing exceptions)
try {
Deployment.Current.Dispatcher.BeginInvoke(() => {
MemoryStream stream = new MemoryStream(e.Buffer, e.Offset, e.BytesTransferred);
BitmapImage im = new BitmapImage();
im.SetSource(stream);
MainPage.page.imagePanel.Source = im;
});
If you want to perform a call on UI thread, then you should save the reference to dispatcher by calling
_dispatcher = Application.Current.Dispatcher;
on UI thread. Then you can access it on non-UI thread and use _dispatcher.BeginInvoke or _dispatcher.Invoke in your watcher_Created method.
If you simply want to make your code threadsafe - you can wrap the call in the lock statement:
private readonly object _sendLock = new object();
void watcher_Created(object sender, FileSystemEventArgs e)
{
System.Diagnostics.Debug.WriteLine(e.FullPath);
lock(_sendLock)
{
sendResponse(e.FullPath);
}
}
Edit: you do not need to use wpf components for opening a bitmap and you should not. Instead, for example, you can do the following:
using (MemoryStream ms = new MemoryStream())
using (Image img = Image.FromStream(File.OpenRead(#path)))
{
img.Save(ms, ImageFormat.Jpeg);
var data = ms.ToArray();
clientStream.Write(data, 0, data.Length);
}
I don't have much of experience with WP8, but I think you should be able to use synchronization context, which is a universal concept. When your ViewModel or a UI element is created on the UI thread, remember the thread's current synchronization context:
SynchronizationContext _uiSyncContext;
// ...
_uiSyncContext = SynchronizationContext.Current;
Later from the background thread, use SynchronizationContext.Post (asynchronous callback) or SynchronizationContext.Send (synchronous callback) to execute code on the UI thread:
_uiSyncContext.Post((_) => { this.textBox.Text = "Hello"; }, null);
I've been trying to get a WebBrowser to draw to a texture in XNA 4.0, and I've found several guides on how to do it. The problem is when I try to implement it, whether I change the Url property or call Navigate() it just won't load the page. I have a feeling I'm being a bit ignorant about the threading required, since my project is not started as an STA thread, so I create a separate thread to start the web browser and render to a bitmap.
Here's how I start it:
public void LoadTexture(GraphicsDevice gfx, ContentManager content, string filename, float duration = -1f)
{
this.gfx = gfx;
this.filename = filename;
this.duration = duration;
_resetEvent = new AutoResetEvent(false);
Thread thread = new Thread(GetWebScreenshot);
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
_resetEvent.WaitOne();
}
And here's GetWebScreenshot:
public void GetWebScreenshot()
{
this.web = new WebBrowser();
this.web.Size = new Size(gfx.Viewport.Width, gfx.Viewport.Height);
this.web.Url = new Uri(this.filename);
while (this.web.ReadyState != WebBrowserReadyState.Complete)
{
if (this.web.ReadyState != WebBrowserReadyState.Uninitialized)
{
Console.WriteLine(this.web.ReadyState.ToString());
}
}
bitmap = new Bitmap(this.gfx.Viewport.Width, this.gfx.Viewport.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
this.web.DrawToBitmap(bitmap, new System.Drawing.Rectangle(0, 0, bitmap.Width, bitmap.Height));
this.texture = BitmapToTexture2D(this.gfx, bitmap);
_resetEvent.Set();
}
The ReadyState property never changes from Uninitialized, I've also tried using the DocumentReady event, and that never gets fired. I've also tried Join() instead of AutoResetEvent, but nothing seems to work.
I was right, it was ignorance on my part. The critical thing about ActiveX controls and Single Threaded Apartments is that the message queue needs to be pumped. So now I've restructured my code to the following:
public void LoadTexture(GraphicsDevice gfx, ContentManager content, string filename, float duration = -1f)
{
this.gfx = gfx;
this.filename = filename;
this.duration = duration;
Thread thread = new Thread(GetWebScreenshot);
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
thread.Join();
}
public void GetWebScreenshot()
{
this.web = new WebBrowser();
this.web.Size = new Size(gfx.Viewport.Width, gfx.Viewport.Height);
this.web.Url = new Uri(this.filename);
this.web.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(web_DocumentCompleted);
Application.Run(); // Starts pumping the message queue (and keeps the thread running)
}
void web_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
Bitmap bitmap = new Bitmap(this.gfx.Viewport.Width, this.gfx.Viewport.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
this.web.DrawToBitmap(bitmap, new System.Drawing.Rectangle(0, 0, bitmap.Width, bitmap.Height));
this.texture = HTMLTextureFactoryMachine.BitmapToTexture2D(this.gfx, bitmap);
Application.ExitThread(); // Exits the thread
}
This works no problem.