How to send touch event in Geckofx? - c#

I use Geckofx in my program.I just want to send touch event (like touchstart , touchmove , touchend) to the document of Geckobrowser.I had try to add the method in "WindowUtils.cs" of "Gecko-core.dll" ,like that:
public Boolean SendTouchEvent(string aType, uint[] aIdentifiers, int[] aXs, int[] aYs, uint[] aRxs, uint[] aRys, float[] aRotationAngles, float[] aForces, uint count, int aModifiers, bool aIgnoreRootScrollFrame)
{
using (nsAString type = new nsAString(aType))
{
return _windowUtils.Instance.SendTouchEvent(type, aIdentifiers, aXs, aYs, aRxs, aRys, aRotationAngles, aForces, count, aModifiers, aIgnoreRootScrollFrame);
}
}
This method uses the the method "SendTouchEvent" in "nsIDOMWindowUtils.cs", like that :
/// <summary>
///Synthesize a touch event. The event types supported are:
/// touchstart, touchend, touchmove, and touchcancel
///
/// Events are sent in coordinates offset by aX and aY from the window.
///
/// Cannot be accessed from unprivileged context (not content-accessible)
/// Will throw a DOM security error if called without chrome privileges.
///
/// The event is dispatched via the toplevel window, so it could go to any
/// window under the toplevel window, in some cases it could never reach this
/// window at all.
///
/// #param aType event type
/// #param xs array of offsets in CSS pixels for each touch to be sent
/// #param ys array of offsets in CSS pixels for each touch to be sent
/// #param rxs array of radii in CSS pixels for each touch to be sent
/// #param rys array of radii in CSS pixels for each touch to be sent
/// #param rotationAngles array of angles in degrees for each touch to be sent
/// #param forces array of forces (floats from 0 to 1) for each touch to be sent
/// #param count number of touches in this set
/// #param aModifiers modifiers pressed, using constants defined as MODIFIER_*
/// #param aIgnoreRootScrollFrame whether the event should ignore viewport bounds
/// during dispatch
///
/// returns true if the page called prevent default on this touch event
/// </summary>
[return: MarshalAs(UnmanagedType.U1)]
[MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
bool SendTouchEvent([MarshalAs(UnmanagedType.CustomMarshaler, MarshalType = "Gecko.CustomMarshalers.AStringMarshaler")] nsAStringBase aType, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] uint[] aIdentifiers, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] int[] aXs, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] int[] aYs, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] uint[] aRxs, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] uint[] aRys, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] float[] aRotationAngles, [MarshalAs(UnmanagedType.LPArray, SizeParamIndex=8)] float[] aForces, uint count, int aModifiers, [MarshalAs(UnmanagedType.U1)] bool aIgnoreRootScrollFrame);
And then , I test my method like that :
private void button2_Click(object sender, EventArgs e)
{
uint[] ident ={1};
int[] ax={200};
int[] ay={200};
uint[] rx ={2};
uint[] ry ={2};
float[] ro ={20};
float[] force={(float)0.8};
bool r = browser.Window.WindowUtils.SendTouchEvent("touchstart", ident, ax, ay, rx, ry, ro, force, (uint)1, 0, true);
Application.DoEvents();
for (int i = 0; i < 100; i++)
{
ay[0]+=10;
r = browser.Window.WindowUtils.SendTouchEvent("touchmove", ident, ax, ay, rx, ry, ro, force, (uint)1, 0, true);
Application.DoEvents();
Thread.Sleep(10);
}
r = browser.Window.WindowUtils.SendTouchEvent("touchend", ident, ax, ay, rx, ry, ro, force, (uint)1, 0, true);
}
But, it always returns false;Could Anyone help me? Thanks in advance.

Related

C# Desktop background displaying incorrect location maybe?

Hi I am currently creating an animated desktop background that uses axWindowsMideaPlayer to play video files in a loop I take the WMP and set its handle to the desktop handle, it works great on single display monitor setup but fails with multiple monitors. Why it is failing is because monitors can have their own set position like left/right/topleft/topright/top/bottom/bottomleft and bottom right of the primary screen which puts their position in the negatives and so forth.. My question is how can I position each WMP correctly on each monitor? Here is what I have so far,
this is how I get each monitor...
public class DisplayInfo
{
public bool isPrimary { get; set; }
public int ScreenHeight { get; set; }
public int ScreenWidth { get; set; }
public Rect MonitorArea { get; set; }
public Rect WorkArea { get; set; }
public string DeviceName { get; set; }
}
/// <summary>
/// Collection of display information
/// </summary>
public class DisplayInfoCollection : List<DisplayInfo>
{
// size of a device name string
private const int CCHDEVICENAME = 32;
/// <summary>
/// The MONITORINFOEX structure contains information about a display monitor.
/// The GetMonitorInfo function stores information into a MONITORINFOEX structure or a MONITORINFO structure.
/// The MONITORINFOEX structure is a superset of the MONITORINFO structure. The MONITORINFOEX structure adds a string member to contain a name
/// for the display monitor.
/// </summary>
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
internal struct MONITORINFOEX
{
/// <summary>
/// The size, in bytes, of the structure. Set this member to sizeof(MONITORINFOEX) (72) before calling the GetMonitorInfo function.
/// Doing so lets the function determine the type of structure you are passing to it.
/// </summary>
public int Size;
/// <summary>
/// A RECT structure that specifies the display monitor rectangle, expressed in virtual-screen coordinates.
/// Note that if the monitor is not the primary display monitor, some of the rectangle's coordinates may be negative values.
/// </summary>
public Rect Monitor;
/// <summary>
/// A RECT structure that specifies the work area rectangle of the display monitor that can be used by applications,
/// expressed in virtual-screen coordinates. Windows uses this rectangle to maximize an application on the monitor.
/// The rest of the area in rcMonitor contains system windows such as the task bar and side bars.
/// Note that if the monitor is not the primary display monitor, some of the rectangle's coordinates may be negative values.
/// </summary>
public Rect WorkArea;
/// <summary>
/// The attributes of the display monitor.
///
/// This member can be the following value:
/// 1 : MONITORINFOF_PRIMARY
/// </summary>
public uint Flags;
/// <summary>
/// A string that specifies the device name of the monitor being used. Most applications have no use for a display monitor name,
/// and so can save some bytes by using a MONITORINFO structure.
/// </summary>
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = CCHDEVICENAME)]
public string DeviceName;
public void Init()
{
this.Size = 40 + 2 * CCHDEVICENAME;
this.DeviceName = string.Empty;
}
}
[DllImport("user32.dll")]
private static extern bool EnumDisplayMonitors(IntPtr hdc, IntPtr lprcClip,
EnumMonitorsDelegate lpfnEnum, IntPtr dwData);
[DllImport("user32.dll", CharSet = CharSet.Auto)]
private static extern bool GetMonitorInfo(IntPtr hMonitor, ref MONITORINFOEX lpmi);
/*
[DllImport("user32.dll")]
static extern bool GetMonitorInfo(IntPtr hMonitor, ref MONITORINFO lpmi);
*/
/// <summary>
/// Returns the number of Displays using the Win32 functions
/// </summary>
/// <returns>collection of Display Info</returns>
public static DisplayInfoCollection GetDisplays()
{
DisplayInfoCollection col = new DisplayInfoCollection();
EnumDisplayMonitors(IntPtr.Zero, IntPtr.Zero,
delegate (IntPtr hMonitor, IntPtr hdcMonitor, ref Rect lprcMonitor, IntPtr dwData)
{
MONITORINFOEX mi = new MONITORINFOEX();
mi.Size = (int)Marshal.SizeOf(mi);
bool success = GetMonitorInfo(hMonitor, ref mi);
if (success)
{
DisplayInfo di = new DisplayInfo();
di.ScreenWidth = (mi.Monitor.Right - mi.Monitor.Left);
di.ScreenHeight = (mi.Monitor.Bottom - mi.Monitor.Top);
di.MonitorArea = mi.Monitor;
di.WorkArea = mi.WorkArea;
di.isPrimary = Convert.ToBoolean(mi.Flags);
di.DeviceName = mi.DeviceName;
col.Add(di);
}
return true;
}, IntPtr.Zero);
return col;
}
}
this is how I have tried to call it and use it but it puts WMP all over the place depending on where the monitors are positioned.
DisplayInfoCollection dic = DisplayInfoCollection.GetDisplays();
int count = 0;
int totalPosX = 0;
int totalPosY = 0;
DisplayInfo dInfo = null;
List<DisplayInfo> di = dic.OrderByDescending(d => d.isPrimary).ToList();
var or = SystemInformation.VirtualScreen;
foreach (DisplayInfo dm in di)
{
bool zeroOutX = false;
bool zeroOutY = false;
if (dm.isPrimary)
{
totalPosX = or.Left > 0 ? or.Left : -or.Left;
totalPosY = or.Top > 0 ? or.Top : -or.Top;
dInfo = dm;
}
else
{
bool left = false;
bool top = false;
bool right = false;
bool bottom = false;
bool topLeft = false;
bool topRight = false;
bool bottomLeft = false;
bool bottomRight = false;
int posY = dm.MonitorArea.Top > 0 ? dm.MonitorArea.Top : -dm.MonitorArea.Top;
if (dm.MonitorArea.Left < 0)
{
left = true;
}
else
right = dm.MonitorArea.Left > 0;
if (dm.MonitorArea.Top < 0)
{
top = true;
}
else
bottom = dm.MonitorArea.Top > 0;
bool center = (dm.MonitorArea.Left > 0 ?
dm.MonitorArea.Left : -dm.MonitorArea.Left) > 0 ||
(dm.MonitorArea.Left > 0 ?
dm.MonitorArea.Left : -dm.MonitorArea.Left) < dInfo.ScreenWidth;
topLeft = left && top;
topRight = right && top;
bottomLeft = left && bottom;
bottomRight = right && bottom;
if (topLeft || topRight || bottomLeft || bottomRight || left || right)
{
if (dm.MonitorArea.Left < 0)
zeroOutX = true;
else
totalPosX += dInfo.ScreenWidth;
if (dm.MonitorArea.Top < 0)
zeroOutY = true;
else
totalPosY += dm.MonitorArea.Top;
}
dInfo = dm;
}
Display display = new Display(dm.DeviceName, dm.ScreenWidth,
dm.ScreenHeight, zeroOutX ? 0 : totalPosX,
zeroOutY ? 0 : totalPosY, Controls, count);
Displays.Add(display);
count++;
}
I cant find much help on this matter and have tried numerous ways to do this its a c# windows form and I am new-ish to programming my knowledge is limited in this any help will be appreciated thanks in advance.
According to the EnumDisplaySettings:
The EnumDisplaySettings function sets values for the following five
DEVMODE members:
dmBitsPerPel
dmPelsWidth
dmPelsHeight
dmDisplayFlags
dmDisplayFrequency
(Excluding dmPosition), You should try to use EnumDisplaySettingsEx, and specify DM_POSITION to get the correct dmPosition value.
thanks for all that helped I found a solution to my problem here it is if anyone need to know.
var or = SystemInformation.VirtualScreen;
foreach (DisplayInfo dm in dic)
{
int x = or.Left > 0 ? or.Left : -or.Left;
int y = or.Top > 0 ? or.Top : -or.Top;
if (dm.isPrimary)
{
Rect rect = new Rect();
rect.Left = x;
rect.Top = y;
rect.Right = rect.Left + dm.ScreenWidth;
rect.Bottom = rect.Top + dm.ScreenHeight;
dm.MonitorArea = rect;
}
else
{
Rect rect = new Rect();
rect.Left = x + dm.MonitorArea.Left;
rect.Top = y + dm.MonitorArea.Top;
rect.Right = rect.Left + dm.ScreenWidth;
rect.Bottom = rect.Top + dm.ScreenHeight;
dm.MonitorArea = rect;
}

Big red X on image and I cannot detect error

To be brief, I'm trying to implement some motion detection algorithm and in my case I'm working on UWP using portable version of AForge library to processing image. Avoiding this issue, I have problem with conversion SoftwareBitmap object (that I get from MediaFrameReader) to Bitmap object (and vice versa) that I use in my code associated with motion detection. In consequence of this conversion, I get proper image with big red X in the foreground. Code below:
private async void FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
{
var frame = sender.TryAcquireLatestFrame();
if (frame != null && !_detectingMotion)
{
SoftwareBitmap aForgeInputBitmap = null;
var inputBitmap = frame.VideoMediaFrame?.SoftwareBitmap;
if (inputBitmap != null)
{
_detectingMotion = true;
//The XAML Image control can only display images in BRGA8 format with premultiplied or no alpha
if (inputBitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8
&& inputBitmap.BitmapAlphaMode == BitmapAlphaMode.Premultiplied)
{
aForgeInputBitmap = SoftwareBitmap.Copy(inputBitmap);
}
else
{
aForgeInputBitmap = SoftwareBitmap.Convert(inputBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore);
}
await _aForgeHelper.MoveBackgrounds(aForgeInputBitmap);
SoftwareBitmap aForgeOutputBitmap = await _aForgeHelper.DetectMotion();
_frameRenderer.PresentSoftwareBitmap(aForgeOutputBitmap);
_detectingMotion = false;
}
}
}
class AForgeHelper
{
private Bitmap _background;
private Bitmap _currentFrameBitmap;
public async Task MoveBackgrounds(SoftwareBitmap currentFrame)
{
if (_background == null)
{
_background = TransformToGrayscale(await ConvertSoftwareBitmapToBitmap(currentFrame));
}
else
{
// modifying _background in compliance with algorithm - in this case irrelevant
}
}
public async Task<SoftwareBitmap> DetectMotion()
{
// to check only this conversion
return await ConvertBitmapToSoftwareBitmap(_background);
}
private static async Task<Bitmap> ConvertSoftwareBitmapToBitmap(SoftwareBitmap input)
{
Bitmap output = null;
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
WriteableBitmap tmpBitmap = new WriteableBitmap(input.PixelWidth, input.PixelHeight);
input.CopyToBuffer(tmpBitmap.PixelBuffer);
output = (Bitmap)tmpBitmap;
});
return output;
}
private static async Task<SoftwareBitmap> ConvertBitmapToSoftwareBitmap(Bitmap input)
{
SoftwareBitmap output = null;
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
WriteableBitmap tmpBitmap = (WriteableBitmap)input;
output = new SoftwareBitmap(BitmapPixelFormat.Bgra8, tmpBitmap.PixelWidth, tmpBitmap.PixelHeight,
BitmapAlphaMode.Premultiplied);
output.CopyFromBuffer(tmpBitmap.PixelBuffer);
});
return output;
}
private static Bitmap TransformToGrayscale(Bitmap input)
{
Grayscale grayscaleFilter = new Grayscale(0.2125, 0.7154, 0.0721);
Bitmap output = grayscaleFilter.Apply(input);
return output;
}
Certainly, I've tried to detect some errors using try-catch clauses. I've found nothing. Thanks in advance.
EDIT (29/03/2018):
Generally, my app targets at providing some features associated with Kinect Sensor. App user has possibility to choose some feature from features list. First of all, this app have to be available on Xbox One, therefore I've chosen UWP. Due to 'scientific' issues, I've implemented MVVM pattern, using MVVM Light framework. As for PresentSoftwareBitmap() method, it comes from Windows-universal-samples repo and I paste FrameRenderer helper class below:
[ComImport]
[Guid("5B0D3235-4DBA-4D44-865E-8F1D0E4FD04D")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
void GetBuffer(out byte* buffer, out uint capacity);
}
class FrameRenderer
{
private Image _imageElement;
private SoftwareBitmap _backBuffer;
private bool _taskRunning = false;
public FrameRenderer(Image imageElement)
{
_imageElement = imageElement;
_imageElement.Source = new SoftwareBitmapSource();
}
// Processes a MediaFrameReference and displays it in a XAML image control
public void ProcessFrame(MediaFrameReference frame)
{
var softwareBitmap = FrameRenderer.ConvertToDisplayableImage(frame?.VideoMediaFrame);
if (softwareBitmap != null)
{
// Swap the processed frame to _backBuffer and trigger UI thread to render it
softwareBitmap = Interlocked.Exchange(ref _backBuffer, softwareBitmap);
// UI thread always reset _backBuffer before using it. Unused bitmap should be disposed.
softwareBitmap?.Dispose();
// Changes to xaml ImageElement must happen in UI thread through Dispatcher
var task = _imageElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
async () =>
{
// Don't let two copies of this task run at the same time.
if (_taskRunning)
{
return;
}
_taskRunning = true;
// Keep draining frames from the backbuffer until the backbuffer is empty.
SoftwareBitmap latestBitmap;
while ((latestBitmap = Interlocked.Exchange(ref _backBuffer, null)) != null)
{
var imageSource = (SoftwareBitmapSource)_imageElement.Source;
await imageSource.SetBitmapAsync(latestBitmap);
latestBitmap.Dispose();
}
_taskRunning = false;
});
}
}
// Function delegate that transforms a scanline from an input image to an output image.
private unsafe delegate void TransformScanline(int pixelWidth, byte* inputRowBytes, byte* outputRowBytes);
/// <summary>
/// Determines the subtype to request from the MediaFrameReader that will result in
/// a frame that can be rendered by ConvertToDisplayableImage.
/// </summary>
/// <returns>Subtype string to request, or null if subtype is not renderable.</returns>
public static string GetSubtypeForFrameReader(MediaFrameSourceKind kind, MediaFrameFormat format)
{
// Note that media encoding subtypes may differ in case.
// https://learn.microsoft.com/en-us/uwp/api/Windows.Media.MediaProperties.MediaEncodingSubtypes
string subtype = format.Subtype;
switch (kind)
{
// For color sources, we accept anything and request that it be converted to Bgra8.
case MediaFrameSourceKind.Color:
return Windows.Media.MediaProperties.MediaEncodingSubtypes.Bgra8;
// The only depth format we can render is D16.
case MediaFrameSourceKind.Depth:
return String.Equals(subtype, Windows.Media.MediaProperties.MediaEncodingSubtypes.D16, StringComparison.OrdinalIgnoreCase) ? subtype : null;
// The only infrared formats we can render are L8 and L16.
case MediaFrameSourceKind.Infrared:
return (String.Equals(subtype, Windows.Media.MediaProperties.MediaEncodingSubtypes.L8, StringComparison.OrdinalIgnoreCase) ||
String.Equals(subtype, Windows.Media.MediaProperties.MediaEncodingSubtypes.L16, StringComparison.OrdinalIgnoreCase)) ? subtype : null;
// No other source kinds are supported by this class.
default:
return null;
}
}
/// <summary>
/// Converts a frame to a SoftwareBitmap of a valid format to display in an Image control.
/// </summary>
/// <param name="inputFrame">Frame to convert.</param>
public static unsafe SoftwareBitmap ConvertToDisplayableImage(VideoMediaFrame inputFrame)
{
SoftwareBitmap result = null;
using (var inputBitmap = inputFrame?.SoftwareBitmap)
{
if (inputBitmap != null)
{
switch (inputFrame.FrameReference.SourceKind)
{
case MediaFrameSourceKind.Color:
// XAML requires Bgra8 with premultiplied alpha.
// We requested Bgra8 from the MediaFrameReader, so all that's
// left is fixing the alpha channel if necessary.
if (inputBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8)
{
System.Diagnostics.Debug.WriteLine("Color frame in unexpected format.");
}
else if (inputBitmap.BitmapAlphaMode == BitmapAlphaMode.Premultiplied)
{
// Already in the correct format.
result = SoftwareBitmap.Copy(inputBitmap);
}
else
{
// Convert to premultiplied alpha.
result = SoftwareBitmap.Convert(inputBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
break;
case MediaFrameSourceKind.Depth:
// We requested D16 from the MediaFrameReader, so the frame should
// be in Gray16 format.
if (inputBitmap.BitmapPixelFormat == BitmapPixelFormat.Gray16)
{
// Use a special pseudo color to render 16 bits depth frame.
var depthScale = (float)inputFrame.DepthMediaFrame.DepthFormat.DepthScaleInMeters;
var minReliableDepth = inputFrame.DepthMediaFrame.MinReliableDepth;
var maxReliableDepth = inputFrame.DepthMediaFrame.MaxReliableDepth;
result = TransformBitmap(inputBitmap, (w, i, o) => PseudoColorHelper.PseudoColorForDepth(w, i, o, depthScale, minReliableDepth, maxReliableDepth));
}
else
{
System.Diagnostics.Debug.WriteLine("Depth frame in unexpected format.");
}
break;
case MediaFrameSourceKind.Infrared:
// We requested L8 or L16 from the MediaFrameReader, so the frame should
// be in Gray8 or Gray16 format.
switch (inputBitmap.BitmapPixelFormat)
{
case BitmapPixelFormat.Gray16:
// Use pseudo color to render 16 bits frames.
result = TransformBitmap(inputBitmap, PseudoColorHelper.PseudoColorFor16BitInfrared);
break;
case BitmapPixelFormat.Gray8:
// Use pseudo color to render 8 bits frames.
result = TransformBitmap(inputBitmap, PseudoColorHelper.PseudoColorFor8BitInfrared);
break;
default:
System.Diagnostics.Debug.WriteLine("Infrared frame in unexpected format.");
break;
}
break;
}
}
}
return result;
}
/// <summary>
/// Transform image into Bgra8 image using given transform method.
/// </summary>
/// <param name="softwareBitmap">Input image to transform.</param>
/// <param name="transformScanline">Method to map pixels in a scanline.</param>
private static unsafe SoftwareBitmap TransformBitmap(SoftwareBitmap softwareBitmap, TransformScanline transformScanline)
{
// XAML Image control only supports premultiplied Bgra8 format.
var outputBitmap = new SoftwareBitmap(BitmapPixelFormat.Bgra8,
softwareBitmap.PixelWidth, softwareBitmap.PixelHeight, BitmapAlphaMode.Premultiplied);
using (var input = softwareBitmap.LockBuffer(BitmapBufferAccessMode.Read))
using (var output = outputBitmap.LockBuffer(BitmapBufferAccessMode.Write))
{
// Get stride values to calculate buffer position for a given pixel x and y position.
int inputStride = input.GetPlaneDescription(0).Stride;
int outputStride = output.GetPlaneDescription(0).Stride;
int pixelWidth = softwareBitmap.PixelWidth;
int pixelHeight = softwareBitmap.PixelHeight;
using (var outputReference = output.CreateReference())
using (var inputReference = input.CreateReference())
{
// Get input and output byte access buffers.
byte* inputBytes;
uint inputCapacity;
((IMemoryBufferByteAccess)inputReference).GetBuffer(out inputBytes, out inputCapacity);
byte* outputBytes;
uint outputCapacity;
((IMemoryBufferByteAccess)outputReference).GetBuffer(out outputBytes, out outputCapacity);
// Iterate over all pixels and store converted value.
for (int y = 0; y < pixelHeight; y++)
{
byte* inputRowBytes = inputBytes + y * inputStride;
byte* outputRowBytes = outputBytes + y * outputStride;
transformScanline(pixelWidth, inputRowBytes, outputRowBytes);
}
}
}
return outputBitmap;
}
/// <summary>
/// A helper class to manage look-up-table for pseudo-colors.
/// </summary>
private static class PseudoColorHelper
{
#region Constructor, private members and methods
private const int TableSize = 1024; // Look up table size
private static readonly uint[] PseudoColorTable;
private static readonly uint[] InfraredRampTable;
// Color palette mapping value from 0 to 1 to blue to red colors.
private static readonly Color[] ColorRamp =
{
Color.FromArgb(a:0xFF, r:0x7F, g:0x00, b:0x00),
Color.FromArgb(a:0xFF, r:0xFF, g:0x00, b:0x00),
Color.FromArgb(a:0xFF, r:0xFF, g:0x7F, b:0x00),
Color.FromArgb(a:0xFF, r:0xFF, g:0xFF, b:0x00),
Color.FromArgb(a:0xFF, r:0x7F, g:0xFF, b:0x7F),
Color.FromArgb(a:0xFF, r:0x00, g:0xFF, b:0xFF),
Color.FromArgb(a:0xFF, r:0x00, g:0x7F, b:0xFF),
Color.FromArgb(a:0xFF, r:0x00, g:0x00, b:0xFF),
Color.FromArgb(a:0xFF, r:0x00, g:0x00, b:0x7F),
};
static PseudoColorHelper()
{
PseudoColorTable = InitializePseudoColorLut();
InfraredRampTable = InitializeInfraredRampLut();
}
/// <summary>
/// Maps an input infrared value between [0, 1] to corrected value between [0, 1].
/// </summary>
/// <param name="value">Input value between [0, 1].</param>
[MethodImpl(MethodImplOptions.AggressiveInlining)] // Tell the compiler to inline this method to improve performance
private static uint InfraredColor(float value)
{
int index = (int)(value * TableSize);
index = index < 0 ? 0 : index > TableSize - 1 ? TableSize - 1 : index;
return InfraredRampTable[index];
}
/// <summary>
/// Initializes the pseudo-color look up table for infrared pixels
/// </summary>
private static uint[] InitializeInfraredRampLut()
{
uint[] lut = new uint[TableSize];
for (int i = 0; i < TableSize; i++)
{
var value = (float)i / TableSize;
// Adjust to increase color change between lower values in infrared images
var alpha = (float)Math.Pow(1 - value, 12);
lut[i] = ColorRampInterpolation(alpha);
}
return lut;
}
/// <summary>
/// Initializes pseudo-color look up table for depth pixels
/// </summary>
private static uint[] InitializePseudoColorLut()
{
uint[] lut = new uint[TableSize];
for (int i = 0; i < TableSize; i++)
{
lut[i] = ColorRampInterpolation((float)i / TableSize);
}
return lut;
}
/// <summary>
/// Maps a float value to a pseudo-color pixel
/// </summary>
private static uint ColorRampInterpolation(float value)
{
// Map value to surrounding indexes on the color ramp
int rampSteps = ColorRamp.Length - 1;
float scaled = value * rampSteps;
int integer = (int)scaled;
int index =
integer < 0 ? 0 :
integer >= rampSteps - 1 ? rampSteps - 1 :
integer;
Color prev = ColorRamp[index];
Color next = ColorRamp[index + 1];
// Set color based on ratio of closeness between the surrounding colors
uint alpha = (uint)((scaled - integer) * 255);
uint beta = 255 - alpha;
return
((prev.A * beta + next.A * alpha) / 255) << 24 | // Alpha
((prev.R * beta + next.R * alpha) / 255) << 16 | // Red
((prev.G * beta + next.G * alpha) / 255) << 8 | // Green
((prev.B * beta + next.B * alpha) / 255); // Blue
}
/// <summary>
/// Maps a value in [0, 1] to a pseudo RGBA color.
/// </summary>
/// <param name="value">Input value between [0, 1].</param>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static uint PseudoColor(float value)
{
int index = (int)(value * TableSize);
index = index < 0 ? 0 : index > TableSize - 1 ? TableSize - 1 : index;
return PseudoColorTable[index];
}
#endregion
/// <summary>
/// Maps each pixel in a scanline from a 16 bit depth value to a pseudo-color pixel.
/// </summary>
/// <param name="pixelWidth">Width of the input scanline, in pixels.</param>
/// <param name="inputRowBytes">Pointer to the start of the input scanline.</param>
/// <param name="outputRowBytes">Pointer to the start of the output scanline.</param>
/// <param name="depthScale">Physical distance that corresponds to one unit in the input scanline.</param>
/// <param name="minReliableDepth">Shortest distance at which the sensor can provide reliable measurements.</param>
/// <param name="maxReliableDepth">Furthest distance at which the sensor can provide reliable measurements.</param>
public static unsafe void PseudoColorForDepth(int pixelWidth, byte* inputRowBytes, byte* outputRowBytes, float depthScale, float minReliableDepth, float maxReliableDepth)
{
// Visualize space in front of your desktop.
float minInMeters = minReliableDepth * depthScale;
float maxInMeters = maxReliableDepth * depthScale;
float one_min = 1.0f / minInMeters;
float range = 1.0f / maxInMeters - one_min;
ushort* inputRow = (ushort*)inputRowBytes;
uint* outputRow = (uint*)outputRowBytes;
for (int x = 0; x < pixelWidth; x++)
{
var depth = inputRow[x] * depthScale;
if (depth == 0)
{
// Map invalid depth values to transparent pixels.
// This happens when depth information cannot be calculated, e.g. when objects are too close.
outputRow[x] = 0;
}
else
{
var alpha = (1.0f / depth - one_min) / range;
outputRow[x] = PseudoColor(alpha * alpha);
}
}
}
/// <summary>
/// Maps each pixel in a scanline from a 8 bit infrared value to a pseudo-color pixel.
/// </summary>
/// /// <param name="pixelWidth">Width of the input scanline, in pixels.</param>
/// <param name="inputRowBytes">Pointer to the start of the input scanline.</param>
/// <param name="outputRowBytes">Pointer to the start of the output scanline.</param>
public static unsafe void PseudoColorFor8BitInfrared(
int pixelWidth, byte* inputRowBytes, byte* outputRowBytes)
{
byte* inputRow = inputRowBytes;
uint* outputRow = (uint*)outputRowBytes;
for (int x = 0; x < pixelWidth; x++)
{
outputRow[x] = InfraredColor(inputRow[x] / (float)Byte.MaxValue);
}
}
/// <summary>
/// Maps each pixel in a scanline from a 16 bit infrared value to a pseudo-color pixel.
/// </summary>
/// <param name="pixelWidth">Width of the input scanline.</param>
/// <param name="inputRowBytes">Pointer to the start of the input scanline.</param>
/// <param name="outputRowBytes">Pointer to the start of the output scanline.</param>
public static unsafe void PseudoColorFor16BitInfrared(int pixelWidth, byte* inputRowBytes, byte* outputRowBytes)
{
ushort* inputRow = (ushort*)inputRowBytes;
uint* outputRow = (uint*)outputRowBytes;
for (int x = 0; x < pixelWidth; x++)
{
outputRow[x] = InfraredColor(inputRow[x] / (float)UInt16.MaxValue);
}
}
}
// Displays the provided softwareBitmap in a XAML image control.
public void PresentSoftwareBitmap(SoftwareBitmap softwareBitmap)
{
if (softwareBitmap != null)
{
// Swap the processed frame to _backBuffer and trigger UI thread to render it
softwareBitmap = Interlocked.Exchange(ref _backBuffer, softwareBitmap);
// UI thread always reset _backBuffer before using it. Unused bitmap should be disposed.
softwareBitmap?.Dispose();
// Changes to xaml ImageElement must happen in UI thread through Dispatcher
var task = _imageElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
async () =>
{
// Don't let two copies of this task run at the same time.
if (_taskRunning)
{
return;
}
_taskRunning = true;
// Keep draining frames from the backbuffer until the backbuffer is empty.
SoftwareBitmap latestBitmap;
while ((latestBitmap = Interlocked.Exchange(ref _backBuffer, null)) != null)
{
var imageSource = (SoftwareBitmapSource)_imageElement.Source;
await imageSource.SetBitmapAsync(latestBitmap);
latestBitmap.Dispose();
}
_taskRunning = false;
});
}
}
}
This is output image, that I've got after conversion issues and grayscale process:
Output image
As for VS version - Visual Studio Enterprise 2017, Version 15.6.1 (to be precise). Once more, thanks in advance for help.

P/Invoke marshaling and unmarshalling 2D array, structure and pointers between C# and unmanaged DLL

Flann C++ library has wrappers for C, C++, Python, Matlab and Ruby but no C# wrapper available. I am trying to create a C# wrapper around flann.dll 32-bit unmanaged DLL downloaded from here.
Being new to PInvoke/marshalling, I am quite certain I am not doing the C# P/Invoke calls to the DLL correctly. I am basically trying to mirror the available Python wrapper in C#. Main areas of confusion are:
I am not sure how to marshal (input) and unmarshal (output) between a 2D managed rectangular array in C# where the argument type is float* i.e. pointer to a query set stored in row major order (according to comments in flann.h).
I am also not sure how I am passing a structure reference to C is correct i.e. struct FLANNParameters*
Is IntPtr appropriate to reference typedef void* and int* indices?
Unmanaged C (flann.dll library)
Public exported C++ methods from flann.h that I need to use are as follows:
typedef void* FLANN_INDEX; /* deprecated */
typedef void* flann_index_t;
FLANN_EXPORT extern struct FLANNParameters DEFAULT_FLANN_PARAMETERS;
// dataset = pointer to a query set stored in row major order
FLANN_EXPORT flann_index_t flann_build_index(float* dataset,
int rows,
int cols,
float* speedup,
struct FLANNParameters* flann_params);
FLANN_EXPORT int flann_free_index(flann_index_t index_id,
struct FLANNParameters* flann_params);
FLANN_EXPORT int flann_find_nearest_neighbors(float* dataset,
int rows,
int cols,
float* testset,
int trows,
int* indices,
float* dists,
int nn,
struct FLANNParameters* flann_params);
Managed C# wrapper (my implementation)
Here is my C# wrapper based on the above publicly exposed methods.
NativeMethods.cs
using System;
using System.Runtime.InteropServices;
namespace FlannWrapper
{
/// <summary>
/// Methods to map between native unmanaged C++ DLL and managed C#
/// Trying to mirror: https://github.com/mariusmuja/flann/blob/master/src/cpp/flann/flann.h
/// </summary>
public class NativeMethods
{
/// <summary>
/// 32-bit flann dll obtained from from http://sourceforge.net/projects/pointclouds/files/dependencies/flann-1.7.1-vs2010-x86.exe/download
/// </summary>
public const string DllWin32 = #"C:\Program Files (x86)\flann\bin\flann.dll";
/// <summary>
/// C++: flann_index_t flann_build_index(float* dataset, int rows, int cols, float* speedup, FLANNParameters* flann_params)
/// </summary>
[DllImport(DllWin32, EntryPoint = "flann_build_index", CallingConvention = CallingConvention.Cdecl, SetLastError = true)]
public static extern IntPtr flannBuildIndex([In] [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.R4)] float[,] dataset, // ??? [In] IntPtr dataset ???
int rows, int cols,
ref float speedup, // ???
[In] ref FlannParameters flannParams); // ???
/// <summary>
/// C++: int flann_free_index(flann_index_t index_ptr, FLANNParameters* flann_params)
/// </summary>
[DllImport(DllWin32, EntryPoint = "flann_free_index", CallingConvention = CallingConvention.Cdecl, SetLastError = true)]
public static extern int flannFreeIndex(IntPtr indexPtr, // ???
[In] ref FlannParameters flannParams); // ??? [In, MarshalAs(UnmanagedType.LPStruct)] FlannParameters flannParams);
/// <summary>
/// C++: int flann_find_nearest_neighbors_index(flann_index_t index_ptr, float* testset, int tcount, int* result, float* dists, int nn, FLANNParameters* flann_params)
/// </summary>
[DllImport(DllWin32, EntryPoint = "flann_find_nearest_neighbors_index", CallingConvention = CallingConvention.Cdecl, SetLastError = true)]
public static extern int flannFindNearestNeighborsIndex(IntPtr indexPtr, // ???
[In] [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.R4)] float[,] testset, // ??? [In] IntPtr dataset ???
int tCount,
[Out] IntPtr result, // ??? [Out] [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.R4)] int[,] result,
[Out] IntPtr dists, // ???
int nn,
[In] ref FlannParameters flannParams); // ???
}
}
FlannTest.cs
using System;
using System.IO;
using System.Runtime.InteropServices;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace FlannWrapper
{
[TestClass]
public class FlannTest : IDisposable
{
private IntPtr curIndex;
protected FlannParameters flannParams;
// protected GCHandle gcHandle;
[TestInitialize]
public void TestInitialize()
{
this.curIndex = IntPtr.Zero;
// Initialise Flann Parameters
this.flannParams = new FlannParameters(); // use defaults
this.flannParams.algorithm = FlannAlgorithmEnum.FLANN_INDEX_KDTREE;
this.flannParams.trees = 8;
this.flannParams.logLevel = FlannLogLevelEnum.FLANN_LOG_WARN;
this.flannParams.checks = 64;
}
[TestMethod]
public void FlannNativeMethodsTestSimple()
{
int rows = 3, cols = 5;
int tCount = 2, nn = 3;
float[,] dataset2D = { { 1.0f, 1.0f, 1.0f, 2.0f, 3.0f},
{ 10.0f, 10.0f, 10.0f, 3.0f, 2.0f},
{ 100.0f, 100.0f, 2.0f, 30.0f, 1.0f} };
//IntPtr dtaasetPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(float)) * dataset2D.Length);
float[,] testset2D = { { 1.0f, 1.0f, 1.0f, 1.0f, 1.0f},
{ 90.0f, 90.0f, 10.0f, 10.0f, 1.0f} };
//IntPtr testsetPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(float)) * testset2D.Length);
int outBufferSize = tCount * nn;
int[] result = new int[outBufferSize];
int[,] result2D = new int[tCount, nn];
IntPtr resultPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(int)) * result.Length);
float[] dists = new float[outBufferSize];
float[,] dists2D = new float[tCount, nn];
IntPtr distsPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(float)) * dists.Length);
try
{
// Copy the array to unmanaged memory.
//Marshal.Copy(testset, 0, testsetPtr, testset.Length);
//Marshal.Copy(dataset, 0, datasetPtr, dataset.Length);
if (this.curIndex != IntPtr.Zero)
{
// n - number of bytes which is enough to keep any type used by function
NativeMethods.flannFreeIndex(this.curIndex, ref this.flannParams);
this.curIndex = IntPtr.Zero;
}
//GC.KeepAlive(this.curIndex); // TODO
float speedup = 0.0f; // TODO: ctype float
Console.WriteLine("Computing index.");
this.curIndex = NativeMethods.flannBuildIndex(dataset2D, rows, cols, ref speedup, ref this.flannParams);
NativeMethods.flannFindNearestNeighborsIndex(this.curIndex, testset2D, tCount, resultPtr, distsPtr, nn, ref this.flannParams);
// Copy unmanaged memory to managed arrays.
Marshal.Copy(resultPtr, result, 0, result.Length);
Marshal.Copy(distsPtr, dists, 0, dists.Length);
// Clutching straws, convert 1D to 2D??
for(int row=0; row<tCount; row++)
{
for(int col=0; col<nn; col++)
{
int buffIndex = row*nn + col;
result2D[row, col] = result[buffIndex];
dists2D[row, col] = dists[buffIndex];
}
}
}
finally
{
// Free unmanaged memory -- [BREAKPOINT HERE]
// Free input pointers
//Marshal.FreeHGlobal(testsetPtr);
//Marshal.FreeHGlobal(datasetPtr);
// Free output pointers
Marshal.FreeHGlobal(resultPtr);
Marshal.FreeHGlobal(distsPtr);
}
}
[TestCleanup]
public void TestCleanup()
{
if (this.curIndex != IntPtr.Zero)
{
NativeMethods.flannFreeIndex(this.curIndex, ref flannParams);
Marshal.FreeHGlobal(this.curIndex);
this.curIndex = IntPtr.Zero;
// gcHandle.Free();
}
}
}
}
FlannParams.cs
Trying to mirror Python FLANNParameters class and C struct FLANNParameters.
using System;
using System.Runtime.InteropServices;
namespace FlannWrapper
{
// FieldOffsets set based on assumption that C++ equivalent of int, uint, float, enum are all 4 bytes for 32-bit
[StructLayout(LayoutKind.Explicit)]
public class FLANNParameters
{
[FieldOffset(0)]
public FlannAlgorithmEnum algorithm;
[FieldOffset(4)]
public int checks;
[FieldOffset(8)]
public float eps;
[FieldOffset(12)]
public int sorted;
[FieldOffset(16)]
public int maxNeighbors;
[FieldOffset(20)]
public int cores;
[FieldOffset(24)]
public int trees;
[FieldOffset(28)]
public int leafMaxSize;
[FieldOffset(32)]
public int branching;
[FieldOffset(36)]
public int iterations;
[FieldOffset(40)]
public FlannCentersInitEnum centersInit;
[FieldOffset(44)]
public float cbIndex;
[FieldOffset(48)]
public float targetPrecision;
[FieldOffset(52)]
public float buildWeight;
[FieldOffset(56)]
public float memoryWeight;
[FieldOffset(60)]
public float sampleFraction;
[FieldOffset(64)]
public int tableNumber;
[FieldOffset(68)]
public int keySize;
[FieldOffset(72)]
public int multiProbeLevel;
[FieldOffset(76)]
public FlannLogLevelEnum logLevel;
[FieldOffset(80)]
public long randomSeed;
/// <summary>
/// Default Constructor
/// Ref https://github.com/mariusmuja/flann/blob/master/src/python/pyflann/flann_ctypes.py : _defaults
/// </summary>
public FlannParameters()
{
this.algorithm = FlannAlgorithmEnum.FLANN_INDEX_KDTREE;
this.checks = 32;
this.eps = 0.0f;
this.sorted = 1;
this.maxNeighbors = -1;
this.cores = 0;
this.trees = 1;
this.leafMaxSize = 4;
this.branching = 32;
this.iterations = 5;
this.centersInit = FlannCentersInitEnum.FLANN_CENTERS_RANDOM;
this.cbIndex = 0.5f;
this.targetPrecision = 0.9f;
this.buildWeight = 0.01f;
this.memoryWeight = 0.0f;
this.sampleFraction = 0.1f;
this.tableNumber = 12;
this.keySize = 20;
this.multiProbeLevel = 2;
this.logLevel = FlannLogLevelEnum.FLANN_LOG_WARN;
this.randomSeed = -1;
}
}
public enum FlannAlgorithmEnum : int
{
FLANN_INDEX_KDTREE = 1
}
public enum FlannCentersInitEnum : int
{
FLANN_CENTERS_RANDOM = 0
}
public enum FlannLogLevelEnum : int
{
FLANN_LOG_WARN = 3
}
}
Incorrect Output - Debug mode, Immediate Window
?result2D
{int[2, 3]}
[0, 0]: 7078010
[0, 1]: 137560165
[0, 2]: 3014708
[1, 0]: 3014704
[1, 1]: 3014704
[1, 2]: 48
?dists2D
{float[2, 3]}
[0, 0]: 2.606415E-43
[0, 1]: 6.06669328E-34
[0, 2]: 9.275506E-39
[1, 0]: 1.05612418E-38
[1, 1]: 1.01938872E-38
[1, 2]: 1.541428E-43
As you can see, I don't get any errors when running Test in Debug mode, but I know the output is definitely incorrect - garbage values as a result of improper memory addressing. I have also included alternative marshaling signatures I tried without any success (please see comments with ???).
Ground truth Python (calling PyFlann library)
To find out the correct result, I implemented a quick test using the available Python library - PyFlann.
FlannTest.py
import pyflann
import numpy as np
dataset = np.array(
[[1., 1., 1., 2., 3.],
[10., 10., 10., 3., 2.],
[100., 100., 2., 30., 1.] ])
testset = np.array(
[[1., 1., 1., 1., 1.],
[90., 90., 10., 10., 1.] ])
flann = pyflann.FLANN()
result, dists = flann.nn(dataset, testset, num_neighbors = 3,
algorithm="kdtree", trees=8, checks=64) # flann parameters
# Output
print("\nResult:")
print(result)
print("\nDists:")
print(dists)
Under the hood, PyFlann.nn() calls the publicly exposed C methods as we can tell from looking at index.py.
Correct Output
Result:
[[0 1 2]
[2 1 0]]
Dists:
[[ 5.00000000e+00 2.48000000e+02 2.04440000e+04]
[ 6.64000000e+02 1.28500000e+04 1.59910000e+04]]
Any help on the correct way to do this would be greatly appreciated. Thanks.
When you're working with p/invoke, you have to stop thinking "managed", and instead think physical binary layout, 32 vs 64 bit, etc. Also, when the called native binary always runs in-process (like here, but with COM servers it can be different) it's easier than out-of-process because you don't have to think too much about marshaling/serialization, ref vs out, etc.
Also, you don't need to tell .NET what it already knows. An array of float is an LPArray of R4, you don't have to specify it. The simpler the better.
So, first of all flann_index_t. It's defined in C as void *, so it must clearly be an IntPtr (an opaque pointer on "something").
Then, structures. Structures passed as a simple pointer in C can just be passed as a ref argument in C# if you define it as struct. If you define it as a class, don't use ref. In general I prefer using struct for C structures.
You'll have to make sure the structure is well defined. In general, you use the LayoutKind.Sequential because .NET p/invoke will pack arguments the same way that the C compiler does. So you don't have to use explicit, especially when arguments are standard (not bit things) like int, float, So you can remove all FieldOffset and use LayoutKind.Sequential if all members are properly declared... but this is not the case.
For types, like I said, you really have to think binary and ask yourself for each type you use, what's its binary layout, size? int are (with 99.9% C compilers) 32-bit. float and double are IEEE standards, so there should never be issues about them. Enums are in general based on int, but this may vary (in C and in .NET, to be able to match C). long are (with 99.0% C compilers) 32-bit, not 64-bit. So the .NET equivalent is Int32 (int), not Int64 (long).
So you should correct your FlannParameters structure and replace the long by an int. If you really want to make sure for a given struct, check Marshal.SizeOf(mystruct) against C's sizeof(mystruct) with the same C compiler than the one that was used to compile the library you're calling. They should be the same. If they're not, there's an error in the .NET definition (packing, member size, order, etc.).
Here are modified definitions and calling code that seem to work.
static void Main(string[] args)
{
int rows = 3, cols = 5;
int tCount = 2, nn = 3;
float[,] dataset2D = { { 1.0f, 1.0f, 1.0f, 2.0f, 3.0f},
{ 10.0f, 10.0f, 10.0f, 3.0f, 2.0f},
{ 100.0f, 100.0f, 2.0f, 30.0f, 1.0f} };
float[,] testset2D = { { 1.0f, 1.0f, 1.0f, 1.0f, 1.0f},
{ 90.0f, 90.0f, 10.0f, 10.0f, 1.0f} };
var fparams = new FlannParameters();
var index = NativeMethods.flannBuildIndex(dataset2D, rows, cols, out float speedup, ref fparams);
var indices = new int[tCount, nn];
var idists = new float[tCount, nn];
NativeMethods.flannFindNearestNeighborsIndex(index, testset2D, tCount, indices, idists, nn, ref fparams);
NativeMethods.flannFreeIndex(index, ref fparams);
}
[DllImport(DllWin32, EntryPoint = "flann_build_index", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr flannBuildIndex(float[,] dataset,
int rows, int cols,
out float speedup, // out because, it's and output parameter, but ref is not a problem
ref FlannParameters flannParams);
[DllImport(DllWin32, EntryPoint = "flann_free_index", CallingConvention = CallingConvention.Cdecl)]
public static extern int flannFreeIndex(IntPtr indexPtr, ref FlannParameters flannParams);
[DllImport(DllWin32, EntryPoint = "flann_find_nearest_neighbors_index", CallingConvention = CallingConvention.Cdecl, SetLastError = true)]
public static extern int flannFindNearestNeighborsIndex(IntPtr indexPtr,
float[,] testset,
int tCount,
[In, Out] int[,] result, // out because it may be changed by C side
[In, Out] float[,] dists,// out because it may be changed by C side
int nn,
ref FlannParameters flannParams);
[StructLayout(LayoutKind.Sequential)]
public struct FlannParameters
{
public FlannAlgorithmEnum algorithm;
public int checks;
public float eps;
public int sorted;
public int maxNeighbors;
public int cores;
public int trees;
public int leafMaxSize;
public int branching;
public int iterations;
public FlannCentersInitEnum centersInit;
public float cbIndex;
public float targetPrecision;
public float buildWeight;
public float memoryWeight;
public float sampleFraction;
public int tableNumber;
public int keySize;
public int multiProbeLevel;
public FlannLogLevelEnum logLevel;
public int randomSeed;
}
note: I've tried to use the specific flann parameters values, but the library crashes in this case, I don't know why...

How to capture a frame of a video using directx

I am fairly new to DirectX and c#I have a new challenge where I am trying to process the frames of video(60 fps) coming as a video stream over HDMI from another PC(using Directx C#). I am using video capture card for capturing video. Moreover my piece of code enables me to capture the video perfectly.
However, I have a requirement where I need to be able to process the frames of video at the same time when it is streaming(may be in a separate thread).
I have tried using AForge library to capture the frames but that only works with the integrated web camera.When I try yo run this with capture card it only shows a black screen
Any pointers or links for reference will be really appreciated.
Finally I found the solution myself..The frames of the video stream can be extracted using Directshow's SampleGrabber method.
/// <summary> Interface frame event </summary>
public delegate void HeFrame(System.Drawing.Bitmap BM);
/// <summary> Frame event </summary>
public event HeFrame FrameEvent2;
private byte[] savedArray;
private int bufferedSize;
int ISampleGrabberCB.BufferCB(double SampleTime, IntPtr pBuffer,
int BufferLen )
{
this.bufferedSize = BufferLen;
int stride = this.SnapShotWidth * 3;
Marshal.Copy( pBuffer, this.savedArray, 0, BufferLen );
GCHandle handle = GCHandle.Alloc( this.savedArray, GCHandleType.Pinned );
int scan0 = (int) handle.AddrOfPinnedObject();
scan0 += (this.SnapShotHeight - 1) * stride;
Bitmap b = new Bitmap(this.SnapShotWidth, this.SnapShotHeight, -stride,
System.Drawing.Imaging.PixelFormat.Format24bppRgb, (IntPtr) scan0 );
handle.Free();
SetBitmap=b;
return 0;
}
/// <summary> capture event, triggered by buffer callback. </summary>
private void OnCaptureDone()
{
Trace.WriteLine( "!!DLG: OnCaptureDone" );
}
/// <summary> Allocate memory space and set SetCallBack </summary>
public void GrapImg()
{
Trace.Write ("IMG");
if( this.savedArray == null )
{
int size = this.snapShotImageSize;
if( (size < 1000) || (size > 16000000) )
return;
this.savedArray = new byte[ size + 64000 ];
}
sampGrabber.SetCallback( this, 1 );
}
/// <summary> Transfer bitmap upon firing event </summary>
public System.Drawing.Bitmap SetBitmap
{
set
{
this.FrameEvent2(value);
}
}
Here is the link to the Article.Might help some one as it took me a lot of time to get to this.

open a page in IE using c#

I'm opening a webpage in IE using c#
Code:
Process.Start("IExplore.exe", "www.northwindtraders.com");
But, how can I resize the page with the following characteritics: toolbar=no, location=no, status=no, menubar=no, scrollbars=yes, resizable=yes, width='622', height='582'
For that level of fine grained control which I do not believe is available from the command line add a reference (COM) to Microsoft Internet Controls and then you can:
var IE = new SHDocVw.InternetExplorer();
object URL = "http://www.northwindtraders.com";
IE.ToolBar = 0;
IE.StatusBar = false;
IE.MenuBar = false;
IE.Width = 622;
IE.Height = 582;
IE.Visible = true;
IE.Navigate2(ref URL);
These are the features for opening Internet Explorer from Command Shell:
http://msdn.microsoft.com/en-us/library/hh826025(v=vs.85).aspx
You need to use the StartInfo property to set some startup parameters.
Check this link for what options are available.
You can do it without a reference to SHDocVw, by using the COM-object directly:
const int FormWidth = 800;
const int FormHeight = 600;
// http://www.c-sharpcorner.com/forums/thread/51998/opening-a-browser-and-hiding-the-address-bar-help.aspx
// http://weblog.west-wind.com/posts/2005/Apr/29/Previewing-HTML-with-InternetExplorerApplication-in-C
// http://social.msdn.microsoft.com/Forums/vstudio/en-US/ab6969c7-0a34-4d88-9a74-b66888d3d88f/ie-automation-navigating-the-soup
// http://superuser.com/questions/459775/how-can-i-launch-a-browser-with-no-window-frame-or-tabs-address-bar
public void OpenBrowserWindow(string strURL)
{
// System.Diagnostics.Process.Start(strURL)
// For Internet Explorer you can use -k (kiosk mode):
// iexplore.exe -k http://www.google.com/
// Internet Explorer Command-Line Options
// http://msdn.microsoft.com/en-us/library/ie/hh826025(v=vs.85).aspx
// Starts Internet Explorer in kiosk mode. The browser opens in a maximized window that does not display the address bar,
// the navigation buttons, or the status bar.
System.Drawing.Rectangle rect = System.Windows.Forms.Screen.PrimaryScreen.Bounds;
// http://stackoverflow.com/questions/5049122/capture-the-screen-shot-using-net
int posX = rect.Width - FormWidth;
int posY = rect.Height - FormHeight;
posX = Convert.ToInt32 (posX / 2.0);
posY = Convert.ToInt32 (posY / 2.0);
posX = Math.Max (posX, 0);
posY = Math.Max (posY, 0);
System.Type oType = System.Type.GetTypeFromProgID ("InternetExplorer.Application");
object o = System.Activator.CreateInstance (oType);
o.GetType ().InvokeMember ("MenuBar", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { 0 });
o.GetType ().InvokeMember ("ToolBar", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { 0 });
o.GetType ().InvokeMember ("StatusBar", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { 0 });
o.GetType ().InvokeMember ("AddressBar", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { 0 });
o.GetType ().InvokeMember ("Visible", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { true });
o.GetType ().InvokeMember ("Top", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { posY });
o.GetType ().InvokeMember ("Left", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { posX });
o.GetType ().InvokeMember ("Width", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { FormWidth });
o.GetType ().InvokeMember ("Height", System.Reflection.BindingFlags.SetProperty, null, o, new object[] { FormHeight });
o.GetType ().InvokeMember ("Navigate", System.Reflection.BindingFlags.InvokeMethod, null, o, new object[] { strURL });
try
{
object ohwnd = o.GetType ().InvokeMember ("hwnd", System.Reflection.BindingFlags.GetProperty, null, o, null);
System.IntPtr IEHwnd = (System.IntPtr)ohwnd;
// NativeMethods.SetForegroundWindow (IEHwnd);
NativeMethods.ShowWindow(IEHwnd, NativeMethods.WindowShowStyle.ShowMaximized);
} catch (Exception ex) {
}
} // OpenBrowserWindow
public class NativeMethods
{
[System.Runtime.InteropServices.DllImport("user32.dll")]
[return: System.Runtime.InteropServices.MarshalAs(
System.Runtime.InteropServices.UnmanagedType.Bool)]
public static extern bool SetForegroundWindow(IntPtr hWnd);
[System.Runtime.InteropServices.DllImport ("user32.dll")]
[return: System.Runtime.InteropServices.MarshalAs(
System.Runtime.InteropServices.UnmanagedType.Bool)]
public static extern bool ShowWindow(IntPtr hwnd, WindowShowStyle nCmdShow);
/// <summary>Enumeration of the different ways of showing a window using
/// ShowWindow</summary>
public enum WindowShowStyle : int
{
/// <summary>Hides the window and activates another window.</summary>
/// <remarks>See SW_HIDE</remarks>
Hide = 0,
/// <summary>Activates and displays a window. If the window is minimized
/// or maximized, the system restores it to its original size and
/// position. An application should specify this flag when displaying
/// the window for the first time.</summary>
/// <remarks>See SW_SHOWNORMAL</remarks>
ShowNormal = 1,
/// <summary>Activates the window and displays it as a minimized window.</summary>
/// <remarks>See SW_SHOWMINIMIZED</remarks>
ShowMinimized = 2,
/// <summary>Activates the window and displays it as a maximized window.</summary>
/// <remarks>See SW_SHOWMAXIMIZED</remarks>
ShowMaximized = 3,
/// <summary>Maximizes the specified window.</summary>
/// <remarks>See SW_MAXIMIZE</remarks>
Maximize = 3,
/// <summary>Displays a window in its most recent size and position.
/// This value is similar to "ShowNormal", except the window is not
/// actived.</summary>
/// <remarks>See SW_SHOWNOACTIVATE</remarks>
ShowNormalNoActivate = 4,
/// <summary>Activates the window and displays it in its current size
/// and position.</summary>
/// <remarks>See SW_SHOW</remarks>
Show = 5,
/// <summary>Minimizes the specified window and activates the next
/// top-level window in the Z order.</summary>
/// <remarks>See SW_MINIMIZE</remarks>
Minimize = 6,
/// <summary>Displays the window as a minimized window. This value is
/// similar to "ShowMinimized", except the window is not activated.</summary>
/// <remarks>See SW_SHOWMINNOACTIVE</remarks>
ShowMinNoActivate = 7,
/// <summary>Displays the window in its current size and position. This
/// value is similar to "Show", except the window is not activated.</summary>
/// <remarks>See SW_SHOWNA</remarks>
ShowNoActivate = 8,
/// <summary>Activates and displays the window. If the window is
/// minimized or maximized, the system restores it to its original size
/// and position. An application should specify this flag when restoring
/// a minimized window.</summary>
/// <remarks>See SW_RESTORE</remarks>
Restore = 9,
/// <summary>Sets the show state based on the SW_ value specified in the
/// STARTUPINFO structure passed to the CreateProcess function by the
/// program that started the application.</summary>
/// <remarks>See SW_SHOWDEFAULT</remarks>
ShowDefault = 10,
/// <summary>Windows 2000/XP: Minimizes a window, even if the thread
/// that owns the window is hung. This flag should only be used when
/// minimizing windows from a different thread.</summary>
/// <remarks>See SW_FORCEMINIMIZE</remarks>
ForceMinimized = 11
}
}
You can also use VirtualScreen
System.Windows.Forms.SystemInformation.VirtualScreen.Width;
System.Windows.Forms.SystemInformation.VirtualScreen.Height;
instead of PrimaryScreen
Another way is to use window messages this is thrown together from PInvoke.net
void Main()
{
Process p = Process.Start(
#"C:\Program Files (x86)\Internet Explorer\iexplore.exe" , "www.google.com" );
while(p.MainWindowHandle == IntPtr.Zero )
{
Thread.Sleep(100);
p.Refresh();
}
SetWindowPos( p.MainWindowHandle , IntPtr.Zero , 100 , 100, 800 ,400 , SetWindowPosFlags.ShowWindow );
}
// Define other methods and classes here
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool SetWindowPos(IntPtr hWnd, IntPtr hWndInsertAfter, int X, int Y, int cx, int cy, SetWindowPosFlags uFlags);
static readonly IntPtr HWND_TOPMOST = new IntPtr(-1);
static readonly IntPtr HWND_NOTOPMOST = new IntPtr(-2);
static readonly IntPtr HWND_TOP = new IntPtr(0);
static readonly IntPtr HWND_BOTTOM = new IntPtr(1);
/// <summary>
/// Window handles (HWND) used for hWndInsertAfter
/// </summary>
public static class HWND
{
public static IntPtr
NoTopMost = new IntPtr(-2),
TopMost = new IntPtr(-1),
Top = new IntPtr(0),
Bottom = new IntPtr(1);
}
/// <summary>
/// SetWindowPos Flags
/// </summary>
public static class SWP
{
public static readonly int
NOSIZE = 0x0001,
NOMOVE = 0x0002,
NOZORDER = 0x0004,
NOREDRAW = 0x0008,
NOACTIVATE = 0x0010,
DRAWFRAME = 0x0020,
FRAMECHANGED = 0x0020,
SHOWWINDOW = 0x0040,
HIDEWINDOW = 0x0080,
NOCOPYBITS = 0x0100,
NOOWNERZORDER = 0x0200,
NOREPOSITION = 0x0200,
NOSENDCHANGING = 0x0400,
DEFERERASE = 0x2000,
ASYNCWINDOWPOS = 0x4000;
}
[Flags()]
private enum SetWindowPosFlags : uint
{
/// <summary>If the calling thread and the thread that owns the window are attached to different input queues,
/// the system posts the request to the thread that owns the window. This prevents the calling thread from
/// blocking its execution while other threads process the request.</summary>
/// <remarks>SWP_ASYNCWINDOWPOS</remarks>
AsynchronousWindowPosition = 0x4000,
/// <summary>Prevents generation of the WM_SYNCPAINT message.</summary>
/// <remarks>SWP_DEFERERASE</remarks>
DeferErase = 0x2000,
/// <summary>Draws a frame (defined in the window's class description) around the window.</summary>
/// <remarks>SWP_DRAWFRAME</remarks>
DrawFrame = 0x0020,
/// <summary>Applies new frame styles set using the SetWindowLong function. Sends a WM_NCCALCSIZE message to
/// the window, even if the window's size is not being changed. If this flag is not specified, WM_NCCALCSIZE
/// is sent only when the window's size is being changed.</summary>
/// <remarks>SWP_FRAMECHANGED</remarks>
FrameChanged = 0x0020,
/// <summary>Hides the window.</summary>
/// <remarks>SWP_HIDEWINDOW</remarks>
HideWindow = 0x0080,
/// <summary>Does not activate the window. If this flag is not set, the window is activated and moved to the
/// top of either the topmost or non-topmost group (depending on the setting of the hWndInsertAfter
/// parameter).</summary>
/// <remarks>SWP_NOACTIVATE</remarks>
DoNotActivate = 0x0010,
/// <summary>Discards the entire contents of the client area. If this flag is not specified, the valid
/// contents of the client area are saved and copied back into the client area after the window is sized or
/// repositioned.</summary>
/// <remarks>SWP_NOCOPYBITS</remarks>
DoNotCopyBits = 0x0100,
/// <summary>Retains the current position (ignores X and Y parameters).</summary>
/// <remarks>SWP_NOMOVE</remarks>
IgnoreMove = 0x0002,
/// <summary>Does not change the owner window's position in the Z order.</summary>
/// <remarks>SWP_NOOWNERZORDER</remarks>
DoNotChangeOwnerZOrder = 0x0200,
/// <summary>Does not redraw changes. If this flag is set, no repainting of any kind occurs. This applies to
/// the client area, the nonclient area (including the title bar and scroll bars), and any part of the parent
/// window uncovered as a result of the window being moved. When this flag is set, the application must
/// explicitly invalidate or redraw any parts of the window and parent window that need redrawing.</summary>
/// <remarks>SWP_NOREDRAW</remarks>
DoNotRedraw = 0x0008,
/// <summary>Same as the SWP_NOOWNERZORDER flag.</summary>
/// <remarks>SWP_NOREPOSITION</remarks>
DoNotReposition = 0x0200,
/// <summary>Prevents the window from receiving the WM_WINDOWPOSCHANGING message.</summary>
/// <remarks>SWP_NOSENDCHANGING</remarks>
DoNotSendChangingEvent = 0x0400,
/// <summary>Retains the current size (ignores the cx and cy parameters).</summary>
/// <remarks>SWP_NOSIZE</remarks>
IgnoreResize = 0x0001,
/// <summary>Retains the current Z order (ignores the hWndInsertAfter parameter).</summary>
/// <remarks>SWP_NOZORDER</remarks>
IgnoreZOrder = 0x0004,
/// <summary>Displays the window.</summary>
/// <remarks>SWP_SHOWWINDOW</remarks>
ShowWindow = 0x0040,
}

Categories

Resources