Retrieve gaze coordinate on screen - c#

I am implementing an eye tracker using emgucv (openCV C# wrapper), So far i was able to detect iris center and eye corner accurately.
As the next step i want to get the screen coordinate where user is focusing (also known as gaze point),As a beginner to image processing , i am completely unaware of gaze mapping and gaze estimation.
I would be thankful , if you provide any code snippets or algoritms to perform gaze mapping to retrieve gaze coordinate on screen .
Thanks in advance

If you want to research the field in more depth, there's this interesting piece of research called "EyeTab: Model-based gaze estimation on unmodified tablet computers". It might have some of the information you want, or at least help you understand the field more.
http://www.cl.cam.ac.uk/research/rainbow/projects/eyetab/
(also GitHub)

Related

Camera calibration and predict the eye focusing point on screen OpenCV / EmguCV

I have a project idea that check web usability using eye tracking. for that I needed to predict the focusing point on the screen(i.e. pixel points on screen) in specific time gap(0.5 second).
Here is some additional information:
I intended to use openCV or emguCV but it causing me a bit of trouble beacuse of my inexperience with OpenCV.
I am planning to "flatten" the eye so it appears to move on a plane. The obvious choice is to calibrate the camera to try to remove the radial distortion.
During the calibartion process the user looks at the corners of a grid on a screen. The moments of the pupil are stored in a Mat for each position during the calibration. So I have an image with the dots corresponding to a number of eye postions when looking at corners of a grid on the screen.
is there any article or example I can refer to get a good idea about this scenario and openCV eye prediction??
Thanks!
Different methods of camera calibration are possible and (similar/as like to corner dots method)
Thesis work on Eye Gaze using C++ and openCV and this should for sure help you.You can find some opencv based c++ scripts as well.
FYI :
Some works are presented where it claims Eye Gaze without calibration
Calibration-Free Gaze Tracking an experimental analysis by Maximilian Möllers
Free head motion eye gaze tracking without calibration
[I am restricted to post less than 2 reference links]
To get precise locations of eyes, you need first to calibrate your camera, using Chessboard approach or some other tools. Then you need to undistort the image, if it is not straight.
OpenCV comes already with an eye detector (Haar classifier, refer to eye.xml file), so you can already locate it and track it easily.
Besides that, there's only math to help you matching the discovered eye to the location it is looking at.

Kinect front light cause grip detection errors

I'm writing an application that use kinect for windows (not the new full hd kinect) with sdk 1.8. I'm using kinect region and kinectScrollviewer. In front of the kinect camera there's a window, so the subject who must be tracked has the sun light source that comes from behind. When the light is intense, kinect loose the ability to detect correctly the grab and release interaction (sometimes the events of grab or release are fired randomly). My application will run in environments where this kind of lights conditions will be possible. My question is, there's some parameters in skeletal stream or something else that i can set to change the sensibility to this kind of light?
The Kinect sensor works using infrared light, which the IR rays in the sun will disturb (you can see this in the depth map when you have direct sunlight, see this link: http://dasl.mem.drexel.edu/wiki/index.php/KINECT_in_direct_sunlight), so unfortunately there isn't much you can do about it except trying to get rid of the sunlight.
Here is another video of how the depth sensing works: http://www.youtube.com/watch?v=uq9SEJxZiUg

kinect c# draw and move skeleton from saved data

What I want to do is draw and animate a skeleton (like we can do with the sensor stream) from saved data (so I have x, y and z value of every joint).
I searched a lot, but I can't find anything that can help me.
I can convert the data to a joints collection, associate it to a skeleton, but then? I don't know how to map the skeleton to the colorImagePoint.
Maybe I have to create a depthImageFrame?
Thank you so much!
Look into the Kinect Toolbox. It offers a recorder and playback functionality which may match your needs as is, or provide you with a starting point:
http://kinecttoolbox.codeplex.com/
If you role your own, I'm not sure why you would need to map it to a color or depth frame, unless I'm missing a requirement of what you are doing.
Have a look at the SkeletonBasics example in the Microsoft Kinect for Windows SDK Toolkit examples. It will show you have to draw a skeleton manually based on skeleton data. From there, you could look into doing the following for you application:
Set up your skeleton tracking callback
At each skeleton frame, or less (if you don't need so many) save the joint positions
Also save a 0-based timestamp
Save data to format of choice when complete
During a playback, read in your recorded data and start a timer. When the timer hits the next skeleton frame's stored timestamp update your drawn skeleton on the screen (using the SkeletonBasics example app as guidance).

fingertip detection from depth Image using kinect (c# and wpf)

I'm making a project for my university, I'm working with kinect and i want to detect the finger tips using microsoft sdk.I made depth segmentation based on the closest object and player so i now have only the hand part in my image. But I have no idea what to do next. I'm trying to get the hand's boundary but not finding the method to do it.
so can anyone help me and suggest some usefull ways or ideas to do that.
I am using c# and wpf
It will be great if anyone can provide a sample code. I only want to detect the index finger and the thumb.
Thanks in advance
Btw about finding the boundaries. Think about this.
Kinect gives you position of your hand. Your hand is on some Z position. It`s the distance from the Kinect device. Now every pixel (from depth stream) of your hand is almoust on the same Z position as the join of your hand. Your boundarie will be an pixel that is much more in background than your hand. This way you can get boundaries in every direction for your hand. You just need to analyze pixel data
Hope it helped
Try this download, it is Kinect finger detection, and get the "Recommended" download on the page, I have found the .dlls in it work well. Good Luck!

How to draw an unfilled square on top of a stream video using a mouse and track the object enclosed in the square automatically in C#?

I am making an object tracking application. I have used Emgucv 2.1.0.0
to load a video file
to a picturebox. I have also taken the video stream from a web camera.
Now, I want
to draw an unfilled square on the video stream using a mouse and then track the object enclosed
by the unfilled square as the video continues to stream.
This is what people have suggested so far:-
(1) .NET Video overlay drawing(DirectX) - but this is for C++ users, the suggester
said that there are .NET wrappers, but I had a hard time finding any.
(2) DxLogo sample
DxLogo – A sample application showing how to superimpose a logo on a data stream.
It uses a capture device for the video source, and outputs the result to a file.
Sadly, this does not use a mouse.
(3) GDI+ and mouse handling - this area I do not have a clue.
And for tracking the object in the square, I would appreciate if someone give me some research paper links to read.
Any help as to using the mouse to draw on a video is greatly appreciated.
Thank you for taking the time to read this.
Many Thanks
It sounds like you want to do image detection and / or tracking.
The EmguCV ( http://www.emgu.com/wiki/index.php/Main_Page ) library provides a good foundation for this sort of thing in .Net.
e.g. http://www.emgu.com/wiki/index.php/Tutorial#Examples
It's a pretty meaty subject with quite a few years and different branches of research associated with it so I'm not sure anyone can give the definitive guide to such things but reading up neural networks and related topics would give you a pretty good grounding in the way EmguCV and related libraries manage it.
It should be noted that systems such as EmguCV are designed to recognise predefined items within a scene (such as a licence plate number) rather than an arbitory feature within a scene.
For arbitory tracking of a given feature, a search for research papers on edge detection and the like (in combination with a library such a EmguCV) is probably a good start.
(You also may want to sneak a peek at an existing application such as http://www.pfhoe.com/ to see if it fits your needs)

Categories

Resources