Gesture Mouse - The Core Program in Sixth Sense

                   

Gesture recognition is a language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Current focuses in the field include emotion recognition from the face and hand gesture recognition. Many approaches have been made using cameras and computer vision algorithms to interpret sign language. However, the identification and recognition of posture, gait, proxemics, and human behaviors is also the subject of gesture recognition techniques. “GESTURE MOUSE” is an invisible computer mouse that provides the acquaintance of interface of a physical mouse without actually needing a real hardware mouse. As we improve our computer vision algorithms, an extensive library of gestures could be implemented in addition to mouse movement and mouse clicks.

         Gesture recognition can be seen as a way for computers to begin to understand human body language, thus building a richer bridge between machines and humans than primitive text user interfaces or even GUIs (graphical user interfaces), which still limit the majority of input to keyboard and mouse. The proposed gesture mouse system consists of a camera to obtain the video and frames of the hand and a CPU for processing the video. The gesture recognition algorithms are quiet complex. A number of contending software tools can be used to implement Gesture recognition. The prospective candidates are MATLAB and Scientific Python (SciPy). Python has the advantage of better interfacing with system level operation, better memory management and efficiency over MATLAB. It was identified to use SciPy with OpenCV library for implementing Gesture Mouse.

     

         Gesture  Mouse  project got  it’s origin  under  the motivation  of  Pranav Mistry’s  Sixth Sense.  So  this is a part of  the big  project done  by Pranav Mistry with  5 years  effort. User  requirement   of  this  project is world  wide.  Sixth Sense is  a  wearable gestural interface that augments the physical world  around us  with digital information  and  lets us use natural hand gestures to interact with that  information. The main purpose of our project is to replace the hardware mouse with the gesture mouse. In gesture mouse we use our hand gestures to employ the mouse functions.

        

Sixth Sense - The Future is Here !!

            Sixth Sense is an awesome project initiated by Pranav Mistry, a Reseach Assistant and Phd candidate at the Massachusetts Institute of Technology (M.I.T) Media Lab.

            Usually we go to ATM and interact with machine or when we go to railway station and search information in kiosk. So what we are doing here is moving our Physical world to the place where we get Digital information. But what sixth sense is offering is that we are able to interact with the digital world on the fly.



A Picture is worth a Thousand words. Moreover, A Video is worth a Thousand Pictures.

Just buffer out this video and you will see what is Sixth Sense and Why everyone is assuring it's the future.

            

                                                                   

Simple definition is that "Sixth Sense is a wearable gestural interface that let us bring our physical world to interact with the digital world on the fly"

  So the core components behind this mega project are:

1>A smart phone capable of processing the data and program execution

2>A pico projector

3>Camera to capture the gestures

4>Colored Markers which will help to detect the fingers.

The Architecture on how the sixth sense works is given below:

Sensor captures the gestures/screen and then it is processed by the program. The result of the program is the decisions and the movement of mouse pointer based on the gesture movement. This result will be given to the applications using like games, web browser etc. User can have a display using projector or on a monitor.

             This Mega Project consists of simple components but an Intelligent Program Code.

Gesture Mouse or Air mouse is the major core in this project. It will be explained in the next post. Wait till then!!....

Birth of SIxth Sense in our B-tech Project

In my post Opening the doors of Linux I have mentioned the inspiration behind choosing python and being familiar with Linux OS for our last semester B-tech Project. Before all these storys we wanted to choose our project topic. Our team had a wish to choose an innovative topic rather than what our H.O.D says "reinventing the wheel".

                   Spend hours in net, went to our project institution Castalia Labs.. had talks with them. They had a list of topics what they  have did with students like stereo vision, traffic signal recognition, measurements using webcam etc. Though it were the best topics to choose we wanted something more. When we were having talk with Mirshad and Jemshid sir suddenly my team mate visakh got an idea from what he read in an article..."Sixth Sense"..Though it was a big master project..they didn't let us go down. They told they can help us with the project "Gesture Mouse".

                             Gesture Mouse is a small major  part of Sixth Sense project which needs digital image processing. So they supported us with two options either MATLAB or PYTHON. As i told later, we chose Python and Ubuntu 11.04 os.  We had a full support for choosing these as the castalia labs were the promoters of open source platform.

                           So..its time to know more about Sixth Sense. Got into the ocean of  net..searched a lot...got more information about Pranav Mistry's Sixth Sense.