Fast Algorithm for Editing Frames of Video Using Vision-Based Hand Gesture Recognition

 

Fast Algorithm for Editing Frames of Video Using Vision-Based Hand Gesture Recognition

Ms.Shaikh Shabnam Shafi Ahmed

Department of Computer Engineering Shri.Jagdish Prasad Jhabarmal Tibrewala University,

Rajasthan, India

Dr. S K Yadav
Director Research
Shri.Jagdish Prasad Jhabarmal Tibrewala University,
Rajasthan, India

Dr.Shah Aqueel Ahmed

Department of Electronics & Instrumentation Engineering Royal Institute of Technology & Science, Hyderabad, India

Prof. Sayyad Farook Bashir

Department of Mechanical Engineering

Genba Sopanrao Moze College of Engineering, Pune, India

Abstract

 

With the rapid emergence of 3D applications and virtual environments in computer systems; the need for a new type of interaction device arises. This is because the traditional devices such as mouse, keyboard, and joystick become inefficient and cumbersome within these virtual environments. In other words, evolution of user interfaces shapes the change in the Human- Computer Interaction (HCI). The aim of this paper is to expand the ways that people are able to interact with their computers to run the video player. Namely, we wanted to enable users to interact more naturally with their video player through using simple hand gestures to perform various functions. In this paper, a machine- user interface is developed which implements gesture recognition using simple computer vision techniques. The interface allows the user to use their bare hands to control the video player. Hand gesture recognition is a challenging problem in its general form. The process is based on vision-based hand-gesture recognition and tracking. Fast algorithm is used for automatically recognizing a limited set of hand gestures. A fixed set of manual commands are considered and a reasonably structured environment and develop a simple and effective procedure for gesture recognition.

Keywords

Hand gesture recognition, Tracking, Fast Algorithm, Skin Detection, Segmenting.

1. IntroductionThe idea was taken from Attila Licsar and Tamas Sziranyi [1]. This interface is simple enough to be run using an ordinary webcam and requires little training. The use of hand gestures provides an attractive alternative to cumbersome interface devices for human- computer interaction (HCI). In particular, visual interpretation of hand gestures can help in achieving the ease and naturalness desired for HCI. The goal of the paper is to develop a real-time system capable of understanding commands given by hand gestures. The main focus of the research was on bare-hands utilizing a simple web camera with a frame rate of approximately 30 frames per second to communicate to computer all basic commands required by a human- computer interface. Hand postures or static gestures are gestures that do not depend on motion. Dynamic gestures on the other hand require motion and are based on the trajectory that is formed during the motion in question.

   The system follows a hybrid approach. It recognizes both motion-based and static hand gestures. To implement the algorithm given real-time constraints was one of the most difficult tasks. The system is a novel application that allows communication of most necessary commands to computer. Further, the system can work with any camera that supports streaming video input to the computer. Its touch less interactive systems and mouse replacement solutions utilizes advanced computer vision to convert simple hand movements into direct mouse control in any environment.

   Thus, the system is being able to provide the following functions:-Stop the video by making using of the four finger count gesture. Play/pause the video by making using of the three count gesture. The hand gesture of three finger count to the right allows to fast playing the video. The hand gesture of three finger count to the left allows to slow play of the video. The hand gesture of two counts allows copying the frame of the video. The two finger count gesture performed to the right allows making the volume of video increase and two finger counts to the left make the volume of the video decrease. The previous and the next frame can be moved to using the finger count one to the left and count one to the right respectively. It is easy to access.

2. Early approaches 

   Asanterabi Malima et. al. [2] approach to the hand gesture recognition problem a robot control context involved the use of markers on the finger tips. An associated algorithm is used to detect the presence and color of the markers, through which one can identify which fingers are active in the gesture. The inconvenience of placing markers on the users hand makes this an infeasible approach in practice. Vafadar and Alireza Behrad [3] used Template Based: In this approach the data obtained is compared against some reference data and using the thresholds, the data is categorized into one of the gestures available in the reference data. This is a simple approach with little calibration but suffers from noise and doesnt work with overlapping gestures.

   Byung-Woo Min et. al. [4] developed Hidden Markov Model (HMM) which is commonly used and has been widely exploited for temporal gesture recognition. An HMM consists of states and state transitions with observation probabilities. For watch gesture a separate HMM is trained and the recognition of the gesture is based on the generation of maximum probability by a particular HMM. This method also suffers from training time involved and complex working nature as the results are unpredicted because of the hidden nature. For the gesture recognition,

   Wing Kwong Chung et.al.[5] has presented a hand gesture recognition modal based on A Real-time Hand Gesture Recognition based on Haar Wavelet Representation.In addition to voice and controller pads, hand gestures can also be an effective way of communication between humans and robots or even between auditory handicapped people and robots.

   Mu-Chun Su [6] suggested a method using Neural Network which is based on modeling of the human nervous system element called neuron and its interaction with the other neurons to transfer the

   information. Each node consists of and the input function which computes the weighted sum and the activation function to generate the response based on the weighted sum.

   Byung-Woo Min et al.[7] developed a method for gesture recognition using Hidden Markov Model. This method has been widely exploited for temporal gesture recognition. Bhuyan, et.al. [8] have proposed the advantage of VOP based method for segmentation of hand image. The proposed acceleration feature works efficiently only when the spatial end position of preceding gesture is different from start position of next gesture in the connected gesture sequence.

   Shewta and Pankaj [9] have proposed that ANN provides a good and powerful solution for gesture recognition. Artificial Neural Networks are applicable to multivariate non-linear problems .It has fast computational ability. Gesture recognition is an important for developing alternative human-computer interaction modalities.

   Zhou Ren et.al. [10] have worked in the direction of hand gesture recognition by making use of kinetic sensor which is very much different from the normal web camera. Hand gesture based Human-Computer- Interaction (HCI) is one of the most natural and intuitive ways to communicate between people and machines, since it closely mimics how human interact with each other. Hamid A Jalab [11] have proposed the succeeds to extract features from hand gesture image based on hand segmentation using both wavelet network an ANN.

   Qing and Nicolas [12] proposed a method that used the formal grammar to represent the hand gstures and postures however limited. This method involves simple gestures requiring the fingers to be extended in various configurations which are mapped to the formal grammar specified by specific tokens and rules. The system involves tracker and glove. This system has poor accuracy and very limited gesture set.

   Lee and Yangsheng Xu [13] developed a glove- based gesture recognition system that was able to recognize 14 of the letters from the hand alphabet, learn new gestures and able to update the model of each gesture in the system in online mode, with a rate of 10Hz. Over the years advanced glove devices have been designed such as the Sayre Glove, Dexterous Hand Master and Power Glove

   Spatio-temporal vector Analysis method was proposed by Vafadar and Behrad [14] which used to track the movement of the hand in the images of the scene and track the motion in the sequence of image. The information about the motion is obtained by the

   derivatives and it is assumed that under static background, hand motion is the fastest changing object of the scene. Then using the refinement and variance constraint flow field is refined. This flow field captures the characteristics of the given gesture.

3. Hand gesture recognitionConsider a navigation problem, in which a user responds to the hand pose signs given by a human, captured through a camera. The interest is in an algorithm that enables to identify a hand pose sign in the input image, as one of five possible commands (or counts). The identified command will then be used as a control input for controlling the video file to perform the desired action or execute a certain task. For examples of the signs to be used in the algorithm, see Figure1.The signs could be associated with various meanings depending on the function. For example, a one count towards left means move previous, a two count is used for adjusting volumes three count is used for play/pause a four count is used for zoom, and finally a five count is used for Alt Tab.

    

   Figure1. Set of hand gestures, or counts considered in the paper.

   The scheme described in the previous section could produce many disconnected regions in the image classified as hand-like. We use ideas from region-based segmentation Asanterabi Malima et. al. [2] to alleviate this problem. Our assumption is that the largest connected white region corresponds to the hand. So we use a relative region size threshold to eliminate the undesired regions. In particular, we remove the regions that contain smaller number of pixels than a threshold value. The threshold value is chosen as 20% of total number of pixels in the white parts. Note that this is an image-size invariant scheme. The ideal outcome is the segmented hand region.

4. System overviewThe picture restoration using hand gestures is developed with the intention to make it easy to control the video files using hand gestures which has features

   of Java like object-oriented methodology, platform independence, and numerous APIs for tasks such as network programming, XML processing, and GUI building. The software is useful for those developers who want to reap benefits of Java.

   The purpose of this method is to collect, analyze and define high-level needs and features of the system for picture restoration using hand gestures. It focuses on the capabilities needed by the stakeholders, and the target users, and why these needs exist. The details of how the system fulfils these needs are detailed in the use-case and supplementary specifications.

5. Objectives of method
   • To capture the image of the hand and identify the finger counts.
   • To correctly count the fingers and perform the desired action associated with each finger count.
   • To perform the action that has been decided to do with the particular finger count.
   • To perform various control operations on the video file.

   The project is mainly concerned with the recognition of hand gestures for controlling the video files. The entire control of the video files will be done using hand gestures without making use of the keyboard or mouse. The project will be beneficial for those developers who want to reap benefits of Java as well as Processing. Results will be in terms of the controlling the operations on video file which can be called in Java.

6. System RequirementsThe product will require the following minimum hardware configuration: IBM Compatible PC ranging from PIII PIV, CD ROM to install Java,128 Mb RAM,40 GB Hard Disk. (Depend on size of data to backup), Web Cam 1.3.The product will require the following minimum software configuration: Windows 2000/NT/XP, J2SE /J2EE, J Creator Editorm, Net Beans IDE 5.0/5.5 and processing beta version.

   The input to the system is the image of the hand shown at runtime which will be captured by the web camera. The appropriate count of fingers will be able to do the action assigned to each finger and the proper orientation of hand. The system will accept the image of the hand through the web camera and count the number of fingers shown by the user making use of the algorithm. After detecting the correct count of fingers the video file which was opened before will be controlled by the recognized hand gesture.This method provides us with the facility of controlling the video file irrespective of the file format with the help of hand

   gesture. Figure 2 shows the class diagram for the system.

   Figure 2 Class Diagram for the system

7. Test results 

   Table 1 shows the test results obtained by present method using fast algorithm with skin detection and also compare with other existing methods. The Figures 3 and 4 shows the test screens and gesture recognition- vector plot.

   Figure 3 Test Screens

   Table 1 Test Results

   Figure 4 Gesture Recognition-Vector Plot

8. ConclusionImplementing gesture recognition using centre of gravity algorithm is an effort in making utilization of computer more unique and natural. With a minimum hardware resource of a Web Camera an attempt to provide a hassle-free attractive solution for graphics designers and movie makers who often are involved in implementing their ideas in a more natural way. Picture Restoration Using Hand Gesture with Learning Capabilities can also be used to edit the frames of the video using hand gesture recognition. This is a powerful tool to change the background and foreground of the video.
9. References

[1] Attila Licsar and Tamas Sziranyi, IEEE Int, Hand Gesture based film Restoration ACM Cybernetics pp. 215-22, 2004.

[2] Asanterabi Malima, Erol Özgür, and Müjdat Çetin A Fast Algorithm For Vision-Based Hand Gesture Recognition for Robot ControlIEEE Int, Conf. Systems, Man, Cybernetics. pp. 289-296, 2008.

[3] Vafadar and Alireza Behrad Human Hand Gesture Recognition Using spatio- Temporal Volume for Human Computer Interaction IEE Int. Symposium on Telecommunications, pp.713- 718,2008.

[4] Byung-Woo Min, Ho-Sub Yoon, Jung Soh, Yun- Mo Yang and Toshiaki Ejima, Hand Gesture Recognition Using Hidden Markov model IEEE Int Conf. systems,Man, and Cybernetics, 1997 , pp 4232-4235.1997.

[5] Wing Kwong Chung, Xinyu Wu and Yangsheng Xu, A Realtime Hand Gesture Recognition based on Haar Wavelet Representation Proceedings of the 2008 IEEE Int. Conf. on Robotics and Biomimetics , pp.336-341,2008.

[6] Mu-Chun Su A Fuzzy Rule-Based Approach to Spatio-Temporal Gesture Recognition IEEETrans on systems, Man, And Cybernetics Part C: Applications and Reviewws, Vol.30No 2, pp276-281,2000.

 

[7] Byung-Woo Min, Ho-Sub Yoon, Jung Soh, Yun- Mo Yang and Toshiaki Ejima, Hand Gesture Recognition Using Hidden Markov model IEEE Int. Conf. systems,Man, And Cybernetics, 1997 , pp 4232-4235,1997.

[8] M K Bhuyan, D Ghosh, P K Bora,Feature Extraction from 2D Gesture Trajectory in Dynamic hand Gesture Recognition 14244 00236/6/ IEEE pp. 432-441,2006.

[9] Shewta Yewale ,Pankaj Bharne,Artificial neural network approach for hand gesture recognition International journal of engineering Science and technology, Vol 3 No.4,pp. 2603-2608, 2011.

[10] Zhou Ren Jing jing Meng Junsong Yuan,Zhengou Zhang, “Robust hand gesture recognition with kinect Sensor ACM multimedia 2011 conference

,pp. 978-979.2011.

[11] Hamid A Jalab,Static hand Gesture recognition for human computer interaction Asian Network for Scientific Information information technology journal, pp. 1-7,2012.

[12] Qing Chen, Nicolas D. Georganas, Hand Gesture Recognition Using Haar-Like Features and a Stochastic Context-Free Grammar, IEEE transactions on instrumentation and measurement, vol. 57, no. 8, pp.1562-1571,2008.

[13] Christopher Lee and Yangsheng Xu, Online, interactive learning of gestures for human robot

interfaces Carnegie Mellon University, The Robotics Institute, Pittsburgh, Pennsylvania, USA,1996.

Vafadar,Alireza Behrad,Human Hand Gesture Recognition Using spatio-Temporal Volume for Human Computer Interaction IEE Int. Symposium on Telecommunications, pp.713- 718,2008.