Operating PC by Handicapped using Face

Operating PC by Handicapped using Face

Pushkar Rane1;Kiran Shinde2;Kaushik Tulaskar3

Department of Computer Engineering,

KC College of Engineering & Management Studies & Research, Kopri,Thane(E)-400 603,India.

Abstract To facilitate people with disabilities to our use computers, scientists have done a lot of studies & research.Using it we are trying to design a PC system that can be used by the handicapped.In our System we analyze the biometric identification and tracking related technologies of human-computer interaction. Based on Haar Cascade face detection algorithm, proposed a position-based head motion detection algorithm is which does not depend on the specific biometric identification and tracking. It uses feature classification method to detect mouths opening and closing actions.A software system to operate computer by image detection of head and mouth is used the combinations of head and mouth movements, are mapped to various mouse events, including move, click, drag, and so on. Our system can be used for the upper limb disabled who fail to use the traditional mouse and keyboard. Furthermore, it can also be used for general computer users to do neck rehabilitation training, computer somatic games, etc.

Keywords-Haar Cascade Algorithm; Face Detection; Mouse; Human-Computer Interaction

1. INTRODUCTION

   The common computer input devices, such as a mouse, keyboard, are designed for normal capable user. It is difficult for the upper limb disabled to manipulate a mouse or keyboard, and then they could not use the computer like normal person. In order to facilitate people with disabilities to use computers, scientists and engineers have done a lot of studies. Some earlier methods used the auxiliary equipments, such as infrared sensors and infrared reflectors, to detect the movements of computer user. Evans et al. used infrared light-emitting diodes and photo detectors as auxiliary equipment to determine the user's head position to operate the computer. Takami et al. invented a computer interface device [2], which places the transmitter over the monitor and uses an infrared reflector that is attached to the users forehead or glasses. Chen et al developed a system that contains an infrared transmitter mounted onto the users eyeglasses, a set of infrared receiving modules that substitute the keys of a keyboard, and a tongue-touch panel to activate the infrared beam.

   Recently, along with the development of image processing technology and the improvement of computer performance, it has been a hot research area of using the non-contact and image processing methods to study human interactions. Betke et al. proposed a system that tracks the facial features and then translates them into the movements of the mouse pointer on the screen. Nabati and Behrad presented a novel approach to estimate the 3D head pose from a monocular camera images for the control of mouse pointer movements on the screen and clicking events. These non-contact methods are more comfortable and convenient for the users and involve less expensive communication devices. However, these methods have high requirements for the camera, as well as high performance of computer image processing and computing. Currently, non-contact technology is mainly used in some special application systems. A specialized software or system to control computer through technology is still not common.

   We study the algorithms of detecting head movements and mouth status. Based on these algorithms, we design a Head-Trace Mouse to operate computer by detecting the movements of head and mouth.

2. LITERATURE SURVEY

   One approach for head movement detection is computer vision-based. M. Nabati and A. Behrad,[6]. introduced a video-based technique for estimating the head pose and used it in a good image processing application for a real-world problem; attention recognition for drivers. It estimates the relative pose between adjacent views in subsequent video frames. Scale-Invariant Feature Transform (SIFT) descriptors are used in matching the corresponding feature points between two adjacent views. After matching the corresponding feature points, the relative pose angle is found using two-view geometry. With this mathematical solution, which can be applied in the image processing.In general, the x, y, and z coordinates of the head position are determined. The accuracy and

   performance of the algorithm were not highlighted in the work and thus more work is needed to prove this algorithm to be applicable in real applications.

   The system analyzes the relationship between different combinations of the detected head and mouth movements, and then maps them to mouse events of computer system.

   1. The Basis of Head Movements

      The face detection is implemented by using Haar Cascade algorithm in our system, in which each frame of video streaming captured by a camera (30 frames per second) is input signal . Then we propose an algorithm for detection of head movements by analyzing face. We defined five motions as the basis of head movements, namely, standard head, head left, head right, head up and head down as shown in fig 1.1

      1. HeadLeft

      2. Head Up (c) Standard Head (d) Head Down

         (e) Head Right

         Fig 1.1 Basic of Head movement

         OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real- time computer vision, developed by Intel, and now supported by Willow Garage and Itseez. It is free for use under the open source BSD license. The library is cross- platform. It focuses mainly on real-time image processing. If the library finds Intel's Integrated Performance

         Primitives on the system, it will use these proprietary optimized routines to accelerate itself. History Officially launched in 1999, the OpenCV project was initially an Intel Research initiative to advance CPU-intensive applications, part of a series of projects including real-time ray tracing and 3D display walls. The main contributors to the project included a number of optimization experts in Intel Russia, as well as Intels Performance Library Team.

         JavaCV first provides wrappers to commonly used libraries by researchers in the field of computer vision: OpenCV, FFmpeg, libdc1394, PGR Fly Capture, video Input, and ARToolKitPlus. The following classes, found under the com.googlecode.javacv.cpp package namespace, expose their complete APIs: opencv_core, opencv_imgproc, opencv_video, opencv_features2d, opencv_calib3d, opencv_objdetect, opencv_highgui, opencv_legacy, avutil, avcodec, avformat, avdevice

         ,avfilter, postprocess, swscale, dc1394, PGRFlyCapture, videoInputLib, and ARToolKitPlus, respectively. Moreover, utility classes make it easy to use their functionality on the Java platform, including Android.

3. METHODOLOGY HAAR CASCADE CLASSIFIERS :

The core basis for Haar classifier object detection is the Haar-like features. These features, rather than using the intensity values of a pixel, use the change in contrast values between adjacent rectangular groups of pixels. The contrast variances between the pixel groups are used to determine relative light and dark areas. Two or three adjacent groups

with a relative contrast variance form a Haar-like feature. Haar-like features, are used to detect an image. Haar features can easily be scaled by increasing or decreasing the size of the pixel group being examined. This allows features to be used to detect objects of various sizes.

Fig 3.1.Edge Feature

Fig3.2 Line Feature

Fig 3.3 Center-surround Fature

Fig 3.3 Face detection work

1. HEAD-TRACE MOUSE SYSTEM

   The principle of Head-Trace Mouse system is to capture the users head images through the camera firstly, then to analyze the users head and mouth movements by using detection algorithm as described above, and finally to convert them to the corresponding mouse events.

   1. Preparation for Head-Trace Mouse System

      Head-Trace Mouse system can either use the computer's built-in camera, or an ordinary CMOS camera via USB port. Fig. 3.4 shows the situation of a user using the system, where a CMOS camera is used and fixed on top of the computer screen.

      In the beginning of the system working, the images captured by the camera are displayed on top right of the screen. So, user can see his head position on the screen, and can adjust the camera angle or his head position to put his head in middle of image whenever necessary. Fig. 3.4 shows the screenshot of the top right part of the computer screen. The left part of Fig.3.4 shows that the rectangles in the image window frame the users head and mouth detected by the system. In the middle part, the circular icon is a mouse cursor specially designed for the system. The right part shows the system control panel.

      Fig 3.4 A user is using Head-Trace Mouse

      Inputs of the system are head and mouth movements. Outputs are mouse events, including still, move, click, drag, etc. When the system detects a users face and mouth image, the user can activate the system by opening mouth. After the system is activated, the image window on the screen disappears, only the circular icon and the control panel are left. Users can operate computer by head and mouth movements. According to the movements of head and mouth, the circular icon and the control panel will make different responses or action tips in the process of operating a computer.

   2. Cursor of Head-Trace Mouse System

   In our system, user does not use a traditional mouse, so we designed a special mouse cursor to replace the traditional cursor in order to help user to operate the system easily. The mouse cursor is a graph includes five parts (up, down, left, right and center) and a traditional cursor at upper left corner, which we call five-direction-graph cursor or five-direction-graph for brief, shown as examples in Fig.

   1. In practice, mouse events are mapped into mouse cursor shown on the screen. Unless noted, we regard the words mouse and mouse cursor the same for the purpose of convenient and uniform description (in fact, it is easy to distinguish them from context). Fig. 3.5 shows part of mouse events.

      Fig 3.5 Control Panel

2. ADVANTAGES

   1. Accurate and Fast Identification

   2. User friendly design

   3. Quick response time

   4. Small memory factor

   5. Really helpful for disabled people 6 Less expensive

3. APPLICATION

   1. We can Apply it In Game Development

   2. We can apply it in security application

   3. We can Apply it in Payment Processing

   4. Gesture recognition can be added to enhance features

4. CONCLUSION

In our System, an eye motion based on low-cost eye tracking system is presented.

In our system we proposed Haar Cascade Algorithms. The user with several disabilities can use our system for handling computer.

A real time eye motion detection technique is presented.

ACKNOWLEDGMENT

We thank Mrs.Asmita Deshmukh, who has been guided and has helped us throughout the seminar work.

We would also like to thank Mrs. Amarja Adgaonkar, Head of Department of Computer Engineering, for her guidance and support.

Also, we would like to expression our sincere gratitude to our Principal, Dr. Hansraj Guhilot for giving us opportunity to enhance our knowledge in the present and futuristic subject like Image Processing.

We would also like to thank our friends and all those who have stood by us throughout the Endeavour.

REFERENCES

1. D. G. Evans, R. Drew, and P. Blenkhorn, Controlling mouse pointer position using an infrared head-operated joystick, IEEE Trans. Rehab. Eng., vol. 8, no. 1, pp. 107117, 2000.

2. O. Takami, N. Irie, C. Kang, T. Ishimatsu, and T. Ochiai, Computer interface to use head movement for handicapped people, in Proc. IEEE TENCON96, Digital Signal Processing Applications, vol. 1, pp.468 472. 1996.

3. Y. L. Chen, F. T. Tang, W. H. Chang, M. K. Wong, Y. Y.

4. , and L. A.Frey, Human-computer interaction using eye-gaze input, IEEE Trans. Syst., Man, Cybern., vol. 19, no. 6, pp. 1527 1533, 1989.

5. M. Betke, J. Gips, and P. Fleming, The Camera Mouse: Visual Shih, and T.S. Kuo, The new design of an infrared-controlled human-computer interface for the disabled, IEEE Trans. Rehab. Eng., vol. 7, pp. 474481, Dec. 1999.

6. T. Hutchinson, K. P. White Jr., W. N. Martin, K. C. ReichertTracking of Body Features to Provide Computer Access for People With Severe Disabilities, IEEE Trans. on Neural Systems And Rehabilitation Engineering.

7. M. Nabati and A. Behrad, Camera Mouse Implementation Using 3D Head PoseEstimation by Monocular Video Camera and 2D to 3D Point and Line Correspondences, 2010 5th International Symposium on Telecommunications (IST'2010), pp.825-830, 2010.

8. P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, in Proc. Conf. Computer Vision and Pattern Recognition, Volume1, pp.511-518, 2001.