EyeMouse: Empowering Computer Interaction through Eye Gaze Tracking

EyeMouse: Empowering Computer Interaction through Eye Gaze Tracking

EyeMouse: Empowering Computer Interaction through Eye Gaze Tracking

Alen Mathew Paul

Dept. of Computer Science and Engineering Mar Baselios Christian College of Engineering And

Technology

Peermade, India alenmp2001@gmail.com

Bibin Binoy

Dept. of Computer Science and Engineering Mar Baselios Christian College of Engineering And

Technology

Peermade, India bibinbinoyvarghese@gmail.com

Achsa Binu

Dept. of Computer Science and Engineering Mar Baselios Christian College of Engineering And

Technology

Peermade, India achsabinu187@gmail.com

Sandra Biju

Dept. of Computer Science and Engineering Mar Baselios Christian College of Engineering And

Technology

Peermade, India sandramariya828@gmail.com

Nissu Simon

Assistant Professor

Dept. of Computer Science and Engineering

Mar Baselios Christian College of Engineering And Technology

Peermade, India nissusimon@mbcpeermade.com

AbstractThis paper presents a novel approach for controlling a virtual mouse using eye movements. The proposed system utilizes a low-cost eye-tracking device to capture the user's eye movements and convert them into mouse pointer movements on the screen. The system has been designed to address the limitations of traditional computer mouse input devices, such as limited mobility and accessibility issues.

The system includes a calibration step that maps the user's eye movements to specific screen locations. A custom-designed algorithm based on machine learning techniques is used to analyze and classify the user's eye movements accurately. The virtual mouse can perform all mouse functions, such as left-click, right- click, and drag, using a combination of eye movements and blinking patterns.

1. INTRODUCTION

   The ability to control computers with eye movements has become an increasingly popular topic of research in recent years. This is due in part to the fact that many individuals with disabilities cannot use traditional computer input devices such as a mouse or keyboard. For these individuals, eye tracking technology represents an important breakthrough that allows them to access computers and other devices more easily.

   In this paper, we present a novel approach to computer control using eye movements. Specifically, we propose a virtual mouse that can be controlled entirely with the eyes. The virtual mouse uses advanced eye tracking technology to accurately interpret the user's eye movements and translate them into mouse movements on the computer screen.

   The virtual mouse system has several advantages over traditional computer input devices. First, it is non-invasive,

   meaning that users do not need to wear any special hardware on their hands or fingers. Second, it is highly accurate, allowing users to perform fine-grained movements with ease. Finally, it is easy to learn and use, making it a great option for individuals who are new to computers or who have limited experience with traditional input devices.

   Overall, the virtual mouse system represents an important step forward in the development of eye tracking technology for computer control. We believe that this system has the potential to greatly improve the lives of individuals with disabilities, as well as to provide a more convenient and efficient input method for all computer users.

   1. Background and motivation

      The idea of using the eye as a virtual mouse is based on the concept of eye tracking technology, which involves using cameras and sensors to track the movement and position of a person's eyes. The utilization of eye-tracking technology has been widespread across different fields, including market research, human-computer interaction, and medical diagnosis.

      The motivation behind using the eye as a virtual mouse is to provide an alternative way of controlling a computer or other digital devices for people with disabilities, such as those with motor impairments that prevent them from using traditional input devices like a mouse or keyboard. By using their eyes to control the computer, these individuals can navigate through software and applications, communicate with others, and access information more easily.

      Overall, the use of the eye as a virtual mouse has the potential to improve accessibility and convenience for individuals with

      disabilities and enhance the user experience for a wide range of applications

   2. Problem statement

      Traditional input devices like a mouse and keyboard are not accessible for people with disabilities or those who cannot use their hands, which hinders their ability to interact with digital devices. This can cause a lot of frustration and limit their ability to perform everyday tasks.

   3. Objective

   The objective of using the eye as a virtual mouse is to provide an alternative means of controlling digital devices for individuals who cannot use traditional input devices. The primary goal is to improve accessibility and convenience for people with disabilities and make it easier for them to navigate through software, communicate with others, and access information. Additionally, the eye as a virtual mouse can also be used in situations where traditional input devices are not practical or available, such as in virtual reality or augmented reality environments. The objective is to enhance the user experience and make it more intuitive and natural for users to interact with digital devices using their eyes.

2. LITERATURE SURVEY

   Aside from facial expressions, the human eye also provides useful information, such as pupil, eye, and eye blink motions.[1] Eye-based HMI framework that allows for human- machine interaction using eye blinks and computer vision techniques. A system for detecting eye blinks in real-time video using the eyes The Haar Cascade Classifier technique can be used to effectively manage computers and home appliances using these predefined blink patterns from the framework.

   A system enabling human-machine interaction without the need of hands that creates a fresh channel of communication. This technique makes it possible for a mouse cursor to move and click using only a human head gesture and eye movement. a deep learning method to categorize each individual eye's degree of opening and shutting.[2] In order to conduct mouse actions, a picture must be cleaned of noise and drift using a complementary filter and an inertial measuring unit (IMU). The user's head movement is tracked using an accelerometer and gyroscope sensor, which also moves the cursor on the screen. A tracking template of the open eye is built online using motion analysis algorithms.[3] The robustness of computer systems that utilize vision is also examined in this way. The Frame Differencing technique is utilized, and the output shows where the moving object is located within the frame.

   For persons with disabilities who are unable to move anything other than their eyes, the technology offers a novel way to manage home appliances by moving the computer mouse pointer with their eyes. [4] By recording the user's eye movements and mapping them to the computer screen, the real- time eye-tracking system used in this project will be able to control the cursor's movement. For every frame, the Haar cascade algorithm is used to find every face in the picture. The technique enables hands-free contact between people and computers by introducing an algorithm to perform mouse-like functions. Eye-Aspect-Ratio (EAR) is tilized for eye detection. [5] It was used to determine whether or not the subject's eye in the video frame was flickering. Mouth-aspect- ratio (MAR), which determines whether or not the mouth is

   open. Physically handicapped people can use it in their schooling because it would enable them to type instead of writing by hand.

   An eye and face movement-based system for hands-free computer interaction is described in the research. To distinguish facial features like the eyes, nose, and mouth, the system makes use of Python, OpenCV, and Dlib. [6] Additionally, it transforms facial expressions like blinking, squinting, and head movements into scroll and mouse gestures. The metrics EAR (Eye aspect ratio) and MAR (Mouth aspect ratio) are used to calculate movements. The technology can be used by people who have suffered amputations or have difficulty using their hands. However, it requires sufficient lighting and may not be suitable for those who have facial problems.

   A self-training method for pupil recognition under low- resolution eye-tracking conditions. [7] Using a webcam with a resolution of 640×480, the system generates an initial pupil pattern based on colour intensity. The system captures a photo of the user's face and eyes, makes a pupil pattern, compares it to the user's eye image, and then evaluates the user's gaze using geometry. The system creates an attention zone in front of the user and logs face and eye motion ranges for tracking. The suggested method increases the precision of eye tracking and pupil recognition while lessening the influence of bright spots.

   A deep learning-based webcam-based eye-gaze identification system that outperforms existing ones in terms of reliability, robustness, and accuracy. [8] The approach addresses gaze detection as a multiclass classification problem and allows user-free head movement. In the study, a dataset of high-definition (HD) face photos was constructed, 468 landmark points were located using a state-of-the-art face mesh technique, and the eye region of the image was removed to produce a special dataset for training a light-weight convolutional neural network (CNN). The dataset was processed, normalized, and divided into nine classes prior to training. Following that, the neural network models for estimating eye gaze were refined. The resulting model was applied to HCI applications including Mouse Pointer Control.

   For individuals with neuro-locomotor impairments or disabilities, a mouse tracking system that can only be operated by eye movements is put forward. [9] It locates faces and their characteristics, such as eyes, noses, and mouths, using OpenCV and Dlib modules for Facial Gesture Detection and Mouse Tracking. After pre-processing, grayscale conversion, and the gathered frames, facial landmarks are detected. The system tracks mouse motions to carry out mouse activities such cursor movement, right-click, left-click, dragging, etc. based on the number of regions of interest and their relative positions in each recorded frame.

   The method enable people with limited range of motion to operate computers using facial and eye motions. [10] A hands- free device that controls the mouse cursor with facial expressions was created using HCL. Using a webcam or the camera on a laptop, it uses techniques including Face Detection, Landmark Detection, Head Pose Estimation, and Gaze Estimation to identify facial landmarks, head position, and gaze direction. The technology can also detect winks and blinks using the Eye Aspect Ratio and Mouth Aspect Ratio, respectively. The PyAutoGUI module controls the mouse pointer based on these actions. The system uses libraries like Numpy, OpenCV, PyAutoGUI, Dlib, and Imutils to initialise

   thresholds and carry out necessary operations based on the drawn bounding box and actions.

3. SYSTEM OVERVIEW

   A system that uses the eye as a virtual mouse typically consists of hardware and software components that work together to enable eye tracking and control of the computer or digital device.Using the built in webcam of the system the position of the eye could be determined. The processor that analyzes the eye tracking data and translates it into commands that can be used to control the computer or digital device.

   The system works by tracking the user's eye movements and translating them into commands that can be used to control the virtual mouse. The virtual mouse driver then emulates the behavior of a traditional mouse, allowing the user to interact with the computer or digital device in a similar way to using a physical mouse. The system may also include additional features, such as eye-based gesture recognition, to provide more advanced control options for the user. This suggests that using eye movements to control computers is possible and that using the eyes as a form of universal computing is an excellent idea.

4. EYE TRACKING AND GAZE ANALYSIS

   Eye tracking and gaze analysis can also be used to control a virtual mouse using a standard webcam. This technology is also known as webcam-based eye tracking and gaze estimation. To use a webcam as an eye tracking device, specialized software analyzes the images captured by the webcam to track the position and movement of the eyes. The software then maps these movements to mouse actions, allowing the user to control the computer using their eye movements.

   Webcam-based eye tracking typically uses computer vision algorithms, such as template matching or feature detection, to locate and track the eyes in real time. These algorithms analyze the color, shape, and movement of the eyes to estimate the gaze location.

   One advantage of using a webcam for eye tracking is that it does not require specialized hardware, such as an infrared eye tracker. This makes it more accessible and affordable for individuals who may not have access to or cannot afford specialized eye tracking equipment.

   Despite these limitations, webcam-based eye tracking has the potential to be a useful tool for a variety of applications, such as accessibility tools, user experience research, and gaming. As the technology continues to develop, it may become even more accurate and reliable, making it a viable alternative to dedicated eye tracking devices.

5. PERFORMANCE EVALUATION Performance evaluation of eye tracking and gaze analysis

   technology can be conducted by measuring the accuracy and efficiency of the virtual mouse interface. This evaluation can help determine the effectiveness of the technology and identify areas for improvement.

   Accuracy can be measured by comparing the estimated gaze location to the actual gaze location on the screen. This can be done using various methods, such as comparing the estimated gaze location to ground truth data or using a target acquisition task, where users are asked to click on specific targets with the virtual mouse.

   Efficiency can be measured by analyzing the speed and accuracy of mouse actions performed using the virtual mouse interface. This can be done by measuring the time it takes to complete a task, such as selecting a specific target, and the number of errors made during the task.

   Other factors that can be evaluated include the user's comfort and satisfaction with the virtual mouse interface, the ease of use of the technology, and any technical issues that may arise during use.

   Overall, performance evaluation of eye tracking and gaze analysis technology as a virtual mouse interface can provide valuable insights into the effectiveness of the technology and inform future development and improvement. It can also help identify potential applications for the technology in areas such as accessibility, gaming, and user experience research.

6. DISCUSSION AND CONCLUSION

In conclusion, webcam-based eye tracking and gaze analysis technology has the potential to make eye tracking more accessible and affordable to a wider range of users. By using a standard webcam, individuals who may not have access to or cannot afford specialized eye tracking equipment can still benefit from this technology.

Webcam-based eye tracking has some limitations, such as potential accuracy and reliability issues caused by lighting conditions and head movements. However, continued development and improvement of computer vision algorithms and webcam technology can help overcome these limitations and improve the accuracy and reliability of the technology.

Performance evaluation of webcam-based eye tracking and gaze analysis technology can be conducted by measuring accuracy and efficiency of the virtual mouse interface. This evaluation can help identify areas for improvement and inform the future development of the technology.

Overall, webcam-based eye tracking and gaze analysis technology as a virtual mouse interface has the potential to significantly improve accessibility and user experience for a variety of applications. As the technology continues to develop, it may become a more viable alternative to dedicated eye tracking devices and expand the applications of eye tracking and gaze analysis even further.

The preferred spelling of the word acknowledgment in America is without an e after the g. Avoid the stilted expression one of us (R. B. G.) thanks …. Instead, try R. B.

G. thanks…. Put sponsor acknowledgments in the unnumbered footnote on the first page.

REFERENCES

[1] S. Bhowmick and H. A. Mustafa, A Framework for Eye-Based Human Machine Interface, 2021 5th International Conference on Electrical Engineering and Information Communication Technology (ICEEICT), Nov. 2021, doi: 10.1109/iceeict53905.2021.9667935.

[2] S. R. Fahim et al., A Visual Analytic in Deep Learning Approach to Eye Movement for Human-Machine Interaction Based on Inertia Measurement, IEEE Access, vol. 8, pp. 4592445937, Mar. 2020, doi: 10.1109/access.2020.2978028.

[3] S. Deb and S. Deb, Designing an intelligent blink analyzer tool for effective human computer interaction through eye, Intelligent Human Computer Interaction, Dec. 2012, doi: 10.1109/ihci.2012.6481806.

[4] S. A. Mathew, A. Sreeshma, T. A. Jaison, V. Pradeep, and S. S. Jabarani, Eye Movement Based Cursor Control and Home Automation for

Disabled People, International Conference on Communication and Electronics Systems, Jul. 2019, doi: 10.1109/icces45898.2019.9002325..

[5] K. Meena, M. Kumar, and M. Jangra, Controlling Mouse Motions Using Eye Tracking Using Computer Vision, International Conference Intelligent Computing and Control Systems, May 2020, doi: 10.1109/iciccs48265.2020.9121137.

[6] Vinay. S. Vasisht, S. Joshi, Shashidhar, Shreedhar, and C. Gururaj, Human computer interaction based eye controlled mouse, 2019 3rd International Conference on Electronics, Communication and Aerospace Technology (ICECA), Jun. 2019, doi: 10.1109/iceca.2019.8822033.

[7] C. Zheng and T. Usagawa, Self-training Pupil Detection Based on Mouse Click Calibration for Eye-Tracking Under Low Resolution, International Conference on Control and Automation, Apr. 2020, doi: 10.1109/iccar49639.2020.9107973.

[8] K. Roy and D. Chanda, A Robust Webcam-based Eye Gaze Estimation System for Human-Computer Interaction, 2022 International Conference on Innovations in Science, Engineering and Technology (ICISET), Feb. 2022, doi: 10.1109/iciset54810.2022.9775896. S. A.

Mathew, A. Sreeshma, T. A. Jaison, V. Pradeep, and S. S. Jabarani, Eye

Movement Based Cursor Control and Home Automation for Disabled People, International Conference on Communication and Electronics Systems, Jul. 2019, doi: 10.1109/icces45898.2019.9002325.

[9] A. Lakshmanarao, B. Lokesh, and K. Sarita, An Efficient Mouse Tracking System Using Facial Gestures, 2022 8th International Conference on Advanced Computing and Communication Systems (ICACCS), Mar. 2022, doi: 10.1109/icaccs54159.2022.9785044.

[10] R. Shashidhar, S. K, A. P. K, A. S. Vishwagna, and P. N, Mouse Cusor Control Using Facial Movements – An HCI Application, 2022 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS), doi: 10.1109/icscds53736.2022.976091