Hand Gesture Recognition Based Presentation System

Hand Gesture Recognition Based Presentation System

Akshaya Ramachandran, Raksha Aruloli, Taran Akshay

Vellore Institute of Technology, Chennai

ABSTRACT

A system that employs hand gestures to control the presentation is called hand gesture recognition for PPT presentations. The user's hand is detected by the system using a camera, which identifies the hand and its landmarks using a hand detection algorithm. The hand motions can be used by the system to control the display once it has recognised the hand and its landmarks. There are several methods in which the presentation can be managed by the system. To continue to the next slide, return to the previous slide, or pause the presentation, for instance, the user can make a gesture. Annotations can also be made on the slides by the user with motions. When compared to conventional presentation control approaches, the system offers several advantages.For instance, the technology allows the user to operate the presentation more naturally by eliminating the need for a keyboard or mouse. Because the user may manipulate the display using movements in a manner akin to controlling a live presentation, the system feels even more natural.

KEYWORDS: Hand gesture recognition, Hand detection algorithm, Presentation control, Keyboardless control,Mouse-free control, Annotation, landmarks

1. INTRODUCTION

   the multidisciplinary discipline of human-computer interaction (hci) examines how people interact with computers. the aim of hci research is to improve the usability, effectiveness, and enjoyment of computer systems and technologies through their design and evaluation.

   because it has the ability to increase everyone's access to and use of computer technology, hci is significant. human-centred computer system design is a way that hci researchers can contribute to making our lives easier and more productive[8]. hci is fundamentally about closing the gap that exists between people and machines. understanding the subtleties of human behaviour, cognition, and emotion and incorporating these understandings into interfaces that are nearly instinctive are key components of human-computer interaction (hci).usability is one of the core principles of hci. an interface that is well-designed facilitates easy communication between the user and the machine, allowing for mutual understanding. usability is the key to releasing the full potential of technology, whether it be the simple elegance of a smartphone's touch interface or the natural motions that operate virtual reality worlds. hci goes well beyond what is often associated with desktop pcs. our interactions in the age of ubiquitous computing take place across a wide range of platforms and devices, including wearables, tablets, smartphones, and more.it takes a thorough grasp of the context, user requirements, and the finer points of each platform to design for these heterogeneous ecosystems.

   a novel approach to interacting with powerpoint presentations, hand gesture detection provides a natural and fluid replacement for the traditional use of mouse or keyboards. an example of a nonverbal and nonvocal communication method is a gesture. it uses physical gestures to convey a specific message to the recipient [7]. the procedure begins with the user's hand being captured by a webcam that uses sophisticated hand detection algorithms to precisely identify and track its movements. hand tracking algorithms take over as the hand interacts interactively, tracking and deciphering the user's complex movements. users are then able to effortlessly manage different components of their powerpoint presentations by having these gestures recognised and converted into instructions. this combination of computer vision technology adds a new dimension of fluidity and engagement to the overall interaction while also streamlining the presentation control process.

   there are many benefits that hand gesture recognition offers the world of powerpoint presentations. it uses hand gestures, the natural and intuitive replacement for more traditional control techniques like a mouse or keyboard, to leverage the universal language of human communication. in addition to the obvious convenience, hand gesture detection adds a hands-free dimension by freeing users from the necessity of holding a mouse or keyboard in their hands. this is a very useful feature for people who are multitasking or have limited mobility. because users can perform numerous operations with a single hand motion, efficiency is a critical characteristic that streamlines tasks like scrolling back, annotating, and moving slides. the applications are cross-domain and improve education by encouraging dynamic and interesting presentations in the classroom.lectronic gadgets.

2. LITERATURE REVIEW

   In the context of Industry 4.0, the paper[1] investigates the usage of hand gesture detection for a user guide utilising the MediaPipe framework. In addition to introducing MediaPipe as a machine learning solution, it highlights the significance of gestures in human- computer interaction. The study trains the system with a variety of inputs, using the MediaPipe architecture for precise recognition, and uses Kinect to record hand motions in real time. Improved interaction is the goal of the user guide application, which is powered

   by hand gesture recognition. The results reveal that the framework is successful in increasing user engagement, with a 95% accuracy rate in hand gesture detection despite various constraints such as lighting conditions.

   in the study[2], george sung et al. present a single rgb camera-based on-device real-time hand gesture recognition (hgr) system. the problem of obtaining reliable hgr in the absence of depth sensors is discussed by the writers. the system consists of two gesture classifiers: heuristic-based and neural network (nn)-based, as well as a hand skeleton tracker based on mediapipe hands. by utilising mediapipe hands' real-time hand tracker, the two-step method optimises complexity while minimising engineering effort. improved keypoint accuracy and three-dimensional key points in a global metric space are two of the hand skeleton tracker's main enhancements. we provide two gesture classifiers: one based on heuristics for ease of use and another on neural networks for increased accuracy. while the nn-based classifier makes use of a three-layer neural network, the heuristics-based classifier depends on predetermined angles between 3d hand keypoints. the article talks about the training dataset, which contains samples for both simple and hard negatives. the mediapipe framework is used for implementation, and possible uses include robot control and virtual touchscreens.

   in this paper [3]we see how they go over the challenges of using gestures made up of both static symbols and dynamic motions to operate computer programmes. each gesture is modelled using either static model data or a dynamic system that is linear in its characteristics. real-time recognition takes place with less computing power and memory required. we look at the appropriate motions, how they may be recognised, and the orders they should govern. the tracking technique is described in depth, as well as how it can be used to supply coordinates for gesture control of a powerpoint presentation.

   this research[4] proposed a real-time method for interactively controlling the presentation through hand gestures. to manage the complicated background and variable illumination presented by the projector, the system makes use of a thermal camera for reliable human body segmentation. it is suggested that the head, torso, and arm be localised successively using a quick and reliable hand localization technique. segmented hand motions are identified as interactional gestures. a web camera is included to map the zones of contact between the thermal camera and the projected contents using a two-step calibration process. according to experiments, the system accurately recognises hand motions and can perform the necessary interactions.

   in this paper[5] we discuss human computer interaction's main objective is to make computers more attentive to user demands in order to enhance user-computer interaction. nowadays, using a computer for personal usage involves more than just using the keyboard and mouse. human interaction is facilitated through a variety of sensory modalities, including gesture, conversation, and body and face expressions. naturally interacting with the system is becoming increasingly crucial in many areas of human-computer interaction.hand gesture recognition has been accomplished using both non-vision and vision-based methods. the literature categorises hand gestures into two categories: static gestures and dynamic gestures. the location and direction of the hand in space do not vary during a static hand gesture, which can be characterised as such. dynamic gestures are those that alter over the course of the allotted period. static hand gestures include combining the thumb and forefinger to form the "ok" symbol, and dynamic hand gestures involve waving of the hands.

   the goal of this research project[6] is to improve powerpoint interactions by developing a presentation system that is controlled by hand gestures. the goal is to make presentations more engaging by doing away with the need for keyboards or other specialised equipment. with the use of a python programme and machine learning, the system can identify small hand gestures to manipulate slides. it is stressed how important gesture recognition is, especially in well-lit environments. one easy and technologically independent way to communicate is through hand gestures. the suggested method uses hand gestures to create a natural and captivating connection between presenters and their audience, making it a useful substitute for laser pens with the goal of facilitating more engaging and interactive presentations.

   this paper[7] discusses issues with microsoft powerpoint presentations, with a particular emphasis on expanding the audience and enhancing slide control. it presents a way to navigate a presentation smoothly by using hand gestures and speech-to-text technology. by combining a python interface with powerpoint, the goal is to increase presenter flexibility by enabling efficient slide control using gestures. in addition to addressing the disruptions brought about by manual slide changes, this method seeks to increase accessibility so that participants in the back row may still understand what the presenter is saying. professional presentations get a new dimension when hand gestures and speech-to-text technologies are used together, which streamlines the presenter-audience interaction.

   this study[8] presents a human machine interaction (hmi) system that uses hand gestures to control powerpoint presentations. the system, which uses opencv and python modules to be developed, uses computer vision techniques and machine learning to accurately recognise gestures as instructions in real-time. by doing away with the necessity for conventional input devices, the invention provides a convenient substitute. presenting a more accessible and interactive experience, the system's average accuracy surpasses 93% at any distance from the camera. its uses include enhancing digital device engagement for those who have restricted access to traditional input techniques. this study offers a fresh and effective method for understanding human-computer interaction and nonverbal communication.

   this study[9] presents a hand gesture control system that enables autonomous engagement with visual material, empowering those with physical disabilities. the device, which functions well within a 2-meter range, uses finger motions to operate functions including play/pause, volume control, and video navigation. because the system is directly trained on finger motions, latency is reduced because no dataset is required. when combined with computer vision algorithms, it recognises different finger combinations as gestures with a low latency of 300 milliseconds and a high accuracy of about 95%. this framework offers a useful technology to improve the level of autonomy in controlling video players for people with physical disabilities.

   this research[10] delves into recent trends in interactive device development, emphasizing the growing significance of sensors and camera-based applications in diverse domains. the paper specifically focuses on enhancing powerpoint presentations through a natural hand gesture-based control tool, leveraging microsoft kinect for stability. introducing a novel concept of gesture recognition through locking and unlocking, the study compares viewer interest, movement frequency, and overall stability against traditional methods. results highlight the gesture-based system's superior efficiency and user experience for both presenters and viewers, eliminating the need for additional devices. the tool's adaptability extends to the development of related applications, showcasing its potential impact on hci advancements.

3. ARCHITECTURE DESIGN

4. TECHNOLOGIES USED

   opencv: in this project, opencv is used to capture video from the webcam, detect the user's hand, and track its landmarks. cvzone.handtrackingmodule: cvzone.handtrackingmodule is a python module that provides a hand tracking algorithm based on opencv. it can detect multiple hands in a frame and track their landmarks over time. in this project, cvzone.handtrackingmodule is used to detect and track the user's hand.

   numpy: numpy is a python library that provides a wide range of functions for scientific computing. in this project, numpy is used to perform mathematical operations on the hand landmarks.

5. FLOW

   • capture video from webcam: use opencv to capture video from the webcam.

   • detect hand and landmarks: use cvzone.handtrackingmodule to detect the user's hand and track its landmarks.

   • draw gesture threshold line: draw a line at a specified height on the screen. if the hand is below this line, it is assumed to be controlling the presentation.

   • hand gesture classification: determine the hand gesture based on the position and orientation of the hand landmarks.

   • control presentation: use hand gestures to control the presentation, such as advancing to the next slide, going back to the previous slide, or drawing annotations etc.

6. TESTING

   Gesture	Description	Action
   [1, 0, 0, 0, 0]	Index finger raised	Advance to the next slide
   [0, 0, 0, 0, 1]	Pinky finger raised	Go back to the previous slide
   [0, 1, 1, 0, 0]	Index and middle fingers raised	Draw annotations on the slides
   [0, 1, 0, 0, 0]	Index finger raised	Start drawing annotations
   [0, 1, 1, 1, 0]	All fingers raised except the pinky	Delete the last annotation

7. RESULTS

   we discovered that the system could navigate the slides with ease and carry out a number of functions, including moving forward and going back to the previous slide.

   FIGURE 1: LEFT SLIDE

   FIGURE 2: RIGHT SLIDE

   FIGURE 3: START ANNOTATIONS

   FIGURE 4: DRAW ANNOTATIONS

   FIGURE 5: DELETE ANNOTATION

8. CONCLUSION

   a useful tool that can improve the presenting experience for users is the hand gesture recognition system for powerpoint presentations. the presenter can concentrate on their delivery and free up their hands since it enables a more organic and engaging method of controlling the presentation. additionally, there are no specific tools or training needs for using the system, making it user-friendly.

9. FUTURE WORK

• incorporate additional gestures: additional gestures, such zooming in and out, swiping slides, and inserting text, can be added to the system to enable more control over powerpoint.

• provide a voice-based interface: enabling a voice-based interface through system integration will enable even greater hands-free control over the presentation.

• provide cross-platform compatibility for the system: this will enable it to function on a larger variety of gadgets, including tablets and smartphones.

REFERENCES

1. harris, m., & agoes, a. s. (2021, november). applying hand gesture recognition for user guide application using mediapipe. in 2nd international seminar of science and applied technology (issat 2021) (pp. 101-108). atlantis press.

2. sung, g., sokal, k., uboweja, e., bazarevsky, v., baccash, j., bazavan, e. g., … & grundmann, m. (2021). on-device real-time hand gesture recognition. arxiv preprint arxiv:2111.00038.

3. cohen, c. j., beach, g., & foulk, g. (2001, october). a basic hand gesture control system for pc applications. in proceedings 30th applied imagery pattern recognition workshop (aipr 2001). analysis and understanding of time varying imagery (pp. 74-79). ieee.

4. zeng, b., wang, g., & lin, x. (2012). a hand gesture based interactive presentation system utilizing heterogeneous cameras. tsinghua science and technology, 17(3), 329-336.

5. haria, a., subramanian, a., asokkumar, n., poddar, s., & nayak, j. s. (2017). hand gesture recognition for human computer interaction. procedia computer science, 115, 367-374.

6. ahamed, s. f., sandeep, p., tushar, p., & srithar, s. (2023, january). efficient gesture-based presentation controller using transfer learning algorithm. in 2023 international conference on computer communication and informatics (iccci) (pp. 1-5). ieee.

7. charan, c. s., meenakshi, k., reddy, v. b., & kashyap, v. (2023, april). controlling powerpoint presentation using hand gestures in real- time. in 2023 7th international conference on trends in electronics and informatics (icoei) (pp. 251-254). ieee.

8. vidya, m., vineela, s., sathish, p., & reddy, a. s. (2023, august). gesture-based control of presentation slides using opencv. in 2023 second international conference on augmented intelligence and sustainable systems (icaiss) (pp. 1786-1791). ieee.

1. mali, c., sayyad, b., ankushe, v., navghane, a., kulkarni, s., & prawin gawande, p. g. (2023). design and implementation of hand gesture assistant command control video player interface for physically challenged people. available at ssrn 4626576

2. kumar, p., jaiswal, a., deepak, b., & reddy, g. r. m. (2018). hand gesture-based stable powerpoint presentation using kinect. in progress in intelligent computing techniques: theory, practice, and applications: proceedings of icacni 2016, volume 1 (pp. 81-94). springer singapore.