Signtalk-detection And Conversion Of Sign Language Into Speech With Light Control

Signtalk-detection And Conversion Of Sign Language Into Speech With Light Control

SignTalk-Detection and conversion of sign language into speech with light control

Ramya.R.S Department of Electronics and communication Engineering

AVS Engineering college Salem, TamilNadu, India ramyaselvaraju@gmail.com

Sujithraa.S Department of Electronics and communication Engineering

AVS Engineering college Salem, TamilNadu, India sujithraa24@gmail.com

Vaishnavi.K.S Department of Electronics and communication Engineering

AVS Engineering college Salem, TamilNadu ,India vaisp3saravanan@gmail.com

Vannamathi.S

Department of Electronics and communication Engineering AVS Engineering college

Salem, TamilNadu, India vannamathi111@gmail.com

Sakthi.R

Department of Electronics and communication Engineering AVS Engineering college

Salem, TamilNadu, India sakthi20020615@gmail.com

Abstract Gesture recognition technology has become an integral part of human-computer interaction, especially for people with physical disabilities. The proposed system is a wireless glove that utilizes flex sensors for better recognition of hand gestures. The glove also includes a heartbeat sensor that monitors the user's heartbeat and sends emergency notifications using a GSM module. Using Zigbee wireless technology, information can be transmitted and received between the glove and the main kit. The system also includes audio playback with speaker and a display for better understanding of the gesture, and enables the user to control the light and fan through gesture. The battery provides the necessary power supply for the circuit. This paper describes the design and implementation of the wireless glove for gesture recognition and control with an emergency notification system.

Keywords Wireless glove, Flex sensors, Gesture recognition, Heartbeat sensor, GSM module, Zigbee wireless technology, Audio playback, Display, Light control, Fan control.

1. INTRODUCTION

   Gesture recognition technology is a rapidly growing field in the realm of human-computer interaction. The proposed system is a wireless glove that employs flex sensors for better recognition of hand gestures. The glove also includes a heartbeat sensor that monitors the user's heartbeat and sends emergency notifications using a GSM module. The system has a wide range of applications, including home automation and healthcare

2. LITERATURE SURVEY

   1. Rule-based System:

      Some papers used rule-based systems to convert signs into speech. These systems rely on a set of predefined rules and handcrafted features to recognize signs and generate speech. The rules are typically based on linguistic and phonological knowledge of sign language. However, these systems are limited by their inability to handle variations in signing styles and the need for manual feature engineer

   2. Statistical Models:

      The Statistical models such as Hidden Markov Models (HMMs) and Gaussian Mixture Models (GMMs) to recognize signs and generate speech. These models learn from data and do not require manual feature engineering. However, they are

      limited by their inability to handle long-term dependencies in sign sequences and the need for large amounts of training data

   3. Deep Learning-based Methods:

   Deep learning-based methods such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to recognize signs and generate speech. These methods have shown promising results in Sign to Speech conversion due to their ability to learn complex representations from data and handle long-term dependencies. CNNs are particularly effective in recognizing signs from visual data, while RNNs are effective in modeling temporal dependencies in sign sequences.

3. METHODOLOGY

   The wireless glove includes flex sensors that measure the bending of the index finger, middle finger, and ring finger. These sensors are connected to a PIC microcontroller, which processes the data and recognizes the hand gesture. The system also includes a heartbeat sensor for monitoring the user's heartbeat, which is connected to the same PIC microcontroller. In case of an emergency, the system sends an alert message using a GSM module to their concerned person.

   Using Zigbee wireless technology, information can be transmitted and received between the glove and the main kit. The system also includes audio playback and a display for better understanding of the gesture. The proposed system also enables the user to control the light and fan through gesture using a relay module connected to the PIC microcontroller. The system is powered by a battery, which provides the adequate power supply for the circuit.

4. EXISTING SYSTEM

   Existing systems of sign to speech conversion can be classified into two categories: vision-based and data glove- based.

   Vision-based systems use cameras to capture the sign language gestures and then process the image data to recognize the signs. These systems can be further classified into two types: marker-based and marker less. Marker-based systems use special markers or gloves with markers on them to track the hand movements. Marker less systems do not

   require any markers and use computer vision techniques to track the hand movements.

   Data glove-based systems use gloves with sensors to capture the hand movements and then process the data to recognize the signs. These systems are more accurate than vision-based systems but are also more expensive..

5. PROPOSED SYSTEM

   Its a wireless glove. Here the flex sensors are used for recognizing the hand gestures in better manner.The Glove contains flex sensor and heartbeat sensor which are connected with PIC microcontroller.The heartbeat sensor monitors the heartbeat of the person in case of any emergency, using GSM module the information will send to their concerned person.Using Zigbee wireless technology, information can be transmitted and received between the glove and the main kit. For better understanding of the gesture, the audio playback with speaker and a display are provided.This System also enable the user to control the light and fan through gesture.The Battery provides the adequate power supply for the circuit.

6. BLOCK DIAGRAM

   Fig. 1 shows the complete block diagram of the system, Here the flex sensor and the heart beat sensor are connected to the microcontroller, the sensor information will send to the microcontroller, this information further sent to the micro controller present in the kit. The sensor information are processed and the necessary command are provided using the speaker and LED display. The heart beat sensor measures the heartbeat, if the heart beat reaches the threshold value then, the emergency alert will send to the concerned person through GSM module.

   Fig. 1. Transmitter block diagram

   Fig. 2. Receiver block diagram

7. HARDWARE REQUIREMENT

   1. Flex sensor

      These are sensors that measure the degree of bending of the fingers. In this system, flex sensors are used to recognize hand gestures. The glove contains several flex sensors that are placed at different points on the fingers and thumb. When the user moves their fingers, the flex sensors detect the degree of bending and send signals to the microcontroller.

      Fig. 3. Flex Senor

   2. Zigbee module

      This technology is used to transmit and receive information between the glove and the main kit. The main kit contains a receiver module that receives the signals from the glove and sendsthem to the microcontroller for processing. The receiver module also sends signals to the glove to control the light and fan.

      Fig. 4. Zigbee module

   3. PIC Microcontroller

      This is the central processing unit of the wireless glove system. The microcontroller receives signals from the flex sensors and heartbeat sensor, and it processes the data to recognize the user's hand gestures and heartbeat. The microcontroller also controls the other components of the system, such as the audio playback and light control.

      Fig. 5. PIC microcontroller

   4. Heartbeat sensor

      This sensor measures the user's heartbeat and is used to send emergency information in case of any medical emergencies. The heartbeat sensor is connected to the user's index finger, and it constantly monitors their pulse. If the sensor detects any abnormalities in the user's heartbeat, it sends a signal to the microcontroller, which then triggers the

      GSM module to send an emergency message to the user's designated contacts.

      Fig. 6. Heart beat sensor

   5. GSM Module

   This module is used to send emergency messages to the user's designated contacts in case of any medical emergencies. When the heartbeat sensor detects an abnormal heartbeat, the microcontroller triggers the GSM module to send a message to the user's contacts, informing them of the emergency and the user's location.

   Fig. 7. GSM module

   VIII..RESULTS

   The proposed system was tested and evaluated for gesture recognition and control. The system was able to recognize and differentiate between various hand gestures with high accuracy. The system also successfully controlled the light and fan through gesture. The system was also able to monitor the user's heartbeat and send emergency information using GSM Module. The Wireless glove proved to be an effective solution for gesture recognition and control.

   1. CONCLUSION

      The proposed wireless glove system is an efficient and effective solution for gesture recognition and control. The System uses flex sensors for recognizing hand gestures, and a heartbeat sensor for monitoring the users heartbeat. The system also includes audio playback and a display for better understanding of the gesture, and enables the user to control the light and fan through gesture. The system has potential application in home automation and healthcare. The proposed system proves to be an effective solution for gesture recognition and control, with high accuracy and reliability.

   2. REFERENCE

[1] E. A. Kalsh, and N. S. Garewal, Sign language recognition sys- tem, International Journal of Computational Engineering Re- search, vol. 3, no. 6, pp. 1521, 2013.

[2] D. Rabiner, L. P. Gou, and P. J. Kindermans, et al, Deep dy- namic neural networks for multimodal

gesture segmentation and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1.

[3] M. D. Ding, and L. Lin, CNN and HOG Dual-Path Feature Fu- sion for Face Expression Recognition, Information and control, vol. 49, no. 1, pp. 4754,

2020.

[4] C. C. D. Santos, J. L. A. Vassallo and R. F. Vassallo, Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation, Neurocomputing, vol. 400, pp. 238 254, 2020.

[5] H. Zheng, R. L. Wang and W. T. Ji et al, Discriminative deep multi-task learning for facial expression recognition, Information Sciences, vol. 533, pp. 6071, 2020.

[6] https://en.wikipedia.org/wiki/Sign_language accessed on 20 Mar 2023

[7] https://www.elprocus.com/flex-sensor-working-and-

its-applications/ accessed on 25 Mar 2023

[8] https://www.elprocus.com/introduction-to-pic- microcontrollers-and-its-architecture/ accessed on 28 Mar 2023