Embedded Based Hand Talk Assisting System for Deaf and Dumb

Embedded Based Hand Talk Assisting System for Deaf and Dumb

J. Thilagavathy Assistant professor, Department of ECE

Dr.Sivanthi Aditanar college of Engineering

1. Jeyapaul Murugan Assistant professor, Department of ECE

   Dr.Sivanthi Aditanar college of Engineering

   1. Darwin

      Assistant professor, Department of ECE Dr.Sivanthi Aditanar college of Engineering

      Abstract – In our day to day life most of the task we carry out involves speaking and hearing. The deaf and dumb people have difficulty in communicating with others who cannot understand sign language and misinterpreters.In this paper, we designed a simple embedded system based device for solving this problem .we have use flex sensor for getting the data from the deaf and dumb people using sign language and microcontroller AT89c51 for controlling all operations and APR 9600 voice chip for voice storage.LCD display and speaker are used as output device to convey the message to deaf and dumb people. Keil and protous software tools are used for compiling software coding and simulating the design.

      1. INTRODUCTION

         Speech and gestures are the expressions, which are mostly used in communication between human beings. Getting the data is the first step. The second step, that of recognizing the sign or gesture once it has been captured is much more challenging, especially in a continuous stream. In fact currently, this is the focus of the research. The objective of this paper is to design a simple embedded system based communicating device for deaf and dumb people. Here two major problems are taken into consideration. First one is deaf and dumb people communicating with normal person and second one is communication between deaf and dumb people. To solve this problem we have use two modes of operation in this system. we are measuring the actions performed by the deaf and dumb people using flex sensor attached to gloves in a hand of the user. Once the glove is placed in the hands, whenever an action for sign language is performed, the bending values are obtained and the corresponding action is identified by the microcontroller AT89c51. It activates the voice chipAPR9600 and the corresponding voice is spelled in speaker and displayed in the LCD.

      2. PROPOSED SYSTEM FOR DEAF AND DUMB PEOPLE

         Our proposal will help the deaf and dumb people who are unable to communicate, or having difficulties in communication. A setup data glove is equipped with five flex sensors, each of the flex sensors is meant to be fixed on each of the finger of the hand glove for the monitoring and sensing of static movements of the fingers of the hand. Whatever the person wants to communicate is activated by two ways either by hand gesture or by keypad in the device. This input is text is processed using a microcontroller. Further, the frequently spoken words can be stored in memory of APR9600 voice chip and can be easily retrieved by using hotkeys. The output from the LCD can be read by the dumb people and Speaker can be heard by the deaf people. This device helps in communication if attached to both the person involved in the communication who may be deaf, dumb, and Normal person.

      3. BLOCK DIAGRAM

         Figure.1 Block diagram

         The flex sensor senses the sign language performed by the deaf people and produces the output. The output of the flex sensor is given to the Microcontroller through ADC. In the Microcontroller we already programmed the particular word for each output of the sensor. This word is recorded in the voice chip and heard from the speaker. If the Controller accepts the input from the Keypad then, the output will be displayed in the LCD.

      4. CIRCUIT DIAGRAM

         Figure.2 Circuit diagram

         In our project we use AT89c51 microcontroller. It consists of four ports. Fig.2 shows the circuit diagram of this device. Here two ports (port 2&3) are used for input and remaining two (port 0&1) are used for output. These two ports are used to get the input from the user. Both the flex sensor and keypad are used to give the input data .The flex sensor is connected to the port 3 is used to get gesture from hand. The keypad is connected to the port 2. The two ports namely port 0 and port 1are used to give the output to the other user. Both the LCD and Speaker are used to produce the output data .The voice chip APR9600 is connected to the port 1 to produce the voice through speaker. The LCD display is connected to the port 0 is used to display the word.

      5. PROJECT DESCRIPTION

         These two are used as the output devices. Two mode of operation is used in our project. Fig .3 explains the flow of this system and its operation.

         Figure.3 Flow Diagram

         The first mode of operation is voice mode. In this mode the input is obtained from flex sensor which senses the sign language performed by the deaf and dumb people or normal person. Then the resistance value of the flex sensor is changed based on the degree of bending of fingers. The output from the flex sensor is multiplexed and is an analog signal .this analog signal is given to the ADC to convert into digital value. This digital value is given to the microcontroller. In the microcontroller we have already programmed a particular word or message indicating for each voltage value coming from flex sensor. It compares the incoming voltage value with already stored value. If it is matched with a particular word that is stored in the voice chip .Then the output from the voice chip is given to the audio amplifier and the word is heard through the speaker, which helps the dumb people to hear the information. vise versa the dumb people can also respond using his gesture. Thus two way communications can be accomplished.

         The second mode of operation is Text mode. In this mode of operation, the controller accepts the input of the deaf, dumb, normal person using keypad. If the particular key in the keypad is pressed, the pressed key is detected by the controller by scanning the each row and column respectively. For each key press a particular word stored in the microcontroller, is displayed in the LCD display which helps the deaf person to see the message. vise versa the deaf people can also respond using his key pad. Thus both the input device can be used by the entire person irrespective of his disability.

      6. Software Description

         In this project two software tools are used for compilation and simulation

         1. KEIL Software

            KEIL software makes c compilers, macro assemblers , real time kernels, debuggers, simulators, integrated environments, and evaluation boards for 8051, 251, ARM and XC16x\C16x\ST10 microcontroller families. The compiler allows writing microcontroller application in C that, once compiled, having the efficiency and speed of assembly language. We compile the program for LCD and Keypad interfacing with microcontroller AT89c51 using this KEIL software.

         2. Proteus software

            PROTEUS is design software which accepts only hex file. Once the machine code is converted into hex code and then that hex code has to be dumped into the microcontroller placed in the programmer kit. It is user friendly one. We simulate the circuit for LCD and Keypad interfacing with microcontroller AT89c51 using this PROTEUS software and get the output.

      7. OUTPUT

         1. LCD and keypad interfacing using protous simulation software

         2. Hardware Implementation

         From fig.4,When any key is pressed by the deaf

         ,dumb or normal person then the corresponding word for communication is stored in microcontroller ill be displayed in LCD so that the dumb people can receive their messages to others.

         Figure.4 LCD With Keypad Interfacing

         The Compilation for the above design using keil software is shown in fig.5

         Figure.5 Compilation output of LCD With Keypad Interfacing using keil software

         Figure 6 Flex sensor output analysis for various bends

         Fig.6 shows that for various bends of the flex sensor the various voltage will be produced which is measured using multimeter and given as input data for microcontroller

         Figure.5 Flex sensor attached to glove in the hand is used to perform Gesture by deaf and dumb people.

         Figure.6 LCD Display output in mode 2

         If the deaf, dumb or normal person, what to communicate some message with the dumb person. He wants to press the key corresponding to a word so that the dumb people can receive the message in the LCD display. From the above output we can see that the user press the first key so that the corresponding word stored is output as please in

         LCD display .so that dumb person can read the message in LCD.

         7.3. Hardware implementation of voice chip

         Figure.7 Voice chip APR 9600 with voice output

         Fig .7 shows the interfacing of voice chip .It can store voice upto few seconds using the Mic as the input device. This voice chip has a playback options, so that we can playback the stored voice and listen through the speaker. This module helps for communication between dumb and deaf people. Deaf people can convey their message through hand gesture, so that the corresponding word stored in the voice chip for the gesture produces output as please sound through speaker

         Figure.8 System output in voice mode1

      8. APPLICATIONS

         This proposed system is cheap, cost efficient and portable. This system uses simple techniques .It helps deaf and dumb people in marking areas, public sectors, working areas for communicating with others. This project plays can also play major role in various fields such as Robotics, Biometrics, Automatic control in industries, Musical instrument by replacing physical buttons and switches by hand gestures.

      9. CONCLUSION

         This paper describes the design and working of a system which is useful for deaf and dumb people to communicate with one another and with the normal people. The deaf and dumb people use their standard sign language which is not easily understandable by common people. This system converts the sign language into voice which is easily understandable by the people. The sign language is translated into text form, to facilitate the deaf people to convey their messages as well to the others. This text is display on LCD and also through speaker by which the other person can understand .In this way our project is very well useful for deaf and dumb people and for other robotics applications etc…

      10. REFERENCES

1. Venketraman.s,T.V.Padmavathi,speech for the disabled

2. John kangchun perng,Seth, HollarAcceleration sensing glove.

3. Salman afghani,Microcontroller and sensors based gesture vocaliser.

4. http://www.engineersgarage.com//microcontroller 8051 project

5. http://www.scribd.com//Hand Talk Assisting system

6. Analog Devices, Norwood, MA, http://www.analog.com