Hand Gesture Recognition and Voice Conversion System for Speech Impaired

Hand Gesture Recognition and Voice Conversion System for Speech Impaired

Dar shan K en y

EXTC

ACE, Malad (W) Mumbai, India

Mousu mi K ar ma kar

EXTC

ACE, Malad (W) Mumbai, India

Ni l esh K a mt e kar

EXTC

ACE, Malad (W) Mumbai, India

A ksha y Meht a

EXTC

ACE, Malad (W) Mumbai, India

Mohan K u mar

EXTC

ACE, Malad (W) Mumbai, India

Abstract:- In our country around 2.78% of peoples are not able to speak ie. dumb and deaf .Their communications with others are only using the motion of their hands and gestures. Our project proposed a new technique called artificial speaking mouth for dumb people. This system is based on motion sensor.For every action the motion sensors get accelerated and give the signal to the microcontroller. The microcontroller matches the gesture with the database and produces the speech signal. The output of the system is using the Audio Module. By updating the database the dumb will speak like a normal person using the artificial mouth. The system also includes a text to speech conversion (TTS) block that interprets the matched gestures.

Keywords:- Sign Language Recognition, Microcontroller, Flex Sensor, Accelerometer Sensor

1. INTRODUCTION

   It is introduced due to the need of developing an electronic device that can translate sign language into speech in order to make the communication take place between the mute communities with the general public possible. A data gloves is used which is normal cloth driving gloves fitted with flex sensors along the length of each finger. Dumb people can use the gloves to perform hand gesture and it will be converted into speech so that normal people can understand their expressionA gesture in a sign language is a particular movement of the hands with a specific shape made out of them. This project aims to overcome the barrier in communication.

   The main contributions of this paper that significantly differ from others are as follows: 1) proposing a framework of hand gesture recognition using fusion of flex, accelerometer; 2) data segmentation is done with the help of data received from accelerometer sensor, that is, change in hand position with respect to time and 3) conducting Scanning experiments with A-Z alphabets comprising of both static and dynamic gestures.4)After matching the signal is given to Audio Module which produces an particular sound.

2. WORKING

   There are two units in this project. One is the transmitter circuit and other is the receiver circuit. Transmitter circuit comprises of Flex sensor, Accelerometer, A-D convertor, Microcontroller. Receiver comprises of Audio Module, Amplifier, Speaker. Depending on the gesture made by with the help of flex sensor and accelerometer corresponding digital output is generated with the help of A to D converter.This values are now send to the microcontroller. Microcontroller matches this values in the database. For specific match particular information is send to the receiver side.

   Which generates particular sound for particular gesture.

   Transmitter

   Receiver

3. COMPONENTS

   The system comprises of the following components

   • Flex sensor

   • Accelerometer (ADXL335)

   • A-D convertor (MCP3208)

   • Microcontroller Atmel (89S52)

   • MP3 Audio module

   1. Flex Sensor

      A Flex Sensor patented technology is based on resistive carbon elements.Flex sensors are normally connected to the glove using needle and thread. They require a 5-volt input& output between 0 and 5 V, the resistivity varying with the sensors degree of bend and the output changing accordingly. It will change resistance in only one direction. An unflexed sensor has a resistance of about 10K ohms. As the flex sensor is bent, the resistance increases to 30- 40 Kohms at 90 degrees.

   2. Accelerometer (ADXL335)

      The ADXL335 is a small, thin, low power, complete 3-axis accelerometer with signal dependent voltage outputs. It measures acceleration with a minimum full-scale range of

      ±3 g. It can measure the static acceleration of gravity in tilt- sensing applications, as well as dynamic acceleration resulting from motion, shock and vibration.The sensor is a polysilicon surface-micromachined structure built on top of a silicon wafer. Polysilicon springs suspend the structure over the surface of the wafer and provide a resistance against acceleration forces. The Deflection of the structure is measured using a differential capacitor that consists of independent fixed plates and plates attached to the moving mass.

   3. A-D Convertor

      The MCP3208 12-bit Analog-to-Digital Converter combines high performance and low power consumption in a small package, making it ideal for embedded control applications.

      Features of analog to digital converter are:

      • It Easy interface to all microprocessors .

      • It Operates ratio metrically or with 5 VDC or analog span adjusted voltage reference.

      • No zero or full-scale adjust required .

      • Provides 8-channel multiplexer with address logic

        .

      • 0V to 5V input range with single 5V power supply..

      • Outputs meet TTL voltage level specifications.

      • Standard hermetic or molded 16-pin DIP package.

   4. Microcontroller

      The AT89S52 is a low-power, high-performance CMOS 8- bit microcontroller with 8000 bytes of in-system programmable Flash memory. This device is manufactured using Atmels high-density nonvolatile memory technology and is compatible with the industry-standard 80C51 instruction set and pinout.This on-chip Flash allows the program memory to be reprogrammed in-system or by a conventional nonvolatile memory programmer. By combining a versatile 8-bit CPU with in-system programmable Flash on a monolithic chip, We get Atmel AT89S52 Which is a powerful microcontroller which provides a highly-flexible and cost-effective solution to many embedded control applications. The Idle Mode stops the CPU while allowing the RAM, timer/counters, serial port, and interrupt system to continue functioning. Power- down mode saves the RAM contents but freezes the oscillator, disabling all other chip functions until the next interrupt or hardware reset.

   5. MP3 Audio module

      The FN-M16P module is a serial MP3 module that is with a perfect integrated MP3 and WMV decoder chip. It comprises micro SD card driver, and supports FAT16 and FAT32 file systems. It is able to play back specified sound files and realize other functions through simple serial commands. It is easy to use, stable and reliable are the most important features of this module.

      Product Features

      • Supports MP3 and WAV decoding.

      • Supports FAT16 and FAT32 file system.

      • 24-bit DAC output and supports dynamic range 90dB and SNR 85dB.

      • Supports AD key control mode and UART RS232 serial control mode.

      • Supports maximum 32GB micro SD card and 32GB USB flash drive.

4. FUTURE SCOPE

   The completion of this project suggests that these data gloves can be used for partial sign language recognition. In future it can support more number of signs and different language mode. We can make this system wireless so that it becomes handy and portable for commercial use. Talking wireless using this we can even transmit the code to a mobile phone. It can also be used for secret military messages.

5. CONCLUSION

   Sign language is a very useful to ease the communicaion between the deaf or mute community and the normal people. Yet there is a communication barrier between these deaf & dumb with normal people. This project aims to overcome the communication gap between the deaf or mute community and the normal world. This project was meant to be a prototype to check the feasibility of recognizing sign language using data gloves.

6. ACKNOWLEDGEMENT

We are very much thankful the our professors from ATHARVA COLLEGE OF ENGINEERING to suggest

various views on our project that not only boasted our confidence but also gave us insights about the different alternatives that could be implemented in our project.

REFERNCES

1. ElectronicsHub.org

2. Slideshare

3. http://people.ece.cornell.edu

4. L. K. Simone, E. Elovic, U. Kalambur, D. Kamper, "A Low Cost Method to Measure Finger Flexion in Individuals with Reduced Hand and Finger Range of Motion", 26th Annual International Conference of the IEEE Engineering.

5. Shoaib Ahmed.V Department of Electronics and Communication C. Abdul Hakeem College of Engineering and Technology Melvisharam, Vellore, Tamil Nadu 632 509, India submitted the thesis report of Hand Gesture Recognition and Voice Conversion System for Differentially Able Dumb People, 2012.