Gesture to Voice Command Glove for Dumb

Gesture to Voice Command Glove for Dumb

Lavish Madaan1 Amogh Bordwekar2 A.N Srikantp Prathamesh Konkar4

Department of Electronics Engineering

K.C. College of Engineering and Management Studies and Research, Thane, India.

Abstract- Sign language is widely used by the dumb people for communicating with people and they find it difficult to communicate with people who dont understand sign language. This project aims to overcome this barrier in communication. The project is based on the need for an electronic device that can translate sign language into speech, for allowing the mute communities to communicate with the general public. A Wireless glove, made out of ordinary cotton cloth is used to perform hand gestures and it will be converted into equivalent speech, so that normal people can understand their expression. Sign language is commonly used by mute people and it is a communication skill that makes use of gestures instead of voice to convey meaning. It also simultaneously combines orientation and movement of the hand and facial expressions to express a speakers thoughts. Signs are used to communicate some message to the audience. A gesture means performing of a specific symbol or shape through the movement of hands. A sign language usually provides a symbol for entire words. Sign language also includes symbols or shapes for individual letters that make up a word or a sentence. Flex sensor [1] plays the major role in this project which changes resistance depending on the amount of bending of the sensor. The communication gap between the dumb and normal people can be eliminated by the prototype we are currently developing.

Keywords-Sign language; Flex sensor; Zigbee; Dev-C++; Express PCB; Flash Magic; Voice IC; AVR ATMEGA 16.

1. INTRODUCTION

   The embedded systems that are rapidly evolving in this modern era are affecting our lifestyle in a positive way. Sign language is an expressive way of communicating that lets the dumb people communicate normally with the ordinary crowds. The intention of developing this project is to curb the difficulties arising for the dumb people to communicate with the normal people who are not aware of the meaning of signs and symbols associated with it. Sign language not a universal language and it varies from country to country.

   America developed American Sign Language (ASL) [2], British developed British sign language system (BSL) and Thailand developed Thai sign language system (TSL) [3]. Most of the spoken English countries follow same sign language but the similar sign represents different meaning and depends upon their own language.

2. SYSTEM ARCHITECTURE AND IMPLEMENTATION

   In this project, the said data glove is implemented to capture hand gestures and movement of a user. The data glove is fitted with flex sensors along the length of each finger. These flex sensors output a stream of data that varies with the degree of bend. The analog outputs from the sensors are then fed to the Atmega 16 microcontroller. The microcontroller processes the signals and performs the corresponding analog to digital signal conversion. The resulting digital signal is encoded and transmitted through a ZigBee system. The ZigBee system receives the signal and feeds to the gesture recognition section through the decoder. In this section the gesture is recognized, the corresponding information is identified and conversion takes place in the voice section which is later played out through the speaker.

   Fig 1. Example of Hand Gestures

3. SYSTEM DESIGN

   1. Hardware design:

      A three axis accelerometer is used in the control unit. Another analog device accelerometer is strapped to the user arm. Zigbee transceivers are used for wireless transmission and reception of data. The data processing is carried using low cost microcontroller rather than digital signal processing. Atmel AT16 microcontroller [4] was used for data processing one in control unit and one at the receiving end to control the robotic arm.

      Fig 2. Block Diagram

      Furthermore, the system reliability and efficiency of ZigBee network will play a key role to achieve the required system performance and stability. ZigBee is low power consumption, low cost and of moderate range. This range is suitable for our application where we want to wirelessly transmit data from human controller to the Database Server. Another most important reason for using ZigBee module is that these modules come with serial interface therefore it will be easier for us to use these modules.

   2. Control unit:

      The work has been chosen with the lower arm joints from the elbow to the hand. The prototype for a control unit is built with the PCB board, microcontroller, accelerometer and ZigBee module. The accelerometer [5] can be strapped to the user arm. The user arm can be moved and the data are measured by the microcontroller from the sensor and send these values to the ZigBee transceiver. The sent values are in a packet with a header bit and a checksum bit. All the measured readings were transmitted from the microcontroller to the Zigbee transceiver [6].

      Fig 3. Glove with Flex Sensors

   3. Electronic Circuit Design and Software Programming:

      The electronic part is used to control the movement of the arm component. For this purpose, Atmel AVR 16 the microcontroller is used. The Atmel AVR 16 needs to have its own electronic circuit and needs to be programmed to enable

      it to control the arm movement and interface with the software to control the arm. Problem occurs in order to program the Visual Basic in PC to interface or communicate with the circuit through Atmel.

   4. Voice IC:

   Offers true solid state storage capability and requires no software or microcontroller support. The Voice IC [7] provides high quality recording and playback with 11 minutes audio at 8 KHz sampling rate with 16 bit resolution. Using on board jumpers, total duration can be divided in individual triggers of 1, 2, 4 & 8 segments which can be triggered by onboard switches or external low trigger like microcontroller pins.

4. ADVANTAGES

   1. The project uses AVR ATMEGA 16, which has 16KB flash memory, and it improves the storage capacity.

   2. The project uses Zigbee module, and it improves the range of the trans-receiver.

   3. The project has accelerometer sensor, so it is possible to accommodate more number of movements and gestures.

   4. The project allows physically handicapped dumb people with the feature to use home automation.

   5. The project uses Voice IC APR9600, which allows storing large number of pre-defined voice outputs for a particular gesture.

5. LIMITATION

   1. Project initial cost will be high.

   2. System is wireless so some time wireless communication may not work in harsh environment.

6. PROJECT APPLICATION

   1. This project can be used in Automobile assembly workshop.

   2. This project can be used in every type of Furness, casting and forging factory.

   3. This project can be used in any type of weight lifting industrial application.

   4. This project can be used in high temperature Furness, Chemical factory, Nuclear Power Plant, Mines, etc.

   5. This project can be used in high level application like space station, satellite telescope, etc.

7. CONCLUSION

Sign language is a useful tool to ease the communication between the deaf or mute community and the normal people. Yet there is a communicatio barrier between these communities with normal people. This project aims to lower the communication gap between the deaf or mute community and the normal world. This project was meant to be a prototype to check the feasibility of recognizing sign language using sensor gloves. With this project the deaf or mute people can use the gloves to perform sign language and it will be converted in to speech so that normal people can easily understand. The main feature of this project is that the gesture recognizer is a standalone system, which is applicable in daily life.

ACKNOWLEDGMENT

It is indeed a matter of great pleasure and proud privilege to be able to present this project on "GESTURE TO VOICE COMMAND GLOVE FOR DUMB". The completion of the project work is a milestone in students life and its execution is inevitable in the hands of guide. We are highly thankful to the Project Guide Mrs. Ashwini Jaywant and Co-Guide Mrs. Pooja Wankhede for his invaluable guidance and appreciation for giving form and substance to this report.

It is due to her enduring efforts; patience and enthusiasm, which has given a sense of direction and purposefulness to this project and ultimately made it a success. We would like to tender our sincere thanks to the staff members for their co- operation. Really it is highly impossible to repay the debt of all the people who have directly or indirectly helped us for performing this project.

REFERENCES

1. www.sparkfun.com/datasheets/Sensors/Flex/FlexSensor.pdf www.nch.com.au/verbose/index.html

2. Steven Douglas Collins. May,ADVERBIAL MORPHEMES IN TACTILE AMERICAN SIGN LANGUAGE. GRADUATE COLLEGE OF UNION INSTITUTE AND UNIVERSITY.1- 131,2004

3. Supawadee Saengsri and Vit Niennattrakul,Thai Finger-Spelling Sign Language Recognition System.IEEE.457-462, 2012K. Elissa, Title of paper if known, unpublished.

4. www.atmel.com

5. Yun Li, , Xiang Chen et al,Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data,IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING,

   VOL. 59, NO. 10,2695-2704, 2012

6. Kluge W, Dresden, Poegel F, Roller H. A fully Integrated 2.4 GHz IEEE 802.15.4-Compliant Transceiver and Zigbee Applications.

7. Liu D, Nilsson R, Norling F. Full Digital Driving voice and audio load on an IC. Vol. No. 7691444, July 2012.