Sign Language Converter for Deaf and Dumb People in Two Way Communication for Regional Languages

Sign Language Converter for Deaf and Dumb People in Two Way Communication for Regional Languages

Chaithraj

Dept. of ISE

Nieit, Mysuru, Karnataka, India

Kavana H R

Dept. of ISE

Nieit, Mysuru, Karnataka, India

Prathima S

Dept. of ISE

Nieit, Mysuru, Karnataka, India

Priyanaka S

Dept. of ISE

Nieit, Mysuru, Karnataka, India

Girish C

Dept. of ISE

Nieit, Mysuru, Karnataka, India

Abstract This project presents a language converter for deaf and dumb people. In the present world it is very difficult for the deaf & dumb people to talk with the ordinary people. So it becomes impossible for them to communicate with the ordinary people unless and until ordinary people like us learn the sign language for the purpose of communication. The sign language of deaf and dumb is quite difficult to learn and it is not possible for everybody to learn that language. So every person cannot come and share their thoughts with these physically impaired people. So here is a system which would enable the deaf and dumb to communicate with each and every one. In this system a webcam is placed in front of the physically impaired person.. The co-ordinates captured will be mapped with the one previously stored and accordingly exact alphabet will be captured. Continuing in this way physically impaired person will be able to go through the entire sentence that he wants to communicate. Later on this sentence will be translated into speech so that it would be audible to everyone.

Keywords Webcam, Audio, Text, Sign

1. INTRODUCTION

   Vision based hand sign signal recognition is one of the emerging recent research is mainly on the human computer machine. However, there are lots of issue are there to recognize the hand gesture, out of them are the variation of the hand gesture appearance, Scaled, rotated version of image and the image processing speed. The main aim of this paper is to develop sign language recognition system that is able to detect and translate the hand gesture (sign signal) from captured images.

   Two Way Communications are mentioned. First with communication between normal people to deaf and dumb people, here normal people can provide the audio that audio should check for matching with the stored audios. Once matching done the corresponding signs has been displayed to

   the deaf and dumb people so that these people can understand the audio language.

   Second with communication between deaf and dumb people to normal people, here webcam will be operated and input from the deaf and dumb people will be given to webcam, webcam captures the deaf and dumb people signs and extract the features of signs and check for matches in the stored signs. Once matching done the audio will played for the particular sign so that normal people can understand the sign language.

2. EXISTING SYSTEM

   Aleem et al, had developed a Gesture Recognition systems examined the input gestures for match with a known gesture in the gesture database. Gesture Database Contained the necessary information required for pattern matching as well as a gesture-to-text dictionary and Speech Synthesis Module Converted word or letters obtained after gesture analysis into corresponding sound [1].

   Byung – woo min et al, presented the visual recognition of static gesture or dynamic gesture, in which recognized hand gestures obtained from the visual images on a 2D image plane, without any external devices. Gestures were spotted by a taskspecific state transition based on natural human articulation. Static gestures were recognized using image moments of hand posture, while dynamic gestures were recognized by analysing their moving trajectories on the Hidden Markov Models (HMMs). Nikhita Praveen; Naveen karanth; M S Megha examined Sign language interpreter using a smart glove.

3. PROPOSED FRAMEWORK

   The Proposed System Gives Two Way Communication System, first with Normal people to Deaf and Dumb people, Second with Deaf and Dumb people to Normal people.For implementation of this systemwe give audio as input so that

   audio matching using pattern matching techniques are used. The signs for corresponding audio matched will be displayed as output. Second, we require web camera is required for capturing the Sign gestures. Sign language gesture is recognised and showed the output in audio forms.

   1. Salient features of the proposed work

      • Camera Interfacing

      • SIFT Feature Extraction

      • Camera Interfacing

      • Grey scale conversion

      • Gesture recognition

      • Color recognition

   1. Camera Interfacing

      At the time of giving gesture it is necessary to interface the camera. Camera interfacing is used to capture the frames continuously and gives the captured frames to the next phase that is RGB to Binary conversion phase.

   2. Scale Invariant Feature Transform(SIFT)

      The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. SIFT is divided into two stages, key point detection and key point description.

   3. Constructing a scale space

      The first stage of computation searches over all scales and image locations. It is implemented efficiently by using a difference-of-Gaussian function to identify potential interest points that are invariant to scale and orientation.

   4. Key point localization

      At each candidate location, a detailed model is used to determine location and scale. Key points are selected based on measures of their stability.

   5. Assigning an orientation to the key points

      One or more orientations are assigned to each key point location based on local image gradient directions. All future operations are performed on image data that has been

      transformed relative to the assigned orientation, scale, and location for each feature, thereby providing invariance to these transformations.

   6. Key point descriptor

   The local image gradients are measured at the selected scale in the region around each key point. These are transformed into a representation that allows for significant levels of local shape distortion and change in illumination.

4. CONCLUSION

The proposed method is tested on different gestures. It produces fairly stable and good results every person cannot come and share their thoughts with these physically impaired people. So we have come up with a system which would enable the deaf and dumb to communicate with each and every one by using the image processing based language converter and sign language recognition system proposed for human computer interaction using Image Processing Technique.The deaf and dumb people use their standard sign language.This system converts sign language in to voice and text form to normal people and audio to sign and text form to deaf and dumb people.

REFERENCES

1. Patel, Y. Srinivas Rao proposes a Automated speech synthesizer and converter in cue symbol generation for physically impairs. International Journal of Recent Trends in Engg, Vol 2, No. 7, Nov. 2009.

2. AnbarasiRajamohan, Hemavathy R., Dhanalakshmi proposes a Deaf-Mute Communication Interpreter. Proceedings of the International MultiConference of Engineers and Computer Scientists 2009 Vol I IMECS 2009, March 18 – 20, 2009, Hong Kog

3. P. SubhaRajam and Dr. G. Balakrishnan Real Time Indian Sign Language Recognition system to aid Deaf – dumb People IEEE International Conference 2013.

4. SoumyaDutta and Bidyut B. Chaudhuri A Color Edge Detection Algorithm in RGB Color Space IEEE transactions vol.8, NO. 5, MAY 2013.

5. David G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, Vol.60, No. 2, pp. 91-110, 2004.