Visual Fuzzy Controller for Gesture Language Recognition System to Aid Physically Impaired People

Visual Fuzzy Controller for Gesture Language Recognition System to Aid Physically Impaired People

M.PALLAVI K.SUBHASHINI

PG Student, ECE Department Asstiant Professor,ECE Department

Sri Sairam Engineering College Sri Sairam Engineering College

Chennai, Tamil Nadu, India Chennai, Tamil Nadu, India

pallavivarman@gmail.com subhashini.ece@sairam.edu

Abstract-Sign language is a method of communication for deaf and dumb people. Normally this sign language is only understood by their family and friends circle alone. So these hearing impaired people are unable to communicate with normal people in the outside world. This work improves the communication between deaf community and normal people for better social and intellectual life .Hand gesture recognition system can be used for interfacing computer and human using hand gesture. Otsu algorithm treats any segmentation problem as classification problem. Total image level is divided into two classes one is hand and other is background. A morphological filtering method is used to effectively remove background and object noise in the segmented image. Canny edge detection technique is used to find the boundary of hand gesture in image. Fuzzy classifier is used for classification purpose and it is used to reduce complexity & provides us less computation time. It leads to work in real time and finally convert the sign language into text using web cam.

Keywords: sign language recognition, pattern classification, fuzzy classifier, segmentation, canny edge detection.

1. INTRODUCTION

   The sign language is the method of communication for deaf-dumb people. Sign language is a movement language which expresses certain semantic information through series of hands and arms motion, facial expressions & head/body postures. Hearing impaired people have over the years developed a gestural language where all defined gestures have an assigned meaning. Normally this sign language is only understood by their family and friend circle alone therefore the Basic communication medium between the deaf-dumb people and an ordinary people usually needs an interpreter for communication. However, finding experienced and qualified interpreters for their day to day affairs throughout life period is a very difficult task and also unaffordable. So these hearing impaired people

   are unable to communicate with normal people in the outside world.

   In order to avoid such a situation for these hearing impaired people one such solution is Human computer interaction system proved to be a reliable and consistent solution to such persons. The idea is to make computers understand human language and develop a user friendly human computer interfaces (HCI). Making a computer understand speech, facial expressions and human gestures are some steps towards it. Gestures are the non-verbally exchanged information. A person can perform innumerable gestures at a time. Since human gestures are perceived through vision, it is a subject of great interest for computer vision researchers. The

   paper aims to determine human gestures by creating an HCI. Coding of these gestures into machine language demands a complex programming algorithm.

   Sign language recognition is multidisciplinary research area involving computer vision, image segmentation, pattern recognition and neural language processing. Sign language recognition is a comprehensive problem because of the complexity of the shapes of hand and head gestures. Sign language recognition requires the knowledge of hands position, shape, motion, orientation and facial expressions. A functioning sign language recognition system could provide an opportunity for the deaf to communicate with non-signing people without the need for an interpreter. It could be used to generate speech or text making the deaf more independent. Unfortunately there has not been any system with these capabilities so far. The aim of this paper is to develop a system which can classify sign language accurately. So this paper helps these physically impaired people using visual fuzzy controller for gesture language recognition system. Fuzzy used as classifier. Fuzzy classifier is also known as fuzzy interference system (FIS) and it used to reduce complexity

   provides us less computation time so this concept leads to work in real time.

2. EXISTING SYSTEM

   Thad starner proposed a real time American Sign Language recognition system using wearable computer based video which uses hidden makov model (HMM) for recognizing continuous American Sign Language system. Signs are modeled with four states of HMMs which have good recognition accuracies. Their system works well but it is not signer independent. M.K.Bhuyan used hand shapes and hand trajectories to recognize static and dynamic hand signs from Indian sign language. Christopher Lee and Yangsheng Xu developed a glove-based gesture recognition system that was able to recognize 14 of the letters from the hand alphabet, learn new gestures and able to update the model of each gesture in the system in online mode, with a rate of 10Hz. Over the years advanced glove devices have been designed such as the Sayre Glove, Dexterous Hand Master and Power Glove. It is based upon patented optical fiber sensors along the back of the fingers. Star-ner and Pentland developed a glove-environment system capable of recognizing 40 signs from the American Sign Language (ASL) with a rate of 5Hz.Althrough this data glove device works in real time application but the signers want wear the data glove so this shows inconvenient to the signers.

3. PROPOSED METHODOLOGY

   Normally the recognition pattern is divided into two phases. First phase is training phase (static image) and another phase is testing phase (dynamic image). In the proposed method consist of four processing stages namely image acquisition, preprocessing, feature extraction and classification.

   Fig 1 block diagram for hand recognition system First stage is image acquisition stage in which images

   are captured at a resolution of 640 x 480 pixels. The runtime images for test phase are captured using USB web camera of LG Smart Cam. The images are captured in a high intensity environment directed to illuminate the image source which is held at black background so as to avoid shadow effects. The images are captured at a specified distance (typically

   1.5 2 ft) between camera and signer. The distance is adjusted by the signer to get the required image clarity. In

   image acquisition stage it captures an input image (hand gesture) in digital form.

   1. PREPROCESSING STAGE

      Preprocessing is very much required task to be done in hand gesture recognition system. Preprocessing is applied to images before extract features from hand images. Preprocessing consist of two steps Segmentation, Morphological filtering. Indexed Video frames are resized to reduce in resolution to lower the processing time. Here the resolution from 640×480 to 128×128 image using interpolation methods. Before segmentation in the preprocessing stage the captured RGB image must be converted into gray color image (binary image). Morphological filter technique is used to remove noise in the segmented image. Morphological techniques consist of four operations: dilation, erosion, opening and closing.

      1. SEGMENTATION

         A very good segmentation is needed to select a adequate threshold of gray level for extract hand from background

         .i.e. there is no part of hand should have background and background also shouldnt have any part of hand. In general, the selection of an appropriate segmetation algorithm depends largely on the type of images and the application areas. The Otsu segmentation algorithm was tested and found to give good segmentation results for the hand gestures and was, therefore, selected. Otsu algorithm is nonparametric and unsupervised method of automatic threshold selection. The segmentation of gray scale image of a hand gesture is performed using Otsu thresholding algorithm. Otsu algorithm treats any segmentation problem as classification problem. Total image level is divided into two classes one is hand and other is background. The optimal threshold value k* is determined by computing the ratio between class variance and total class variance. After determined the optimal threshold k* and hand pixel is assigned 1 and the background pixels are assigned 0 thus we get a binary image. A morphological filtering method is used to effectively remove background and object noise in the segmented image.

         On the filtered image apply canny edge detector. The Canny edge detection algorithm is known as the optimal edge detector. The first is low error rate. Low error rate means edges occurring in images should not be missed and that there are NO responses to non-edges. The second criterion is that the edge points be well localized. In other words, the distance between the edge pixels as found by the detector and the actual edge is to be at a minimum. A third criterion is to have only one response to a single edge. This was implemented because the first 2 were not substantial enough to completely eliminate the possibility of multiple responses to an edge.

         Based on these criteria, the canny edge detector first smoothes the image to eliminate the noise. It then finds the

         image gradient to highlight regions with high spatial derivatives. The algorithm then tracks along these regions and suppresses any pixel that is not at the maximum (non maximum suppression). The gradient array is now further reduced by hysteresis. Hysteresis is used to track along the remaining pixels that have not been suppressed. Hysteresis uses two thresholds and if the magnitude is below the first threshold then, it is set to zero (made a non edge). If the magnitude is above the high threshold, then it is (made an edge)

         Fig (a) Resized image Fig (b) Edge detection

   2. FEATURE EXTRACTION STAGE

      The edge images are further taken through scan process and detection phase. The procedure of scan process includes

      1. marking of feature points

      2. determination of heights of fingers in UP position is denoted as maximum peak point

      3. determination of angle between the line joining the feature point of UP fingers with the reference point and the horizontal line passing through the reference point is denoted as minimum point.

      After the extraction of maximum peak point and minimum point of hand gesture. Another feature shape of the hand gesture is also determined. Peak point and minimum point are determined using reference point (x0,y0).The tip of the finger is measured with respect to reference point. Below fig (d) shows the parameter of the hand gesture.

      Fig (c) System architecture Fig (d) parameter of hand gesture

      Using these parameter the shape of the hand gesture is also determined .finally this result to binary image (i.e.) when the finger in UP POSITION is represented as 1and the finger in DOWN POSITION is represented as 0. Finally the sign is detected using this binary image. In the testing phase the above computed values are compared with the static image in the training phase are classified using fuzzy classifiers.

   3. DATA BASE

      Training phase consists of database containing a collection of set of 26 alphabets and (0-9) numeric sign language. Totally set of 36 sign language are stored in the database. Each sign is assigned with codes. During testing phase (dynamic image) when the signer shows the sign infornt of web cam then the sign image captured by the web cam is compared with the database image. Therefore finally this sign language is discriminated using fuzzy classifier in the classification stage.

   4. CLASSIFICATION STAGE

      After the features are extracted these features are used as input in a classification algorithm for classification. The goal of classification is to identify characteristic features, patterns or shape of the sign image and use these to detect the sign. Classifier is final stage in sign language recognition system. There are different types of classifier to discriminate the sign image one of such classifier method is Euclidean distance method. This method is determined by calculating the minimum distance. In this paper presents a Fuzzy classifier to discriminate the images based on dissimilarity between two images.

      1. FUZZY CLASSIFIER

   Pattern classification is mainly to recognize the hand gesture for sign detection. These hand gestures are classified based on their feature like maximum peak point, minimum point, and shape of the hand gesture. Pattern classification can be done by many methods they are linear classifier, support vector machine, Euclidean distance but here this paper proposes fuzzy classifier to recognize the sign based on fuzzy interference system (FIS).

   For pattern classification consider Takagi-sugeno- kang (TSK) or simply sugeno type Fuzzy inference system because the output membership functions are linear or constants. Sugeno fuzzy inference system consists of five steps, fuzzification of input variables, applying fuzzy

   andor oroperator, calculating the rule weights, calculating the output level and finally defuzzification. Many methods are proposed to generate fuzzy rule base. The basic idea is to study and generate the optimum rules needed to control the input without compromising the quality of control. In this paper the generation of fuzzy rule base is done by subtractive clustering technique in sugeno fuzzy method for classification video. Cluster center found in the training data points in the feature space whose neighborhood maps into the given class. Each cluster center is translated into a fuzzy rule for identifying the class. A generalized type-I TSK model can be described by fuzzy IF-THEN rules which represent input output relations of a system. For multi input single output first order type-I TSK can be expressed as IF xi is Q 1k and x 2 is Q 2K and … and x nis Q nk, THEN Z is

   W=p0 +p1x1+p2x2+..+pnxn

   Where x1,x2,..xn and Z are linguistic variables;Q1K,Q2K..and Qnk are the fuzzy sets on universe .Using clustering method features are grouped and inputs are mapped with the membership function of the features .Finally the hand gesture is recognized using fuzzy classifier.

4. CONCLUSION

   Sign language recognition system is implemented successfully in real time application. Several algorithms and techniques are involved to give accurate hand gesture recognition. The aim of the project is to recognize the sign language in real time application. The processing steps to classify a gesture included gesture acquisition, segmentation, morphological filtering, and feature extraction and classification technique. Both training and testing phase sign image are classified using fuzzy classifier. Fuzzy classifier is used to discriminate the images based on dissimilarity between two images. Reduced complexity provides us less computation time so that this system will work in real time.

5. EXPERIMENTAL RESULTS

The proposed method is implemented with a set of 36 sign language. Which include 26 alphabets and (0-9) numeric values of Indian sign language. During real time when the signer shows the sign infornt of the web cam that sign is compared with the training phase image and finally produce output

Fig (4.1) Input sign Fig(4.2) Masked sign image

Fig (4.3) output sign detected as a

Similarly all the sign are classified and recognised using fuzzy classifier.

5. REFERENCE

[1]P. SubhaRajam, Dr. G. Balakrishnan, Indian Sign Language Recognition System to aid Deaf Dumb People,IEEE International Conference on Computing Communication and Networking Technologies (ICCCNT), 2010, Pages(s):1-9.

1. P.V.V.Kishore , P.Rajesh Kumar A Model for Real time sign language recognition system, International journal of advanced research in computer science and software engineering.

2. T. Shanableh, K. Assaleh,_Arabic sign language recognition in user independent mode, IEEE International Conference on Intelligent and Advanced Systems 2007.

3. T. Starner, J. Weaver, and A. Pentland, Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video, IEEE Trans. Pattern Analysis Machine Intelligence, vol.20, no. [5]Chan Wah Ng, Surendra Ranganath, Real-time gesture recognition system and application, Image Vision Computing,

1. A survey of document image classification: problem statement, classifier architecture and performance evaluation Nawei Chen · Dorothea Blostein Received: 1 June 2004 / Accepted: 20 December 2004 / Published online: 3 August 2006.

2. A Model For Real Time Sign Language Recognition System P.V.V.Kishore Dept. of electronics and communications engineering Miracle Educational Society Group of Institutions Miracle City, Bhoghapuram, INDIA P.Rajesh Kumar Dept. of electronics and communications engineering Andhra University College of Engineering Visakhapatnam.

8] Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Computer Interaction Archana S. Ghotkar archana.ghotkar@gmail.com. Gajanan K, Kharate .Department of Electronics and Telecommunication Engineering Matoshri College of Engineering and Research Centre University of Pune.

1. A survey of document image classification: problem statement, classifier architecture and performance evaluation Nawei Chen , Dorothea Blostein

2. Remote Sensing Image Classification: A Neuro-fuzzy MCS Approach B. Uma Shankar, Saroj K. Meher, Ashish Ghosh_, and Lorenzo Bruzzone.