Nirvatha Vadathi- An App to Assist Deaf and Dumb

Nirvatha Vadathi- An App to Assist Deaf and Dumb

1Pallavi A, 2Amrita I, 3 Pranitha B.L, 4Darshini S (8th sem CSE),

5 R. Paramesh (Associate Professor)

Department of Computer science Engineering Global Academy of Technology, Bengaluru, India

Abstract This application introduces the gateway for deaf dumb and blind to communicate with each other through an Android app. Educating the dumb people would be a difficult task and hence there are self-learning-based applications in the App store. The main focus of this application is to provide one such useful app to the community. Usually Voice-based email and chatting systems are available to communicate with each other by blinds.

Keywords–Android Application, communicate, Speech recognition, Indian Sign Language;

1. INTRODUCTION

   Android applications have indicated a sensational improvement in their usefulness to a point where it is presently conceivable to have mobile phones execute Java programs. Therefore, cell clients all through the world are currently ready to peruse and compose messages, peruse website pages, and play java games utilizing their PDAs. This pattern has been elevated to propose the utilization of the android application for better correspondence. Our social setup believes in dealing with every community equally. The goal of our project is to offer a social responsibility to improve the usability of android applications for deaf and dumb so that they can enjoy ordinary experience of communication though these android devices. One such application is "Nirvatha Vadathi. Nirvatha Vadathi" (Silence speaks) is an android app that helps deaf and dumb user to communicate with normal people around him/her. It provides an environment for the user to send message through a sign language keyboard. It recognize the hand gestures of sign language through image processing. The app also has feature to learn Indian Sign Language. Other feature converts the speech spoken into text and vice versa.

2. RELATED WORK

   There are few android based mobile applications that has been developed for Deaf and dumb like speak it, Signily etc., The features of these application enable only the communication between deaf and dumb through American sign language. The [1] paper introduced an application that proposes a sign language keyboard that translate word from sign language to English or Arabic vice versa. It also introduces quizzes and games for deaf and dumb to identify Arabic and English words. The paper [2] has introduced a sign language

   converter system using hand gesture recognition feature to recognize the gestures in Indian sign language. The paper [3] has demonstrated an application of android that will help speech disabled deaf and dumb people to get trained and start communication with normal people. This app will focus on using Google Offline speech recognition system, which converts the voice spoken to text. The converted text will be searched in the dictionary built in app to convert that text into its appropriate Hand-sign. Through this paper, we inferred to implement Google offline speech recognition for voice to text conversion in our Android app. The [4] is a web resource that introduces Live Relay that is a app developing by Google that will allow speech disabled users to conduct a phone call. There will be a voice to text conversion in an ongoing call. This app targets differently disabled person to have a normal conversation. The paper [5] gives us an insight of how to design user-friendly app for speech disabled people by evaluating through Heuristics method. Heuristics evaluation is a systematic usability inspection method. This paper provides an insight to an android app called Speak it. The application ease of use is assessed through heuristics assessment and proposals are given to defeat the shortcomings.

3. PROPOSED APPLICATION

   This application has given a way for the deaf person to interact with normal people anywhere. Our proposed system includes a number of technologies like voice to text conversion speech to text conversion. Being a completely software based application, the app uses Google- speech API to achieve that result. In our application, for building this the main software which we have used here is Android Studio. Frontend designs are made using XML connected using Java along with SQLITE as a database. This application has user login for the first time following displaying different options like sending text message, voice to text message, emergency message, sending GPS location and also a feature that assists the Indian Sign Language to the normal people. There are many such applications available ready in the Google play store. But we have integrated all such available features in our Android app for the benefit of mankind

   1. Architecture diagram

      System architecture defines the structural and behavioural views of the system. It is a formal and brief representation of the system. The architectural model that we have discussed is shown in the fig. 1.

      Fig 1: Architecture diagram of the system.

      The general architecture of this system is a three-tier architecture. The first phase is the data input phase and the next phase is modules split ups followed by the output phase. Now lets look into the description about each and every module.

   2. Google speech recognition Module

      The proposed app takes voice as input and produces the corresponding text as output. The converted text is displayed on the screen, in English language. Thus, by this way the deaf will be able to communicate easily with normal people.

      Google offline speech API is a speech recognition system that has been used for voice to text conversion. The Google specialists have built up this framework in android which can identify the words verbally expressed by the communicator and convert those words into a book. It enables the voice to text conversion by applying powerful neural network models for automatic speech recognition. Google Speech API recognizes 120 languages and variants to support a global user base.

   3. Text Messaging Module

      The Text Messaging Module helps the user to send the SMS/Text messages to any user. The module takes phone numbers and the text through the in-built feature i.e., Google keyboard.

      Text message is an act of composing electronic message to the right person and at the right time using real-time. The messaging feature acts a platform to communicate between deaf and dumb users with normal person in this digital platform. It is not a bot and there is response from the right person. This helps the deaf and dumb user to experience normal messaging feature in the proposed app itself.

   4. Sign Language Module

      The SMS option is featured with a sign language keyboard (Sign language keyboard is a keyboard where each alphabet key is replaced with an equivalent hand-sign of Indian sign language). This keyboard is also provided with some emojis that has a particular hand-sign to communicate general sentences like hello, how are you etc., This helps the deaf and dumb user to communicate his thoughts over SMS by using the hand sign. This combination of hand sign keys are either converted to text or sent as it is to the intended user.

      The proposed sign language keyboard is built using Keyboard designer. Keyboard designer is a tool to create keyboards. With this tool, one can freely place all the buttons and provide them with functions and characters as we want.

      The emoji are created using Bitmoji and added to the virtual keyboard called Gboard (Gboardis an all-in-one keyboard that includes glide and voice typing, a predictive word engine and stickers). Bitmoji is an app that integrates into Gboard. One can create an avatar from scratch and it creates a sticker library, this library can be accessed within Gboard.

   5. Emergency Module

   In the situation of any emergency the proposed app has the emergency module. This module helps the person to send an emergency message along with present GPS location to the added emergency contact.

   The users GPS location will be notified to the emergency saved contact. In this way deaf and dumb can intimate their emergency situations and get the help without feeling any helplessness.

4. FEATURES

   • The voice message is converted to text and it is displayed on the screen.

   • A SMS option has a feature, where the voice converted text can be sent to the intended user. The SMS feature also contains a Sign language keyboard where hand-sign keys are used to type the message.

   • The emojis are provided as a substitute to general sentences conveyed through one hand. The typed message can be sent as it is or converted to text and sent to the intended user.

   • The user is provided with an Emergency feature, where the user can store some emergency contacts. When a power button is pressed thrice, his current location is sent via SMS to those contacts numbers. The user also has some other templates such as help me, call me, text options that are sent via SMS.

   • The Help Center option is a learning tool that teaches Indian sign Language.

5. CONCLUSION AND FUTURE WORK

As talked about above, there are fewer investigations that done on assessing the ease of use of the hard of hearing and unable to speak portable application. Numerous applications that have been created will in general be assessed by and large instead of containing the assessment into more profundity to test the ease of use. This could be a result of the point of the talk about examinations is fixated uniquely on the advancement that they will in general overlook the ease of use significance. To satisfy the improvement need, the specialists assess the application by and large whereas in the genuine circumstance general assessment probably won't have the option to create persuading results since the prerequisite of hard of hearing individuals are diverse contrasted and typical individuals. Studies that were talked about before likewise shows that crippled are additionally needing innovation to guarantee they are not confined. Incapacitated need innovation particularly cell phones to speak with each other. In the future, the framework should be redesigned for voice to text and the other way around

in a continuous call and this is the progressing research in the Google people group for better correspondence utilization.

REFERENCES

1. Dalia Nashat, Abeer Shoker, Fowzyah Al-Swat and Reem Al-Ebailan, An Android Application to Aid Uneducated Deaf-Dumb people, in Dalia Nashat et AL, International Journal of Computer Science and Mobile Applications, Vol.2 Issue. 9, September-2014.

2. Pranali Loke, Juliee Paranjpe, Sayli Bhabal Ketan Kanere, Indian Sign Language Converter System Using An Android App. International Conference on Electronics, Communication and Aerospace Technology ICECA 2017.

3. Rohit Prakash, Emerging Mobile Applications: An App to Improve Literacy and Communication for Disabled Persons,2017 2nd International Conference for Convergence in Technology.

4. https://www.google.com/url?sa=t&source=web&rct=j&url=https://www

   .blog.google/outreachinitiatives/accessibility/live-relay-phone-calls-io

5. Emerging Voice controlled wheel chair Sufia Noori, R.paramesh International Journal of Education and Research Publication (IJERP)- 2019, vol 2 Issue 1 2019.

6. Saadia Ahmed, Fahim Arif, Evaluation of Speak_ it Application to Identify Usability Heuristics Loopholes with Suggestions , Department of Computer Software Engineering, National University of Science and Technology, (NUST), Islamabad, Pakistan.