Voice Based Mail System for Visually Impaired

Voice Based Mail System for Visually Impaired

1st M. R. Pradhicsha, 2nd M. Vasanth, 3rd V. Renita, 4th Dr. P. Lakshmi Harika

Department of Information Technology, Madras Institute of Technology

Chennai-600 025

AbstractIn day today life, internet has become one of the most important thing. Every individual person is widely accessing the knowledge, information and all the available features for the communication via internet. The visually impaired, however, have trouble using those functions and require outside assistance to make their task easier. The development of computers has given visually handicapped people all around the world numerous new opportunities. Audio based environment, screen readers and many helpful features have helped blind people to take the advantage of the workspace.Today, sending private information requires the use of email. Email is a form of technology that helps with corporate communication as well as allowing users to send mail to others. The major goal is to create a voice-based email system for those who are blind or visually impaired, allowing them to send and receive emails using computers.It will use the latest features to provide an environment which is helpful for the visually challenged people to work without the need for an external help.

KeywordsScreen readers; Audio Based environment; Voice Based Technology

1. INTRODUCTION

   Although our culture blames those who are blind for these reasons. The visually impaired may now use the Internet thanks to a range of dependable and efficient technology. As a result, we have developed this design concept for a voicemail system that is email-friendly and can be used by an eyeless person in order to advance society and grant similar especially suitable people an equal status and respect. This gives blind people the ability to send and receive messages using voice commands. Other visual enhancements or a keyboard are not necessary for this system. This is based on the usage of STS and TTS transformers, or speech to text and text to speech, respectively, for translating spoken words into written and audible forms. The person utilizing this method wouldn't have any prior knowledge of the keyboard's layout, the alphabet's positions, or the locations of the keys.The user will respond after being automatically fed voice commands to carry out certain actions by the system. The user merely needs to correctly follow the directions that the system offers. With the use of voice commands and the suggested technology, visually impaired persons might access emails on their own.

   1. Overview

      The internet has become as one of the desired or practical things for daily life. By gaining access to information, facilitating interpersonal interactions, and growing enterprises and associations, it has made people's lives more comfortable. Internet becomes the first luxury for a 24-hour lifestyle. everyone who uses the data and information on the internet. People's lives are made simpler when they use the internet for communication. Internet has significantly altered the realms of communication. Global e-mail drug users were four billion in 2021, and the figure is expected to rise to 4.6 billion by 2025.In 2021, there were over 306 billion emails sent and received per day globally. For sharing and inputting sensitive or private information, electronic correspondence communication has shown to be the most secure and safest method. To utilise the Internet, a person must meet the prerequisite of being able to see, which is a need that must be satisfied. Because surfing the internet involves visual sense, it has become a scourge for those who are visually impaired. More than 250 million individuals, according to a check, do not know how to use email or Internet installations for communication.The only way an eyeless person can utilise all these internet features is if they dictate the entire material to a third party who isn't visually impaired. The third party will then send, receive, and read out the messages at the visually impaired person's request. Although the average person assists the blind in accessing their mail, that isn't the best method to approach the problem. Because the visually impaired individual demands assistance whenever he or she can find it.

   2. Research challenges

      Emailing doesn't provide a large challenge to individuals with the gift of sight, but since it interferes with so many work obligations, it presents a considerable challenge to those without the gift of sight. Emails with audio attachments are only preferred by those without eyes. They can easily reply to the spoken commands. Therefore, there are less odds of making this audio-based email available to those who are blind. For those without eyes, this voice-based email system is helpful since it enables them to understand their location.On the other hand, people need to remember mouse clicks and keyboard shortcuts. With this method, the usability of all users, including those who are typically blind, is given more priority. Only desktop computers may use this system, and occasionally it is unable to properly decode the material. Future updates to the system's UI may also include new functionality.

   3. Objectives

      The major goal of this article is to create a voice-based emailing system that will enable visually impaired and illiterate persons to use everyday technologies like sending and receiving emails and accessing the internet. With the use of this method, blind users may easily log in by speaking. The primary goal of developing the sort of system outlined in the study is to increase the sense of community among those who are blind or visually impaired. Any user of any age may easily access our email system. In the future, this technology may be

      improved and used for additional services, such as texting and using voice commands to operate other applications, in addition to email

   4. Scope of the project

      The development of computer-based accessible solutions has given the blind and visually impaired many new opportunities all around the world. However, because utilizing them involves visual sight, visually impaired persons find it extremely difficult to utilize this equipment. The weaknesses of the current system are remedied by the proposed system. In contrast to the current systems, the proposed approach is voice command anchored. The availability of the suggested system is the most crucial factor that has been taken into consideration. Since the present system is completely different from the one we're designing.

      Python is used to implement the requested task. Ideas focuses on providing basic features like create, read, transfer, and read emails as well as voice-based instructions. Unlike the existing system, which places a strong emphasis on user friendliness and compatibility. By using this system, the user is prompted to give the system certain commands, such as "compose the mail," "display the email id of the previous three unread messages," and "select one mail id" before reading the topic and content of that particular email id. Direct user- system interaction is used. Our method is favourable to all sorts of individuals, whether they are normal, visually handicapped, or illiterate, whereas the current system concentrates more on normal people.

   5. Contributions

   Our contributions are summarized as follows:

   1. The STT ( Speech- to- text ) module , converts the speech recognized through microphone to the text and stores the written formattext in the variable. Or else, it throws an exception

   2. The TTS ( text to speech ) module converts the text given to that function to the audio that is .mp3 file and plays the file created .

   3. GTTS ( Google text to speech) library in python .This is platform independent that is it is compatible with Windows, Linux, and MacOS speech library. This module which automatically produces an audio file while accepting the input which is in text format along with language and speed parameter. And also read out the messages in emails for the user.

   4. SMTP ( Simple mail Transfer Protocol) module is imported using the keyword smtplib. It is used to connect the Gmail This protocol is demanded to compose the emails and act as a connection to Gmail helping in performing other functions also.

2. LITERATURE SURVEY

   1. Voice Based Technology for Blind

      Pranjal Ingle et al.(2016) [1] , used three types of technologies to create the application namely STT(speech-to- text) where the speech is converted to text,TTS(text-to- speech) to convert the text to speech,and thirdly

      IVR(interactive voice response) which describes the interaction between the user and the technology in many ways like keyboard or voice message. It also allows the user to interact with the mail system. The main disadvantage includes the usage of high sensitive mics which are mostly not available to all the users

      Jain. V. et al., (2021)[2] ,this research proposes a voice- based email system that visually impaired people can use to easily access email. With the aid of technology, this initiative aims to assist people who are blind in sending and receiving voice mails. The advancements in text-to-voice email delivery for people who are blind or visually impaired are the main topic of this study. This study offers a text-to-voice and voice- to-text email access method for those who are blind. This enables persons who are blind to send mail using voice control instead of a keypad.

      Divesh Jethani et al.,(2018)[3] proposed a voice based system for the visually blind with multi-lingual facility,the system provides a good GUI for all types of users. The user will be able to send,receive,read,delete the mail from the mail system. But the main disadvantage includes the usage of mouse clicks, which is necessary at some places of the proposed application.

      Dr.S.Brintha et al.[4], proposed a system with TTS and STT to read and record symbolic linguistic representations like phonetic transcriptions. The architecture of the system includes two modules namely interface selection and mailing option, the first module selects the type of users that is blind user or sighted user. And the second module includes the simple mailing options to perform all tasks.

      Parkhi Bhardwaj et al.,(2016) [5], uses an extra speech recognition technology along with all other converters and IVR to develop the application. The proposed system provides more features than existing GUI. Java was the core programming language used. The application can be used by all types of handicapped people and illiterate people.

   2. Summary of the Literature survey

      In literature survey various technology to develop a voice based E-Mail system was discussed. Since the existing system like screen readers and so on, has some disadvantages. The proposed work in this model takes the best advantage of technology to use features which has the highest accuracy in capturing voice and displaying contents which are needed by the visually impaired people.

3. SYSTEM ARCHITECTURE

   The following system architecture Figure 3.1 shows the flow from the starting where the basic options are spoken out by the google-text-to-speech and the voice commands from the user is recognized and the command is executed, that is if the user wants to compose mail, then the subject and receiver mail id is asked and the message is sent to the receiver. Python has a yagmail module and many other modules which support the SMTP.

   If the user wants to know the unread messages from mail, then the messages from the selected mail id and read out to the user.

   Figure 3.1 – Architecture for Voice based email system for visually impaired

4. ALGORITHM

   1. User Login

      The login module used in our project is purely voice- based where the user need to tell their email id and password . The login system in turn tells email id and password that the user told and asks whether it is correct or not , if the user says correct , the user can proceed with the further operations using the email id and password that he/she told. If the user says not correct , the login system again asks for the email id and password for the login purpose.

      We used python to recognize voice using Speech recognition module and used Google text-to-speech module to convert text to speech.The front-end for the login page is developed using tkinder module in python. The recognized email id and password through speech will be shown in corresponding text boxes.corresponding text boxes.

   2. Option Choosing

      After the successful login , the email system asks for the option whether the user wants to compose a mail or wants to check the inbox . To compose a mail user need to press ENTER . To check Inbox user need to press the space bar .

      This module is developed using a menu-driven type of program in python, the program ask the user to press Enter to compose a mail or ask the user to press Space bar to check the inbox of the email id of logged in by the user.If the user press the Enter , call the compose_mail() function to compose a mail.If the user press the Space bar , call the check_inbox() function to check the inbox of the logged in mail id by the user.The front-end of this module is also developed using tkinter GUI module.

      Figure 4.1 – Option choosing page

   3. Composing Mail

      This module first asks for the subject of the mail to be sent and then asks the content of the mail to be sent . Then it asks the sender mail address to which the subject and the contents to be sent . Finally the composed mail is sent to the destination mail address using SMTP and the acknowledgement of the sent mail will be provided by the voice commands.

      After pressing the Enter in the option choosing , the user can compose a mail . First the user is asked for the subject of the mail to be sent , then the user is asked for the content of the message to be sent and finally the sender mail address is asked from the user to which the mail needs to be sent. A bit of file handling is done on the sender mail address to make it as the perfect mail address . All these inputs were got from the user through voice commands . Then SMTP is used to send mail to desired mail address.The frontend of this page is designed using tkinter.

      Figure 4.2 – Composed Mail

      Figure 4.3 – Acknowledgement at sender side

   4. Checking Inbox

      After pressing the Space Bar in the option choosing , the user can check the inbox of the logged in mail . First the user is asked for the specific user whose message need to be read. Then the subject and content of the message is read by the system .For reading mechanism Google text-to-speech(gTTs) is used. All these inputs were got from the user through voice commands . The front end of this page is designed using tkinter.

      After pressing the Space Bar in the option choosing , the user can check the inbox of the logged in mail . First the mail id of the first three unread messages are stored in keys of the dictionary and the subject and contents are stored in the python list and mapped as the value to the corresponding key value in the dictionary . The user is asked to use a mail id among those stored mail ids . Then the user chosen mail ids values in the dictionary (i.e) The list which contains subject and content of the received mail is been read. The front end of this page is also developed using tkinter.

      Figur 4.4 – Reading message in inbox

      Figure 4.5 – First three Unread messages

   5. Text to Speech

      This module is almost used in all other modules.This function converts the text given to that function to the .mp3 file.And plays the .mp3 file created.

      To use this function first import the gTTs (Google text- to-speech) module.This function converts the text given to that function to the .mp3 file.And plays the .mp3 file created.

   6. Speech to Text

   This module is almost used in all other modules.This function converts the speech recognized through system microphone to the text and stores it in the variable. If not recognized it would rise an exception.

   This function converts the speech recognized through system microphone to the text and stores it in the variable. If not recognized it would rise an exception.

5. RESULTS AND DISCUSSIONS

   1. Libraries used by the system

      The propose system speech_recognition package from python library which is installed using the pip command. Speech Recognition it is an important feature in several day- to-day application used in home application and many artificial intelligence etc. Yagmail is another important package which is used to send mail from the sender mail address to the receivers mail id. The text to speech is done using a module called gTTS from google-text-to-speech. Pythons client side library called imaplib is used for accessing emails over IMAP protocol. It allows the client program to manipulate the email message. For providing a good interface for user tkinter and pyglet is imported which provides functions for developing interface for multimedia application.

   2. Speech to Text -Proposed approach

      Speech recognition includes the technology and linguistic to identify the spoken words by the user and converts them into text. It helps the computer to understand the human language. The speech recognizer in python is used to convert the spoken words into text and make a query and reply them with the required functions. The proposed system utilizes this feature to hear the voice commands from the visually impaired people to get the inputs and does the needed function accordingly by checking the condition mentioned in the code.

      The speech input is done using the external microphone. The external microphone is configured and even blue tooth devices can be used to take the input from the user. Since the environment around the user can be noisy,the program is allowed to adjust a second according to the threshold value of the external noise adjustments.

      Finally the speech to text translation is done using Google Speech Recognition which requires an active connection to the internet. But there are few offline recognition system which requires very difficult installation steps and requires many dependencies. Google speech to text is one of the easy and efficient way to hear the user commands and convert them to text properly.

      In our system, the voice commands of the user is directly listened from the microphone. The spoken commands of the user is collected with the help of listen command which takes the parameter of defined source which the microphone in our system.The voice is converted into text by recognize_google command which converts the audio file to text. These function are available in Recognizer function from the speech_recognition package from python.

   3. Text to Speech -Proposed approach

      The proposed model converts human language text into human like speech. Python provides many application interface to do the same. The Google Text to Speech is one the most efficient method and this API is used in our system. It is very easy to use the tool and provides many built in function which is utilized to save the text.We dont need an external neural network to train the file into speech, instead we can use the API to complete the work easily.

      In our system the text is converted into mp3 file, the language of the text is specified as the function parameters in the gtts function. The mp3 file is saved directly in the same directory as in the project file. The filename of the audio can be specified at the time of code specification.

      Later using the pyglet module, the saved mp3 file is loaded and is played using th speakers which are available in the personal computers of the user. Using the time package available in python library, the speed of the voice can be specified, the system can also wait for some time after converting. And later the saved mp3 file from the directory can be removed by the remove command available in the OS package in the python library.

   4. Filtering of Mails from Inbox

      Our proposed system will read first three unread message form inbox of the user. The filtration process is an added feature which is implemented by the data structure called dictionary which is available in python library. The email id of messages are kept as keys and the content is specified as value to that particular key in the dictionary. When the user chooses the mail id, the particular message from the dictionary is read out loud to the user.

      This is an added novelty by our system to facilitate the user to choose the required message without the need for hearing all the messages from the inbox. This feature is very helpful and effective in making the working process a step easier for the visually impaired people.

   5. To send and receive Mails

      The library used in the proposed system is email and imaplib from python library. Email is pythons built in smtp library which is used to send basic emails. Email package helps with HTML content and attachments using the available email package. Also the system uses Yagmail package to send mail through specific users who have Gmail account using very few lines of code.

      IMAP is an email retrieval protocol which saves the storage space by not downloading the mail. It helps to read the inbox message and displays them. This package is very helpful in low bandwidth. IMAP provides various functions like deleting,reading,replying and many more. In our module we utilize the read function.

      IMAP has a function named login where we can specify the username and password of the senders mail. Later the messages from the inbox of specified mail can be retrieved using the select function. We can parameters like inbox,archived in the select function.

      The get_payload function helps in retrieving messages from the html junks. This is very helpful to extract the message from the inbox without any unnecessary values.

   6. GUI for the Voice Based Mail System

      Python offers a wide range of possibilities for creating graphical user interfaces. One such interface for the tk GUI toolkit included with Python is tkinter. It offers a quick and simple solution to develop GUI apps. The Tkinter module has to be imported before we can develop a GUI application in the main window. Additionally, widgets can be created to do the required task.

      The proposed uses simple tools for developing interface, as the visually interface does not require the graphical content for sending mail. Hence the simple tool was implemented to provide an environment for the visually impaired people to send and receive mail.

      If the cursor is moved on the screen then the function will start working and will ask for the necessary options, that is either to read or compose message. And simple keyboard shortcuts are used which enables the user to choose the specified option.

   7. Result And Performance Analysis

   The proposed method for voice based email system when compared to some pre exitising model has very few disadvantages. The only disadvantage is the use of very few keyboard clicks. Only two moves are needed to work with this interface. But the system has many advantages that is it allows the user to filter the contents from the mail and allows the user to choose the mail id from which the contents can be read. Its effective with imple microphone available in the personal computers.

6. CONCLUSION AND FUTURE WORK

1. Conclusion

   We have designed a method that will make it easier for those with visual impairments to use email services effectively. This approach may assist in overcoming several obstacles that blind people previously encountered while trying to access emails. Screen readers, which might lessen the cognitive strain of memorising tasks, have been removed. The major goal of developing the kind of system outlined in the study is to increase the sense of community among those who are blind in this little environment. The decision tree takes a distinct path since each operation has a distinct consequence, making the system considerably more compatible. People who are physically disabled will be able to access the world with the aid of our system. Anyone may use this email system with ease, regardless of their age group. It contains features of speech to content as well as content to speech with discourse reader, making planned structure manageable for those who are externally disabled as well. Now, visually challenged individuals may send and receive mail with ease using only voice instructions and very little keyboard or mouse use. It has helped eliminate the challenges that blind people experience and transformed them into more typical persons. It has eliminated the notion of deploying screen readers in addition to console shortcuts, which would have lessened the cognitive

   load associated with remembering console shortcuts. In the future, this programme can be improved and used for additional services in addition to email, such as messaging, taking notes, and using other voice-activated applications.

2. Future work

In the future, this application may be improved and used for various services in addition to email, such as messaging, taking notes, and voice-operated operation of other applications. Furthermore, even the smallest amount of keyboard shortcuts may be removed with the help of high-tech tools. In the future, voice's capabilities could be expanded to include picture attachments and other features included in regular E-Mail, including indentation and font selection.

REFERENCES

[1] Ingle, P., Kanade, H. and Lanke, A., 2016. Voice based e-mail System for Blinds. International Journal of Research Studies in Computer Science and Engineering (IJRSCSE), 3(01), pp.25-30.

[2] Jain, V., AK, K., Shenoy, R.N. and Ahmed, M., 2021. Voice Based Email for the Visually Impaired.

[3] Divesh Jethani, Simmi Saluja, Saurabh Gautam Published 26 March 2018 Computer Science International Journal of Advance Research,

Ideas and Innovations in Technology

[4] Biruntha, S., Priya, M.G., Kiruthika, R., Indupriya, N. and Ashwini, R., 2021. Voice Based Email for Blind People Using Speech Recognition through Artificial Intelligence. International Journal of All Research Education and Scientific Methods (IJARESM), 9(04).

[5] Parkhi Bhardwaj, Gunjan Sethi Voice Based E-mail System for Visually Impaired: A Review published in International Research Journal of Engineering and Technology (IRJET) on December 12th, 2020.

[6] Pathan, N., Bhoyar, N., Lakra, U. and Lilhare, D., 2019. V-Mail (Voice Based E-Mail Application). International Research Journal of Engineering and Technology (IRJET), 6(03).

[7] Tharani K K, Shalini R, Jeyanthi I, Dr.Deepalakshmi R (2017), Voice Based Mail Attachment For Visually Challenged People, in International Journal of Scientific and Engineering Research, Vol.8, Issue.5, pp. 126-130.

[8] D Kiran kumar, User Interface for Visually Impaired People, IOSR Journal of Electronics and Communication Engineering (IOSR-JECE) (Jan.-Feb. 2017), PP 65-71

[9] Shoba, G., Anusha, G., Jeevitha, V. and Shanmathi, R., 2014. An interactive email for visually impaired. International Journal of Advanced Research in Computer and Communication Engineering (IJARCCE), (5089-5092).

[10] Rohit Rastogi, Anshika Rajput, Archana, Komal, An Application of Voice Mail: Email Services for the Visually Challenged Individual

,International Journal of Software and Computer Science Engineering Volume 4 Issue 1

[11] Bhowmick, A. and Hazarika, S.M., 2017. An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends. Journal on Multimodal User Interfaces, 11(2), pp.149- 172.

[12] Ummuhanysifa, U. and Nizar Banu, P.K., 2013. Voice based search engine and web page reader. Internationa Journal of Computational Engineering Research (IJCER), pp.1-5.

[13] Tiwari, P.A., Zodawan, P., Nimkar, H.P., Rotke, T., Wanjari, P.G. and Samarth, U., 2020, February. A Review on Voice based E-Mail System for Blind. In 2020 International Conference on Inventive Computation Technologies (ICICT) (pp. 435-438). IEEE.

[14] Hari Priya, S.L., Karthigasree, S. and Revathi, K.,2015. VoiceBased E- Mail (V-Mail) for blind. International Journal.

[15] Kulkarni, O., Alhat, A., Tejankar, N. and Patil, M., 2019. Voice based e- mail system for blind people. Open access international journal ofscience and engineering, 4(01).