Smart Mirror using Raspberry Pi

Smart Mirror using Raspberry Pi

Bhuvaneswari T, Aishwarya C, Aishwarya HA, B Abhinaya Nalina HD (Assistant Professor)

Department of Electronics and Communication Engineering, GSSSIETW, Mysore

Abstract In this fast running world where time has become the major constraint, each and every minute becomes important to the busy individuals. Hence, Smart Mirror could be an added advantage for the busy individuals who believe in multitasking and feel a need to get the daily updates while on the hustle. Instead of repeatedly checking the devices for information, one could be informed while grooming oneself. This paper gives the details of the hardware and the software requirements and the techniques which are used to display the various information such as weather, time, date, and news updates, music, email notifications in the form of widgets, along with voice control and therefore controlling the mirror updates using voice for news feeds, e-mail notification and other information using the Raspberry Pi. Embedding a computer screen in mirror appears very futuristic. The raspberry Pi, at the backend controls and manages the data displayed on mirror. When one looks in the mirror, one can view various notifications from date, time and news to weather forecast and more things.

Keywords Smart Mirror, Raspberry Pi, Voice control

I INTRODUCTION

The growth in Science and the scientific innovations in this field has contributed greatly towards the growth in the technology, one among which, are smart devices. With the advancements in technology increasing, equipping these smart devices in the houses shows the increase in interest shown by people. The use of smart devices, redefines the experience of living throughout the world, which is of great help in finding efficient solutions of day to day problems. The demand and commercial availability of smart devices have increased recently. Among the busy schedules, it becomes a problem for people these days to be updated with the daily news. The proposed Smart Mirror helps in solving this problem.

One of the sources that can be used to obtain the daily updates are the Newspapers which are at time creating inconvenience considering the facts that

1. Do not have dynamic information, i.e. there are no updates on information.

2. The speed at which the information is conveyed via digital media is greater than that at which the newspaper conveys information (usually, newspapers are distributed once a day). There are upcoming devices with the idea of providing comfortable services to users in their own houses, workplaces as well as for use in the hospitality industries. The touch interaction technology with integrated application software support is utilized in the smart mirror, which may not be pocket friendly for the consumers. As per the current standards all products in the smart-mirror market are either too expensive or limited to general hobbyists. Smart mirror is

   one of the applications of Raspberry Pi. The basic design of a smart mirror begins with the glass that is to be used. Embedding a computer screen in mirror appears very futuristic. The raspberry Pi, at the backend controls and manages the data displayed on mirror. When one looks in the mirror, one can view various notifications from date, time and news to weather forecast and more things. This interface is easy to use, can be accessed from any Wi-Fi enabled device, which allows the user to easily setup the connection with the Wi-Fi at their home, and view location- based weather reports, and daily headlines form the selected sources. By building these features into a mirror that is already been used on daily basis for grooming tasks, it is possible to display this information in such a way as to seamlessly blending it together with the task of grooming.

   Most of the papers have focused on displaying various kinds of information such as users calendar schedule, news, music, email notifications and weather. Some of the papers have focused on specific application such as for medical health, college navigation. Almost all the papers have used Raspberry Pi as their computer as it is convenient and small computer with greater functions.

   The system proposed here is developed based on three objectives:

   1. Design a prototype on Smart Mirror using Raspberry PI.

   2. Facilitating the implementation of Smart Mirror by developing a voice recognition system.

II LITERATURE SURVEY

Using the MCU and PC techniques weather, Twitter, news, to-do list, and calendar is displayed to provide the user- specific information along with grooming feature of the mirror. This was proposed by Varsha Singh [1]. Raju Nadaf designed and implemented a smart mirror to provide Security by detecting any of the intuitions passing around the area of the mirror using Raspberry pi [2]. Derrick Gold designed and implemented smart magic mirror to allow users to interact with the mirror and to access information [13]. Kun Jin designed and implemented Smart Mirror for home automation system to meet consumers need towards intelligent life using Raspberry pi which is helpful for the Home Communication [3]. Design and implementation of smart mirror for intellectual purposes which is time saving and affordable assistant which displays weather, news, date and time, this was proposed by Ayushman Johri [4]. Design and implementation of Smart Mirror to enhance such mirrors with intelligence and security was done by Charles Njaka [5]. Design and implementation of the smart mirror to provide an improvement attempt to reinforce the mirror with magnified features was proposed by R Akshaya [6]. Design of mirror

that offers the services to the home environment was proposed by Mohammed Ghazal [10].

Fatima Ok designed and implemented Smart Mirror which majorly concentrated on providing users to access information via internet [11]. Sun Young developed design and implementation of an intelligent mirror based on Raspberry Pi designed for home of internet of things [7]. Mohammed Yusri designed and implementation of smart mirror for intellectual purposes [9]. Oihane designed and implementation of a smart mirror which can be used by multiple users to provide healthier work environment [14]. V E Pawar designed and implementation of a smart mirror to provide both mirror and computer aided information services to its users using Raspberry Pi 3 Microcontroller card [15].

III. HARDWARE AND SOFTWARE REQUIREMENTS

1. Hardware Components

   1. Raspberry Pi: Raspberry Pi is the key hardware component used in the smart mirror to display user-specific information on the monitor. It is a small single-board computer which runs code on the operating system installed in it. This component has a pre-installed Operating system called RaspbianOS (Operating system), which is debian- based Operating system. The code can be written in any of the supporting language. This paper is based on JAVA Script. This helps the monitor to display weather-forecast, date and time, email notifications, calendar, news-feed and music with the written code dumped to the device. This also includes the usage of web-based services to display weather, news and other information by extracting the information from the internet as tokens to present on the monitor in order to provide it to the users. To accomplish this, the Raspberry Pi module has a Wi-Fi module to connect to the internet. The voice recognition is achieved by receiving the input from the microphone connected via USB card which allows user to give the voice input to the mirror to set up reminders and for cnversations.

      Fig 1. Raspberry Pi

   2. LED Monitor: LED is a Light Emitting Diode monitor which acts as an interface between the raspberry Pi and the mirror to display the user specific information on the screen via a HDMI cable. This displays information such as weather-forecast, date, time, calendar, complements to the user, news-feed and email notifications. These information are displayed under the control of the voice input given by the users. This display is visible to the users with the use of Two-way mirror which has both reflective and refractive property. Hence, by placing Two-way mirror in front of the

      display the user can view the information while grooming themselves.

   3. Microphone: The voice input to the mirror is provided by connecting a microphone to the Raspberry Pi module via the USB card. Hence, by connecting microphone to the device the user can input voice commands to display certain specific information and also command the mirror to do tasks like setting up of alarm, reminders and for voice conversations.

   d)

   1. Speaker: A speaker is connected to the Raspberry Pi in order for the Two-way conversation between the mirror and the user by giving the audio output.

   2. Two – Way Mirror

   Fig 2.Working of Two-Way mirror

   Daily use mirrors have a reflection property which reflects the entire light entering the glass and hence allows only users to view their reflection. But, Two-way mirrors have the property of both reflection and refraction where only some part of the light is reflected and the rest is refracted passing through the glass surface and hence allowing the user to view the information display on the monitor.

2. Software components

Raspbian Operating System: Raspbian OS is a Debian- based Operating system which is pre-installed in the raspberry pi component hence, making the users easy to code.

JAVA Script: The coding language used to display information in this paper is JAVA Script. JAVA Script or JS is a programming high level language, just in-time compiled and prototype-based object-orientation.

IV DETAILED DESIGN

Fig 3. Block Diagram of Working of Smart Mirror.

The working of the model is according to the above Block Diagram. The voice input, which is the audio is taken as an input by the system.

The Speech recognition module recognizes the input audio and hence converts it to a computer understandable language using algorithms through knowledge extraction and Natural Language Generation. This recognized Speech is now converted to text Using speech to text converter code. The given text is now analyzed and based on the analysis the control is sent to the respective module which contains the code for the display of requested input.

Audio: This is the voice input given by the users in order for the response from the mirror.

Speech recognition: it is the capability of a machine or program to indicate words and phrases in spoken and convert them into a machine readable form. Rudimentary speech recognition software consists of minimal or limited vocabulary words and phrases. Also these are identified only if they are spoken very clearly. If the software is more sophisticated then it has the ability to accept natural speech. The speech recognition works with the help of algorithms through acoustic and language modelling. The acoustic model shows the relationship between linguistic units of speech and also using audio signals. The language modelling matches sounds with word sequence so as to help us differentiate between words that sound alike.

Knowledge extraction and natural language generation: it is the generation of knowledge from structured (relational databases) and unstructured sources (text, images). The resulting knowledge has to be machine readable as well as machine interpretable form and should represent knowledge in a way that helps in inferencing.

Speech to text conversion: this is the process of converting spoken words into written texts. This can also be called as speech recognition. The terms available here are almost synonymous. It is sometimes used to explain the wider process of extracting meaning from the speech which means the speech understanding. An ADC will convert the analog waves of your voice into digital by sampling the sound. As higher the sampling and precision rates increase, the quality also increases simultaneously. In order to convert speech into on screen text or computer command, a computer has to perform several steps.

Text Analysis: This can be of two methods. Storing the predefined texts and hence comparing the Speech input texts with the stored texts. The text which matches is then taken to the respected modules. The generated text can be analyzed by comparing the texts with the Google words using naïve Bayes Algorithm.

Modules: The analyzed text is then sent to the respective modules for their display. This includes:

Weather Display: Displays the information of weather of the given city along with future weather prediction.

Time, Date Display: Displays time and date of the given city.

Gmail Notification: Notifies with the pop-up message If there are any mails received.

Calendar Display: Displays the calendar with upcoming holidays.

News Feed: Displays the news happening around the city. Music: Plays the music stored inside the module.

Our proposed model can show various capabilities which it can perform:

It can work as an intelligent mirror with an aim that the client can use it as an ordinary mirror. The two way reflection can work as an intelligent as well as a see through mirror. This is connected to a LED screen and gives two important functions; they are: Emulating a mirror as a filling in as a show for ongoing information.

Fig 5. Overall Subsystem Flow-diagram

V.IMPLEMENTATION

The entire Module works on the Voice input through switch case statement. Each of the information displayed on the mirror is divided into separate modules which is activated based on the specific voice input given by users. Each module is given a name, when this module name is given as a voice input the respective module gets activated and executes the code in order to display the respective information that has to be displayed.

Fig 6. Flow chart of the Voice Module.

• Date and Time: When the user gives the Voice input to display Date and Time, this module gets activated and fetches the code of this module. The Time and date format will be pre-written in the code which is accessed by Google according to the region specified (This mirror displays Date and time according to Indian Standard Time). The position to be displayed is also accessed by the code which is Top Left. Once the module is executed the output of date and time will be displayed on the mirror in the Top Left Corner.

• Calendar: When the user gives the Voice input to display Calendar, this module gets activated and fetches the code of this module. This module displays all the upcoming Indian Holidays. This information is fetched from google whose location is given as a link in the code. Once this module gets activated, the link will be accessed and fetches the information accordingly. The position to be displayed is also accessed by the code which is Top Left. Once the module is executed the output of upcoming Indian Holidays will be displayed on the mirror in the Top Left Corner.

• Compliment: This module is to display a compliment message to the user. This message is not through a voice input, but displays just as an additional feature to improvise the mirror usage. The module will contain pre-written compliments as a print statement, when this is accessed these messages will be displayed on the Lower Third position of the mirror and these compliment messages are changed on a timely basis according to the input print statements in the code.

• News feed: When the user gives the Voice input to display News Updates, this module gets activated and fetches the code of this module. This module displays the Headlines of The New York Times. This information is fetched from google whose location is given as a link in the code. This location will contain the News Updates of New York Times. Once this module gets activated, the link will be accessed and fetches the information accordingly and each update changes on a timely basis. The position to be displayed is also accessed by the code which is Bottom bar. Once the module is executed the output of the headlines from New York Times will be displayed on the mirror in the Bottom bar Corner.

• Current Weather: When the user gives the Voice input to display Current Weather, this module gets activated and fetches the code of this module. This module displays the current weather of Mysore. This information is fetched from google whose location is given as a link in the code. This link contains the information of current Weather of Mysore whose id is 1262321. Once this module gets activated, the link will be accessed and fetches the information accordingly. The position to be displayed is also accessed by the code which is Top Right. Once the module is executed the output of Current Weather of Mysore will be displayed on the mirror in the Top Right Corner.

• E-mail notification: Email notification module uses IMAP protocol to access the emails from the respective email ID through the server outlook. Office 365.com. The position to be displayed is also accessed by the code which is Bottom Right. Once the module is executed the output of email notifications will be displayed on the mirror in the Bottom Right Corner.

  VI CONCLUSION AND FUTURE WORK

  This paper presented a smart mirror system that has integrated several impressive features using the Voice recognition. A reliable and easy to use design was implemented.

  Smart mirror are known for its best user experience to fetch information and to have an interactive conversation with the mirror along with self-grooming. They not only are known for exchange of information but also can be integrated to a bigger system providing users better experience which includes home automation, intrusion detection, medical usage and much more. This paper majorly concentrates on providing user a better experience by Two-way interaction with voice control and to fetch user-specific information.

  To alleviate this problem using the above techniques display of News, date, time, calendar, music, weather and email notifications on the two-way mirror with voice control is achieved.

  As for future work, the system can be extended as commercial product. Using the Artificial Intelligence this can be used to control Home Appliances and lighting. For the security purpose Face detection can be used.

  ACKNOWLEDGEMENT

  We express our sincere thankfulness to our project guide Mrs. Nalina HD for our successful guidance to our project. We thank our guide for constant support, encouragement throughout the period of work. We also thank our Head of Department (ECE) Mr. Rajendra R Patil providing us all the necessary facilities and constant motivation.

• Weather Forecast: When the user gives the Voice input to display Weather Forecast, this module gets activated and fetches the code of this module. This module displays all the Weather Forecast. This information is fetched from google whose location is given as a link in the code. This link contains the information of Weather of Mysore whose id is 5128581. Once this module gets activated, the link will be accessed and fetches the information accordingly. The position to be displayed is also accessed by the code which is Top right. Once the module is executed the output of Weather Forecast will be displayed on the mirror in the Top right Corner.

REFERENCES

1. Varsha Singh, 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (Com-IT-Con), India, 14th – 16th Feb 2019.

2. Raju Nadaf, 2019 Third International Conference.

3. Kun Jin, 2018, 2nd IEEE Advanced Information

   Management, Communicates, Electronic and Automation Control Conference (IMCEC).

4. Ayushman Johri, 2018, 4th International Conference on Computing Communication and Automation (ICCCA).

5. Adokiya Charles Njaka, 2018, Department of Computer Science Prairie View A&M University Prairie View, Texas 77446, USA.

6. R Akshaya, 2018 International Conference on Emerging Trends and Innovations in Engineering and Technological Research (ICETIETR)

7. Sun Yong, 2018 International Conference on Intelligent Transportation, Big Data & Smart City.

8. Misaki Tani, 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE 2018).

9. Mohamed Yusri, 2018, Universiti Tun Hussein Onn Malaysia, Johor, Malaysia.

10. Mohammed Ghazal, 2017 5th International Conference on Future Internet of Things and Cloud Workshops.

11. Fatima Ok, 2017, Bilecik Ueyh Edebali University, Bilecik.

12. Paul Hagelin 2016, University of California, Davis.

13. Derrick Gold, 2016, MacEwan University, Alberta.

14. Oihane Gohmez, Diego Casado-Mansilla DeustoTech, University of Deusto Avda Universidades, 24, 48007 Bilbao.

15. V E Pawar, Bharati Vidyapeeth College Of Engineering, Sector-7, C.B.D, Belpada, Navi Mumbai-400614, India.