A Modular Smart Mirror Google Assistant based

A Modular Smart Mirror Google Assistant based

Smart Google Assistant based Mirror

Dr. Ajitkumar Khanchane

CSE

VIT, Vadala

Mr. Ammar Fitwalla

CSE

VIT, Vadala

Abstract Efficiency and productivity are two qualities that are increasingly establishing their dominance as keywords companies are using to market their products. The fact that their product can multitask or increase productivity better than the competition has become a real selling point. This is due to the fact that effective time management is an essential factor in increasing production of day-to-day life. The best time management strategies involve being able to find time where there was no time before. Integration of technology into peoples daily lives has made that time management possible. The use of products such as tablets, PCs, and smartphones have given people access to the tools needed to be productive. However, though successful technological products have been used to increase productivity, it has done its fair share to stifle it as well. The use of technology has become another task on everyones daily to- do list. Technology should mold to our schedule, not the other way around. That is where the smart mirror idea originated. The smart mirror idea aimed to integrate technology seamlessly into peoples lives by putting it where everyones routine eventually collides, the bathroom. The goal of the smart mirror is to increase a users productivity by saving their time.

Keywords (1) Google assistant (2) smart mirror (3) IoT (4) Raspberry Pi (6) Virtual system (7) glass

1. INTRODUCTION

   A smart mirror displays applications so that you can check the weather, local news, etc. while you are getting ready in the morning. The mirror will do the thinking for the user in a smart way. This smart mirror aims to reduce and possibly eliminate the need for the user to make time in their daily morning or nightly routine to check their PC, tablet, or smartphone for the information they need. The making of this smart mirror includes a micro-controller called Raspberry Pi which will act as a brain of the interactive system. The micro- controller will be powered using python scripts for mirror software as well as personal assistant. For making of this personal assistant an Amazon Web Services (AWS) account is required. Using the AWS, the Alexa function and Alexa skills using skill kit will be created. Once the function is setup, the function can be accessed using a command line after the software is installed. An 18 LED Monitor will be placed below an acrylic two-way mirror sheet and the model will be placed in a wooden frame. Well build a separate personal assistance that will be able to recognize and perform speech to text operations. The personal assistance is named ALEXA. There is a huge scope for this project and AI is much needed today.

   Intelligent mirrors, which continue the works today and will take its place in the future technology, provide both mirror and computer aided information services to its users. Thanks to the microcontroller cards on board, these systems, which can connect to the internet and take data from the

   internet, can show this information on the places located on the mirror. In the scope of the study, the developed intelligent mirror system includes the weather information, time and location information, current event information, user information, and camera image taken from web services using Raspberry Pi 3 microcontroller card. Some equipment can be controlled by voice commands via the microphone on the smart mirror. With the introduction of system on chip (SOC), such as the Raspberry Pi, the notion of creating smart devices is a relatively new craze that has taken over hobbyist communities. One currently popular project using a SOC is the development of smart mirrors [1][3]. A smart mirror is a mirror with smart capabilities much like how cell phones have become smart. That is, it is a display that looks and acts like a mirror, but has the capability of displaying multimedia data through the mirror glass as if the mirror was a screen on its own accord. The major appeal of a smart mirror is that its physical design embeds a computational device in an ordinary piece of furniture that can integrate seamlessly into a home or working environment. A common approach to building a smart mirror is to use a pane of two-way glass, a monitor, a frame to hold the glass and monitor, and a web browser with JavaScript to provide the software features and drive the display. The main limitation of this setup is related to the use of a browser as the displays method of information presentation. A browser creates a sandbox for the code that runs within it, that is, all interactions and processes are isolated from other running processes and hardware interactions on the computer. Furthermore, web applications are typically driven through user events generated on a web page (e.g., mouse clicks). This feature poses limitations in smart mirror applications. First, user events cannot be generated naturally in a browser when one interacts with the browser as one would with a mirror. Second, a sandbox limits the use of external hardware to generate events based on typical user-mirror interaction. Third, only JavaScript runs natively in a browser.

   Consequently, such smart mirror platforms are typically limited in the following ways. First, they are not truly modular. Plugin systems exist, but require JavaScript knowledge to en-able, disable, or configure plugins. Second, they use server-side solutions geared for web sites and RESTful (representational state transfer) API (application programming interface). The limitations of a RESTful API are inherent by the fact that users typically have no way of generating events to obtain data or to specify where on the server to obtain the requested data via natural mirror interactions. Third, the platforms are not inclusive for all programmers and programming methodologies. Only JavaScript is supported, which is geared for event-driven programming. No solutions exist for supporting other programming languages with their vast libraries of features

   and user base, hence, fragmenting the potential pool of developers for extending smart mirror features.That being said, a web browser is still a necessary feature for providing and displaying information, as it has built-in support for multiple media formats, such as text, images, and videos. The information presentation can be made interactive with JavaScript and is customizable with CSS (Cascading Style Sheets). Furthermore, hyperlinking and web connectivity allows for borrowing and sharing of resources. We designed and developed Smart Reflecta smart mirror platform that offers three main benefits [5]. First, it is modular and extensible. Developers can add plugins to customize their smart mirror applications. Second, it utilizes a server design that allows one to sidestep a sandbox created by a web browser. In our prototype, we demonstrated this feature by enabling users to interact with a smart mirror through an external hardware interface. Third, it allows for plugins to be created in all programming languages. With these problems addressed, an extensible platform is attainable, allowing for growth in smart mirror application development.

2. LITERATURESURVEY

   Paramet ers	Paper 1: Fatma Ok	Paper 2: Murat Can	Paper 3: HakanUcgun	Paper 4: UgurYugec
   Technol ogy	Raspberr y Pi 3	Raspberr y Pi 2	Pineappl e Pi	Raspberr y Pi 3
   Languag e	Python	Java	Python	GoLang
   Display applications	Weather , news, date, time, temperature	Weather , news, in- built applications	Weather , news, things to do reminder, temperature	Weather , news, in- built alarm, date, time
   Personal assistant	Yes	Yes	No	No

   Smart mirrors usually allow users some customization. To provide a personalized service, a smart mirror needs to identify the user who is standing in front of it, so that it can access and display the users personal information, such as his/her schedule, to-do list, and appointments. Automatic methods for recognizing users include face recognition tag- based identification, biometric data, and personal belongings (e.g., toothbrush) [8]. To select the most appropriate method, one should consider the location of a smart mirror and its intended uses. For example, the use of a camera (for face recognition) may not be suitable for a smart mirror installed in a washroom due to privacy reasons.

   Compared to existing systems, SmartReflect is designed to be simple, lightweight, and extensible. It does not require a lot of computational resources and can run on a Raspberry Pi. Our platform allows users to interact with a smart mirror through two interfaces. First, users can use a touchscreen mounted on top of a Raspberry Pi to write notes or control the smart mirror through a menu on the screen (see Fig. 1). Second, users can also use their mobile devices to access a web interface to configure the available plugins.

3. SYSTEMDESIGN

   The system is designed to offer all the required services using various components. An Acrylic two mirror is the most important component used which provides a reflection layer on front surface and a transparent layer on the back surface. Beneath the mirror a standard LCD monitor is employed, which is the displaying component of our mirror. A System on Chip (SoC) microcontroller, Raspberry-Pi 3b is used that has computational capability and ability to control other devices with the help of a relay device. An USB microphone is connected to RPI board to receive input audio from the user. A speaker is utilized toplay audio feedback of Alexa assistant. Power module is used to supply AC power for the operation of smart mirror. On receiving voice command from the user, the system fetches the corresponding information from the cloud or executes any requested services like locking the door, turning on lights, showing CCTV footage of door and controlling fans etc. Systems interface is designed in

   user to stand in front it. Once the user comes in front of mirror, it detects the authorized user and loads his personalized modules. The voice recognition system is now ready for user queries or commands. Modular fashion and it comprises of many modules like date, time, weather and many more. User may also get multimedia playback facility like playing music and videos.

   Raspberry Pi3:

   The Raspberry Pi is a small, credit card- sized computer that powers the whole thing, displaying the Magic Mirror interface and running Alexa, the voice-control system. The Raspberry Pi 3 is a single board computer which features a quad core ARM cortex A-5 processor, with the capability to run Linux based operating systems. The Raspberry Pi 3 has an integrated LAN port which can be used to connect to the internet. It also comes with double row GPIO (General Purpose Input Output) pins which can be used for interfacing it with a variety of I/O devices. We will be installing Debian-

   based Raspbian OS Jesse on the Raspberry Pi and download and install node.js for the packages.

   The two-way mirror:

   The two-way mirror is made of acrylic and sits flush over the monitor, allowing the graphics on the monitor to come through while maintaining a mirror effect. I ordered my mirror through Amogh Aluminium.The thicker mirror option (3/16") to prevent a "funhouse mirror" effect.

   The Monitor Display:

   An LED monitor for minimal power consumption, maximum crispness, and to prevent mirror glow at night. The required information for the user will be displayed on the LED monitor.

   The box/frame:

   The box/frame houses all the components, including the microphone, and looks pretty.

   Speakers for Output:

   A 3.5mm jack speaker will be required for speech output of the personal assistant.

4. PROPOSEDSYSTEM

   The mirror will do the thinking for the user with the help of an in-built personal assistant. It will update with the users calendar schedule, to-do lists, news, and weather etc. The information wouldnt be thrown in the users face, but unobtrusively displayed on the edges of the mirror to still allow use of the actual mirror. The use of personal assistant will keep things simple and easy to use. No keyboards to try to keep dry and clean. The personal assistant will also allow the user to still use the mirror despite their hands are wet or dirty. The mirror provides common information most people check their smartphones or tablets for, such as weather, news, Twitter and schedules. This allows the users to read, think, and plan their day while getting ready in the morning or night.

   Finally, the mirror must be smart enough to protect itself from the wet and humid conditions. It will feature a humidity protection system where itwill monitor the temperature and humidity levels near the hardware.

   Fig 2: Architecture design of Smart mirror

   The project is designed to develop INTERNET CONNECTED MIRROR /SMART MIRROR, A smart mirror is a device that works like a mirror with additional ability to display multimedia data, such as text, images and videos. This device allows users to access and interact with contextual information, such as weather forecast, date, Time, seamlessly

   as part of our daily routine & local news feed while you are getting ready in the morning, & upcoming events in calendar and a complimentary message. It's also modular, so you can easy move it around or hang it on the wall if you like. It is a one-way mirror with an LCD screen behind it. However, getting the screen blacked out and set to provide the best possible effect is the trick which will give you a Mirror finish. The mirror consists of a wooden frame and a one-way monitor which is sandwiches to the back of the frame. It can display user data such as calendar, and to do list based on Facial Recognition. With the help of HD webcam smart mirror will recognize your face & give you a complimentary message. The Facial-Recognition is based on local binary pattern histogram. We have also implemented Fingerprint as an alternative option for security purpose

5. RESULTSANALYSIS

   This project (smart Mirror) has now been flourish inside the vicinity of a time where every day, we see more and more interconnected or connected devices. The Internet has revolutionized our world by linking us and making us more agile to the information and other people in this working world. Since then evaluation of Mobile phones which later became smartphones and since then this theory has exploded and changed smoothly into what we called as Internet of Things (IoT), there are various things that can be connected with every object, more or less every purpose can be made smarter, for example, Smart Mirror which provides large surface for displaying information and we can interact with it. But what we wanted to accomplish was something which would be easy for a person to interact with it. My objective was to learn about raspberry pie3 and to use it how to merge hardware and software to build a new projects and thus with the help of raspberry pie and some more physically connected devices we were able to establis the smart Mirror, we had spent long hours to calibrate the sensors to Interface it with software[6]. Then we started designing the wooden frame which will give the project a spectacular look. It has a facial reorganization in collaboration with Artificial Intelligence, which will help you to recognize your face and give you an appreciative message. It is designed especially for people who spend much time in front of Mirror without wasting any time they can do the productive work such as having a glance on emails and read the news by using our smart Mirror.

6. CONCLUSION

   The principal focus of this project is that it is a very new kind of smart device that is not available in the worldwide market and we dont see it each day at it looks very much eye- catching This Internet-connected Mirror does the work way beyond our imagination; it helps parents to set a reminder for their children like what they need to do and their extra activities with the help of notifications send on Mirror.With enhancement Internet of things (IoT) and with the help of millions of devices connected to the Internet every day, this product is going to be an optimistic technology for the future.

7. REFERENCES:

1. E. Cohen, Smart mirror, http://smart-mirror.io/, accessed: 2016- 07-14.

2. H. Mittelstaedt, HomeMirror, https://github.com/HannahMitt/HomeMirror, accessed: 2016-07-14.

3. M.Teeuw, Xonay labs, http://michaelteeuw.nl/tagged/magicmirror, accessed: 2016-07-14.

4. H. Sukeda, Y. Horry, Y. Maruyama, and T. Hoshino, Information- accessing furniture to make our everyday lives more comfortable, IEEE Transactions on Consumer Electronics, vol. 52, no. 1, pp. 173 178, 2006.

5. D. Gold and D. Sollinger, Smart reflect server, https://github.com/ DerrickGold/SmartReflectServer, accessed: 2016-07-30.

6. L. Ceccaroni and X. Verdaguer, Magical mirror: Multimedia, interac-tive services in home automation, in Proceedings of the Workshop on Environments for Personalized Information Access, 2004, pp. 1021.

7. J. R. Ding, C. L. Huang, J. K. Lin, J. F. Yang, and C. H. Wu, Interactive multimedia mirror system design, IEEE Transactions on Consumer Electronics, vol. 54, no. 3, pp. 972980, 2008.

8. K. Fujinami, F. Kawsar, and T. Nakajima, AwareMirror: A personalized display using a mirror, in Proceedings of the Third International Conference on Pervasive Computing, 2005, pp. 315 332.

9. M. A. Hossain, P. K. Atrey, and A. E. Saddik, Smart mirror for ambient home environment, in Proceedings of the Third IET International Conference on Intelligent Environments, 2007, pp. 589596.

10. T. Lashina, Intelligent bathroom, in Proceedings of the Workshop on Ambient Intelligent Technologies for Wellbeing at Home, 2004.

11. C. Sethukkarasi, V. S. HariKrishnan, and R. Pitchiah, Design and development of interactive mirror for aware home, in Proceedings of the First International Conference on Smart Systems, Devices and Technologies, 2012, pp. 18.

12. Y. C. Yu, S. D. You, and D. R. Tsai, Magic mirror table for social- emotion alleviation in the smart home, IEEE Transactions on Consumer Electronics, vol. 58, no. 1, pp. 126131, 2012.

13. Internet Engineering Task Force (IETF), The WebSocketProtocol, https://tools.ietf.org/html/rfc6455, accessed: 2016-07-16.

14. R. W. Scheifler and J. Gettys, The x window system, ACM Trans. Graph., vol. 5, no. 2, pp. 79109, 1986.

15. D. Saakes, H.-S. Yeo, S.-T. Noh, G. Han, and W. Woo, Mirror mirror: An on-body t-shirt design system, in Proceedings of the Conference on