Augmented Reality & Deep Learning Application to Design Vastu Compilant House

Augmented Reality & Deep Learning Application to Design Vastu Compilant House

Freny Jariwala

Computer Enginnering Sarvajanik College of Engineering

and Technology Surat, India

Harsh Jariwala

Computer Enginnering Sarvajanik College of Engineering

and Technology Surat, India

Prof. (Dr.) Dipali Kasat

Computer Enginnering Sarvajanik College of Engineering

and Technology Surat, India

Abstract Vastu Shastra is a traditional Indian system of architecture originating in India. The principles of Vastu Shastra can be successfully tuned, loosened up and modernized to address the issues of man because the key needs of man never remain same. Since, Indian individuals are profoundly intrigued by Vastu so keeping unblemished with Indian Culture we have chosen to give them a cordial application just as give them an Exclusive Exposure to Vastu so everyone can discover the exit plan without anyone else. One may receive a blend of both Vastu Shastra and current structure science. A few requirements like thickness of settlement, accessibility of framework, accessibility of plot, rules and guidelines of specialists, cost of plot, development, individual preferences and convictions influence an ultimate conclusion and afterward the proposals of Vastu Shastra can be fused with present day building science and may improve personal satisfaction. So bringing this idea into light and utilizing the cutting edge innovation for example Augmented Reality, Image Processing, and Deep Learning. Following all Vastu Shastra suggestions may have positive mental impact on the proprietors and clients.

Keywords Room, Augmented Reality, Compass, Direction, Vastu, Household Amenities, Ancient Architecture, Home, Construction.

1. INTRODUCTION

   Vastu Shastra is a traditional Indian system of architecture originating in India. Texts from the Indian subcontinent describe principles of design, layout, measurements, ground preparation, space arrangement, and spatial geometry. Vastu Shastras incorporate traditional Hindu and Buddhist beliefs. Vastu is architecture and much more. Many factors govern the life of a human being like his fate, Karma and surroundings. But vastu can make sweet things sweeter and bitter less bitter. Thus there is a great need for the architects and Vastu Engineers to coordinate; since an architect can build a posh house but cant assure happy life to the people living in that house, whereas Vastu -science assures peace, prosperity and progress to the owner as also the inmates. According to a study conducted across eight major cities, nearly 93 % of home buyers seek vastu compliant homes."[1]

   Four out of every five prospective home buyers in India would like to know the vastu (ancient science of architecture as per Hindu tradition) of the property, even before physically checking the property", the study said. According to the study, south has the highest vastu compliancy requests followed by north, east and west." Nearly 87 % of the respondents readily agreed to forego the size of the house for vastu, it said. [1] Vastu experts charge handsome amount for providing vastu suggestions and recommendation and many a times they may cheat as any layman doesnt have vastu knowledge so the solution is to develop one stop vastu application that provides everything related to vastu with great ease so that any layman without any knowledge and get to check vastu of their house that too at low cost.

2. LITERATURE OVERVIEW

   Oliver Clayton, David Morris, et al. (2019) [2] developed a system for accurately positioning AR content within a coordinate system, it may include AR content tethered to trackable physical features. As the system is used by mobile computing devices, each mobile device may calculate and compare relative positioning data between the trackable features.

   Mckinnon, David, Wnuk, Kamil, et al. (2019) [3] published embodiments of the inventive subject matter can obtain an initial map of an area, derive views of interest, obtain AR content objects associated with the views of interest, establish experience clusters and generate a tile map. A user device could be configured to obtain and instantiate at least some of the AR content objects based on at least one of a location and a recognition.

   Sunil Jagadish (2010) [4] designed an astrological profile of the user is determined electronically in a computer system based on the plurality of inputs. Further, contents are selected based on the astrological profile. The contents are then provided to the user.

   Zhao Chenjunl (2016) [5] developed the interaction method and the interaction device provided by the invention, a user can position operable controlled equipment intuitively by marking the controlled equipment on the real-time scene image to realize obtaining and controlling for the user when in seeing, so that the operation difficulty of the user is obviously reduced and the user experience is improved. The implementation mode of the invention also provides an interaction device of the intelligent household equipment.

3. METHODS AND TECHNOLOGY

   Vastu encapsulates immense extension and it can't be learned inside specified time it requires long time alongside appropriate confirmed information and generally individuals with vastu information accuse us of attractive measure of cash and furthermore at times even individual might be false in this manner to determine such issues identified with vastu we have planned Doctor Vastu application which would give whole presentation of vastu on only a single tick .There are different usefulness given to client, for example, : 2D vastu (For client with less framework configuration),Plot selection(Selecting plot to check whether its as indicated by vastu or not),Floor plan Analysis(Examine whether the arrangement is in agreement to vastu or not),Room Scanning(To examine the room and recognize existing items and afterward show them in understanding to vastu),Vastu Expert(To settle for questions identified with essential vastu).Thus, by actualizing such usefulness any layman can undoubtedly know about vastu without paying attractive add up to anyone.

   Fig. 1. Flow Diagram of Application

   1. COMPASS MODULE

      This module is the heart of our entire project as Vastu is completely dependent upon the 8 major direction i.e. East , West , North , South , North-East , South-East , South-West , North-West .Thus , the major task is to identify the accurate direction. As higher the accuracy of compass more will be the accuracy of places amenities within the house.

      Fig. 2. Compass Module

   2. 2D VASTU MODULE

      This Module is specially implemented for the user whose phone doesnt have ARCore supported within it as well as compass sensor is not available. So to make the Application available to all we decided to give 2D exposure of Vastu in the following way. Here , the user simply need to drag and drop the items from above scrollbar into any of the specified compass direction and click on the check button if the dropped item would in the favorable position in accordance to vastu then it would turn green if not then red.

      Fig. 3. 2D Vastu Module

   3. LAND/PLOT SELECTION AND EVALUATION MODULE

      The user is expected to select plot which he/she wants to validate and further our shape detection algorithm identifies the shape of the plot and declares result of whether the plot is according to vastu or not, if its not according to vastu certain measures would be displayed to user and if its according to vastu the user will be redirected to vastu bot developed usig AWS Sumerian, AWS Lex, AWS Lambda, AWS DynamoDB, AWS Polly which would ask couple of questions related to plot environment and user is expected to answer accordingly once done it will set up a point by point examination report which unmistakably gives you whether to purchase the plot or not

      .Also it gives the investigation of what all elements in your

      encompassing influences the plot and give you certain proposal and finally it gives the general rate identified with plot rightness as indicated by vastu.

      Fig. 4. Land/Plot Selection and Evaluation Module.

   4. ROOM SCANNING MODULE

      The examination of your room is done dependent on the video you provided. Basically the working follows Object detection API which detects the available objects within the room and provides vastu suggestion on where to place those objects. We use transfer learning to re-train model to detect multiple types of household amities, despite there only being one household amities in the original training data. Here we used COCO MoblieNet V2 pre-trained model with training at 50000 steps by 660 Images (800*600 pixels) of Households amities in 18 Class. The detected objects are suggested to user out of which user can simply view them as 3D objects in AR view. The 3D objects we used are low poly 3D model.

      Fig. 5. Room Scanning Module

   5. FLOOR PLAN ANALYSIS MODULE

      The user is expected to select floor plan from his/her device and then the entire floor plan would be sent to server for processing ,this processing is mainly done using various Image

      Processing techniques like image segmentation and also Text detection API is used for detection of text within floor plan, once the text is detected user needs to specify the initial location of the segment and afterward solicit client bearing from that segment of the house so rest of the segments could be masterminded as needs be and afterward calculation checks whether each area of house is in agreement to vastu or not and if not it gives a portion of the cures with respect to it followed by certain proposals.

      Fig. 6. Floor Plan Analysis Module

   6. VASTU EXPERT BOT MODULE

      In this module to make our application more easy to understand we have incorporated a chat bot framework named as vastu expert where client can essentially ask different questions related to vastu and the bot would react dependent on its whole information the bot is fit for giving reactions dependent on fundamental vastu. It identified with home, building as well as individual's way of life.

      Fig. 7. Vastu Expert Bot Module

4. RESULT

   Land/Plot Selection and Evaluation Module

   End user is provided entire analysis of his/her plot through E-mail. The document would contain the entire analysis based on the queries answered by user asked to them by our vastu expert and at the end also provides the estimation of how much the selected plot is in accordance to vastu which would be depicted through pie chart attached within the analysis document itself.

   Fig. 8. Analysis Report of Land/Plot Selection And Evaluation Module

   Fig. 9. Pie Chart of Analysis of Land/Plot Selection And Evaluation Module

   The correctness of plot is calculated as the ratio percentage of queries answered which are in synchronization with vastu to total no of queries asked by vastu bot. Thus, the remaining percentage give us the incorrectness of plot.

   Room Scanning Module

   I. TABLE

   No	Pertained Pipeline Name	No of Images for Training	No of Images for Testing	Steps of Training (Epoch)	Accuracy In (mPA)
   1.	SSD Mobilenet v2 coco_2018_ 03_29	174	46	10000	0.546
   2.	SSD Mobilenet v2 coco_2018_ 03_29	530	133	50000	0.632

   Floor Plan Analysis Module

   End user is provided entire analysis of his/her floorplan through E-mail. The document would contain the entire analysis and also provides user with the remedies if certain section is not in accordance to vastu also provides suggestions to end user and at the end also provides the estimation of how much is the floorplan in accordance to vastu which would be depicted through pie chart attached within the analysis document itself.

   Fig. 10. Analysis Report of Floor Plan Module

   Fig. 11. Pie Chart of Analysis of Floor Plan Module

   The correctness of floorplan is calculated as the ratio percentage of various section of floorplan which are in synchronization with vastu to total no of section in floorplan. Thus, the remaining percentage give us the incorrectness of plot.

5. CONCLUSION

Doctor Vastu application based on augmented reality, deep learning and image processing focuses to provide exclusive exposure of vastu to end user which allows user to access various functionalities like 3D placement of household objects according to vastu with great ease, Checking floor plan, Evaluating plot before buying. Thus the application provides various functionalities to end user one just few clicks and with great ease.

For future improvement, more focus would be on getting high accuracy in object detection and also to increase the scope to every vastu affected sector and further to make this application region specific as vastu affects according to region. For example placement of kitchen in south would be favourable in India but It may not be favourable in America. Thus, Vastu depends on region. Thereby, the flexibility of the application is very important.

The concept of this work indicates its potential for future development as it incorporates emerging technologies with the ancient architecture to provide end user with the great satisfaction of designing the modern house which is vastu compliant too.

REFERENCES

1. https://economictimes.indiatimes.com/wealth/personal-finance- news/over-90-buyers-prefer-vastu-compliant-homes- survey/articleshow/50077412.cms

2. Oliver Clayton, David Morris, et al. Accurate positioning of augmented reality content June 2019 , 15/592047.

3. Mckinnon, David, Wnuk, Kamil, et al. Wide Area Augmented Reality Location-based services February 2019 , 16/168419.

4. Sunil Jagadish Astrology based content selection May 2010 , US20100125503A1.

5. Zhao Chenjunlin Interaction method and interaction device of intelligent household equipment, July 2016 , CN105760106A.

6. https://en.wikipedia.org/wiki/Vastu_shastra

7. https://www.dwarkadheeshvastu.com/Free-Vastu-Book-Download-

   PDF.aspx

8. https://www.subhavaastu.com/scientific-analysis.html

9. https://www.academia.edu/36111609/Vastu_English

10. https://developers.google.com/ar/discover

11. https://en.wikipedia.org/wiki/Augmented_reality

APPENDIX

Shape detection: It is feature to detect the shape of plot using original plot/land image convert into HSV its thresholding to get rid of as much background as possible and extract shape from image according to point of shapes.

AWS Sumerian: Amazon Sumerian makes it easy to create engaging 3D front-end experiences and is integrated with AWS services to provide easy access to machine learning, chat bots, code execution and more. As a web-based platform, your immersive experiences are accessible via a simple browser URL and are able to run on popular hardware for AR/VR.

AWS Lex: Amazon Lex is a service for building conversational interfaces into any application using voice and text. Amazon Lex provides the advanced deep learning functionalities of automatic speech recognition for converting speech to text, and natural language understanding to recognize the intent of the text, to enable you to build pplications with highly engaging user experiences and lifelike conversational interactions. With Amazon Lex, the same deep learning technologies that power Amazon Alexa are now available to any developer, enabling you to quickly and easily build sophisticated, natural language, conversational bots (chat bots).

AWS Lambda: AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

AWS DynamoDB: Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It's a fully managed, multi region, multi master, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications.

AWS Polly: Amazon Polly is a service that turns text into lifelike speech, allowing you to create applications that talk, and build entirely new categories of speech-enabled products. Amazon Polly is a Text-to-Speech service that uses advanced deep learning technologies to synthesize speech that sounds like a human voice.

SMTP: SMTP is an application layer protocol. The client who wants to send the mail opens a TCP connection to the SMTP server and then sends the mail across the connection.

The SMTP server is always on listening mode. As soon as it listens for a TCP connection from any client, the SMTP process initiates a connection on that port. After successfully establishing the TCP connection the client process sends the mail instantly.

Tensorflow Based Object detection: Given a video stream, an object detection model can identify which of a known set of objects might be present and provide information about their positions within the image/video frame.

Customize model with Transfer Learning: We can use a technique known as transfer learning to re-train a model to recognize classes not in the original set. For example, you could re-train the model to detect multiple types of household amities, despite there only being one household amities in the

original training data. To do this, you will need a set of training images for each of the new labels you wish to train.

ARCore: Scene Viewer is an immersive viewer that enables 3D and AR experiences from your website or Android app. It lets users of Android mobile devices easily preview, place, view, and interact with web-hosted 3D Low Polly models in their environment.

Python-Tesseract: It is an optical character recognition tool for python. That is, it will recognize and read the text embedded in images. It is a wrapper for Googles Tesseract- OCR Engine. It is also useful as a stand-alone invocation script to tesseract, as it can read all image types supported by the Pillow and Leptonica imaging libraries, including jpeg, png, gif, bmp, tiff, and others.