Furnished: An Augmented Reality based Approach Towards Furniture Shopping

Furnished: An Augmented Reality based Approach Towards Furniture Shopping

Syamantak N. Dhavle

Student

Department of Computer Engineering Atharva College of Engineering Mumbai, India

Chaudhary Mohammed Qais

Student

Department of Computer Engineering Atharva College of Engineering Mumbai, India

Prof. Bhavna Arora

Assistant Professor Department of Computer Engineering

Atharva College of Engineering Mumbai, India

Khan Mohd Saif Tabarakallah

Student

Department of Computer Engineering Atharva College of Engineering Mumbai,India

Abstract The world as we know it today, is run on technology. If servers of tech giants like google, facebook etc stop working for a few minutes, the world encounters huge losses, usually in millions. There are multiple types of technology used to run any organization, associated with its hardware and software. Yet in the world dominated by technology, we see that Augmented Reality has not been used widely in business applications. AR has been used by several organizations to create revenue from the field of entertainment, for example Snapchat and Pokemon Go, but rarely used in any business sectors like e-commerce. Furnished application is created on the basis of the fact that Augmented Reality technology is grossly underutilized in the e- commerce industry. We hope to create a more comfortable environment of AR technology platforms for the everyday user for them to inculcate the use of such technologies as a daily utility, especially for traditional users.

Keywords:- Augmented Reality, AR app, AR Furniture, ArCore, Android App

1. INTRODUCTION

   The ecommerce industry in India is growing exponentially. There is no growth without evolution, thus the ecommerce industry needs to constantly add new increments of technology to itself if it plans to expand. One such increment would be the use of Augmented Reality to appeal to its customers and new users alike. For the sake of the Furnished application, the sector of the ecommerce industry chosen by us is the Furniture industry.

   There are already a couple of big players in the furniture industry yet there would be a lot of potential for anyone who can bring something unique to the table. The Furnished app aims to bring all the products that are displayed in their online furniture store to its customers in a way that no other stores have pulled off successfully before, through the use of Augmented reality.

   Augmented reality and Virtual Reality seem to be two sides of the same coin. Augmented Reality allows you to put objects in your own environment, whereas Virtual Reality puts you in the environment created by the developer. Although AR is somewhat used in industries like design and modeling, VR has been known to be an efficient training

   technology for industry workers, especially for the ones working in factories with harmful environments.

   Following are the major contributions of the paper:

   1. Proposed an e-commerce application with augmented reality support.

   2. Using Sceneform SDK consisting of a PBR (physically based renderer) to aid in the process of displaying products in the real environment.

   3. Using the GLB file format to render models at runtime with greater efficiency.

2. BACKGROUND

   In this section you can understand the details of the software requirements, concepts used and a brief explanation about the overall result. [1]

   1. Android SDK

      Android SDK (Software Development Kit) offers all the components required to build applications on the Android Platform.

      The major components of the Android SDK include :

      1. SDK Build tools : These consist of all the tools which are required to build the components of the application separately.

      2. Android Emulator : Used for testing the android application in the development environment itself, if a physical device is not present.

      3. Platform Tools: These are the tools which provide the support for running the application on the current android api.

      4. SDK Platform : Target API level (Android level) for running the application.

      5. Google APIs – Provides simplicity of building applications, by providing apis for different interfaces.

   2. Sceneform SDK [3]: Building AR applications in android traditionally requires a person to learn OpenGL, which is an API for developing 2D & 3D vector graphics. However, Sceneform SDK allows for the user to build dynamic AR applications without learning about OpenGL, and makes the process inherently seamless. Sceneform SDK consists of 3 major components, An high-level scene graph API, a Physically Based Renderer provided to it by filament & an Android Studio plugin which embeds the sdk to Android Studio for the said development process.

   3. Scene graph: [4] A scene graph is essentially a data structure used in vector graphics applications, which carries the data of the scene into it in the form of nodes. In 3D graphics, nodes can be used to define the relationship between digital objects in the virtual environment. In Furnished application, transformable nodes will be used to anchor the 3D objects into the scene, along with scaling and transformation ability.

   4. Physically Based Rendering : [5] It is a type of rendering technique which makes all of the 2D/3D models in the view accurate with respect to the lighting conditions. Thus PBR allows to display the objects in the environment by enhancing the textures, shadows, reflections etc. Different types of surfaces like metals, wooden surfaces also appear photorealistic.

   5. ARCore : ARCore enables AR functionality in android devices without the use of any external sensors. Previously Tango was used to provide AR support to android devices, which is enhanced by ARCore today. ARCore works with sceneform to input 3D assets into the application and build models at runtime which is displayed into the environment with the help of the on-device cameras. There are three basic concepts used in ARCore [2] :

   1. Motion Tracking: The models displayed in AR are first generated in a virtual environment and placed into the real environment by using the device cameras. Motion tracking estimates where the location of the device relative to the world in real time, for the models to be anchored accurately within the world, relative to the user.

   2. Environmental Understanding: To place the models in any environment, ARCore requires a plane to place the object on. It finds a suitable plane by finding a cluster of points which appear to be at the same level horizontally and provides that plane surface to place the object on.

   3. Light Estimation: ARCore has the ability to estimate the light information in the given environment, i.e the amount of light present or the lighting conditions in the scene where the model is to be displayed.

   This allows the model to be displayed with the proper lighting, which makes the scene more realistic. If light estimation is not carried out, the model will look out of place in the given environment.

3. SYSTEM DESIGN

   Fig. 1 System Architecture

   1. Android Device & Camera

      The mobile device used to run the application should have a minimum Android level Android 7.0 (Nougat) with an API level 24 in order for the application to work properly. The device should have a good quality camera & the associated sensors to get the best result.

   2. User Interface

      The user interface designed fr this application uses a modern e-commerce layout. Inter font from google fonts is used across all the pages – Login, Home, Product details etc. The UI is interactive enough for customers to easily navigate through the conversion funnel.

   3. Database

      Fig. 2 Home page of the application

      Fig. 3 : Block Diagram to display AR model.

      1. Icon Press : An UI icon button has been provided on the product details page. On clicking that icon, the user is able to open the AR Fragment page.

         The data concerning the users, the transactions and the products needs to be stored in a consistent database, thus in the case of Furnished application, Firebase database is chosen. Firebase is easily compatible with mobile applications and is also free for use if under some standard limits of data, thus is also perfect for testing the various segments of the application.

   4. Sceneform

      Consists of the several sceneform dependencies added to the project, namely sceneform core, plugin, ux & assets. This works with Googles ARCore and builds a model at runtime instead of creating a separate asset file (.sfb) for each model.

   5. ARCore

   To display the model through arcore, several steps are followed. The brief understanding of all the steps is given below.

   1. DisplayModel method : A method is called which completes 3 further steps.

      1. Firebase database is initialized.

      2. A child folder in the database consisting of the product 3D models is referenced.

      3. A Temp file is created, for the model file to be downloaded to this file, with a .glb extension.

   2. BuildModel method : [6] After the model file is downloaded from the database & received into the temp file, the 3D model is built at runtime with the buildmodel method.

      1. ModelRenderable within the BuildModel method builds the 3D model with the file provided at runtime.

      2. The model is attached to a node in the scene graph.

      3. The model is then returned to the display model method.

      4. A message is provided to the user which indicates that the model has been built.

   3. On Tap Listener : When a node is tapped in the AR fragment, the model is simply displayed on that node in the fragment. Since Transformable Node is used, it is also possible to scale and move the model through the scene graph horizontally.

4. USER FLOW

   Fig. 4 : Flow of control for the user

   1. Login/Register : To enable a safe and secure shopping experience, the user has to register themselves into the application using their phone number. Upon successful registration, the user can login to the application through the login page using their phone number as the user id and the password they have entered during registration.

   2. Home Page : The user is greeted with a personalized message with their name concatenated with the text Hi on the home page after a successful login. The home page features the best selling product or any product on offer, which is currently the black leather chair. The home also contains a recycler view ui element which automatically gets updated after any new product is added. The categories page allows the user to browse through several categories which can help them to find their required product.

   3. Categories : Currently 6 categories are present in the categories page, but changes can be made as required. The user can comfortably browse through each of those categories, each of which when pressed opens its own recycler view fetching products from the database with the unique category name.

   4. Select Product : When a product is selected from either the recycler view on the home page or from any of the categories, an activity containing the product details is opened, called the product details activity. The activity receives the unique product id from the previous activity in the form of an intent string extra. The product details class fetches the

      product information, i.e the product name, description, price, image and the link to the ar object located in the firebase storage.

   5. AR View : The procedure to explain the AR object into the view is mentioned above. The AR object is received from the database also by its unique product key, i.e the timestamp of when the product was added into the database. When the AR Fragment is opened, the user receives a model built toast message, and the ARCore plugin finds the horizontal points on which the model can be placed. The points can then be tapped in order to display the model, which is scalable as well as transformable to fit into any particular area of the customers house.

   6. Checkout : After proper inspection of the product, the customer can add the product to their cart. When the add to cart button is pressed, the product is added to a cart list database, along with the information of the user who has added it to their cart. The cart total is displayed at the bottom of the cart page. The user can checkout if they are satisfied with the product(s).

5. IMPLEMENTATION

   The application is implemented and deployed in 3 stages

   explained below:

   1. Data / 3D models collection : The application requires a 3D model in .glb format to be displayed for each product in the catalog. Since a physical store is not currently available for testing purposes of the application, the models required were obtained from cgtrader.com, a repository of 3d models of various formats. Namely models of chairs, table sets, coffee tables etc. were obtained for testing of various categories of furniture.

   2. User Interface creation : The user interface of the application consists of several pages – Login page, Registration page, Home page, Categories page, products page, product details page & Cart page. The UI has been created considering the modern design requirements and the appeal needed for the applications customer base.

   3. Adding Products : After collection of product data, an interface was created in order to simplify the process of uploading the data to the firebase database. The permission to add any product data to the database is limited to only the administrator, whose credentials can be created on the server side of firebase. The administrator is able to login through a separate login page than the user login page. Upon successful authentication and login, the administrator is able to add the product information to the database through the add product page. The products added to the database can then be accessed by all registered users via the application itself.

   Fig. 5 : Screenshot of Admin Login page (Left) and Add product page (Right).

6. RESULT

   Fig. 6 : Screenshot of Chair AR model Prototype in AR

   Fragment.

   The result achieved in the application with the use of GLB models is of good quality along with its textures and colours. The reflections and shadows of the chair depicted in the AR Fragment are accurate due to proper light estimation. Applied PBR allows the leather on the chair to appear shiny. The resulting model has an average density regarding its complexity, since it has been generated at runtime. If a higher complexity model is to be achieved, devices with a higher processing power and a higher bandwidth database can be used.

7. CONCLUSIONS

Traditional shopping industries have been very successful in the past, however todays world requires a convenient way of doing things. May it be shopping or getting a cab. This change may be due to a different generation of individuals as a customer base for most of the industries, or due to the technological revolution in the past few years. Thus the Furnished application fits perfectly in this world where convenience is paramount for the customers. We are able to achieve the goal of saving time, costs and hassles by the use of AR. The vrious plugins and sdk(s) used also provides the user with a faster shopping experience.

REFERENCES

1. Nowacki, Pawe & Woda, Marek. (2020). Capabilities of ARCore and ARKit Platforms for AR/VR Applications. 10.1007/978-3-030- 19501-4_36.

2. Akshay Singh, Sakshi Sharma and Shashwat Singh. Article: Android Application Development using Android Studio and PHP Framework. IJCA Proceedings on Recent Trends in Future Prospective in Engineering and Management Technology RTFEM 2016(1):5-8, July 2016.

3. Google Developers. 2021. Sceneform overview | Sceneform (1.15.0)

   | Google Developers. [online] Available at: https://developers.google.com/sceneform/develop [Accessed 11

   May 2021].

4. xu, Pengfei & Chang, Xiaojun & Guo, Ling & Huang, Po-Yao & Chen, Xiaojiang & Hauptmann, Alexander. (2020). A Survey of Scene Graph: Generation and Application. 10.13140/RG.2.2.11161.57446.

5. Marmoset. 2021. Basic Theory of Physically-Based Rendering. [online] Available at:https://marmoset.co/posts/basic-theory-of- physically-based-rendering Accessed : 10 May 2021.

6. Grid Dynamics Blog. 2021. How to build an AR app with ARCore and Sceneform. [online] Available at: https://blog.griddynamics.com/latest-arcore-and-sceneform- features-take-creation-of-ar-apps-to-the-next-level [Accessed 10 May 2021].