Complete the in Complete an Augmented 3D Re-Construction of Historical Monuments

Complete the in Complete an Augmented 3D Re-Construction of Historical Monuments

Sanchitsagar Singh

(Author)

Dept. of Information Technology Vidyavardhinis College of Engg. And Tech., Maharashtra, INDIA

Shreyans Gosalia

(Author)

Dept. of Information Technology Vidyavardhinis College of Engg. And Tech., Maharashtra, INDIA

Sushmita Mahajan

(Author)

Dept. of Information Technology Vidyavardhinis College of Engg. And Tech., Maharashtra, INDIA

Prof. Sainath Patil

(Author)

Dept. of Information Technology Vidyavardhinis College of Engg. And Tech., Maharashtra, INDIA

Abstract The masterful or chronicled estimation of a structure, for example, a monument, a mosaic or generally speaking, an artifact, arises from the novelty and the development it represents in a certain field or at a certain point of time in the human history. More faithfully speaking, the more the structure safeguards its unique status, the more prominent its masterful and recorded esteem it has. Consequently, it is our crucial obligation to protect its unique condition, keeping up it as authentic as conceivable over time. The conservation of a structure can't be constantly conceivable or has not generally been acknowledged, thus, bringing about carelessness, deficiency or even blameworthy reluctance. Thus, shockingly, the status of such not superfluous number of structures can run from awful to even calamitous conditions. In such a time allotment the present innovation outfits a major help for recreation/reclamation of these landmarks. Therefore, our point is to convey back such structures to its unique authentic condition with the assistance of technology. Among the advanced offices, new conceivable outcomes emerge from the Augmented Reality (AR) apparatuses, which consolidate the Virtual Reality (VR) settings with the real-world physical environment. This incompleteness we feel while visiting any historical or tourist attracted monument is when we get to see is the degraded or deteriorated remains of that structure and we remain unaware of the original beauty of that monument. The thought is to make a virtual recreation/rebuilding before tangibly following up on the structure itself. We, virtually with the help of augmented reality and image processing we can complete the incomplete structure by drawing on a screen when pointed towards the structure.

Keywords Augmented reality(AR); Virtual Reality(VR); Historical Monuments; Tourism.

1. INTRODUCTION

   The solution we focus on is by designing a complete system which includes a camera for the raw data input, an Unity3D environment for processing, Vuforia SDK for augmented reality software development, Blender and Maya for 3D model reconstruction, MySQL for storage and a centralized server using Spring Framework. The user at the site would point a camera equipped device, in our case a smartphone, to a broken or deteriorated structure which will then be completed in real time. The incomplete parts of the structure are then augmented with images from the database wherein

   the feature points are evaluated to get an appropriate match. This technique results in more immersive experience of the user into the historical aspect of the structure.

2. RELATED WORK

   Desai, Nilams Recreation of history using augmented reality[1] sheds light on augmented reality. Augmented Reality is a technology that composites virtual objects into real world. This perception can be applied to rebuilding of historical object to have realist view of time. Latest technologies associated to cultural and archaeological historical sites have given rise to new concepts such as Virtual History, Virtual Archaeology, Virtual Museums and many more. MAR (Mobile Augmented Reality) can be used that provide way to reconstruct damaged or lost part of heritage sites. Efficient AR visualization scenarios for complementing broken or damaged real objects. This is being used in AR game for Philippine history. Xavier, R. S., da Silva, B. M. F., & Goncalves, L. M. G.[2] in Accuracy Analysis of Augmented Reality Markers for Visual Mapping and Localization talks about an important problem that can be solved by Computer Vision algorithms is that of computing a 3D reconstruction of a scene captured by an ordinary camera. The algorithm to be used here is Simultaneous Localization and Mapping (SLAM). Along with SLAM using ARUCO marker. The process being divided into two stages, the first stage visualizes the artificial markers are captured and processed. It also produces a map composed of a network of artificial markers that are in global reference frame. In the second stage, images of the camera are used as inputs along with the computed map in an algorithm that localizes each image within the network of markers, thus providing the position and orientation of any camera capturing one or more markers in the environment. Putra, E. Y., Wahyudi, A. K., & Dumingan, C.[3]- A proposed mix of photogrammetry, Augmented Reality and Virtual Reality Headset for legacy perception. The examination on the reproduction of 3D scanned Models was started utilizing photogrammetry procedures and joining of multimedia components. 3D models are shown utilizing Augmented Reality by utilizing Virtual Reality Gear to amplify the experience of the client. It portrays the strategies

   and procedures utilized in gathering the information and giving development in the perception of social legacy. The outcome is that the framework can give new encounters to clients and a decent 3D perception. Main method used here is 3D-Model Re-Construction using software Autodesk Remake. Images were captured using DSLR cameras and Unmanned Aerial Vehicle (UAV).

3. PROPOSED SYSTEM

   Implementation involves Unity3D environment along with various modules with their library supported by Android. Also, a mobile application along with server and database configuration is necessary.

   1. Hardware

      • DSLR Camera

      A Canon 700D DSLR Camera will be used to get the images which will given as an input to the software for the process of photogrammetry. Photographs will be taken at an angle of 10o from the adjacent click in a 360-degree rotation.

   2. Software

      • Mobile application

        Unity 3D is a software used for development of three- dimensional and two-dimensional games and simulations. It is also a cross-platform engine for gaming. The Unity Engine offers a primary scripting API in C#, for both the Unity editor in the form of plugins, and games themselves. It also provides drag and drop functionality. The application we are using is based on Mobile Augmented Reality (MAR) using Unity Game Engine. Unity is used to superimpose digital media like image, video, text, sound, etc. on physical locations or objects and can be experienced through electronic media like smartphones and tablets. Unity 3D is easy to design, build, and maintain efficient, reusable, and reliable code.

      • Android SDK

        The process to create Android application, running on Android operating system is Android software development. The software kit of Unity includes development tools like debugger and libraries and many more Platform that are supported for development contains computer running OS X 10.5.8, Windows and Linux.

      • Vuforia SDK[4]

        Vuforia is a software development kit based on Augmented Reality used for mobile devices. This software is used for creating application based on Augmented Reality. It uses the concept of computer vision which helps in recognizing and tracking images and also simpl 3D boxes. The Vuforia SDK helps the developer for positioning virtual objects like 3D models with real world images when it will be seen through mobile phones camera using image registration capability. Then position and orientation tracking of real time image is done by virtual object which helps the viewer to see the object from image perspective Thus the virtual object appears to be part of real-world image. With this SDK various 3D and 2D targets can be made. Advanced features of this SDK are dynamic image target selection and the

        capability to construct and rearrange targets sets dynamically.

      • Server o MySQL Database

        MySQL is free and open source database management system which uses structured query language. It is popular language for managing and accessing content of the relational database. The advantages of using MySQL, it is fast, flexible and quick processing. MySQL is most preferably used with PHP server. It is compatible with platforms including Linux, UNIX and Windows.

        o Spring REST API

        The 'REST' API is utilized to get/send values in a key- esteem pair from/to the server. For the conveyance of information from the server to the devouring assets, API must be created. For this situation, RESTful based APIs for example 'REST' APIs are utilized to get/send values in a key- esteem pair from/to the server. The innovation utilized for creating REST APIs here is Spring MVC as it gives heartiness and security to different system dangers. Spring connectivity to the MySQL Database is done by mysql- connector v8.5 library and for web-services jackson v2.2.3 library is used in creation of REST based APIs.

   3. Methods

      Apache Tomcat Server

      It is utilized to convey .war records made utilizing the spring system. It is utilized for sending of RESTful APIs made by Spring MVC in the best possible condition.

      Organization is as simple as replicating the .war record of our code made utilizing spring structure. Easy Apache (right now v4) is to be introduced on the facilitating gave to you, after which you can transfer your .war record after that unfasten it and you are as great to go.

      • 3D Reconstruction

        3D reconstruction is the method in which we capture the shape and appearance of real objects. This process can be achieved using active or passive methods. Active methods provide depth map and it also gives insight about reconstructing the 3D profile by numerical approximation approach and helps in building objects. On the other hand, Passive methods does not tamper the reconstructed object, here there is usage of sensors which helps in measuring the radiance reflected from object's surface to guess its 3D structure using image understanding and processing.

      • Photogrammetry Methodology

      Photogrammetry is the process which gathers measurements in the physical world with the help of computer analysis of images captured by camera and is a well-known scientific tool. Photogrammetry is used to capture measurements of fixed points for estimating measurements of the photographed subject which also can be used in 3D.

   4. Algorithms

   • SLAM [8]

   Slam is an acronym for Simultaneous Localization and Mapping. SLAM is used to dynamically map and mark the feature points of the target image. Now the advantage of using SLAM is that it maps unknown spaces and it is drift

   free. It also is used extensively as it is real time. The architecture of SLAM consists of following components.

   1. Sensor data: They are basically devices like mobiles which includes camera gyroscope and accelerometer. This can be augmented by some sensors like GPS, depth sensor, light sensor etc.

   2. SLAM estimate: This phase shows the result of our algorithm implementation which is tracked features and camera position.

   A very good example of SLAM is Orb-SLAM which gives excellent results. Also, SLAM has features like loop detection and loop closing which helps in detecting the target even from different angles of the camera.

4. EXPECTED OUTCOME

   We point our camera of the mobile phone to the deteriorated structure it will detect it and send it to the repository. Then repository will find the appropriate match for the structure and after mapping and scaling it with its ancient picture which is stored in repository it will only send only deteriorated part of the structure which will be morphed on the mobile phone screen. In this way it will display an ancient image at the place of original image.

   Fig 1. Workflow Diagram

5. EXPERRIMENTAL RESULTS

The experimental results are expected to be as follows:

• The number of tourists increasing.

• The harassment by the local guides done to tourist and the general audience reducing over the time.

• Income due to tourism increasing.

• The local businesses flourishing.

• The immersive nature of the system rejuvenates the mind and soul of the user.

CONCLUSION

Result will be increment in number of audiences at the historical sites and monuments as the system will create a never-ending hunger for knowledge and inquisitiveness. Interest will spark in the minds of youth to learn more about the culture and traditions of the local surrounding hence beneficial to keep the majority of the audience close to our roots.

For future advancement, we will be adding few more functionalities. We will try to replicate the full 3D affect through augmentation. Suppose, there was a door so you will get a virtual 3D door in front of you and you can enter into it experiencing the actual scenario of the monument.

REFERENCES

1. Desai, Nilam. (2018). Recreation of history using augmented reality. ACCENTS Transactions on Image Processing and Computer Vision. 4. 15. 10.19101/TIPCV.2017.39019.

2. Xavier, R. S., da Silva, B. M. F., & Goncalves, L. M. G. (2017). Accuracy Analysis of Augmented Reality Markers for Visual Mapping and Localization. 2017 Workshop of Computer Vision (WVC).doi:10.1109/wvc.2017.00020

3. Putra, E. Y., Wahyudi, A. K., & Dumingan, C. (2016). A proposed combination of photogrammetry, Augmented Reality and Virtual Reality Headset for heritage visualisation. 2016 International Conference on Informatics and Computing (ICIC).doi:10.1109/iac.2016.7905687.

4. VuforiaTH API References. Vuforia SDK Information and integration with Unity3D. URL: https://library.vuforia.com/content/vuforialibrary/en/reference/unity/i ndex.html.

5. R. Muñoz-Salinas, M. J. Marín-Jiménez, E. Yeguas-Bolivar, and R.

   M. Carnicer, Mapping and localization from planar markers, Arxiv Computing Research Repository (CoRR), vol. abs/1606.00151, 2016. [Online]. Available: http://arxiv.org/abs/1606.00151

6. A. Wahyudi, R. Ferdiana, and R. Hartanto, Pengujian dan Evaluasi Buku Interaktif Augmented Reality ARca 3D, in Seminar Nasional Teknologi Informasi dan Multimedia 2014, Stimik Amikom, Yogyakarta, 2014, vol 2.

7. J. Kang, AR Teleport: Digital Reconstruction of Historical and Cultural-Heritage Sites Using Mobile Augmented Reality, in 11th International Conference on Trust, Security and Privacy in Computing and Communications, South Korea, 2012, p. 1666.

8. Basics of AR: SLAM Simultaneous Localization and Mapping, August 14, 2018 by Andijakl. https://www.andreasjakl.com/basics-of- arslam-simultaneous-localization-and-mapping/