Augmented Reality – an Application for Kid’s Education

Augmented Reality – an Application for Kid’s Education

Surabhi Nanda

Dept. of Computer Science Engineering, HMRITM, Delhi, India

Shailendra Kumar Jha

Dept. of Computer Science Engineering, HMRITM, Delhi, India

Abstract The paper explores the field of augmented reality with the central focus on the education for kids. The Augmented Reality bridges the gap between theoretical and practical knowledge. Generally, an AR system generates, in real time, a composite view which is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments it with additional information. It allows the users to control and manipulate the objects that are not real, in an augmented environment. The alphabets are the basic building blocks of any language. Such application enables the students to have a pre-school learning experience, to have a better understanding of the letters, their corresponding sounds and thus help in improving the pronunciation skills and memorization of English alphabets. It has a positive impact on students learning experience. The application includes some new and interesting features like navigation buttons for previous alphabet, next alphabet, home screen. Several words starting from the same alphabet have also been added in each scene. The hardware can beany smart device with the downloaded AR application. The software used in the application are Unity3D, Vuforia and SketchUp.

KeywordsAugmented Reality; Virtual Reality; Unity3D Vuforia; SketchUp; Image Target

1. INTRODUCTION

   According to the reality-virtuality continuum [1] by Milgram and Kishino (1994), AR is one significant part of the general representation of mixed reality. The area between two extremes, where both the real environment and the virtual environment are mixed, is called mixed reality [2] or hybrid reality. It consists of both augmented reality and augmented virtuality. Augmented Reality is becoming an emerging multi-platform application. Augmented Reality is an extension of virtual reality with more advancement in our day to day life. VR attempts to create an artificial world a person can experience and explore, through his or her sense of vision, audio, and other different forms of feedback. AR brings about a similar interactive experience, but also aims to supplement the real world, instead of creating an entirely artificial environment. The physical objects in the individuals real life surroundings acts as the backdrop and target images for computer-generated illustrations[3]. In Augmented Reality, the user is more aware of the real world environment than in Virtual Reality.

   Augmented Reality is a relatively young technology and is the way of future technologies and innovations. It has large number of applications in the fields of medical[4][5], marketing[6], advertisement[7], entertainment, education and training, industrial design[8], manufacturing and maintenance[9], architecture[10], navigation[11], emergency,

   search and rescue systems[12], interactive gaming[13][14] and tourism[15].

   Scripting is the most important aspect of unity 3D and has the advantage of quick prototyping and better organization of code. Unity supports two programming languages which are C# and Unity Script. Everything from the creation of 3D models to the enhancement of features like rotation, Scaling, Zooming and showing the simple movements of 3D models, applying materials and textures require the use of scripts.

   This paper focuses on Augmented Reality and its related application in the field of education. We attempt to create a new education tool that combines a mobile application for the learning of 26 alphabets of English language with augmented reality technique [16]. The textbooks, newspaper, magazines, comics and other reading materials can contain certain images or markers when scanned by any AR device such as mobile phone, tablet or a computer system supported by and platform produces additional information in the multimedia format. The textual, graphical, audio and visual information superimposes on the students real time environment. Such application enable the students to actively participate and interact with computer generated simulations. We introduce an application that presents new and exciting ways to get children involved in learning alphabets .This application can change the way children learn alphabets with the technology of Augmented Reality. The technology helps interleaving the concepts of theoretical and practical learning. Augmentation can serve to aid and enhance individuals knowledge and understanding of concepts. It enhanced the educational potential that scholars, teachers and students are embracing. AR is bringing a revolution to the entire education system and way we teach and learn.

   The paper is organized in the following manner: Section II explains related works, Section III briefly explains the methodology, the procedure for creating an app based on augmented reality, the hardware and software requirements, section IV is based on the results and conclusion for the entire application and section V presents the future work to modify and enhance the applications for education and other related areas.

2. RELATED WORK

   There are various examples of research on the Augmented Reality applications in the field of education. Therefore, before describing our contribution and its related methodology we provide a brief overview of the related projects that utilize AR technology for developing educational applications.

   1. A 3D pop-up book has been developed. It is a Augmented Reality 3-Dimensional pop-up book namely the Seed Shooting Game, for Grade three students, in Bangkok, Thailand. This Augmented Reality (AR) 3D pop-up Book employed a research and development process through storytelling technique [17].

   2. Augmented Reality for academic purposes is growing in the same upward direction as the expansion of smart multimedia into education and lifelong learning. This paper reports on an AR curriculum materials research and development project which employs storytelling as a teaching technique. This technique has a long lasting effects on its user due to it liveliness and interactivity. The primary rationale supporting this research was the high demand and appreciation of the potential to integrate Augmented Reality into classrooms effectively to enhance learning [18].

   3. Semantic and content-based multimedia exploitation for European benefit is developing a solution for museums to create and manipulate digital representations of museum objects. The Sculpture project utilizes multistereo and silhouette techniques to create 3D museum object reconstructions stored in a database together with other multimedia data. The project is also aimed at developing a semantic layer providing a variety of search and content analysis opportunities. A system that allows museums to build and manage Virtual and Augmented Reality exhibitions based on 3D models of artifacts is presented [19].

   4. There has been a growing interest in augmented reality with the recent advancements in computer technologies. Head mounted displays which were once very expensive hardware devices are now available at low cost, which has led to the increase in the applications of AR technology. Methods/Statistical Analysis: In this paper, we describe an AR coloring book with a transitional user interface. It comprises a paper book and a smart phone application. The proposed AR coloring book enables the user to enjoy content by switching between real world, augmented reality, virtual rality, and networked virtual reality [20].

   5. Teaching English and correct way to pronunciate word to children who do not come from an English speaking background is a challenge for educators. In this paper, the authors present an augmented reality tool, TeachAR, for teaching Basic English words, colors, shapes, and prepositions to children for whom English is not a native language. In this paper the author also compares AR system to a traditional non-AR system. The results indicate a potentially better learning outcome using the TeachAR system than the traditional system [21].

   6. A case study was conducted in a typical kindergarten in Hong Kong, with a class of four- to five-year- old children. The results indicated that all the children could design, control and interact with the animated objects generated by the AR application, all the stakeholders interviewed were supportive of this pedagogical innovation and the principal, teachers and parents were concerned about the side effects of using AR in early childhood[22].

3. METHODOLOGY

   1. Software and algorithms

      The software must derive the real world coordinates, independent from the camera, from the images captured by the camera. This process is called dynamic image registration [23]-[24] which uses different methods of computer vision, and consists of 2 stages. The first stage is the feature detection method, used to detect fiducial markers, interest points, or optical flow in the camera images. The second stage restores a real world coordinate system from the data obtained in the first stage using some mathematical representations [25].

      Some of the software that we have used are Unity 3D, Vuforia and Sketchup.

      1. Unity 3D: Unity 3D is one of the most efficient gaming platform which allows us to build 3D and 2D models. It also has various features like adding animation, sound, text, images and many more.

      2. Vuforia: Vuforia is one of the well-known SDKs for the rapid development of the AR applications. It include impressive digital features like recognize, analyse and improve user designed target image for better performance of application.

      3. Sketch Up: Sketch Up is computer software, provides a large number of interesting tools, used to generate or to create 3D models for a wide ranging applications. The users may upload and download 3D models to provide a user friendly environment. The models created using this software can be directly imported to unity solves some kind of modeling problems.

   2. Hardware

      The Augmented Reality applications can be implemented by varied technologies such as desktop computer, hand held devices, head mounted displays, optical projection systems and so on. The application described in this paper requires only one hardware device that can either be a smart phone, a tablet with pre-installed AR application. Our application focuses specifically on the android devices.

   3. Stages of App Development

   Fig. 1. Working of Vuforia

   Fig. 2. Working of Unity 3D

   D. PICTURES

   Fig. 3. Target Image. (By- https://pageborders.org/download/alphabet- blocks-border/)

   Fig. 4. Application Homepage

   Fig. 5. Screenshot Alphabet A

   Fig. 6. Screenshot Alphabet B

   and learning environment through augmented reality. The results indicate that AR systems are a positive step towards reducing the distance between children and knowledge. This prototype of AR application enables the children to increase the familiarity with the recognition of letters, pronunciation of letters and enhance their skills and improve the memorization of students.

   V. FUTURE WORK

   The potential of AR technology in education is just the beginning to be explored. Since AR interfaces provides seamless interactions between the real and the virtual world, therefore educators should work with the researchers to explore how these characteristics can best be incorporated in the school environment. For future development in this application various extra features can be added like direct navigation to a particular letter and the same application can be developed for various platforms which is now restricted for an android. Presently, the application is dependent on a single target images. The application can however be developed for multiple image targets or applications which does not require any image target in order to make it more interesting, more interactive and less dependent on a single or multiple image targets. Image target detection and rendering of images captured by the camera can be improved. Gesture and voice control can be added for navigation between various scenes. Also it is possible to develop 3D digital books for higher education.

   REFERENCES

   Fig. 7. Screenshot Alphabet Z

   Fig.3 This is the target image for our application. The user must scan the target image by any smart device, using an application created by unity and the result will be 3D model for the alphabets.

   Fig.4 The front page contains a button START LEARNING. This button will directly connect with the first alphabet A.

   Fig.5 This is the first alphabet. It consists of three buttons at the bottom. The previous button connecting to the home screen, the home screen button, the next button connecting to next alphabet B.

   Fig.6 This is the second alphabet. It also consists of three buttons at the bottom. The previous button connecting to previous alphabet A, the home screen button, the next button connecting to next alphabet C.

   Fig.7 This is the last alphabet. It also consists of three buttons. The previous button connecting to the alphabet Y, the home screen button, the next button connecting to the

   home screen

4. RESULTS AND CONCLUSION

Augmented reality allows the user to create layers and layers of digital information over the top of physical information in 2D configuration. This paper tries to create a more efficient

1. Paul Milgram, Haruo Takemura, Akira Utsumi and Fumio Kishino, Augmented Reality: A class of displays on the reality-virtuality continuum, Proceedings of Telemanipulator and Telepresence Technologies, SPIE vol. 2351, pp. 282-292, October 1994.

2. Paul Milgram and Fumio Kishino, Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information Systems (IECE special issue on networked reality), vol. E77-D, no. 12, pp. 1321-1329, December 1994.

3. Tobias H. Hollerer and Steven K. Feiner, Mobile Augmented Reality, Telegeoinformatics: Location-Based Computing and Services, H. Karimi and A. Hammad (eds.), Taylor & Francis Books Ltd., 2004.

4. Tobias Sielhorst, Marco Feuerstein, Nassir Navab, Advanced Medical Displays: A Literature Review of Augmented Reality, Journal of Display Technology, vol. 4, pp. 451-467, December 2008. Yoshitaka Masutani, Takeyoshi Dohi, Fumitaka Yamane, Hiroshi Iseki and Kintomo Takakura, Augmented Reality Visualization System for Intravascular Neurosurgery, Journal of Computer Aided Surgery, vol. 3, pp. 239-247, October 1998.

5. Alexander Bornik, Reinhard Beichel, Bernhard Reitinger, Georg Gotschuli, Erich Sorantin, Franz W. Leberl and Milan Sonka, Computer-aided liver surgery planning:an augmented reality approach, Proceedings SPIE 5029, Medical Imaging:Visualization, Image-Guided Procedures and Display, May 2003.

6. X. Zhang, N. Navab, S. P. Liou, E-commerce direct marketing using augmented reality, IEEE International Conference on Multimedia and Expo- ICME 2000, Proceedings, Latest Advances in the Fast Changing World of Multimedia, vol.3, 30 July-2 August 2000

7. Fadi Chehimi, Paul Coulton and Reuben Edwards, Augmented Reality 3D Interactive Advertisements on Smartphones, IEEE Sixth International Conference on the Management of Mobile Business, 9-11 July 2007.

8. H. Regenbrecht, G. Baratoff and W. Wilke, Augmented reality projects in the automative and aerospace industrie, IEEE Computer Graphics and Applications, vol. 25, pp. 48-56, Nov.-Dec. 2005.

9. U. Neumann and A. Majoros, Cognitive, performance and systems issues for augmented reality applications in manufacturing and maintenance, Virtual Reality International Symposium, IEEE, 14-18 March 1998.

10. Bruce Thomas, Wayne Piekarski and Bernard Gunther, Using Augmented Reality to Visualise Architecture Designs in an Outdoor Environment, In DCNET 1999- Design Computing on the Net, November 1999.

11. B. Thomes, V.Demczuk, W. Piekarski, D. Hepworth and B. Gunther, A wearable computer system with augmented reality to support terrestrial navigation, Second International Symposium on Wearable Computers, IEEE, 19-20 Oct. 1998.

12. Junho Ahn, Richard Han, RescueMe: An Indoor Augmented-Reality Evacuation System by Personalized Pedometry, Services Computing Conference (APSCC), IEEE Asia-Pacific, 12-15 Dec. 2011.

13. Diogo Cordeiro, Nuno Correia and Rui Jesus, ARZombie: A mobile augmented reality game with multimodal iteraction, IEEE, 2015.

14. Zhihan Lv, Alaa Halawani, Shengzhong Feng, Shafiq ur Rehman and Haibo Li Touch-less interactive augmented reality game on vision- based wearable device, Personal and Ubiquitous Computing, vol. 19, pp. 551-567, May 2015.

15. Feng Tian, Feifei Xu and Jiacai Fu, Augmented reality technology overview for tourism app development, IEEE International Conference on Machine Learning and Cybernetics, IEEE, 14-17 July 2013.

16. Nipat Thiengtham and Yingyos Sriboonruang, Improve Template Matching Method in Mobile Augmented Reality for Thai Alphabet Learning, International Journal of Smart Home, vol. 6, no. 3, July 2012.

17. Poonsri Vate-U-Lan, AAugmented Reality 3D Pop-Up Children Book:Instructional Design for Hybrid Learning , IEEE, 2011.

18. Poonsri Vate-U-Lan, An Augmented Reality 3D Pop-Up Book: The Development of a Multimedia Project for English Language Teaching , IEEE International Conference on Multimedia and Expo, 9-13 July 2012.

19. Rafa Wojciechowski, Krzysztof Walczak, Martin White and Wojciech Cellary, Building Virtual and Augmented Reality Museum Exhibitions, Proceedings of the Ninth International Conference on 3D Web Technology, Association for Computing Machinery, January 2004.

20. Kwangmoon Cho, Hadong Kim and Youngho Lee, Augmented Reality Coloring Book with Transitional User Interface, Indian Journal of Science & Technology, May 2016.

21. Che Samihah Che Dalim, Thammathip Piumsomboon, Arindam Dey, Mark Billinghurst and Shahrizal Sunar, TeachAR: An Interactive Augmented Reality Tool for Teaching Basic English to Non-native Children, IEEE International Symposium on Mixed and Augmented Reality (ISMAR Adjunct), 19-23 Sept. 2016.

22. Yujia Huang, Hui Li and Ricci Fong, Using Augmented Reality in early art education: A case study in Hong Kong Kindergarten, Early child development and care, August 2015.

23. M. Bajura and U. Neumann, Dynamic registration correction in video- based augmented reality systems, IEEE Computer Graphics and Applications, vol. 15, pp. 52-60, Sep. 1995.

24. S. You and U. Neumann, Fusion of vision and gyro tracking for robust augmented reality registration, IEEE, 13-17 March 2001.

25. J. Rekimoto, Matrix: a realtime object identification and registration method for augmented reality, Computer Human Interaction, Proceedings, 3rd Asia-Pacific, IEEE, July 1998.