Augmented Reality in Learning System using Contour Analysis

Augmented Reality in Learning System using Contour Analysis

Namrata S. Mandvikar

E&TC Dept.

Saraswati Collage of Engg.,Khargar New Mumbai,India

Sunita M. Jadhav

E&TC Dept.

Saraswati Collage of Engg.,Khargar New Mumbai,India

Abstract Augmented reality (AR) has been increasingly applied in various fields. It is a new technique of the computer vision application used to facilitate interaction in the digital arts. Augmented Reality (AR) employs computer vision, image processing and computer graphics techniques to merge digital content into the real world. It enables real time interaction between the user, real objects and virtual objects. We have used here contour analysis for processing an image. As Contour Analysis allows to describe, store, compare and find the object presented in the form of exterior outlines, solve the main problems of a pattern recognition – transposition, turn and a rescaling of the image of object. CA methods are invariant to these transformations. It provides more realistic interaction. It is an advance method and could be a promising technology for motivating users to engage in learning systems.

Keywords Augmented reality, Pattern recognition, Image processing, Contour analysis.

1. INTRODUCTION

   Augmented Reality aims at simplifying the users life by bringing virtual information not only to his immediate surroundings, but also to any indirect view of the real-world environment, such as live-video stream. AR enhances the users perception of and interaction with the real world. AR can potentially apply to all senses, augmenting smell, touch and hearing as well. AR can also be used to augment or substitute users missing senses by sensory substitution, such as augmenting the sight of blind users or users with poor vision by the use of audio cues, or augmenting hearing for deaf users by the use of visual cues.

   The objective of this research is to provide tool to learn different patterns that will be inputted into the system. And augment the assigned object to these patterns. By doing this we will be able to enhance the user perception for the respective pattern. The first part of the article contains reviews of definitions and algorithms in augmented reality, image enhancement, edge detection and image segmentation as in [1]. These algorithms are applied to use on our problem domain where we try to augment the learning image. The second part is the proposed model that designed to extract the image by using inbuilt library and close contour of small elements of the characters. Third part includes augmentation.

2. AUGMENTED REALITY

   AR is a type of virtuality which aims to duplicate the worlds environment in a computer.AR generates a composite view for user which combines real scene viewed by user and the virtual scene generated by computer that augment the scene with additional information. Encyclopaedia Britannica gives the following definition for AR: Augmented reality, in computer programming, a process of combining or augmenting video or photographic displays by overlaying the images with useful computer-generated data.

   Simple augmented reality system as shown in figure 1 consists of a camera, a computational unit and a display. The camera captures an image, and then the system augments virtual objects on top of the image and displays the result as in [4]. Figure 2 shows flowchart for simple augmented reality system, The capturing module captures the image from the camera. The tracking module calculates the correct location and orientation for virtual overlay. The rendering module combines the original image and the virtual components using the calculated pose and then renders the augmented image on the display.

   Figure 1.Example of a simple augmented reality system setup.

   Figure 2. A flowchart for a simple augmented reality system

   The tracking module is the main part of the augmented reality system, it calculates the relative pose of the camera in real time.

3. CONTOUR ANALYSIS

   1

   1-i

   1-i

   -i

   -1-i

   -1

   -1+i

   -1+i

   i 1+i

   The contour is the boundary of object. In a CA the contour is encoded by the sequence consisting of complex numbers. On a contour, the starting point is fixed. Then, the contour is scanned and each vector of offset is noted by a complex number a+ib. Where a – point offset on x axis, and b – offset on y axis. Offset is noted concerning the previous point. Owing to the physical nature of three-dimensional objects, their contours are always closed and cannot have self- intersection. Hence we can define unambiguously a way of bypass of a contour. The last vector of a contour always leads to the starting point. Each vector of a contour name(EV).

   START POINT

   ELEMENTARY VECTOR (EV)

   CONTOUR PIXEL

   Figure3. contour concept

   Sequence of complex-valued numbers – vector-contour (VC) Normalised scalar product (NSP) of a contour can be defined as

   The norm of the normalized scalar product of contours gives unity only in the event that these two contours are equal to within turn and a scale. Otherwise, the norm of NSP it will be less unity. Actually, the norm a NSP is an invariant on transposition, rotation and scaling of contours. If there are two identical contours their NSP always gives a unity, is not

   dependent on where contours are, what their angle of rotation and a scale. Similarly, if contours are various, their NSP will be strict less 1, and also independent of a place, rotation and a scale.

   If contours are identical, but the EV reference begins with other starting point the norm the NSP of such contours will not be equal to a unity.

4. PRACTICAL APPLICATION OF CA

   Contour analysis is used for pattern recognition task as in [17] on the image. Let us take the image a size n*n pixels. Then breed its uniform grid with a step s. The total length of all grid lines is:

   The CA allows to process the image in a progressive mode. It means that we can sort contours on any to an indication (for example, by square or on a gradient of boundaries, or on brightness, etc.). And then to treat the first contour, and to produce outcome. Remaining contours to process in a background mode. As contours are independent from each other algorithms of a recognition it is easy to parallelize. Besides, algorithms are very simple and can be executed on graphic processors.

   For fast searching of templates, it is necessary to introduce the certain descriptor characterizing the shape of a contour. An ACF invariantly to transposition, rotation, scaling and a starting point choice. And besides, the ACF is a function of one contour, instead of two, as an ICF.Hence the ACF can be selected as the descriptor of shape of a contour. The close contours will always have the close values an ACF.

   In pictures, the norm the ACF is represented by dark blue color (an ACF it is represented only for an interval from 0 to k/2).

   Figure 4. ACF for different shapes

   In the real image contours have arbitrary length. Therefore, for searching and comparing of contours, all of them should be led to uniform length. This process is called equalization.

5. LEARNING SYSTEM

   Web cam access module

   Feature matching module

   Template loading and saving module

   Image/object recognition module

   Image processing module

   Augmented reality module

   Figure 5. different modules

   1. Web cam access module

      It consists of program to access web camera in order to take input from he web cam. Here C#.net and openCV software is used to access camera. As soon as the software starts, this module of the software starts executing. By clicking on start button in GUI learner can allow the web cam to take input. Output of this shows camera started. By clicking on stop button in GUI learner can allow the web cam to stop processing. Output of this shows camera stopped.

   2. Template loading and saving module

      This module fetches previously stored or learned templates from the database. Database consists of table in which augmented picture for particular template is there. Contour templates with assigned pictures are stored in table form. When a given pattern is matched with the contour template, assigned picture is display for that contour. User can store new template by clicking on learn template button in GUI.

      In learn template module, we can learn new templates to the system. The contours for given real time images are detected first, and particular pattern as a template is selected. We can assign new jpg or png picture to that particular pattern, by clicking on add selected contour as template button.

   3. Image processing module

      It takes the image from the camera module and processes it. It involves smoothing, dialation, erosion and thresolding etc.

   4. Feature matching module

      After detecting the features, the system needs to match them,

      i.e. it needs to find corresponding features in different images. For feature matching, tracking systems use commonly two different approaches: they compare the small image areas around the features and find similar areas (template matching), or they calculate image characteristics around the features and compare them (descriptor matching).

      In the template matching approach, a feature detector matches image patches against regions in the image. This is also called the patch matching approach. Template markers and image markers can

      be considered image patches, with the distinction that marker detection is a separate process and matching is used only for identification of the marker.

      The second matching approach where the system calculates image characteristics around the feature that describe it appearance distinctively. These calculated characteristics are called feature descriptors.

      A good feature descriptor is invariant under image deformations, it is scale and illumination invariant and is capable of distinguishing between different features.

      An ACF invariant to transposition, rotation, scaling and a starting point choice. And besides, the ACF is a function of one contour, instead of two, as an ICF. Hence the ACF can be selected as the descriptor of shape of a contour. The close contours will always have the close values an ACF.

      This module contains set of codes matching features of the recognized object/image which is pointed out by the learner and image in the database.

   5. Image/Object recognition module

      It contains code for Image/Object recognition. Image/Object recognition is done by the following procedure. When we have all pairs of matched features, we can recognize the images or objects. The simplest method is to count the number of matched features. Without loss of generality, the image pairs that have the largest number of matched features are the same.

   6. Augmented reality module

   In this database all augmented reality shapes which has to be augmented out when certain condition is matched ,for example when images are matched is stored. This is also updated whenever a new pattern is encounter. These augmented reality patterns are changes from Images to images / Objects to Objects matches. When the camera is focused on the particular character on paper the student can study that by watching augmented reality pattern.

6. CA LIMITATION

   CA makes sense, only in that case when the object contour is defined unambiguously correctly in all points.CA methods assume that the contour describes all object bodily, and does not suppose any intersections with other objects or incomplete visibility of object.

7. SOFTWARE

   Some softwares for image processing are matlab which is very slow in response, AForge.net which dont have more libraries for human computer interaction (HCI) and OpenCV. OpenCV as in [18] is an open source computer vision library written in C and C++ and runs under Linux, Windows and Mac OS X. Image going to be processed is capture through web camera using open CV software.

   As C#.net is based on C++ and is compatible with OpenCV library. For implementing base function of CA we propose C#. Contour Analysis Processing uses library OpenCV (EmguCV

   .NET wrapper) for operation with the image.

   C# is easy to use and is graphical user interface (G.U.I.) based. Emgu.CV is used as a wrapper for the C#.net and OpenCV environment.

8. RESULT

9. CONCLUSION

A simple AR learning system is designed and implemented. CA methods are attractive for the simplicity and high-speed performance. High-speed performance of a CA allows to process video. Object is recognized and related picture is augmented.

REFERENCES

1. Jay Acharya1,et al. ,Segmentation techniques for image analysis : a review, C U Shah College Of Engineering & Technology, Wadhwan, India, International Journal of Computer Science and Management Research ,Vol 2, Issue 1, January 2013.

2. Nipat Thiengtham and Yingyos Sriboonruang Improve Template Matching Method in Mobile Augmented Reality for Thai Alphabet Learning,Sciences and Social Science Burapha University, Sakaeo Campus, Sakaeo,International Journal of Smart Home, Vol. 6, No. 3,

   July, 2012

3. Mohamed SAKKARI ,Hands tracking for Augmented Reality Applications, dept. of computer science Faculty of Sciences of Gabes, FSG Cité Erriadh 6072, Zrig, Gabes, Tunisia,2012.

4. Chetan Kumar, G. Shetty ,Expeinn E-Learning System, ExperInn Research, KVG College of Engineering, Sullia (D.K), Karnataka, Academic Research International, Vol. 2, No. 1, January 2012.

5. Alfa Sheffildi Manaf, Riri Fitri Sari, Color Recognition System with Augmented Reality Concept and Finger Interaction, Case Study for Color Blind Aid System, Electrical Engineering Department Universitas Indonesia Depok, Indonesia,Ninth International Conference on ICT and Knowledge Engineering ,2011.

6. D. Prochazka, T. Koubek ,Augmented Reality Implementation Methods in mainstream Applications, Received: August 31, 2010.

7. R. Deepak Kumar. K.Ramareddy, B.Madhusudan Rao ,A Simple Region Descriptor based on Object Area per Scan Line , Interdisciplinary Science and Technology I2IT, Pune, International Journal of Computer Applications (0975 8887),Volume 3 No.7, June 2010

8. Sang Hwa Lee, Junyeong Choi, and Jong-Il Park ,Interactive E- Learning System Using Pattern Recognition and Augmented Reality, Member, IEEE,2009.

9. Taehee Lee, Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality, transactions on visualization and computer graphics,VOL. 15, NO. 3, MAY/JUNE 2009.

10. Jong Bae Kim and HeeSung Jun, Vision-Based Location Positioning using Augmented Reality for Indoor Navigation, IEEE Transactions on Consumer Electronics, Vol. 54, No. 3,August 2008.

11. Kyusung Cho & Juho Lee et al ,A Realistic E-Learning System based on Mixed Reality, Advanced Institute of Science and Technology, 373- 1 Guseong-dong, Yuseong-gu, Daejeon 305-701, Republic of Korea, VSMM 2007, Brisbane, Australia, 23-26 Sept 2007.

12. Vlad Popovici , Jean-Philippe Thiran,Pattern recognition using higher- order local autocorrelation coefficients, Signal Processing Institute, Swiss Federal Institute of Technology (EPFL), CH-1015 Lausanne, Switzerland , Available online 15 April 2004.

13. Luiz Reginaldo A. F. Curado et al. ,Using PCA to Recognize Characters in Vehicle License Plates Electrical Engineering Dept. University of Brasilia Brasília, Brasil.

14. Herbert Bay1, Tinne Tuytelaars2, and Luc Van Gool12 1 ETH Zurich, SURF: Speeded Up Robust Features, Katholieke Universiteit Leuven.

15. Ildar Farkhatdinov1, Jee-Hwan Ryu2, Development of Educational System for Automotive Engineering based on Augmented Reality, Korea University of Technology and Education, Cheonan, Korea.

16. Sanni Siltanen,Theory and applications of marker-based augmented reality,Chapter 4,64-75.

17. Lanjakorn Sewata Salil Boonbrahm ,The extracting of shirt composition using Canny edge detection with close contour,Walailak University Nakorn Si Thammarat, Thailand.

18. Gary Bradski and Adrian Kaehler , Learning OpenCV,1st ed., OReilly

    ,2008.

19. Namrata S. Mandvikar Augmented reality using contour analysis in e- learning, IJIRSET, Volume 2, Issue 5, May 2013.

20. Namrata s. Mandvikar,Mrs. Sunita M. Jadhav Augmented reality learning system with contour analysis,IJIIT,Volume 2,issue 3,2013.

BIOGRAPHY

Mrs. Namrata Mandvikar is working as a lecturer in B.L.Patil Polytechnic Khopoli, Raigad. She is pursuing M.E. in Electronics & Telecommunication from Saraswati College of Engineering, Mumbai University. She has seven years of teaching experience. Her research interest is in Image processing and Microwave.

.

Mrs. Sunita M. Jadhav is working as an Assistant Professor in Saraswati College of Engineering Kharghar, Navi mumbai, Mumbai University, India. She obtained M.E. (Electronics/Computer Application) in 2006. She received MBA (Production & Marketing) in 1995 and currently pursuing Ph.D. at JJTU, Jhunjhunu, Rajasthan, India. She has eleven years of teaching experience. Her research interest is in Image processing, Microprocessors & Microcontrollers. She has published number of papers in National and International Journals.