Panoramic Image Mosaicing

Panoramic Image Mosaicing

Mrs. Hetal M.Patel1, Asst. Prof. Pinal. J. Patel 2, Asst. Prof. Mr. Sandip G. Patel 3

1 C.S.E. Department, Government College of Engineering, Gandhinagar Gujarat Technology University, Gujarat, India.

2 C.S.E. Department, Government College of Engineering, Gandhinagar Gujarat Technology University, Gujarat, India

3 Shrimad Rajchandra Institute of Management and Computer Application Uka Tarsadiya University,Gujarat, India.

Abstract

Image Mosaicing is the process of combining two or more images of the same scene into one image and generate panorama of high resolution image. In this paper , we have used SIFT (Scale Invariant Feature Transform) based algorithm to generate panoramic image. SIFT is used to extract feature points by searching over all scale and image locations, then the descriptors defined on the keypoint neighborhood are computed. Next step for homography calculation we have used RANSAC algorithm which eliminate flase feature point pairs. Finally to get smooth image we used blending and completed image mosaic process generating panoramic image .

Keywords: Mosaicing, Panorama, Image blending, Warping

1. Introduction

   Image mosaicing in normal term also called image stitching where we stitch two or more images and generate one single image. Nowadays almost all digital cameras come with the feature of image panorama. Still it is not giving the very nice result and lots of improvement can be done. So this field of image processing also required efforts and still many new algorithm can be developed. A panoramic image gives people a realistic experience about the location and is useful in a wide variety of fields. Image mosaicing is used in many

   applications like video conferencing, from multiple node create 3D view, astronomy, telemedicine, cartoons, virtual museums, architectural Walkthroughs.

   Different steps required for doing image mosaicing are feature extraction, registration, stitching (merging images) and blending. Image registration is the process of aligning two or more images taken from one point or same thing is captured from different point. There are many different algorithms available to do image registration. Main purpose of doing image registration is to create geometric correspondence between images so that we can compare images and apply other steps appropriately. Some known registration algorithms are SUSAN, Harris, SIFT. SUSAN and Harris algorithms are sensitive to scale and noise. In our proposed scheme we have used SIFT , which is having rotation and scale invariant properties. Also not sensitive to noise[8].

   After registration next is stitching. In stitching or image merging, all images are transformed according to registration parameter on single big canvas and final step is to do image blending which make the transition from one image to another image smoother so that joint between two images can be removed. RANSAC is used to estimate the translational parameter. To blend the two images feathering blending we have used.

   Paper is organized as follows. In section II we have describe SIFT Algorithm for image registration.

   Section III contains description of RANSAC Algorithm to delete wrongly match points. In Section IV we have described Image Blending and Stitching technique as well as discussed Warping of image on cylinder. In section V we present our proposed scheme for panoramic image generation. Section VI consists of experimental result and Finally section VII contain conclusion.

2. Image Registration using SIFT [1]

   1. Feature Extraction

      Scale Invariant Feature Transform termed as SIFT is used to identify locations and scales that can be repeatably assigned under different views of the same object. Detecting locations that are invariant to scale changes of the image can be accomplished by searching for stable feature across all possible scales, using continuous function of scale known as scale space. The scale space of an image is defined as a function,

      L(x, y, ) , that is produced from the convolution of a variable – scale Gaussian, G( x, y , ), with the input image, I( x, y ):

      L( x, y , ) = G( x, y , ) * I( x, y ) (1)

      To efficiently detect stable keypoint locations in scale space, David G. Low proposed using scale-space extrema in the difference of Gaussian function convolved with the image:

      D( x, y, ) = (G( x, y ,k ) G( x, y , )) * I( x, y ) (2)

      = L( x, y ,k ) L( x, y , )

      The difference-of-Gaussian function will have a strong response along edges, even is the location along the edge is poorly determined and therefore unstable to small amounts of noise.

      In Order to detect local maximum and minimum of D(x, y, ), each sample point is compared to its eight neighbors in the same scale and nine neighbors in the scale above and below (Fig. 2).

      It is selected only if it is larger than all of these neighbors or smaller than all of them.

      To reject edge points, a 2X2 Hessian matrix computed at the location and scale of the each keypoint is:

      (3)

      Rejecting the keypoints for which,

      (4)

      Here,

      tr (H) = DXX + DYY,

      det (H) = DXX DYY (DXY)2

      Fig. 2 : Maxima and minima detection in the difference-of- Gaussian image [1]

   2. Feature Descriptor

      By assigning a consistent orientation to each keypoint based on local image properties, the keypoint descriptor can be represented relative to this orientation and therefore achieve invariance to image rotation. The gradient magnitude, m(x, y) ,and orientation, (x, y) , can be computed using pixel differences:

      m(x,y) =

      (5)

      (x, y) =

      (6)

      Fig. 1 : Difference Of Gaussian Pyramid[1]

      These samples are then accumulated into orientation histograms summarizing the contents over 4 X 4 sub regions. David G. Low proposed using a 4 X 4 array of histograms with 8 orientation bins in each. Therefore, the feature vector has 4 X 4 X 8 = 128 elements. The descriptor is formed from a vector containing the values

      of all the orientation histogram entries. Finally, the vector is normalized to unit length to reduce the effects of illumination change[8].

   3. Feature Matching

      We have used Lowe's initial keypoint matching method to find a list of pairs for two images. The best candidate match for each feature point is found by identifying its nearest neighbor in the data set of feature vectors from input image. The nearest neighbor is defined as the feature point with minimum Euclidean distance for the invariant descriptor vector [1]. However, the descriptor has a high dimension, matching the feature point by comparing the feature vector one by one is done using k-d tree[11].

3. Homography Calculation using RANSAC

   For deleting wrongly detected points RANSAC algorithm is used in image mosaicing. RANSAC is an abbreviation for "RANdom SAmple Consensus". It is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers . The algorithm was first published by Fischler and Bolles at SRI international in 1981[16].

   A basic assumption is that the data consists of "inliers", i.e., data whose distribution can be explained by some set of model parameters, and "outliers" which are data that do not fit the model. In addition to this, the data can be subject to noise. The outliers can come, e.g., from extreme values of the noise or from erroneous measurements or incorrect hypotheses about the interpretation of data. RANAC also assumes that, given a (usually small) set of inliers, there exists a procedure which can estimate the parameters of a model that optimally explains or fits this data.

   The idea is to repeatedly guess a set of model parameters using small subsets of correspondences that are drawn randomly from the input set . With a large number of draws, there is a high probability to draw a subset of correspondences that are part of the same model. After each subset draw, the model parameters for this subset are determined and the number of correspondences in that are consistent with these parameters is counted. The set of model parameters with the largest support is considered to be the correct homography [8].

4. Blending and Warping

   After computing best transformation, blend the input image together with the reference image. To reduce the visible artifacts blending is used. Warping of image is projection of panaromic image on any surface like cylinder, spherical surface.

   1. Image Blending

      Blending is applied to make seamless stitching. Two popular methods of blending the images are, One is called alpha blending also called feathering blending , which takes weighted average of two images. and another is Gaussian pyramid.

      In alpha blending ,the weighting function is usually a ramp. At the stitching line, the weight is half and half, while away from the stitching line one image is given more weights than the other. The cases that alpha blending works extremely well is when image pixels are well aligned to each other and the only difference between two images is the overall intensity shift. Alpha blending will merge two images seamlessly. However, if the images are not aligned well, the disagreements will show in the blended image [12].

   2. Cylindrical Warping

   Cylinderical projection , where the stitched image shows a 360° horizontal field of view and a limited vertical field of view. Panoramas in this projection are meant to be viewed as though the image is wrapped into a cylinder and viewed from within.

   To build a cylindrical panorama, a sequence of images is taken by a camera mounted on a leveled tripod. If the camera focal length or field of view is known, each perspective image can be warped into cylindrical coordinates. Two types of cylindrical warping are, Forward warping and Inverse warping. In forward warping the source image is mapped onto cylindrical surface, but it can holes in the destination image (because some pixels may never get mapped there). therefore we use inverse mapping where each pixel in the destination image is mapped to the source image. Since the mapping is unlikely to be exactly on the pixel values, bilinear interpolation is used to calculate the colors at the destination pixels.

5. Proposed Scheme

   Here , we have presented our proposed scheme for panoramic image generation.

   Step 1: Take sequence of images with far and near view.

   Step 2: Input the set of images in our program.

   Step 3: Project each image on to cylindrical co- ordinate system.

   Step 4: After projection, SIFT is applied to extract the features from each image.

   Step 5: Given the features of two image, find the list of matching pairs using Low's matching method.

   Step 6: Translation matrix T is calculated to translate one image to its previous image.

   Step 7: From given translational model T, outliers are removed using RANSAC algorithm and final translation matrix T is estimated.

   Step 8: Based on Translational parameter images are stitched together.

   Step 9: Next linear blending is applied to achieve a smooth transition between images.

   Step 10: Affine transformation is used for drift correction.

   Step 11: Finally, the panoramic image is cropped to remove the black edge because of cylindrical projection.

   Applying above steps to sequence of images we had generated panaroma of 360 degree or less than it.

6. Experimental Result

   For the experiment purpose, we have taken sequence of images. We applied our implementation on tested images as well as images captured by us. Following are the result of our experiment.

   Fig. 3 Sequence of Images for Input

   Fig. 4 Feature Extraction using SIFT

   Fig. 5 Homography Calculation using RANSAC

   Fig. 6 Blending

   Fig. 7 Final Panoramic Image after crop

7. Conclusion

   Due to the wide range of applications, image mosaicing is one of the important research area in the field of image processing. In this paper, to generate panaroma, our proposed system uses the rotation and scale invariant property of SIFT. From the experiment it is proved SIFT is robust to rotaion, scaling and noise. We have generated panaroma taking sequence of images to cover wide scene. We have projected our panaroma on cylindrical surface. Our generated panaroma can be used as an input to the virtual reality application as well as walkthrough application.

8. References

1. David G Lowe, "Distinctive image features from scale – invariant keypoints " International Journal of Computer Vision, 2204, 60(2) : 91 – 110.

2. D.I. Barnea, H.F. Silverman, A class of algorithms for fast digital registration, IEEE trans.comput., vol c-12.

3. C.D. Kuglin, D.C. hines, The phase correlation image alignment method, Proc IEEE 1975,pp.163-165.

4. Lisa, A survey of image registration techniques, ACM, 1992.

5. Yanli Wan, Zhenjiang Miao, Jing Chen, A Robust Image Mosaic Algorithm, 2008 Congress on Image and Signal Processing.

6. Deepak Jain, Gaurav Saxena, Vineet singh, Image Mosaicing using corner techniques, 2012 International Conference on Communication Systems and Network Technologies.

7. Yi Zhuang, Xinrong Hu, Jianchun Wang, The Implement of an Image Stitching Algorithm based on Feature Extraction, 2009 Second International Conference on Education Technology and Training.

8. Yang zhan-long Guo bao-long, IMAGE MOSAIC BASED ON SIFT, International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IEEE 2008.

9. S. M. Smith and J. M. Brady. "SUSAN – a new approach to low level image processing". International Journal of Computer Vision 1997, 23 (1): 4578.

10. C. Harris and M. Stephens, "A combined corner and edge detector". Proceedings of the 4th Alvey Vision Conference. pp. 147151,1988.

11. Andrew Moore. An introductory tutorial on k-d trees. Efficient Memory-based Learning for Robot Control, PhD.Thesis.No.209, University of Cambridge. 1991: Chapter 6.

12. Yining Deng and Tong Zhang."Generating Panorama Photos".SPIE Digital library doi:10.1117/12.513119 Wells, Sarah et al,. IATH Best Practices Guide to Digital Panoramic Photography. Retrieved 2008-06-01.2007.

13. Wells, Sarah et al,. IATH Best Practices Guide to Digital Panoramic Photography. Retrieved 2008-06-01.2007.

14. Hugin.sourceforge.net, hugin manual: Panini.Groups.google.com, hugin-ptx mailing list, December 29, 2008.

15. Groups.google.com, hugin-ptx mailing list, December 29, 2008.

16. Martin A. Fischler and Robert C. Bolles (June 1981). "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography". Comm. of the ACM 24 (6): 381 395. doi:10.1145/358669.358692

17. David A. Forsyth and Jean Ponce (2003). Computer Vision, a modern approach. Prentice Hall. ISBN 0-13-085198-1.

18. P.H.S. Torr and D.W. Murray (1997). "The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix". International Journal of Computer Vision 24 (3): 271300.doi:10.1023/A:1007927408552.