Optimizing Image Stitching for Panoramic View

Optimizing Image Stitching for Panoramic View

Ridhi Arora Dr. Parvinder Singh

Dept. of Comp. Science & Engineering Dept. of Comp. Science & Engineering M.Tech Associate Professor

DCRUST Murthal DCRUST Murthal

Sonepat, India Sonepat, India

Abstract-: Image Stitching or mosaicing is an important aspect of research in the field of computer vision. It involves various techniques of joining images together to form a mosaic of high resolution. Stitching images generally require complete overlap in order to generate high resolution panoramas. As these panoramas become increasingly popular, there arises a need for the software to create mosaics. These mosaics are used for variety of applications like in digital maps and satellite photos. In recent years various techniques have been generated to do stitching in order to obtain data from images. Such techniques which have been used to successfully combine images are discussed in this paper.

Keywords: Image Stitching, Panorama, Feature based techniques, Direct based techniques, image blending, Image registration, Image calibration.

1. INTRODUCTION

   Image processing has been the most exploited field of computer vision and computer graphics. It has been in use since the age of digital computers. Earlier, to capture a large region, various different images were taken at different angle with different cameras and then manually pierced together. But as photography evolved with time, need for panoramas increased. This led to the development of improved computer software for combining images together. As photography evolved, many people gained interest in it and started to apply different methods that could achieve considerable results. This led to the formation of mosaics which is also called Image Stitching. Therefore stitching is the process of combining multiple images with overlapping region and a wider field of view. It is generally divided into three parts: calibration, registration and blending. In calibration, images are acquired from different sources and decisions are taken for position and acquisition according to the resultant panorama. After calibration comes registration of images. It is done to find the translation between images which align them. Finally a blending algorithm is chosen to appropriately stitch the aligned images together. A good blending of pictures is required for better results.

   The use of image stitching in real time applications has proved to be a challenging field for image processing. Image stitching has found variety of applications in microscopy, video conferencing, video matting, fluorography, 3D image reconstruction, texture synthesis, video indexing, super resolution, scene completion, video compression, satellite imaging, and several medical applications. Stitched images (mosaics) are also used in

   topographic mapping and stenography [1]. Stenography is a technique that is used to hide information in images. Various other operations can also be applied on images being stitched [2][3][4][5].

   For videos, additional challenges are imposed on image stitching. As videos require motion of pictures with varying intensities so, camera feature based techniques which aim to determine a relationship zoom, and to visualize dynamic events impose between the images is used which poses additional challenges to image stitching.

   This paper is organized as follows: in section 2, we present main steps required for image stitching. Furthermore, section 3 presents a detailed summary on approaches to tackle image stitching. In section 4 we present the discussion on various approaches to image mosaicing and in section 5, future of image stitching is discussed.

2. IMAGE STITCHING MODEL

Image Stitching is the process of merging images to form a high resolution panoramic image. Various steps are being followed for joining images together.

The very first step in generating a panoramic image is image acquisition. It is mainly the step of calibrating images. It requires selection of position and acquisition of images. Here, a decision is to be made on the type of the resultant panorama. According to the resultant panoramic image, various acquisition methods are used. After successful acquisition of images, registration is performed. Registration is done on the overlapping region to find the translation which aligns them. It is done in two ways:

1. Feature matching

2. RANSAC estimation

   Feature matching is the integration of direct and feature-based methods. Direct method is generally used with large overlapping regions, small translations and rotations while feature-based is used over small overlapping region [12]. Feature-based uses corner detection and matching, SIFT, SURF etc. SIFT is robust feature matching technique which extracts 128-dimentional feature vector for every pixel [9]. As it extracts 128- dimentional it becomes very complex. So, to reduce its complexity, SURF came into picture which improves efficiency with a similar invariant nature to rotation and transformation by making grids around the key-point detected and making histograms around it [10]. The count of histograms increased according to the degree of the

   gradient weighted by Gaussian. This average Gaussian is responsible for clarity in mosaic. After we have feature matching of all the pictures, we now make feature sets according to RANSAC estimation [8]. It eliminates the need to extract features from unnecessary points in the

   images and thus reduces computation and improves efficiency. RANSAC estimation is an important step in stitching as it reduces computation and improves performance and efficiency.

   Image Acquisition

   Registration

   Feature Matching

   RANSAC estimation

   Image merging

   Global Alignment

   Blending and Composition

   Output Panorama

   Fig1. Typical flow diagram of Image Stitching

   These processes play an important role in stitching of images. Once the feature sets are computed, the images need to be stitched together along the overlapping regions using a blending algorithm. Prior to blending, alignment is necessary. Alignment minimizes the miss-registration that

   removes double edges and blurring. Now the aligned images are stitched together using a blending algorithm. Blending can be done by projecting images onto some composition surface like cylindrical, planer, fisheye etc.

   Fig2. Input images with varying intensities

   1. Input image I1 (b) Input image I2

      Fig3. Various blending composition surfaces on which the aligned images are projected

      (a) Cylindrical (b) spherical (c) fisheye (d) stereographical ( Panini )

      Author	Technique			Process
      Parul M. Jain Shandliya	, Prof. Vijaya K.	Computer vision mosaicing		Initially, when images were used as maps, these images were used in background and also mapped as shadows upon synthetic objects for performing complex computations.
      Richard Szeliski	Feathering Pyramid Blending			In feathering, mosaic image is created from the input weighted images. Their weighted coefficients, known as alpha mask, vary as a function of their distance with the seam in between. Here images are joined according to their frequencies bands. Different frequency bands are combined with different alpha masks. To remove edge duplications, images with lower frequencies are merged together in narrow region with fine details. Images joined in different resolution helps in mapping textures at different scales.
      Richard Szeliski	Optical seam			This method searches for a curve in the overlapping region with the minimal color difference between them. Then each input image is copied on both sides of seam. If the difference of the images with seam on the curve is zero, then no seam gradients are produced. But in applications which used contrasting images this application is not used because of vast color differences among them.
      Anat Levin, Assaf Zomet, Shmuel Peleg, and Yair Weiss	GIST: Gradient Domain Image stitching (GIST 1, GIST 2)			This method brings in a new way of obtaining mosaic images, i-e. using cost functions[3]. These cost functions are optimized to get greater similarity of mosaic with the input images and also the least visibility of seam. These cost functions reduces the photometric and geometric inconsistencies significantly. Mainly, the mosaic image is produced from the gradients of input images and later compared with them in order to find similarity artifacts. This gradient matching is a complex phenomenon but it proves to be effective in not introducing double edges in mosaics.
      David G. Lowe	SIFT			Scale Invariant Feature Transform is a very effective and robust algorithm developed by Lowe in 1999 [9]. It extracts invariant features from images which are invariant to scale, rotation, illumination, noise etc. The problem is that it extracts 128-dimentional feature vector which causes huge computation in stitching process.
      Herbert Bay, Tinne Tuytelaars, and Luc Van Gool	SURF			Speed Up Robust Features algorithm is achieved by utilizing integral images for computing image convolutions, using a Hessian matrix-based measure for the detector and a distribution-based descriptor which both are simplified and approximate [9].
      Chris Harris and Mike Stephens	Harris Corner method			This method of feature extraction is robust and invariant to scale, rotation, illumination and noise. In this method, gradient of every pixel is calculated and if the absolute gradient values in two directions are both great then that pixel is regarded as the corner and process continues.
      Ioana S. Sevcenco, Peter J. Hampton and Pan Agathoklis	Haar Wavelet Method	2D	Integration	Haar Wavelet has proved to be an effective method to produce mosaic image from the gradients of input images [5]. The gradients for the mosaic image are created by blending the individual gradients of the source images in the overlapped region. Later, mosaic image will be reconstructed from the gradient data of the input image gradients.

      Author	Technique			Process
      Parul M. Jain Shandliya	, Prof. Vijaya K.	Computer vision mosaicing		Initially, when images were used as maps, these images were used in background and also mapped as shadows upon synthetic objects for performing complex computations.
      Richard Szeliski	Feathering Pyramid Blending			In feathering, mosaic image is created from the input weighted images. Their weighted coefficients, known as alpha mask, vary as a function of their distance with the seam in between. Here images are joined according to their frequencies bands. Different frequency bands are combined with different alpha masks. To remove edge duplications, images with lower frequencies are merged together in narrow region with fine details. Images joined in different resolution helps in mapping textures at different scales.
      Richard Szeliski	Optical seam			This method searches for a curve in the overlapping region with the minimal color difference between them. Then each input image is copied on both sides of seam. If the difference of the images with seam on the curve is zero, then no seam gradients are produced. But in applications which used contrasting images this application is not used because of vast color differences among them.
      Anat Levin, Assaf Zomet, Shmuel Peleg, and Yair Weiss	GIST: Gradient Domain Image stitching (GIST 1, GIST 2)			This method brings in a new way of obtaining mosaic images, i-e. using cost functions[3]. These cost functions are optimized to get greater similarity of mosaic with the input images and also the least visibility of seam. These cost functions reduces the photometric and geometric inconsistencies significantly. Mainly, the mosaic image is produced from the gradients of input images and later compared with them in order to find similarity artifacts. This gradient matching is a complex phenomenon but it proves to be effective in not introducing double edges in mosaics.
      David G. Lowe	SIFT			Scale Invariant Feature Transform is a very effective and robust algorithm developed by Lowe in 1999 [9]. It extracts invariant features from images which are invariant to scale, rotation, illumination, noise etc. The problem is that it extracts 128-dimentional feature vector which causes huge computation in stitching process.
      Herbert Bay, Tinne Tuytelaars, and Luc Van Gool	SURF			Speed Up Robust Features algorithm is achieved by utilizing integral images for computing image convolutions, using a Hessian matrix-based measure for the detector and a distribution-based descriptor which both are simplified and approximate [9].
      Chris Harris and Mike Stephens	Harris Corner method			This method of feature extraction is robust and invariant to scale, rotation, illumination and noise. In this method, gradient of every pixel is calculated and if the absolute gradient values in two directions are both great then that pixel is regarded as the corner and process continues.
      Ioana S. Sevcenco, Peter J. Hampton and Pan Agathoklis	Haar Wavelet Method	2D	Integration	Haar Wavelet has proved to be an effective method to produce mosaic image from the gradients of input images [5]. The gradients for the mosaic image are created by blending the individual gradients of the source images in the overlapped region. Later, mosaic image will be reconstructed from the gradient data of the input image gradients.

3. RELATED WORK Table 1 Review of techniques involved

4. DISCUSSION AND CONCLUSION

   Various techniques of stitching evolve the enhancements of photographs making them usable in variety of applications like video conferencing, video stabilization, 3D image reconstruction, video summarization, and several medical applications. Haar wavelet technique discussed above uses gradient method which provides solution to local adjustments in image by making it an optimization problem. Energy factor could also be taken to optimize the process. This type of mosaic development from Haar wavelet can be enhanced so that complex images can be combined easily in future. Other techniques are also there which provide seamless stitching of images. For summary, it would suffice that various techniques of stitching like gradient-based, optical seam, Harris corner method etc. have evolved the enhancement of pictures making them used extensively in todays research and development.

5. FUTURE SCOPE

   Stitching with th simple normal images is easy as they can be readily aligned for joining, but this process can be extended with MRI images which will provide high definition view along with immense clarity.

6. COMPETING INTERESTS

The authors declare that there are no competing interests.

REFERENCES

1. P. Singh, S. Batra, and H. R. Sharma, Steganographic Methods Based on Digital Logic, pp. 157162, 2007.

2. A. V. Krishna, Performance Evaluation of New Encryption Algorithms with Emphasis on Probabilistic Encryption & Time Stamp in Network Security, vol. 3, no. 5, pp. 3946, 2009.

3. D. Hooda and P. Singh, A Comprehensive Survey of Video Encryption Algorithms, Ijca, vol. 59, no. 1, pp. 1419, 2012.

4. R. Singh, P. Singh, and M. Duhan, An Effective Implementation of Security Based Algorithmic Approach in Mobile Adhoc Networks, Hum.-centric Comput. Inf. Sci., vol. 4, no. 1, pp. 7:17:14, 2014.

5. D. Singh and D. Sethi, Reduction of Noise from Speech Signal using Haar and Biorthogonal Wavelet, vol. 7109, pp. 263269, 2011.

6. K. Pranoti, K.R.Singh, A Technical Analysis of Image Stitching Algorithm, International Journal of Computer Science and Information Technologies, Vol. 6(1), pp. 284-288, 2015

7. A. Shikha, A Review on Image Stitching and its Different Methods, International Journal of Advanced Research in Computer Science and Software Engineering, Vol. 5(5), May 2015, pp. 299-303

8. L. Anat, Z. Assaf, P. Shmuel, W. Yair, Seamless Image Stitching in the Gradient Domain, School of Computer Science and Engineering , The Hebrew University of Jerusalem

9. Brown, L.G., A survey of image registration techniques, ACM Computing Surveys, 24(4), 325-376, 1992.

10. I. S. Sevcenco, P. J. Hampton, P. Agathoklis, Seamless stitching of images based on a Haar Wavelet Haar wavelet 2D integration method, 17th International Conference for Digital Signal Processing (DSP 2011), 2011, July 6-8, 2011, Piscataway, NJ, USA, 2011, pp. 6.

11. K.Shashank, N.SivaChaitanya, G.Manikanta, Ch.N.V.Balaji, and V.V.S.Murthy, Survey and Review Over Image Alignment and Stitching Methods, International Journal of Electronics and Communication Technology, vol. 5 (3), 2014.

12. A. Ebtsam, E. Mohammed, E. Hazem, "Image Stitching based on Feature Extraction Techniques, International Journal of Computer Applications (0975 8887) Volume 99 No.6,

    August 2014

13. Abdelfatah R., Omer H.,Automatic Seamless Image Stitching, Computer Engineering and Intelligent Systems, vol. 4, no. 11, pp. 7-13, 2013

14. Ze-lang Wang, Fang-hua Yan, Ya-yuZheng. An Adaptive Uniform Distribution SURF for Image Stitching, IEEE 6th International conference on Image and Signal Processing (CISP), vol. 2, pp. 888-892, 2013

15. G.R. Licu et. al. E. Sybingco , Image Stitching of Dissimilar Images, 8th IEEE International Conference Humanoid, Nanotechnology, Information Technology, IEEE 9-12 December 2015 Water Front Hotel, Cebu, Philippines

16. http://www.gutenberg.us/articles/photo_stitching

17. P. M. Jain, V.K. Shandliya, A Review Paper on Various Approaches for Image Mosaicing, International Journal of Computational Engineering Research, vol 3 (4)