Efficient Video Stabilization Using SURF Features & Filtering

Efficient Video Stabilization Using SURF Features & Filtering

NCIETM – 2017 Conference Proceedings

Kawaljeet Kour1, Deepti Ahlawat2

Department of Electronic and Communication Engineering NCCE, Israna, Haryana, India1,2

Abstract Video stabilization techniques have gathered a great interest in recent years. With handheld camera, motion and vibration are difficult to be avoided, so we have a need of that algorithm that gives a high quality of video. A framework of video stabilization is based on point feature extraction. Video stabilization is a process to remove the unwanted motion variation from video. This paper present three operation motion estimation, motion filtering, motion correction. Global motion is related with the motion of background i.e. remove with the help of Kalman filter. In this method feature points are extracted from the input video based on the Speeded Up Robust Feature (SURF). Random Samples Consensus (RANSAC) is used to remove the local motion. Weighted Least Square (WLS) algorithm is used to remove global motion and finally Kalman filter to remove unwanted motion. Experimental result show that proposed technique provides great deal of stabilization and good robustness.

Index Terms: SURF, Kalman Global motion

1. INTRODUCTION

   The increasing availability of micro digital cameras and mobile phones has attracted everybody to photography. The video stabilization technique is a convenient technique that is mechanically executed in high-end digital cameras as a special function to minimize hand-shaking motion vibrations. The video stabilization technique can implement by using software and hardware device. The video stabilization system can be grouped into three types (1) electronic image stabilizer (EIS) (2) optical image stabilizer (OIS) (3) Digital image stabilization (DIS) [1]. Both EIS and OIS required hardware system that put restrictions on them. In contrast DIS worked without hardware updates. Methods used for the motion estimation can be divided into two grouped (1) intensity based motion estimation (2) feature based motion estimation. Feature based motion estimation is more error free then the intensity based motion estimation. Two feature point extraction algorithms are (1) SIFT (Scale Invariant Feature Transform) (2) SURF (Speeded Up Robust Features).RANSAC Random Samples Consensus) is normally used to eliminate local. motion vectors and false correspondences. SFM (structure from motion) can be used to redesign the scene in 3D. However, thereare two demerits of using RANSAN and SFM: (firstly, RANSAC can only discard

   feature points on fast moving objects. If the object is moving with very slow speed in contrast to the background, RANSAC cannot debar feature points accurately. Secondly, SFM can lead to overkill. In this paper main contribution is to develop motion estimation algorithm for video stabilization.

2. RELATED WORK

   Video stabilization procedure have been deliberated for a long period of time and attracted even great attentiveness in recent years. Uomori et al [2] progressed an automatic image-stabilizing technique for camcorders; make use of only digital signal processing. Kinugasa et al. [3] on the basis of scanning region selection of imager have realized a dense electronic image stabilizer. However, the stabilization rate becomes very bad at high-level frequency. Paik et al [4] presented a DIS for video cameras. It is made up by an edge detection module, a motion detection module and a digital zooming module. The preferred DIS system is model mainly for reducing the hardware in a video camera device. Censi et al [5] extracted and tracked corner features points in order to calculate global motion. With regard to some image transformations, such as scaling and rotation the features are not strong. To overcome the difficulty in scaling and rotation, SIFT features [6], and PCA-SIFT [7] are being mainly used for calculating the global motion[8-11]. Recently, Wang et al. [12] proposed a DIS algorithm locate on feature point tracking. They used the Kanade-Lucas-Tomasi (KLT) tracker to calculate the global motion between two successive frames. For speed up the further tracking process the motion prediction by the Kalman filter (KF) is incorporated into the KLT tracker. Amanatiadis et al.

   [13] proposed a novel digital-image stabilization method based on independent component analysis (ICA). In this process ICA and data from the image frame sequence are deconvolved and unwanted motion from the frame sequence can be removed. Huang et al [14] proposed a algorithm to stabilize blurring video for vehicular applications based on feature point analysis.

3. PROPOSED WORK

   The video stabilization algorithm consist of following steps: reading the frame memory, Extraction of feature point and matching of feature point, determine the local motion by RANSAC, Global motion calculation based on particle

   filtering, intentional and unintentional motion calculation by Kalman filtering, illustrated in Fig 1. Here we discuss various step of video stabilization algorithm. Extraction of feature point and matching of feature point .In this by using SURF and SIFT , selecting feature point for motion estimation is very important because unstable feature cause unwanted motion SURF is a image interest point detector, first presented by Herbert SURF is Bay et al [15].RANSAC based on local motion estimation RANSAC is used to eliminate the outlier

   NCIETM – 2017 Conference Proceedings

   Start

   Reading frame memory

   Intentional motion calculation

   Fig2. SURF correspondences filtering by RANSAC

   Extraction of feature point and matching

   Unintentional motion recompense

   Global motion calculation based on

   Determine the local motion by RANSAC

   Last frame

   Stop

   Fig1:Flow chart of video stabilization algorithm

   feature points. RANSAC decide inliers and outlier depending upon the input data fits the model or not. After the kth iterations, the result that has minimal outliers is used as the origin value of the parameters in the affine transform model. The corresponding filtered result obtain by RANSAC is shown in Fig 2.

   Intentional and unintentional motion is come under the global motion. We only want to compensate the transformation created by unintentional camera moves; transformations due to intentional motion should be recognised. We used the Kalman filter to determine the intentional motion of the camera.

4. EXPERIMENTAL RESULTS

   In this paper we purposed a efficient video stabilization method using RANSAC, SURF and filtering. The key insight of this paper is to estimate the feature point and estimate the local motion using RANSAC and SURF, global motion by filtering. By using the K iterations of RANSAC error is minimized. To calculate the performance of the algorithm, we

   adopted ITF (Inter-frame Transformation Fidelity) [11] measure used in equation 1:

   ITF = 1/N frame k=1Nframe PSNR (k) (1)

   N frame no of video frames. PSNR (k) is peak signal to noise ratio defined by equation 2:

   PSNR (k) =10log10 Imax/MSE (k) (2)

   IMAX is maximum pixel intensity and MSE (k) is mean square error between successive frames.

   We compared our technique with other one approach based on particle filtering, which are presented by Yang et al. [10]. We chose SURF in this series of experiment as the feature is used in the two algorithms. The total number of feature points in a frame sequence and the number of particles are both 30. In Yangs algorithm the total number of particle is 30.

   	Original	Yangs	Proposed
   Sequence 1	17.03	20.15	20.75
   Sequence 2	15.01	19.72	20.38
   Sequence 3	17.64	21.21	21.15
   Sequence 4	15.02	16.27	17.04
   Sequence 5	16.07	19.41	20.03
   Sequence 6	18.24	19.93	20.81
   Sequence 7	15.63	18.11	18.96
   Sequence 8	14.95	18.20	18.62

   Table 1: ITF results of the three algorithms

5. CONCLUSION

In this paper we propose a efficient video stabilization method based on some algorithm and particle filtering. The key insight this paper is to extracting the feature point removing the local motion and global motion that contain the intentional and unintentional motion. Kalman filter are used to obtain better global motion estimation. Experiments have confirmed the effectiveness and robustness of the purposed algorithm.

REFERENCES

1. W. H. Cho and K. S. Hong, A Fast CIS Still Image Stabilization Method without Parallax and Moving Object Problems, IEEE Trans. On Consumer Electron. vol. 54, no. 2, pp.197-205, May.2008.

2. K. Uomori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Kitamura Automatic image stabilizing system by full-digital signal processing, IEEE Trans. on Consumer Electron., vol. 36, no. 3, pp. 510-519, Aug. 1990.

3. T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T. Imaide, Electronic image stabilizer for video camera use, IEEE Trans. on Consumer Electron., vol. 36, no. 3, pp. 520-525, Aug. 1990.

4. J. K. Paik, Y. C. Park, and S. W. Park, An edge detection approach to digital image stabilization based on tri-state adaptive linear neurons, IEEE Trans. Consumer Electron., vol. 37, no. 3, pp. 521- 530, Aug. 1991.

5. A. Censi, A. Fusiello, and V. Roberto. Image stabilization byfeatures tracking, International Conference on Image Analysis and Processing, pp.665- 667, Sep. 1999.

6. D. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no.2, pp.91-110, 2004. [1] Christopher Geyer, Marci Meingast, and Shankar Sastry, Geometric models of rolling-shutter cameras, IEEE Workshop on Omnidirectional Vision 2005.

7. Y. Ke, and R. Sukthankar, PCA-SIFT: A More DistinctivRepresentation for Local Image Descriptors International Conference on Computer Vision and Pattern Recognition, vol.2, pp.506-513, Jun. 2004.

   NCIETM – 2017 Conference Proceedings

8. O. Urhan and S. Erturk, Single sub-image matching based locomplexity motion estimation for digital image stabilization using constrained one-bit transform, IEEE Trans. on Consumer Electron., vol. 52, no. 4, pp. 1275-1279, Nov. 2006.

9. H. Okuda, M. Hashimoto, K. Sumi, and S. Kaneko, Optimum motion estimation algorithms for fast and robust digital image stabilization, IEEE Trans. on Consumer Electron., vol. 51, no. 1, pp. 276-280, Feb. 2006.

10. J. L. Yang, D. Schonfeld, and M. Mohamed, Robust Video Stabilization Based on Particle Filter Tracking of Projected Camera Motion, IEEE Trans. Circuits and Systems for Video Technology, vol.19, no.7, pp.945-954, July.2009.

11. S. Yao, G. Parthasarathy, and D. Thyagaraju, Video stabilization using principal component analysis and scale invariant feature transform in particle filter framework, IEEE Trans. Consumer Electron, vol.55, no.3, pp.1714- 1721, August. 2009.

12. C. T. Wang, J. H. Kim, and K. Y. Byun, Robust digitalimage stabilization using the Kalman filter, IEEE Trans. Consumer Electron, vol.55, no.1, pp.6-14, Feb. 2009.

13. A. A. Amanatiadis, I. Andreadis, Digital Image Stabilization by Independent Component Analysis, IEEE Trans. Instrumentation and Measurement, vol.59, no.7, pp.1755 – 1763, July. 2010.

14. K. Y. Huang, Y. M. Tsai, C. C. Tsai, L. G. Chen, Video stabilization for vehicular applications using SURF-like descriptor and KD-tree, 2010 17th IEEE International Conference on Image Processing (ICIP 2010), pp.3517- 3520, Sept. 2010.

15. H. Bay, T. Tuytelaars and L. V. Gool, Surf: Speeded Up Robust Features, Computer Vision and Image Understanding (CVIU), vol.110, no.3, pp.346-359, 2008.