Implementation of Blind Separation based ME in Independent Component Analysis for fast Digital Image Stabilization

Implementation of Blind Separation based ME in Independent Component Analysis for fast Digital Image Stabilization

V.Kishore1, S V Rajtilak2, A. Vamseekrishna3

1Assistant professor, vignan institute of information technology, Guntur, A.P., India

2Post graduate scholar, Gitam University, Vizag. A.P., India

3Assistant professor, vignan institute of information technology, Guntur, A.P., India

Abstract

In this paper, an independentcomponent analysis (ICA)is proposed for image stabilization scheme. During capturing the image sequences using electronic products such as mobile phones and the cameras two independent motions occurs: 1 the hand jitter motion (high frequency motion) and the other is camera motion (ego motion). By using ICA, the information obtained from image sequence is deconvolve the hand jitter motion and the ego motion of the image sequence there by providing the stabilized image sequence. The simulation results show the deconvolution of two different motions, through simulations the performance of the proposed method is evaluated in providing the image stabilization.

Index Terms Motion estimation, Digital image stabilization (DIS), image sequence analysis, ego- motiondetermination, independent component analysis (ICA)

1. Introduction

   Over the past few years, there is a huge demand and interest in research field in image stabilization method which a usually used in digital imaging device. During capturing the image sequence or video, an unwanted motion occurs in the image sequence which is referred as jitter motion, so to remove the unwanted jitter motion image stabilization techniques are used. In many applications in image processing there is requirement of stabilized sequences for better performance.In an Intelligent transportation system integrated with the vision system utilizes the digital image stabilization for reducing the computational complexity in the algorithm [1-2]. DIS is used for improving the efficiency and the performance in the video communication system with high performance codec [3-4]. In medical images to remove the unwanted motion [5] proposed a stabilization scheme. For quality performance the [6-7] proposed a dedicated DIS scheme for video surveillance and the motion tracking applications.

   In order to extract an individual signal from a mixture of signal, a method called independent component analysis [8-9] is said to be used. In other words it split the signal into

   independentcomponents without any knowledge of the signal which is also said to be as blind source separation [10-11]. The physical process produces unrelated signal with a physically realistic assumption. . This assumption makes sure that the ICA can be applicable for various research fields. For analyzing and separating the speech signal in the application of electro encephalographic [8],[12], the ICA is proposed.For feature extraction and face recognition [14 -15] of images the ICA is proposed. In the application of medical brain imaging [16-17], ICA is proposed.

   In this paper, independent component analysis (ICA) is proposed for image stabilization scheme. For an image sequence to evaluate, we first apply the local motion vectors (LMVs) for calculating the motion in some regions by using the motion estimationalgorithm. By using the blind separation we evaluate the LMVs of the image sequence into independent components so as to deconvolve the camera motion from the unwanted sequence. . For deconvolution The ICA method is used. By further processing, two signals are identified namely the high frequency motion and the ego motion from the independent signal. After resolving the identification of the ego signal and high frequency motions signals. The energy and right sign is assigned for the ego motion which helps in providing the stabilized image sequence.

   This paper is organized as follows. In Sections II illustrates about the digital image stabilization and in section III, wedescribed a briefly about the ICA. In Section IV,we illustrate the implementation of the proposed method. InSection V Simulation results and the performanceevaluation of the proposed stabilization method is presented. Finally, we provide conclusions

2. Digital Image Stabilization (DIS) Stabilization of a video constitutes three stages namely local motion estimation, global motion estimation and the image compensation. In the first stage,with in a specific frame regions the LMVs are estimated which consist of unwanted and wanted

   motions of the image sequence. In terms of accuracy and computation time, the LMV stage leads the entire stabilization process. And the second stage is global motion estimation where precise discrimination should be performed among the estimated LMVs. A global motion vector (GMV) is determined fro each local vector. This global motion vector can be moving object motion or an error vector caused due irregular condition of unwanted jotter motion, global motion and the moving object motion.

   Moreover the estimated GMVs should be further classified into unwanted high frequency GMVs and the ego GMVs [18]. By using the final stage which is nothing but image compensation which helps in eliminating the unwanted motion from the input image there by resulting a stabilized image as an output of the given input image sequence.

3. Independent Component Analysis (ICA)

   ICA method is specifically meant for finding the components from the multidimensional data which means it helps in finding those components which are non-gaussian and statically independent.

   A. ICA Review

   The ICA model from [9] is given as

   = 1 1 + 2 2 +, . , (1) Where s represents the random vector whose elements are 1, 2 , . , similarly, x represents the random vector with elements 1 , 2 , . , and the

   represents the elements of the matrix A. The

   above mention mixing model can be written as

   x = As (2)

   In order to apply the ICA model for certain problems, based on the problem some assumptions and restrictions need to address.

   Primarily, The ICs are said to be statically independent. Suppose random variables

   1 , 2 , , are said to be independent only when the value on does not support to offer any kind of information on the value of for .

   By using the probability density function (pdf) the independence can be defined. Let us represent the joint pdf of by (1 , 2 , . , ) and the marginal pdf of by . then is said to independent only if joint pdf is factorable as given below eqn.(4)

   1 , 2 , . , = 1 1 , 2 2 , . (4)

   The second property that the ICs should have is non-Gaussian distribution. Usually in the ICA model the high order estimation is needed for the estimation. As it is impossible for the ICA model to perform if the observed variable are Gaussian distribution. Moreover it is unknown how the non gaussian distributions of ICs look like. The problem can be easy if the non gaussian distribution of ICs is known.The gaussian has the maximum entropy for the pdfs whose standard deviation value is fixed. Non Gaussian is measured by using negentropyapproximations.

   C. Ambiguities of ICA

   Some uncertainty or ambiguity is observed in the final mixed signal after applying the model. Some of the uncertainty is not important. Based on the application some ambiguity or uncertainty is required for post processing. The model with ICA extracts set of mixed signal components in an irregular order which is also said as permutation

   ambiguity or permutatio uncertainty.

   or as

   x = 1 ai (3)

   =

   =

   i/p

   LMV

   estimation

   LMV

   estimation

   GMV

   estimation

   GMV

   estimation

   Image compensation

   Image compensation

   Stabilized

   Where represents the column of Amatrix.

   image

   image

   from the definition is said as statically independent. The ICA model illustrates how the observed data cane be obtained by the process of mixing the components of . The independent components can be determined

   directly so they said to be as latent variables. Moreover, assume that the mixing matrix is said to be unknown. By using the random vector x, the vector s and the A matrix need to be estimated.

   ICA method is very similar to a method which is said to be as blind source separation. Where the source represents the original signal (IC signal) and the blind represents the knowing very little of the mixing matrix

   B. Restrictions in ICA

   Fig.1. Typical architecture of a DIS technique

   LMV

   estimation

   LMV

   estimation

   MVs de- convolution

   MVs de- convolution

   Unmixed signal

   Unmixed signal

   ICA

   i/p image

   Fig.2. Proposed architecture of DIS by ICA

   Since the mixing matrix and the source signals are unknown so one can change the order terms in the eqn. (3) and can call any of the ICs as the first one. Since the new matrix is said to be unknown and is solved by using the ICA algorithm.

   Stabilized image

   Moreover, upto the scaling factor the resultant energy of the ICs are identified. As the mixing matrix and the source signal is unknown, in any one of the source the scalar multiplier can be cancelled by same scalar of the related

   column of the matrix A.The above fact leads to the ambiguity sign, without affecting the model one can multiply an IC by -1.

4. Proposed Algorithm

   Fig.3 shows the proposed model is compared with the cocktail party problem [13]. In the proposed approach, the micro phones are scattered around the room and this represents the different frame regions and recorded samples symbolize as LMVs estimations in the selected frame regions.

   As shown in fig.2, in the proposedmethod first we apply the local motion estimation algorithm for estimating the LMVs and then by using blind separation categorize the independent components to de-convolve the ego-motionfrom the unwanted signal. In the last stage, from the mixed signal based on ambiguities we find two unmixed signals such as high frequency motion and the ego motion. After the permutation ambiguity is determined suitable energyand sign is assigned to the ego- motion vector which results in providing thestabilized image sequence.Where as in conventional DIS shown in Fig.1 in image compensation , the image compensation vector is obtained in which high frequency image sequence still holds the ego motion in the image sequence due to which performance is low in comparison to the proposed method.

   Fig.3. Proposed model compared to the cocktail- party problem.

   1. LMV Estimation

      As this step is very common in stabilization schemes any algorithm can be applied as shown in fig.1 and fig.2. For LMV estimation we apply algorithm proposed in [19]. This algorithm has combinational features such as both the compression and the stabilization, so the performance is very effective in terms of computational complexity and the accuracy.

      In an image, we chose two frame regions for deriving the LMVs. The LMV is selected from a frame region which is not included foreground region moving in various directions from the global motion. Many algorithms have been proposed for the detectionof sub image which is included in foreground or not [20].

      The best value of LMVs are obtained by matching each frame with the reference image if that match perfectly i.e. It means that the sum of absolute differences is said to be minimum in the quarter-pixel motion estimation, which is shown in fig.4.

      From the block diagram of fig.2 we search for best motion estimation match points in a grid. And then later we still; search for the best matches in half sample and quarter sample for further improvement in the best matches.

      With the broad search for best matches one can provide enhanced motion compensation performance but comes with a cost of increased complexity. With the increase in the search step size the performance of the gain reduces. By increasing the step size from half sample to quarter sample there is further enhancement in the gain.

      Fig.4. Integer, half-pixel, and quarter-pixel motion estimation

      Dual pattern refinement and multiple predictors are used for avoiding trapped into the local minima in the aforementioned algorithms and also helps in further enhancing the speed of motion estimation by applying early termination criteria.

   2. ICA of LMVs

      The following frames are time varying signal represented by (). The linear mixture of the ICs is represented by (). The LMVs are estimated between the linear mixture of ICs and the

      following frames of time varying signals. By removing the timing index .The LMVs are estimated with the linear mixture of ICs and the consecutive frame regions proper frame regions denoted by and the independent motion vectors represented by . For estimating the LMVs we define the weighing factor for each motion represented by defined as the amplitude of the each independent motion vector with respect to

      the related frame region. We interfere either the unwanted motion of high frequencies or ego motion, so i=2 is referred. So the resultant contribution of the mixture represented by and each motion represented by from (1) can be

      rewritten as

      = 1 1 + 2 2 for all (5) For = 2 frame regions, for different amplitude at each frame represented by and the independent motion vector of each frame represented by .So the LMV at each frame region of eqn. (5) represented as eqn.

      (6) and (7).

      1 = 11 1 + 12 2 (6)

      2 = 21 1 + 22 2 (7)

      From the two independent motion vectors along with its mixture , results in contributing of weighting factors represented by

      11 , 21 , and12 , 22 .

      The ICA used, is based on the central limit theorem which supports the mixture of independence and the non gaussianity [9].

      Lets consider x=Aand a weight vector represented by w. the ICs can be one defined as the linear product of x and w with a condition where any of the row of 1 is a

      = x = s (8) Here represents the source vector of linear

      combinations. With the help of central limit theorem we try to improve the non-gaussianity of in terms of w, which helps in estimating the one of the ICs in x.

      Firstly we calculate the first un-mixing vector w or first IC. And later by applying the learning rule, where after each iteration the previous estimated vector is kept orthogonal by using gramschmidt orthogonalization scheme proposed in [21]. Before applying ICA, This means that the data is pre-whitened.

      Kurtosis with fourth order is used for measuring the non gaussianity of a random variable. With zero mean the kurtosis of the random variables given by the formula shown

      in eqn. (9)

      = 4 3 2 2 (10) The pre-whitening of the data information guarantees that the source signal are uncorrelated i.e. orthonormal with a unit variance and later by maximizing the kurtosis we find the find angle of wT x,which means that we are finding the angle of the most non Gaussian component.

      For maximizing the kurtosis we use Newton type algorithm proposed in [22]. By using Lagrange multipliers the following fixed point algorithm is derived.

      + z z 3 3w (11)

      The algorithm is illustrated briefly

      1. Firstly pre-whiten the data , i.e.,

         = V .

      2. Initiate with the random initial vector represnted by = 1.

         3) Updating the + z z 3

         3w

         1. Later normalizing the w+ w+ w+

         2. Repeat step 3 until it converges.

         As only two independent motion vectors are estimated there is no need to keep the new estimated component in orthogonal with the previous component.

   3. Processing of the Unmixed Signals

   In this section we focus on addressing the solution for sign, magnitude uncertainty or ambiguity and the permutation.

   Firstly, for the given two unmixed signals we try to recognize into unwanted motion and the ego motion.The unwanted motion is subjugated by higher frequencycomponents. And the ego motion is represented by the smooth trajectory. This certifies that the two unmixed signal into unwanted motion and the ego motion. By this the permutation uncertainty or ambiguity is solved. Now we need to address the magnitude uncertainties and the sign in order to map the sources in the observation space [23]. One can delete or eliminate ant scale deformation which is achieved by the unmixing matrix w by mapping the individual sources back on the

   2 2

   2 2

   = { 4}

   ({ } )

   3 (9)

   space of the LMV regions. Since permutation ambiguity is performed in two source two

   For the random variable , the 2 and 4 represents the second and the fourth moments

   of the variable.A simple principle of the

   output case. From the eqn. (6) and (7) we obtained the following analysis

   1 = 11 12 × 1 (12)

   kurtosis is, for non Gaussian variables the

   2

   21 22

   2

   kurtosis is non zero where as for the Gaussian variables the kurtosis is zero. Suppose if the

   The signal are represented by X1 andX2 where as the1 and2 represents the signal in the observed

   signal sub Gaussian then its value becomes negative and its values decreases where as if

   frame regions.

   X1 =

   11 X = 12 (13)

   1 2

   2

   1 2

   2

   the signals said to be more super Gaussian then its said to be positive and its value

   As a result

   21 22

   X = X1 + X2 (14)

   increase in order. The following equation is obtained by multiply with positive squared variances of 2 2 and is given as

   After finding the solution for the permutation problem and estimating the un-mixing matrix,

   now we represent the w matrix as (A )1 where is a diagonal matrix with scaling elements initiated from the algorithm and the resultant output is separated and scaled and is given as

   1 = wx = (A )1As =1 s = 1 1 (15)

   components. The frame regions are of 8*8 pixels size.

   The obtained LMVs are shown in fig.6 (a) and (b) for both frame regions. After evaluating the LMVs then it is further processed by ICA for de- convolution operation.

   2

   2 2

   lmv for region 1

   10

   lmv for region 2

   10

   As we now know the un-mixing matrix w, by moving the separated signal to the frame regions one can able to cancel or eliminate the faulty scaling, i.e. analytically we are calculating the 1 and 2

   (1)11

   8 8

   6 6

   vertial displacement

   vertial displacement

   vertical displacement

   vertical displacement

   4 4

   2 2

   0 0

   -2 -2

   -4 -4

   -6 -6

   -8 -8

   1 =

   1 21

   1

   0 10 20 30 40 50 60 70 80 90 100

   frame

   0 10 20 30 40 50 60 70 80 90 100

   frame

   = 11 1

   21 1 1 1

   = 11 (16)

   1

   1

   21

   (a) (b)

   Fig.6. Obtained LMVs (vertical component) for

   2

   = (1)12

   1 22 2

   frame regions (a) frame region 1 and (b) frame region 2 of fig.5 (a) and (b)respectively.

   12 2

   12

   deconvolved motion vector 1

   deconvolved motion vector 2

   =

   2 2 =

   2

   (17) 2 2

   22 2 22

   1.5

   1.5

   In this way, by mapping the signals into the frame region domain we are able to preserve the unmixed signal and also by removing the sign and the magnitude.

5. Simulation Results and Evaluation

   1

   vertical displacement

   vertical displacement

   0.5

   0

   -0.5

   -1

   -1.5

   -2

   0 10 20 30 40 50 60 70 80 90 100

   frame

   1

   vertical isplacement

   vertical isplacement

   0.5

   0

   -0.5

   -1

   -1.5

   -2

   -2.5

   -3

   0 10 20 30 40 50 60 70 80 90 100

   frame

   Simulations have been carried out by using the proposed algorithm. For image stabilization architecture and evaluation procedures have been presented. We evaluate the performance of the two schemes. In this simulation part we considered the vertical displacements. One can consider the horizontal movements as the procedure is same for the vertical and the horizontal, displacements.

   In the simulation, a non stabilized image sequence is considered. And then to this image sequence we use the application of ICA for de-convolving the unwanted motion and the ego motion from the non stabilized image sequence. From the sample image sequence a totalof 100is used a sequence and a

   1. (b)

      Fig.7.(a) and (b) De-convolved motion vectorsfor

      (a) motion vector 1 and (b) motion vector 2

      Fig. 7 (a) and (b) illustrates two unmixed signal de- convolution operation is done by using blind separation. These two signals represent two independent motion vectors. At this stage one cannot differentiate the characterization of the unmixed signal due to uncertainties or ambiguities. So firstly we need to recognize them into unwanted motions and the ego motions. After recognizing the motions appropriate amplitude and sign need to be assigned.

      frame region represented by the square in the corners of Fig.5 (a) and (b).

      8

      6

      4

      vertical displacement

      vertical displacement

      2

      0

      -2

      -4

      -6

      -8

      -10

      final motion after processing

      vertical diplacement

      vertical diplacement

      0 10 20 30 40 50 60 70 80 90 100

      frame

      final motion ater processing

      10

      8

      6

      4

      2

      0

      -2

      -4

      -6

      0 10 20 30 40 50 60 70 80 90 100

      frame

      1. (b)

         Fig.5. Experimental results for vertical are displacement. (a) And (b) Sample frame and the regions used for obtaining the LMVs.

         These (a) and (b) are frame regions which are used for acquiring the LMVs in the vertical

         1. (b)

      Fig.8. (a) and (b) final motions after processing

      After assigning the amplitude and the sign to the motions its is further processed, we obtain the final motion the fig.8(a) and (b) shows the final motions of the image sequence shows the compensated image sequence of two frame regions in the vertical displacement.

6. Conclusion

In this paper, a new DIS scheme based onICA has been proposed. From the image sequence the ICA uses the obtained information and de-convolvesinto unwanted motion and the ego-motion. Based on the independence property the unwanted motion and the ego motion, the ICA helps in deconvoluting the motions. Simulation results shows that the ICA application for deconvoluting the two different motions is successful and also shows that the performance of the proposed scheme reveals better performance inimage stabilization.

References

1. Y. Tyan, S. Liao, and H. Chen,Video stabilization for a camcordermounted on a moving vehicle, IEEE Trans. Veh. Technol., vol. 53, no. 6,pp. 16361648, Nov. 2004.

2. J. Jin, Z. Zhu, and G. Xu, A stable vision system for moving vehicles,IEEE Trans. Intell. Transp. Syst., vol. 1, no. 1, pp. 3239, Mar. 2000.

3. A. Amanatiadis and I. Andreadis, An integrated architecture for adaptiveimage stabilization in zooming operation, IEEE Trans. Consum.Electron., vol. 54,no. 2, pp. 600608, May 2008.

4. H. Chen, C. Liang, Y. Peng, and H. Chang, Integration of digital stabilizerwith video codec for digital video cameras, IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 7, pp. 801813, Jul. 2007.

5. R. Zoroofi, Y. Sato, S. Tamura, and H. Naito, An improved method forMRI artifact correction due to translational motion in the imaging plane,IEEE Trans. Med. Imag., vol. 14, no. 3, pp. 471479, Sep. 1995.

6. A. Censi, A. Fusiello, and V. Roberto, Image stabilization by featurestracking, in Proc. Int. Conf. Image Anal.Process., 1999, pp. 665667.

7. L. Marcenaro, G. Vernazza, and C. Regazzoni, Image stabilizationalgorithms for video- surveillance applications, in Proc. Int. Conf. ImageProcess., 2001, vol. 1, pp. 349352.

8. A. Hyvärinen and E. Oja, Independent component analysis:Algorithms and applications, Neural Netw., vol. 13, no. 4/5, pp. 411430,Jun. 2000.

9. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis.New York: Wiley, 2001.

10. A. Bell and T. Sejnowski, An information- maximization approach toblind separation and blind deconvolution, Neural Comput., vol. 7, no. 6,pp. 11291159, Nov. 1995.

11. J. Cardoso and P. Cnrs, Blind signal separation: Statistical principles,Proc. IEEE, vol. 86, no. 10, pp. 20092025, Oct. 1998.

12. S. Makeig, A. Bell, T. Jung, and T. Sejnowski, Independent componentanalysis of electroencephalographic data, in Advances in Neural InformationProcessing Systems, vol. 8. Cambridge, MA: MIT Press, 1996,pp. 145151.

13. Y. Ainhoren, S. Engelberg, and S. Friedman, The cocktail partyproblem, IEEE Instrum.Meas.Mag., vol. 11, no. 3, pp. 4448, Jun. 2008.

14. O. Déniz, M. Castrillón, and M. Hernández, Face recognition usingindependent component analysis and support vector machines, Pattern Recognit.Lett., vol. 24, no. 13, pp. 21532157, Sep. 2003.

15. M. Bartlett, J. Movellan, and T. Sejnowski, Face recognition by independentcomponent analysis, IEEE Trans. Neural Netw., vol. 13, no. 6,pp. 14501464, Nov. 2002.

16. M. McKeown, S.Makeig, G. Brown, T. Jung,

    S. Kindermann, A. Bell, andT. Sejnowski, Analysis of fMRI data by blind separation into independentspatial components, Hum. Brain Mapp., vol. 6, no. 3, pp. 160188,1998.

17. R. Vigario, J. Sarela, V. Jousmiki, M. Hamalainen, and E. Oja, Independentcomponent approach to the analysis of EEG and MEG recordings,IEEE Trans. Biomed. Eng., vol. 47, no. 5, pp. 589593, May 2000

18. Y. Matsushita, E. Heung, and Y. Shum, Full- frame video stabilizationwith motion inpainting, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28,no. 7, pp. 11501163, Jul. 2006.

2003.

1. C. Stauffer and W. Grimson, Adaptive background mixture models forreal-time tracking,inProc. IEEE Comput. Soc. Conf. Comput. Vis.PatternRecog., 1999, vol. 2, pp. 246 252.

2. T. Moon and W. Stirling, Mathematical Methods and Algorithms forSignal Processing. Englewood Cliffs, NJ: Prentice-Hall, 2000.

3. A. Hyvarinen and E. Oja, A fast fixed-point algorithm for independentcomponent analysis, Neural Comput., vol. 9, no. 7, pp. 14831492,Oct. 1997.

4. J.-F. Cardoso, Multidimensional independent component analysis, inProc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1998, vol. 4pp. 19411944.