Comparison of Various Thresholding Techniques of Image Denoising

Comparison of Various Thresholding Techniques of Image Denoising

Shivani Mupparaju 1, B Naga Venkata Satya Durga Jahnavi 2

Department of ECE, VNR VJIET, Hyderabad, A.P, India

Abstract

Denoising using wavelets attempts to remove the noise present in the signal while signal characteristics are preserved, regardless of its frequency content. It can be handled using three steps: a linear forward wavelet transform, nonlinear thresholding step and a linear inverse wavelet transform. Wavelet denoising is a lot different from smoothing; smoothing is used to remove the high frequencies and retains the lower frequencies. Wavelet shrinkage is a non-linear process and it is used to distinguish from entire linear denoising technique. Wavelet shrinkage depends on the choice of a thresholding parameter and the choice of how the threshold is determined, and the efficacy of denoising various techniques can be used for choosing denoising parameters and so far there is no best universal threshold determination technique. So, Various denoising techniques such as Sure Shrink, Bayes Shrink and Visu Shrink determines the best one for image denoising.

1. Introduction

   During acquisition and transmission, image denoising can used to remove the additive noise while keeping the important signal features. In the recent years wavelet thresholding and threshold selection for signal de- noising has gain more interest because wavelet gives an appropriate basis for separating noisy signal from the image signal. The motivation is the wavelet transform is good at compacting energy, the small coefficient are more likely due to noise and large coefficient due to important signal features . These small coefficients can be thresholded without affecting the main features of the image.

   A simple non-linear technique called 'thresholding' which operates on one wavelet coefficient at a time. Each coefficient is thresholded by comparing against threshold. And if the coefficient is smaller than the threshold, set it to zero; otherwise it is modified. Replacing the small coefficients by zero and applying inverse wavelet transform on the result may lead to

   reconstruction retaining back the essential signal characteristics and giving less noise image.

2. Objectives and Tools Employed

   1. Objective of the project

      The main objective of this paper is study various thresholding techniques such as Sure Shrink, Visu Shrink and Bayes Shrink and determine the best one for image denoising.

   2. Tools Used

      Software: MATLAB

3. Types of Noise

   1. Gaussian Noise

      Because of its mathematical tractability in both spatial and frequency domains, Gaussian (also called normal) noise models are used frequently in practice. In fact, this tractability is so convenient that it often results in Gaussian models being used in situations in which they are marginally applicable at best.

      The Probability density function of Gaussian random variable z, is given by:

      (1.1)

      where z represents intensity, is the mean (average) value of z , and is its standard deviation. The standard deviation squared , is called variance of z.

      3.2 Rayleigh Noise

      The probability density function of Rayleigh Noise is given by

      P(z) (z-a) for z >= a

      = 0 for z < a

      3.5. Uniform Noise

      The Probability density function is given by P(z) = if a <= z <= b

      = 0 otherwise (1.11)

      The mean and variance of this density are given by

      And

      (1.2)

      (1.3)

      The mean of this density function is given by And its variance by

4. Denoising

   (1.12)

   (1.13)

   1. Erlang (Gamma) Noise

      (1.4)

      In many cases, additive noise is evenly distributed over the frequency domain (i.e., white noise), whereas an image contains mostly low frequency information. The noise is a characteristic at high frequencies and its effects can be reduced using low-pass filter. So a

      The probability density function of Erlang noise is given by

      P(z) = for z >= 0

      = 0 for z <0 (1.5)

      Where the parameters are such that a > 0, b is a positive integer. The mean and variance of this density are given by

      (1.6)

      And

      (1.7)

   2. Exponential Noise

      The Probability density function of exponential noise is given by

      P(z) = for z >= 0

      = 0 for z < 0

      (1.8)

      Where a > 0. The mean and variance of this density function are

      (1.9)

      And

      (1.10)

      frequency filter or with a spatial filter can also be used. Often a spatial filter is preferably used, as it is computationally less expensive than a frequency filter. Denoising can be done in various domains and by using

      various methods

      1. Spatial domain

      2. Frequency domain

      3. Wavelet domain and

      4. Curvelet domain

5. Thresholding

1. Motivation for Wavelet thresholding

   The plot of wavelet coefficients suggests that small coefficients are decreased due to noise, while coefficients with a large absolute value carry more signal information. Replacing noisy coefficients (small coefficients below a certain threshold value) by zero and an inverse wavelet transform may lead to a reconstruction that has lesser noise. Stated more precisely, we are motivated to this thresholding idea based on the following assumptions:

   1. The deco-relating property of a wavelet transform creates a sparse signal: most coefficients, which are zero or close to zero when they are left untouched.

   2. Noise is spread out equally along all coefficients.

   3. The noise level is not too high so that we can distinguish the signal wavelet coefficients from the noisy ones.

   As it turns out, this method is indeed effective and thresholding is a simple and efficient method for noise reduction. Further, inserting zeros creates more sparsity in the wavelet domain and here we see a link between wavelet de-noising and compression.

2. Hard and soft thresholding:

   Hard and soft thresholding with threshold ¸ are defined as follows The hard thresholding operator is defined as D(U,) = U for all |U| >¸

   = 0 otherwise

   (2.1)

   The soft thresholding operator can be defined as D(U,) = sgn(U)max(0,U|-)

   (2.2)

   Hard threshold is a keep or kill procedure and is more intuitively appealing. The transfer function of the same is shown. The alternative, soft thresholding (whose transfer function is shown), shrinks coefficients above the threshold in absolute value. While hard thresholding may seem goodl, the continuity of soft thresholding has some advantages. Moreover, hard thresholding does not even work with some algorithms such as the GCV procedure. At times, pure noise coefficients may pass the hard threshold and appear as annoying blips in the output. Soft thresholding shrinks these false structures.

3. Threshold determination

   Threshold determination is an important in image denoising. A small threshold gives a result close to the input, but the result can still have the noise component. Whereas a large threshold, produces a signal with a large number of zero coefficients. This results in smooth noiseless signal.

   The effect of smoothness, however, destroys details and in image processing may cause blur and artifacts.

   To investigate the effect of threshold selection, we performed wavelet denoising usin hard and soft thresholds on four signals popular in wavelet literature: Blocks and Doppler.

   The setup is as follows:

   1. The original signals have length 2048.

   2. We step through the thresholds from 0 to 5 with steps of 0.2 and at each step denoise the four noisy signals by both hard and soft thresholding with that threshold.

   3. For each threshold, the MSE of the denoised signal is calculated.

   4. Repeat the above steps for different orthogonal bases, namely, Haar, Daubechies 2,4 and 8.

   Fig:Table 1

   The results are tabulated in the Table 1 and represents Blocks and Doppler of both hard and soft for different filters.

4. Comparison with Universal threshold:

   The threshold UNIV = 2ln N (N being the signal length, 2being the noise variance) is well known in wavelet literature as the Universal threshold. It is the optimum threshold and minimizes the cost function of the difference between the function and the soft threshold version of the same in the L2.In our case, N=2048, = 1, therefore theoretically,

   2ln(2048)(1) 3.905

   As seen from the table, the best empirical thresholds for both hard and soft thresholding are much lower than this value, independent of the wavelet used. It therefore seems that the universal threshold is not useful to determine a threshold. However, it is useful for obtain a starting value when nothing is known of the signal condition. One can surmise that the universal threshold may give a better estimate for the soft threshold if the numbers of samples are larger.

5. Image Denoising using Thresholding:

   An image is often corrupted by noise in its acquisition or transmission. The underlying concept of denoising in images is similar to the 1D case. The goal is to remove the noise while retaining the important signal features as much as possible.

   The noisy image is represented as a two-dimensional matrix {xij},i ,j=1,.N. The noisy version of the image is modeled as

   yij=xij+nij ij=1,.N. (2.3)

   Where {nij} are iid as N (0, 2). We can use the same principles of thresholding and shrinkage to achieve denoising as in 1-D signals. The problem again boils down to finding an optimal threshold such that the mean squared error between the signal and its estimate is minimized.

   The wavelet decomposition of an image is done as follows: In the first level of decomposition, the image is split into 4 sub bands, namely the HH, HL, LH and LL sub bands. The HH sub band gives the diagonal details of the image the HL sub band gives the horizontal features while the LH sub band represents the vertical structures. The LL sub band is the low- resolution residual consisting of low frequency components and it is this sub band, which is further split at higher levels of decomposition. The different

   methods for denoising we investigate differ only in the selection of the threshold. The basic procedure remains the same:

   1. Calculate the DWT of the image.

   2. Threshold the wavelet coefficients.

   3. Compute the IDWT to get the denoised estimate.

Moreover, it is also found to yield visually more pleasing images. Hard thresholding introduces artifacts in the recovered images.

The three thresholding techniques- Visu Shrink, Sure Shrink and Bayes Shrink and investigate their performance for denoising various standard images

1. VisuShrink: Visu Shrink was introduced by Donoho . It can be defined as 2log I where is the noise variance and I is the number of pixels in the image.The maximum of any I values can be given by N(0,2) with the probability approaching 1 as number of pixels in the given image increases. Therefore if it has high probability, a pure noise signal is calculated as being identically zero.

   However, for denoising images, Visu shrink is found to yield an overly smoothed estimate as seen. This is because the universal threshold (UT) is derived under the constraint that with high probability the estimate should be at least as smooth as the signal. The universal threshold is high for large values of I, killing many signal coefficients along with the noise. Thus, the threshold doesnt perform well at discontinuities in the signal.

2. Sure Shrink: Let = (i : i = 1,. d) be a length-d vector, and let x = {xi} (with xi distributed as N(¹i,1)) be multivariate normal observations with mean vector . Let = be a fixed estimate based on the observations x. SURE can be defined as Steins unbiased Risk Estimator). It is a special method for estimating the loss in an unbiased fashion is the soft threshold estimator. Steins result to get an unbiased estimate of the risk is applied. For an observed vector x is the set of noisy wavelet coefficients in a sub band we find out the threshold ts that minimizes SURE (t,x) ,i.e.

   (2.4)

   The above optimization problem is computationally straightforward. Without loss of generality, we can reorder x in order of increasing . Then on intervals

   of t that lie between two values of , SURE (t) is strictly increasing. Therefore the minimum value of ts is

3. Threshold Selection in Sparse Cases :The drawback of SURE in situations of extreme sparsity of the wavelet coefficients. In such cases the noise contributed to the SURE profile by the many coordinates at which the signal is zero swamps the information contributed to the SURE profile by the few coordinates where the signal is nonzero. Consequently,

   d

   d

   Sure Shrink uses a Hybrid scheme. The idea behind this hybrid scheme is that the losses while using an universal threshold, tend to be larger than SURE for dense situations, but much smaller for sparse cases .So the threshold is set to tF in dense situations and to tS in sparse situations. Thus the

   (4.12)

   Where and

   (2.5)

   being the thresholding operator.

   estimator in the hybrid method works as follows

4. SURE applied to image denoising: The wavelet decomposition of the noisy image is obtained. The SURE threshold is determined for each sub band using the above equations. We choose between this threshold and the universal threshold using the equation .The expressions and in the equations, given for = 1 have to suitably modified according to the noise variance and the variance of the coefficients in the sub band. The results obtained for the image lina (512*512pixels) using Sure Shrink are shown in results. The `Db4' wavelet was used with 4 levels of decomposition. Clearly, the results are much better than Visu Shrink. The sharp features of the image are retained and the MSE is considerably lower. This is because Sure Shrink is sub band adaptive- a separate threshold is computed for each detail sub band.

5. Bayes Shrink: In Bayes Shrink we determine the threshold for each subband assuming a Generalized Gaussian Distribution (GGD) . The GGD is given by

   x, x x

   x, x x

   GG (x) C( , ) exp[( , ) | x |]

   one of the data values there are only d values and the threshold can be obtained.

   x , 0

   (2.6)

   Where

   (3

   (3

   1 2

   x X

   x X

   , 1

   (1 )

6. Parameter Estimation to determine the Threshold:

   The GGD parameters, x and , need to be estimated to compute TB (x) . The noise variance 2 is estimated

   and

   C x ,

   . x ,

   21

   (2.7)

   (2.8)

   from the subband HH1 by the robust median estimator ,

   median | Yij |

   0.6745

   and

   (t) euut 1du

   (2.9)

   Yij subbandHH1

   (2.11)

   0

   Y X

   Y X

   The parameter x is the standard deviation and is the shape parameter It has been observed that with a shape parameter ranging from 0.5 to 1, we can describe the distribution of coefficients in a sub band fr a large set of natural images. Assuming such a distribution for the

   The parameter does not explicitly enter into the expression of TB (x). Therefore it suffices to estimate directly the signal standard deviation x. The observation gives Y = X + V, with X and V as independent of each other, hence

   wavelet coefficients, we empirically estimate and x for each sub band and try to find the threshold T which minimizes the Bayesian Risk, i.e., the expected value of the mean square error.

   2 2 2

   (2.12)

   x y x

   x y x

   T E( X X )2 E E

   X X 2

   Where

   2 is the variance of Y. Since Y is modeled

   Y

   Y

   Y

   Y

   as zero-mean, 2 can be found empirically by

   Where

   X = T(Y ); Y/X ~ N(x,2) and X ~ G Gx,. 1 n

   2 Y 2

   Y

   Y

   The optimal threshold T* is then given by

   n i, j1

   i, j

   T*(x, ¯) = arg min (T) (4.20)

   (2.13)

   It is a function of the parameters x and . Since there is no closed form solution for T*, numerical calculation is

   Where nXn is the size of the subband under consideration.

   X

   X

   2

   used to find its value.

   It is observed that the threshold value set by

   2

   Thus TB (x )

   (2.14)

   TB ( x )

   (2.10)

   Where

   x 2

   max 2 2 , 0

   (2.15)

   is very close to T*. The estimated threshold TB = =x is not only nearly optimal but also has an intuitive appeal. The normalized threshold, TB/. is inversely proportional to x, the standard deviation of X, and proportional to x, the noise standard deviation. When /x<<1, the signal is much stronger than the noise, Tb/

   x Y

   Y

   Y

   x

   x

   In the case that 2 > 2 ,

   B x

   B x

   T ( ) is , or, in practice,

   is taken to be zero, i.e,

   B x

   B x

   T ( ) = max| Yij |

   is chosen to be small in order to preserve most of the

   signal and remove some of the noise; when = x>>1, the noise dominates and the normalized threshold is chosen to be large to remove the noise which has

   and all coefficients are set to zero.

   To summarize, Bayes Shrink performs soft thresholding, sub band dependent threshold,

   X

   X

   2

   exacted the signal. Thus, this threshold choice adapts to both the signal and the noise characteristics as reflected in the parameters and x..

   TB (x )

   (2.16)

   The reconstruction using Bayes Shrink is smoother and more visually appealing than the one obtained using

   Sure Shrink. This not only validates the approximation of the wavelet Coefficients to the GGD but also justifies the approximation to the threshold to a value independent of .

   2.17)

   2.17)

7. Image Reconstruction:

Image reconstruction can be carried out by the following procedure:

1. Lets sample the data by a factor of two on all the four sub bands at the coarsest scale

2. Filter the sub bands in each dimension.

3. Sum the four filtered sub bands to reach the low- low sub band at the next finer scale.

This above process can be repeated until the image is fully reconstructed.

r2 F(r, )drd[ r /(2 )] (

Figure 2: Noisy Image

0 2 0

xp(|z z |)1J (r )J (r )d.

j

i

1

2

0

i

j

i

1

2

0

i

e

0

1. Results

   Figure 3: Visu Shrink

   Figure 1: Original Image

   Figure 4: SNR of Visu shrink

   Figure 5: Bayes Shrink

   Figure 6: SNR of Bayes shrink

   Figure 7: Sure Shrink

   Figure 8: SNR of SURE shrink

2. Conclusion

   We have seen that wavelet thresholding is an effective method of denoising noisy signals.Then we

   investigated many soft and hard thresholding schemes using Visu Shrink, Sure Shrink and Bayes Shrink for denoising images. It was found that sub band adaptive thresholding performs better than a universal thresholding. Among these, Bayes Shrink gave the best results. This validates the assumption that the Generalized Gaussian Distribution (GGD) is a very good model for the wavelet coefficient distribution in a sub band.

   An important point to be noted is that although Sure Shrink performed worse than Bayes Shrink, it adapts well to sharp discontinuities in the signal. This was not evident in the natural images that were used for testing purpose specially while comparing the performance of these algorithms on artificial images with discontinuities (such as medical images).

   Image denoising using thresholding by wavelets provides an extension for research into fast and robust multi-frame super resolution, exemplar-based inpainting and image feature extraction. Wavelet transform can also be used in the analysis and synthesis of multi-scale models of stochastic processes. The concept of wavelet transform finds an important application in speech coding, communications, radar, sonar, denoising, edge detection and feature detection

3. References

1. Digital Image Processing, 3rd edition, by Rafael C.Gonzalez, Richard E.Woods ,Pearson Publications

2. Digital Image Processing using MATLAB, 2nd edition, by Rafael C.Gonzalez,

3. Martin Vetterli , S Grace Chang, Bin Yu. Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, 9(9):15321546, Sep 2000.

4. David L Donoho. De-noising by soft thresholding. IEEE Transactions on Information Theory, 41(3):613 627, May 1995.

5. Iain M.Johnstone David L Donoho. Adapting to smoothness via wavelet shrinkage. Journal of the Statistical Association, 90(432):12001224, Dec 1995.

6. David L Donoho. Ideal spatial adaptation by wavelet shrinkage . Biometrika, 81(3):425455,August 1994.

7. Maarten Jansen. Noise Reduction by Wavelet Thresholding ,volume 161. Springer Verlag, United States of America, 1 edition, 2001.

8. Carl Taswell. The what, how and why of wavelet shrinkage denoising. Computing in Science and Engineering , pages 1219, May/June 2000.

9. Sachin D Ruikar and Dharmpal D Doye . Wavelet based image denoising technique . (IJACSA)

International Journal of Advanced Computer Science and Applications,Vol. 2, No.3, March 2011