A Comparative Study of FFDNet and Curvelet Thresholding for Image Denoising

A Comparative Study of FFDNet and Curvelet Thresholding for Image Denoising

N. Pavan Srinivas

Department of Electronics and Communication Engineering

K L Educational Foundation Vijayawada, Andhra Pradesh

Praneeth Akkala

1. Pavan Sai

   Department of Electronics and Communication Engineering

   K L Educational Foundation Vijayawada, Andhra Pradesh

   Department of Electronics and Communication Engineering K L Educational Foundation

   Vijayawada, Andhra Pradesh

   Abstract: – Image denoising as an inverse problem plays an indispensable role as a pre-processing step for many high end computer vision and image processing applications. The applications of image denoising includes but not limited to: image restoration, visual tracking, image registration, image segmentation, and image classification, where retrieving the original image content from its raw noisy form is crucial for best results. While many algorithms have been proposed in the literature, the problem of image noise reduction stays an open challenge, especially in situations where the images are captured under low light conditions. In this paper, we study the inverse problem in-terms of the state-of-the-art methods that includes neural network based FFDNet and the dual domain approach of multiscale NLM filtering based Curvelet thresholding technique. The analysis indicates that these approaches have reached the limits of denoising, under low noise power, however the scope of denoising still require further study under high noise strength.

   Keywords: Curvelet Thresholding, Image denoising, NLM Filtering, Neural Network, FFDNet.

   1. INTRODUCTION

      Image denoising is preliminary task to perform on any image before doing the operation on it. During data acquisition and transmission noise is added. In image denoising, we remove the noise while retaining the important image features. This problem can be because of camera instruments, transmission medium, or discrete source of radiation. There are different algorithms to de-noise different noises. Most of the natural images have additive Gaussian noise added to them, Poisson noise, speckle noise are few other noises added to the list.

      There are spatial filters like mean and median filter to remove the noise from images, but the problem is that they reduce the noise, smoothen the image, and also they blur the edges. Therefore we use domain transform techniques, In this there are formats like Wavelet/Curvelet transform, it is a powerful tool as it has multi-resolution properties, With wavelets having popularity since last two decades, lot of algorithms were introduced and focus moved from spatial to frequency transformed domain.

      In this paper we show that, the latest neural network techniques will be better performing than the present existing thresholding techniques.

      1. FFDNet

         Because of the better performance and quality in the algorithm, discriminative learning strategies have been generally utilized in image denoising field. Despite the fact that, these techniques for the most part will in general get familiar with a specific model for each noise force and require multiple models for denoising images with different noise levels. They additionally need highlight to manage spatially variant noise, limiting their applications continuously denoising. To determine these issues, we present a quick and adaptable denoising convolutional neural network, named FFDNet, with a variable noise level guide as the info. The proposed FFDNet works on down sampled images, showing signs of improvement exchange off between derivation speed and denoising performance. In contrast to the current image denoise, FFDNet shows a few extra properties, including:

         Fig.1.The design of FFDNet for image denoising is proposed. The input image is reshaped to four sub-images, which are then contribution to the CNN layers together with a noise level map.

         • The method can handle huge band of noise (i.e., [0, 75]) effectively in only one network.

         • The capability to reduce noise varying spatially by creating a non-uniform noise map with levels.

           Many works on system generated and real-time noisy images are led to test FFDNet in examination with best in class denoisers. The outcomes show that FFDNet is better, progressively proficient and compelling, making it profoundly valuable for real-time de-noisng applications.

      2. Curvelet Transform

      Multi-resolution methods are mostly related to image processing, biomedical and computer vision, and scientific computing. The Curvelet transform is a multi-scale directional transform that permits a near optimal non-adaptive sparse representation of objects with edges. It has created a spike in the interest in the area of applied mathematics and signal processing over the years. As we see the recent applications in image/video processing, fluid mechanics, seismic/topological exploration, processing of partial different equations, and compressed sensing.

      One of the starter tasks in computer vision is to separate features from an image or a request for images. The features of an image can be points, curves, lines, edges, and surfaces. A given component is situated by position, bearing, scale, and other property parameters. The naval forces procedure, utilized in early vision for separating of such features, is linear filtering, which is additionally reflected in models utilized in biological visual frameworks, i.e., human visual movement detecting. Items at various levels can emerge out of unmistakable physical procedures. This watches out for the utilization of scale-space filtering process and multi-resolution wavelet change in this application. An essential task for computer vision is to get directional portrayals that catch anisotropic lines and edges while giving sparse decompositions.

   2. LITERATURE SURVEY

      In this section, we present a concise audit on the development of CNNs for image de-noising, SISR, JPEG image artifacts expulsion, and other image reclamation tasks. In particular, more exchanges are given to the pertinent chips away at expanding open field and joining FFDNet in CNNs.

      1. Image denoising

         From a decade ago, CNNs have been applied for image denoising. These crude techniques for the most part can't accomplish best in class denoising results. As of late, multi- layer perception (MLP) idea has been taken to take in the mapping from noise fix regarding clean pixel and got equivalent output with BM3D. By mix of residual learning

         with batch normalization, the Duncanmodel by Zhang et al. can be superior to innocent non-CNN based strategies. Mao et al. additionally propose including symmetric skip associations with FCN to improve de noising performance. For better connection among speed and performance, Zhang et al. Fig 1 shows the present a 7-layer FCN with dilated filtering. Santhanam et al. Introduce a recursively branched de- convolutional network (RBDN), where pooling/ unspooling is adopted to obtain and aggregate multi-context representation.

         Deep neural networks show better results for image restoration. The leading works include a multilayer perceptron for image de noising and a three-layer CNN for image. De convolution is gotten to save estimation cost and enliven derivation speed. Deep CNNs are designed to help SR precision. Dense relationship among various residual blocks are included.

      2. Curvelet performance

         In order to retrieve the image from the noise effected version and get the original image, is a big task in getting the output for image processing and computer vision. The characteristics of noise generally depend upon the type of sensor used, pixel dimensions, ISO, brightness levels, and exposure of the environment. Thereby in many real-time applications the type of noise is additive white Gaussian noise.

         (1)

         Here, In (1) y is the noise image, z is the original image and (0, 2) is the Gaussian noise of zero mean and 2 variance.

         The NLM filter is used as a denoising framework, To apply this, The noise need to be additive in analysis domain, it uses tight frames to represent integrable function f in Curvelet domain and will obey parsevals identity.

         f = , , o (2)

         here in (2) ,,o indicates the Curvelet basis function.

         Because of the linearity property NLM filter can be implemented for Curvelet in approximation and the finer scales, From experimentation we obtain the required number of decomposition levels (N_) for restoration of image using Curvelet transform at maximum peak signal to noise (PSNR) and structural similarity index(SSIM), for the noise levels, the minimum level of decomposition scales can be known.

         Fig.2. (a) Noisy Image with =20, (b) De-noised image, (c) Original image.

         Fig.3. (a) Barbara, (b)Boat, (c) Building, (d) Cameraman(e) Couple, (f) Gold hill, (g) House, (h) Lake, (i) Lena (j) Peppers

         Fig.4. The flow chart of Curvelet Thresholding

      3. Grayscale image denoising

         The image is initially broke down into three different scales from which low frequency and high frequency components can be analyzed, the noise in spatial scale is treated separately using approximations in multi-scale NLM filter. The reconstructed image from approximation sub-band are taken remaining scales are made zero, and also the noise in coarser scale is suppressed due to filtering. The fine grain noise is removed and the fine details, edge details, texture of image are restored by the NLM Filtering. Fig 2 shows the thresholding levels that are made to process the algorithm.

         For the testing of FFDNet, We used a predefined data set which is made of 10 test images, also for the testing of Curvelet values and the FFDNet values both the sets are resized to a fixed predefined size, The network has a set of 15 layers with defined links, It also has 64 channels, the range of noise levels are [0, 75], It also has a Training patch size of 70x70in Fig 3,

      4. Colour image Denoising

      In the case of Curvelet thresholding the best way to perform multichannel image de-noising is to use the same grayscale algorithm for each channel (RGB), separately. But, because of

      the heavy correlation between the RGB-color channels, The best way to do this, Is to shift to different colour space without correlation, previous works also shows that using of YCbCr or YUV color space will improve the performance by 1.1 dB that RGB color-space. Based on this the thresholding of the noise strengths is taken and transform is done.

      For the FFDNet color images, the pre trained model is used, we used the standard Kodak24 images dataset, The size of images was fixed and 12 layers were used for the algorithm, The number of channels used are 96 and the level of noise ranges from [0, 75], It also has Training patch size of 70×70.

   3. EXPERIMENTATION

      In order for these models to be tested for how they perform, several key requirements must be met. To begin, anyone who will be using these models will be using MATLAB R2017a, Cuda-8.0 ,cuDNN v-5.1, MatConvNet and Python 3 with pytorch, keras and tensorflow. Furthermore, A background knowledge of Convolutional Neural Network and understanding of Multi-level Wavelet- CNN model and Non-Local Recurrent Network model is necessary to compare the architectures.

   4. RESULTS AND DISCUSSION

      FFDNet and Curvelet transform are used here for image restoration. From this we could suggest that in future, there

      can be a technique to de-noise the images more efficiently by using both the methods combined. The parameters in which both the methods are compared are also same.

      Noise Levels (in dB)	PSNR		SSIM
      	CTu NLM	FFDNet	CTu NLM	FFDNet
      =10	35.682	37.314	0.9318	0.951
      =20	32.142	33.808	0.874	0.908
      =30	30.154	31.853	0.829	0.870
      =40	28.953	30.515	0.777	0.838
      =50	28.143	29.511	0.745	0.810
      =75	26.204	27.759	0.673	0.756

      Noise Levels (in dB)	PSNR		SSIM
      	CTu NLM	FFDNet	CTu NLM	FFDNet
      =10	35.682	37.314	0.9318	0.951
      =20	32.142	33.808	0.874	0.908
      =30	30.154	31.853	0.829	0.870
      =40	28.953	30.515	0.777	0.838
      =50	28.143	29.511	0.745	0.810
      =75	26.204	27.759	0.673	0.756

      TABLE I. COMPARISON OF RESULTS BETWEEN CTU NLM AND FFDNET FOR GREY SCALE IMAGES

   5. CONCLUSION

FFDNet and Curvelet transform are mostly used for image restoration. This paper presents a comparative study of FFDNet and Curvelet transform. From the above results Table I, Table II, we conclude that both the methods are equally beneficial for the restoration of an image for lower level noises.

TABLE II. COMPARISON OF RESULTS BETWEEN CTU NLM AND FFDNET FOR COLOR IMAGES

REFERENCES

1. Susant Kumar Panigrahi, Supratim Gupta, Prasanna K. Sahu, Curvelet-based multiscale denoising using non-local means & guided image filter, The Institute of Engineering Technology Image Processing, S1:006.

2. M. Lebrun, M. Colom, A. Buades, and J.-M. Morel, Secrets of image denoising cuisine, ActaNumerica, vol. 21, pp. 475576, 2012.

3. P. Milanfar, A tour of modern image filtering: New insights and methods, both practical and theoretical, IEEE Signal Processing Magazine, vol. 30, no. 1, pp. 106128 2013.

4. A. Buades, B. Coll, and J.-M. Morel, A review of image denoising algorithms, with a new one, Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 490530, 2005.

5. M. Zhang and B. K. Gunturk, Multiresolution bilateral filtering for image denoising, IEEE Transactions on image processing, vol. 17, no. 12, pp. 23242333, 2008.

6. K. Dabov, A. Foi, V Katkovnik, and K. Egiazarian, Image denoising by sparse 3-d transform-domain collaborative filtering, IEEE Transactions on image processing,vol. 16, no. 8, pp. 20802095, 2007.

7. H. C. Burger, Modelling and learning approaches to image denoising, Ph.D. disserta-tion, Universit¨atT¨ubingenT¨ubingen, 2013.

8. Y. Romano, M. Elad, and P. Milanfar, The little engine that could:Regularization by denoising (RED), submitted to SIAM Journal onImaging Sciences, 2016.

9. K. Zhang, W. Zuo, S. Gu, and L. Zhang, Learning deep CNN denoiserprior for image restoration, in IEEE Conference on Computer Visionand Pattern Recognition, 2017, pp. 39293938.

10. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, Imagedenoising using scale mixtures of gaussians in the wavelet domain, IEEE Transactions on Image processing, vol. 12, no. 11, pp. 13381351,2003.

11. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising bysparse 3-D transform-domain collaborative filtering, 3-D transform- domain collaborative filtering, IEEE Transactionson Image Processing, vol. 16, no. 8, pp. 20802095, 2007.

12. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, Non- localsparse models for image restoration, in IEEE International Conference on Computer Vision, 2009, pp. 22722279