A Review of Non-Linear Direct Multi-Scale Image Enhancement Techniques

A Review of Non-Linear Direct Multi-Scale Image Enhancement Techniques

Mirza Ahmed Ahsan

M. Tech. Scholar,

Department of Electronics & Telecommunication, Shri Shankaracharya Group of Institutions, Faculty of Engineering and Technology (SSCET) Bhilai, India

Ravi Mishra

Senior Assistant Professor, Department of Electrical and Electronics, Shri Shankaracharya Group of Institutions,

Faculty of Engineering and Technology (SSCET) Bhilai, India

Abstract- Image enhancement improves the quality of an image to an extent so that the resultant image gives more information to the observer. Image enhancement procedures are application specific. Depending upon the application, various image enhancement techniques are available. The image enhancement technique which is implemented for the enhancement of satellite image may not be suitable for enhancement of X-ray images. Therefore, to design any new technique for image enhancement, it is now necessary to study various enhancement techniques which are evolved over the years. In this paper, we compare the performances of the well- known image enhancement techniques which are based on nonlinear direct multiscale transforms.

Keywords – Image enhancement, direct image enhancement, human visual system (HVS), multiscale methods.

1. INTRODUCTION

   The aim of image enhancement is to modify images in such a way that the visual information contained in the image is improved for human perception to a specific application [1], [5], [7]. The performance of image enhancement is always judged subjectively [2]. There is no specific set of criteria which can universally define an ideal enhancement for all circumstances or requirements, so many image enhancement techniques have been proposed [5], [7]. Comparison of image processing techniques is critically important in deciding which algorithm, method or scale to use for enhanced image assessment [3]. Image enhancement is required for many essential areas and applications such as vision, remote sensing, autonomous navigation, dynamic scene analysis, biomedical image analysis, and other image processing fields [5], [6].

   Image enhancement techniques are classified as direct enhancement or indirect enhancement technique. In direct enhancement techniques, the contrast measured is defined and the contrast is enhanced by modification of image contrast directly [5], [17]. Indirect image enhancement techniques enhance the image without measuring the contrast [5].

   Two types of non-linear direct multi-scale techniques have been used for image enhancement: The Laplacian Pyramid (LP) and wavelet methods [4].

   The image enhancement techniques are based on mathematical formulations, but the choice of suitable

   technique depends upon the human analysis and intuition. Hence understanding the HVS is very important [11]. In contemporary times, various models of the HVS have been used for image enhancement. In one method attempt is made to model the transfer functions of the parts of the HVS, such as the visual cortex, optical nerve, and so forth. This method then attempts to realize filters which recreate these processes to model human vision [9], [10]. Another method uses a single channel to model the entire HVS and then processing the image with a global algorithm [10].

   The aim of HVS-based direct multiscale image enhancement techniques is to emulate the way in which the HVS discriminates between useful and useless data [2]. The important characteristics of HVS are: sensitivity to contrast changes, varying sensitivity to stimulus at different spatial frequencies, masking, which refers to the inability to detect a stimulus on a spatially or temporally complex background [5], [25].

2. HUMAN VISUAL SISTEM

   The HVS is an excellent image processor capable of detecting and recognizing the image information, it is only natural to bridge the gap between the psychophysics attributes and the way in which images are represented and manipulated [5], [12]-[15].

   For a large range of background intensities, the HVS is sensitive to relative, and not absolute, luminance changes [5], [16]. This phenomenon is known as the luminance masking (LM) characteristic of HVS [5]. Another important characteristic of HVS is that it is sensitive to relative changes in contrast. This feature is known as contrast masking (CM) [5].

   Image enhancement techniques which use the characteristic of luminance masking are discrete wavelet transform (DWT) [17]-[19] and LP transform [20]. These techniques are suitable in the enhancement of radiographies and detection of breast cancer in mammograms [21].

   Image enhancement techniques which use both, the LM and CM characteristic of HVS are: single scale Just Noticeable Difference (JND) models [22], [23]. JND algorithms are suitable for the compression and watermarking applications. Another technique includes parametric edge detection algorithm which improves the performance of standard Canny edge detector [16], [24], [26]. While in

   multiscale image enhancement techniques based on the LM and CM characteristics are Laplacian pyramid and the some wavelet based techniques.

   In the technique of luminance as well as contrast masking, first the image gradient is measured. This gradient is then masked by the background luminance, yielding the LM gradient. And then, the LM gradient is masked by a narrow activity map, which is a function of the LM gradient, resulting in the luminance and contrast-masked (LCM) gradient [5].

3. CHARACTERISTICS OF HUMAN VISUAL SISTEM

   An important step in a direct image enhancement approach is to create a suitable performance measure. The improvement found in the resulting images after enhancement is often very difficult to calculate. This problem becomes more noticeable when the enhancement algorithms are parametric, and one needs to choose the best parameters, to choose the best transformation among a class of unitary transforms, or to automate the image enhancement procedures. The problem becomes especially complicated when an image enhancement procedure is used as a preprocessing step for other image processing purposes such as object detection, classification, and recognition [6].

   There is no universal measure which can specify both the objective and subjective validity of the enhancement method [27]. However, in [28], the three necessary characteristics of a performance measurement are given. First, it must measure the desired characteristics in some holistic way. In our case, this is contrast masking and luminance masking. Second, it must show a proportional relationship between increase and decrease of this characteristic. Finally, it must have some resulting characteristic to find optimal points.

   In this section, important characteristics of the HVS are reviewed. The extent of the discussion is by no means an all encompassing explanation of the inner workings of the HVS and its many subtleties. Here, key concepts are established which will be pertinent in developing HVS-inspired tools for image enhancement. The HVS can be regarded as a multi- scale device, and analyzing images of their many scales therefore emulates the early stages of image formation by the HVS. Moreover, the contrast which is perceived by the HVS is both a function of the local background luminance and local activity. These characteristics will later be integrated into the proposed multi-scale contrast measures and transforms, which are ultimately utilized to achieve direct image enhancement.

   1. Luminance Masking

      The HVS perceives relative luminance changes for a large range of background intensity values [29]. The degree to which the HVS is sensitive to relative, and not absolute, luminance differences varies with background illumination. This response has been characterized by a piecewise function which divides contrast sensitivity into four regions, namely the dark, Devries- Rose, Weber, and saturation regions, and defines the dependency of perceived contrast on the background illumination for each region [6], [13], [30]. In this case, the contrast is not expressed as a ratio of a

      difference and an average, but the difference of logarithms is consistent with the LM phenomena in that the contrast measure is less sensitive to the case of the higher base luminance.

   2. Contrast Masking

      The HVS is sensitive not only to relative changes in luminance, but also to relative changes in contrast [16]. This CM phenomenon of HVS is one in which the visibility of a certain stimulus is reduced due to the presence of another one. This is to say that if a certain pattern is placed near no other stimuli, a given amount of contrast is perceived by the human eye, and when this pattern is surrounded or superimposed by other stimuli, the perceived contrast of the pattern decreases. An example of this spatial masking effect is that the HVS is susceptible to additive white noise in smooth areas of an image than in regions of high contrast which contain more details [12], [22], [24], [31]. In this case, the underlying true image signal reduces the visibility of the noise based on its local image content. In general, the CM effect is a function of spatial frequency, where the signal threshold elevation is maximal when the signal and masker have the same frequency.

   3. Higher Level Perception Factors

   Higher level perception factors are: attention, eye movements and our different objection ability to different types of coding artifacts [25]. Human only possess high visual acuity over a small area of viewing, and our acuity drops off rapidly in the periphery. Some areas of images are also more important to us and distortion in these important regions is more objectionable than distortions in background region [25].

4. MULTISCALE IMAGE ENHANCEMENT TECHNIQUES

   Recently, various multiscale algorithms have been proposed, where the image is split up into a larger number of frequency channels, which can then be processed separately. In the field of medical image processing, multiscale methods have been used for many purposes, e.g., in the context of segmentation [32], registration [33], noise reduction [34], or compression of images [35][37]. Mostly, medical applications used wavelet methods for the multiscale decomposition of the signal [4]. Two multiscale-methods are the Laplacian Pyramid and the wavelet methods.

   The LP was introduced by Burt and Adelson in the context of compression of images [38]. It has the advantage that the image is only expanded to 4/3 of the original size and that the same (small) filter kernel can be used for all pyramid levels. The image is filtered with a small kernel. In each filter step, the previous low-pass image is smoothed by the small kernel and sub-sampled by a factor of two to give the next low-pass image. This new low-pass image is up sampled again by inserting zeros after each pixel and smoothed once more with the small kernel before it is subtracted from the previous low- pass image. The sequence of low-pass images is termed a Gaussian Pyramid, while the sequence of the subtracted (bandpass) images is termed an LP. For enhancement of

   images by means of LP decomposition, the bandpass images are mapped by a nonlinear function [5].

   The wavelet transform transforms the signal from time domain to wavelet domain. The new domain consists of more complicated basis functions called wavelets or mother wavelets. The basic idea of the wavelet is to analyze according to scale. The DWT allows splitting the signal in two parts: the high frequency part and the low frequency part. The high frequency part consists of an edge component.

   SWT is a novel method for the fusion of spatially registered images and image sequences, based on a shift invariant wavelet transform. The shift invariant fusion scheme is highly desirable for the fusion of image sequences. The SWT method outperforms the standard wavelet fusion scheme in both the fusion of still images and image sequences [45].

5. PERFORMANCE OF VARIOUS IMAGE ENHANCEMENT TECHNIQUES

   In this section, the study of the evaluation of image quality by various techniques and comparison of their results is performed. Generally, there are several ways how to evaluate image quality. Key approaches are: subjective testing, objective testing and image quality evaluation using a HVS. The subjective testing is based on human perception, the objective testing on a mathematical computing and the human vision models on mathematical modelling of the human vision of the properties of human perception.

   This work compares the most commonly used multiscale image enhancement techniques. LP uses differences between successive scales of a Gaussian pyramid to provide a multi- scale representation for an image [31], [38]. With nonlinear Multiscale enhancement based on the LP, we obtained very satisfactory results in two clinical trials [39]. There are reasons why one might expect that wavelet-based enhancement could be even more powerful than the LP: perfect decomposition due to orthogonality of the wavelet bases, direction sensitivity, and superior noise-reduction potential [4].

   LP based image enhancement techniques are applied to X- ray images in general [40], [39], and wavelet-based methods are generally used in the context of mammography [41][43], although there are also some isolated applications to magnetic resonance (MR) and computed tomography (CT) images [41] or chest radiographs [44].

   Discrete wavelet DWT is able to provide perfect reconstruction while using critical sampling. Each analysis stage consists of filtering along rows, downsampling along columns, filtering along columns, and downsampling along rows in order to generate the approximation coefficient [5].

   We find that image enhancement based on the wavelet transforms suffers from one serious drawbackthe introduction of visible artifacts when large structures are enhanced strongly. On the other hand, the LP allows a smooth enhancement of larger structures, such that visible artifacts can be avoided. Only for the enhancement of very small details, for deploying applications or compression of

   images, the wavelet transform may have some advantages over the LP [4].

   Enhancement based on the wavelet transforms suffers from another drawback: the lack of directionality. Lack of directional selectivity greatly complicates modeling and processing of image [46].

   The DWT is shift-variant due to the down-sampling step which they employ. Therefore, the alteration of transform coefficients may introduce artifacts when processed using the DWT. As it is desired that artifacts not be introduced due to any further processing, stationary wavelet transforms (SWT) was consequently formulated [5].

   The SWT is a shift-invariant; over-complete wavelet transform which attempts to reduce artifact effects of the DWT by upsampling the analysis filters rather than down- sampling the approximation coefficient sub-bands at each level of decomposition [5], [45]. However, because the filter outputs are not decimated, and remain the same dimension of the original image regardless of the decomposition level, its high level of redundancy makes it highly memory intensive.

   The paper compares the direct multiscale image enhancement techniques on the basis of directionality, phase information, artifactual problem. At the same time, we studied which technique is how much memory intensive.

   TABLE I

   COMPARISON OF VARIOUS TECHNIQUES

   Directionality

   Features	LP	DWT	SWT
   No	Some	Some
   Phase Information	Less	Less	Less
   Artifacting Problem	No	Yes	No
   Memory Intensive	Less	Least	Most

6. CONLUSION

The comparative study of some well-known image enhancement techniques, performed in this work, which permits recognition of weaknesses and strengths of the techniques and also their applicability restrictions for different specific conditions. In this paper, we compared direct image enhancement techniques based on characteristics of HVS. The techniques which are compared are LP, DWT and SWT [5].

We believe the technique for direct image enhancement based on characteristics of HVS including LP, DWT, and SWT can be extended so that we get more of the directionality, the phase information, and other parameters.

REFERENCES

1. M. Kumar and S. Dass, A total variation-based algorithm for pixel- level image fusion, IEEE Trans. Image Process., vol. 18, no. 9, pp.2137-2143, Sep. 2009.

2. E. Wharton, S. Agaian, and K. Panetta, A logarithmic measure of image enhancement, in SPIE , Defense Security Symp., Orlando, FL, Apr. 20, 2006, p. 62500P.

3. Z. Liu, E. Blasch, Z. Xue, J. Zhao, R. Laganiere, and W. Wu,

   Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study, IEEE Trans. Pattern Analysis and Machine Intelligence., vol. 34, no. 1, pp.94-109, Jan. 2012.

4. S. Dippel, M. Stahl, R. Wiemker, and T. Blaffert, Multiscale contrast enhancement for radiographies: Laplacian pyramid versus fast wavelet transform, IEEE Trans. Med. Imaging., vol. 21, no. 4, pp.343-353, Apr. 2002.

5. S. Nercessian, K. Panetta, and S. Agaian, Non-linear direct multi-scale image enhancement based on the luminance and contrast masking characteristics of human visual system, IEEE Trans. Image Process., vol. 22, no. 9, pp.3549-3561, Sep. 2013.

6. K. Panetta, E. Wharton, and S. Agaian, Human visual system based image enhancement and logarithmic contrast measure, IEEE Trans. Syst. , Man, Cybern. B, Cybern., vol. 38, no. 1, pp.174-188, Feb. 2008.

7. J. Tang, E. Peli, and S. Acton, Image enhancement using a contrast measure in the compressed domain, IEEE Signal Process. Lett., vol. 10, no. 10, pp.289-292, Oct. 2003.

8. B. Bhanu, J. Peng, T. Huang, and B. Draper, Introduction to the special issue on learning in computer vision and pattern recognition, IEEE Trans. on Systems, Man, and Cybern. B: Cybern., vol. 35, no. 3, pp. 391-396, Jan. 2005.

9. R. A. Nobakht, and S. A. Rajala, An image coding technique using a human visual system model and image analysis criteria, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 12, pp.1358- 1361, Apr. 1987.

10. J. Dusek , and K. Roubik, Testing of new models of the human visual system for image quality evaluation, in Proc. IEEE 7th Int. Symp. Signal Process. Appl., vol. 2, pp.621-622, Jul. 2003.

11. R. Gonzalez, and R. Woods, Digital Image Processing, 2nd edition, Prentice Hall, 2012.

12. S. Agaian, Visual Morphology, Proc. SPIE, vol. 3646, pp. 139- 150, Mar. 1999.

13. K. Panetta, Y. Zhou, S. Agaian, H. Jia, Nonlinear unsharp masking For mammogram enhancement, IEEE Trans. Inf. Technol.

    Biomed., vol. 15, no. 6, pp. 918928, Nov. 2011.

14. K. Panetta, S. Agaian, Y. Zhou, and E. J. Wharton, Parameterized logarithmic framework for image enhancement systems, IEEE Trans. Man, Cybern., B, Cybern., vol. 41, no. 2, pp. 460473, Feb. 2011.

15. E. Wharton, K. Panetta, and S. Agaian, Human visual system based multi-histogram equalization for non-uniform illumination and shoadow correction, in Proc. IEEE Int. Conf., vol. 1. Apr. 2007, pp. 729732.

16. F. Yang, Y. Chang, and S.Wan, Gradient-threshold edge detection based on the human visual system, Opt. Eng., vol. 44, no. 2, p. 020205,Feb. 2005.

17. J. Tang, X. Liu, and Q. Sun, A direct image contrast enhancement algorithm in the wavelet domain for screening mammograms, IEEE Sel. Topics Signal Process., vol. 3, no. 1, pp. 7480, Nov. 2009.

18. S. DelMarco and S. Agaian, The design of wavelets for image enhancement and target detection, Proc. SPIE, vol. 7351, pp. 735103- 1735103-9, Apr. 2009.

19. J. Xia, K. Panetta, and S. Agaian, Wavelet transform coefficient histogram-based image enhancement algorithms, Proc. SPIE, vol. 7708, pp. 770812-1770812-12, Apr. 2010.

20. Liu, J. Tang, S. Xiong, and Z. Feng, A multiscale contrast enhancement algorithm for breast cancer detection using Laplacian Pyramid, in Proc. Int. Conf. Inf. Autom., Jun. 2009, pp. 11671171.

21. J. Tang, R. M. Rangayyan, J. Xu, I. E. Naqa, and Y. Yang, Computer aided detection and diagnosis of breast cancer with mammography: Recent advances, IEEE Trans. Inf. Technol. Biomed., vol. 13, no. 2, pp. 236251, Mar. 2009.

22. A. Liu, W. Lin, M. Paul, C. Deng, and F. Zhang, Just noticeable difference for images with decomposition model for separating edge and textured regions, IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 11, pp. 16481652, Oct. 2010.

23. X. K. Yang, W. S. Ling, Z. K. Lu, E. P. Ong, and S. S. Yao, Just noticeable distortion model and its applications in video coding, Signal Process., Image Commun., vol. 20, no. 7, pp. 662680, Aug. 2005.

24. K. Panetta, E.Wharton, and S. Agaian, Logarithmic edge detection with applications, J. Comput. Process., vol. 3, no. 9, pp. 1119, Sep. 2008.

25. W. Osberger, A. Maeder, and D. McLean, A computational model of the human visual system for image quality assessment,

26. K. Panetta, S. Agaian, S. Nercessian, and A. Almunstashri,

    Shapedependent canny edge detector, Opt. Eng., vol. 50, no. 8, pp. 087008- 1087008-12, 2011.

27. J. K. Kim, J. M. Park, K. S. Song, and H. W. Park, Adaptive mammographic image enhancement using first derivative and local statistics, IEEE Trans. Med. Imag., vol. 16, no. 5, pp. 495502, Oct. 1997.

28. S. Agaian, B. Silver, and K. Panetta, Transform coefficient histogram based image enhancement algorithms using contrast entropy, IEEE Trans. Image Process., vol. 16, no. 3, pp. 741758, Mar. 2007.

29. M. K. Kundu and S. K. Pal, Thresholding for edge detection using human psychovisual phenomena, Pattern Recognit. Lett., vol. 4, no. 6, pp. 433441, 1986.

30. G. Buchsbaum An analytical derivation of visual nonlinearity, IEEE Trans. Biomed. Eng., vol. 27, no. 5, pp. 237242, May 1980.

31. S. Nercessian, K. A. Panetta, and S. S. Agaian, A multi-scale nonlocal means algorithm for image de-noising, Proc. SPIE, vol. 8406, pp. 84060J-184060J-12, May 2012.

32. W. Qian, L. P. Clarke, and B. Zheng, Computer assisted diagnosis for digital mammography, IEEE Eng. Med. Biol. Mag., vol. 14, pp. 561 569, Sept./Oct. 1995.

33. M. Unser, P. Thévenaz, C. Lee, and U. E. Ruttiman, Registration and statistical analysis of pet images, IEEE Eng. Med. Biol. Mag., vol. 14, pp. 603611, Sept./Oct. 1995.

34. J. B.Weaver, X. Yansun Jr., D. M. Healy, and L. D. Cromwell,

    Filtering noise from images with wavelet transforms, Magn. Reson. Med., vol. 21, no. 2, pp. 288295, 1991.

35. P. Saipetch, B. K. T. Ho, R. Panwar, M. Ma, and J. Wei, Applying wavelet transforms with arithmetic coding to radiological image compression, IEEE Eng. Med. Biol. Mag., vol. 14, pp. 587593, Sept./Oct.1995.

36. A. Manduca, Compressing images with wavelet/subband coding,

    IEEE Eng. Med. Biol. Mag., vol. 14, pp. 639646, Sept./Oct. 1995.

37. Z. Yang, M. Kallergi, R. A. eVore, B. Lucier, W. Qian, R. A. Clark, and L. P. Clarke, Effect of wavelet bases on compressing digital mammograms,IEEE Eng. Med. Biol. Mag., vol. 14, pp. 570577, Sept./Oct.1995.

38. P. J. Burt and E. H. Adelson, The Laplacian pyramid as a compact image code, IEEE Trans. Commun., vol. COMM31, pp. 532540, Apr.1983.

39. M. Stahl, T. Aach, T. M. Buzug, S. Dippel, and U. Neitzel, Noise- resistant weak-structure enhancement for digital radiography, in Proc. SPIE Med. Imag., 1999, vol. 3661, pp. 14061417.

40. P. Vuylsteke and E. Schoeters, Multiscale image contrast amplification (MUSICA), in Proc. SPIE Image Processing, vol. 2167, 1994, pp. 551560.

41. J. Lu, D. M. Healy, and J. B.Weaver, Contrast enhancement of medical images using multiscale edge representation, Opt. Eng., vol. 33, no. 7, pp. 21512161, 1994.

42. A. Laine, J. Fan, and W. Yang, Wavelets for contrast enhancement of digital mammography, IEEE Eng. Med. Biol. Mag., vol. 14, pp. 536 550, Sept./Oct. 1995.

43. X. Zong, A. F. Laine, E. A. Geiser, and D. C. Wilson, De-noising and contrast enhancement via wavelet shrinkage and nonlinear adaptive gain, in Proc. SPIE Wavelet Applications III, vol. 2762, 1996, pp. 566574.

44. J.-P. Bolet, A. R. Cowen, J. Launders, A. G. Davies, G. J. S. Parkin, and R. F. Bury, Progress with an all-wavelet approach to image enhancement and de-noising of direct digital thorax radiographic images, in Proc. 6th Int. Conf. Image Processing and its Applications, vol. 1, Dublin, Ireland, 1997, Conf. Publ. 443, pp. 244 248.

45. O. Rockinger, Image sequence fusion using a shift-invariant wavelet transform, in Proc. IEEE Int. Conf. Image Process., Oct. 1997, pp. 288291.

46. W. Selesnick, R. G. Baraniuk, and N. G. Kingsbury, The dual-tree complex wavelet transform, IEEE Signal Process. Mag., vol. 22, no. 6, pp. 123141, Nov. 2005.