An Advanced Approach towards Shadow Detection and Restoration in VHR Images

An Advanced Approach towards Shadow Detection and Restoration in VHR Images

Nishna Soman

1. ech Computer Science and Engineering, Sree Narayana Gurukulam College of Engineering,

   Kadayirippu Ernakulam, Kerala

   Sneha Sudhakaran

   Asst. Prof, Dept of Computer Science and Engineering, Sree Narayana Gurukulam College of Engineering, Kadayirippu

   Ernakulam, Kerala

   AbstractThe presence of shadows can cause a serious problem for their full exploitation in Very High Resolution (VHR) images. This paper proposes to face this problem through a combination of advanced shadow detection, removal and restoration techniques. A hierarchical supervised classification scheme is used for performing the shadow detection task. The classification task is performed using the state-of-the-art support vector machine approach. Shadow removal is done using a combination of both additive and model-based methods. Three different algorithms are presented in order to restore the detected shadow areas. These algorithms are: Gamma Correction Method, Linear-Correction Method and Histogram Matching Method. In order to determine the best restoration method, a comparative analysis is done based on their Peak Signal-to-Noise Ratio (PSNR).

   Keywordsshadow detection; shadow reconstruction; very high resolution (VHR) images

   1. INTRODUCTION

      An innovative era for very high resolution (VHR) satellite images are discussing everywhere especially in the remote sensing field. Shadows can create a serious obstacle for the full exploitation of satellite images. It is important to identify shadow areas in an image and restore their brightness for performing successful change detection on a single or time series of images. VHR images show resolutions which distinguish detailed features from very small objects like little building structure, trees, vehicles, and roofs. On the other side VHR images have some drawbacks like the unwanted presence of shadows, long and variable shadows. Shadows strongly affect the information present in the images and cause severe problems like false color tones, wrong size and shape of an object. This will create problems for both remote sensing image seller and user. An example of the importance of getting shadow-free images is the tsunami in 2004 where it was important to obtain such images in a very short time in order to take quick and crucial decisions in rescue missions. Shadows can lead to erroneous classification or interpretation (e.g., biophysical parameters such as vegetation, water, or soil indexes), due to the partial or complete loss of information in the image [1].

      In order to overcome this drawback and to increase the accuracy, two necessary steps are required. They are 1) Shadow detection and 2) Shadow reconstruction.

   2. RELATED WORK

      Two methods are generally used to detect shadows, namely model-based and shadow-property-based. Model-based approach needs prior information about the scenario and the sensor. However, since, usually, such information is not available, most of the detection methods are based on shadow properties.

      Method [2] is based on invariant color models to identify and classify shadows in digital images. This method consists of two steps 1) candidate region extraction 2) separation self and cast shadow points using invariant color features. The method [3] addresses the problem of shadow detection in high-resolution satellite images. A simple and efficient method based on local MSER (Maximally Stable Extremal Region) detector to gain shadow regions on high-resolution satellite images is used. These methods provide limited performance in eliminating the impact of shadow in high- resolution remote sensing. Method [4] is based on the physical properties of the illumination source and uses the blackbody radiator model for the description of the illumination process. Unlike static methods, this method adaptively calculates the parameters for a particular scene and allows one to work with different images and sensors obtained with different illumination conditions. Method [5] analyzes the properties of shadows in HIS color space and a robust method to segment shadows in remote sensed color images are presented. Segmented shadows are then compensated directly in HIS color space using histogram matching, and converted back to RGB color space. Method [6] effectively separates shadows out from the image by the method based on HIS Space Transformation and NDVI Index. The recognition accuracy was improved by this method. The method [7] uses normalized saturation-value difference index (NSVDI) in Hue-Saturation-Value (HSV) color space for shadow detection and employs histogram matching to recover the information under shadows. In this approach, there is no specific strategy for dealing with the borders between shadow and non-shadow regions.

   3. PROBLEM FORMULATION

      In urban areas, the presence of shadows in VHR optical images may completely destroy the information present in it. The missing information in shadow areas can directly affect the processing and analysis operations, such as the classification map generation. Shadows appear when objects occlude the direct light from the illumination source, usually,

      the sun. Shadows are not all the same. There are two different classes of shadows: cast shadows and self shadows (see Fig. 1). Cast shadow occurs due to the projection of light source in the direction of the object. Self shadow represents the part of the object that is not illuminated directly by the light source. For simplicity, this paper does not distinguish between self shadows and cast shadows and assumes that most of the shadows in an image belong to the cast type, producing homogeneous dark areas with a loss of information that we desire to recover.

      In this paper, the shadow detection is done through a hierarchical supervised classification procedure for the following reasons: 1) to separate between shadow and non- shadow areas and 2) to identify the different non-shadow classes as well as their corresponding shadow counterpart.

      Fig 1: Illustration of self and cast shadows

   4. PROPOSED METHOD

      Fig. 2 shows a flowchart with the principal steps of the proposed methodology

      Fig 2: Flow chart of proposed method.

      1. Shadow Detection

         1. Mask Construction: The shadow versus non-shadow mask can be obtained by binary classification followed by post-processing [8]. The binary classification is implemented in a supervised way. The feature space is defined by the original image bands and wavelet transform is used to extract

            features. In order to maximize the sparseness of the transformation, symlet wavelet is used. Due to the presence of noise in the image, the binary image may be characterized by a salt and pepper effect. To overcome this problem, an opening by reconstruction, followed by a closing by reconstruction, is applied on the image [9]. The use of morphological filters to deal with this problem is motivated by their effectiveness and better shape preservation capabilities and by the capacity to adapt them according to the image filtering requirements as is the case in the border creation.

         2. Border Creation: The transition between shadow and non-shadow areas can create problems like boundary ambiguity, color infidelity, and illumination variation [8]. Presence of the penumbra induces mixed pixels which make classification difficult. A border between the shadow and non-shadowed classes is defined to handle the border pixels appropriately. Gamma correction is then done by converting each RGB component o each pixel into its corresponding light, mixing these light levels together as particular image processing operation requires, and converting them back to RGB. After this color based clustering and shape modeling is also done.

         3. SVM Classification: Given a set of training examples, each belonging to one of two categories, an SVM training algorithm builds a model that assigns new examples into one category or the other, making it a non-probabilistic binary linear classifier [8]. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on. Here the previously obtained mask is used to guide a further classification level applied separately to both shadow and non-shadow regions.

      2. Shadow Removal

         For the majority of remote sensing applications, high- resolution satellite images could be acquired when lighting conditions were at their optimum. This is not always possible, and thus, other techniques have to be developed to deal with the problems caused by shadows and to obtain shadow free images. Here we use a combination of both additive and model-based techniques. The images are first converted to the YCbCr color-space. Additive method is used for the correction on Y channel and a model-based for the correction on Cb and Cr channels. Shadow removal procedure involves various steps:

         1. Creating shadow segmentation if no mask is available.

         2. Structuring element for the shadow mask blurring, and the shadow/light core detection.

         3. Computing shadow/light core (pixels not on the blurred edge of the shadow area).

         4. Smoothing the mask.

         5. Combined additive and light model-based shadow removal in YCbCr color space and conversion to YCbCr.

         6. Computing average channel values in YCbCr space.

         7. Computing ratio and difference in YCbCr space.

         8. Shadow correction and

         9. Conversion back to RGB color space.

      3. Shadow Restoration

      Image reconstruction is one of the most important steps in our methodology. To restore shadow areas, there exist essentially three different methods as shown in Fig 3: 1) gamma correction; 2) histogram matching; and 3) linear correlation.

      (a)

      (b)

      (c)

      (d)

      Fig 3: Shadow restoration using three methods. (a) Input image. (b) Gamma correction. (c) Histogram matching. (d) Linear correlation.

   5. COMPARATIVE ANALYSIS

      Three different algorithms are used in order to restore the detected shadow areas. These algorithms are: 1) gamma correction; 2) histogram matching; and 3) linear correlation.

      1. Gamma Correction

         This method considers the shadow as a multiplicative noise source that corrupts the underlying pixel brightness [10]. Therefore, we can introduce the recovered DN values of the shadow regions are given by Eq. 1:

         DN recovered = (DNshadow) 1/ (1)

         where is the parameter of the algorithm obtained from the training data set. DN values should be normalized. In the case of an image with 11-bit dynamic range, Eq. 1 can be written as:

         (DNrecovered 2047) = (DNshadow)1/ (2)

         The parameter should be applied only to the class for which the value is computed. To determine the coefficient, the mean value of shadow pixels and neighboring non- shadow pixels that are known to represent the shadow pixels are used.

      2. Linear -Correlation Correction

         If the shadow is modeled as a combination of both additive and multiplicative noise, the brightness of shadow pixels to the first order can be restored by using a linear function [10]. Using minimum square error criterion, we can describe this linear function as:

         DNrecovered= non-shadow shadow (DNshadow – µshadow) + µnon-shadow

         (3)

         where and denotes the mean and the standard deviation value of the shadow or non-shadow region.

      3. Histogram Matching

      Histogram matching is one the typical methods that are used in order to make brightness distribution of two given images as close as possible to each other [10]. This method recovers the DN values of the shadow-covered pixels by matching the histogram of the shadow regions to the histogram of the non-shadow areas of the same class. This operation depends on the window size in which the histograms are matched. In order to select the appropriate window sizes automatically, the Quad-tree partitioning method is used.

      TABLE I. RESULTS OF DIFFERENT RESTORATION TECHNIQUES

      RESTORATION TECHNIQUES	TEST CASE 1 PSNR VALUE	TEST CASE 2 PSNR VALUE	TEST CASE 3 PSNR VALUE
      GAMMA CORRECTION	26.4259	26.9719	25.871
      LINEAR CORRELATION	29.5482	32.6058	29.456
      HISTOGRAM MATCHING	26.3585	27.0814	26.123

   6. CONCLUSION

The proposed method has dealt with the challenging problem of restoration of VHR images masked by the presence of shadows. The shadow areas are not only detected but also classified allowing their customized compensation. The classification tasks are performed by using the state-of- the-art SVM approach. The proposed method yields shadow- free and visually realistic images while preserving the spectral and textural properties of the masked objects. Classification accuracy is also improved by using this method.

A comparative analysis of three restoration methods is done based on their Peak Signal-to-Noise Ratio (PSNR). The best results are obtained using the Linear Correlation Correction method. The Gamma Correction and Histogram Matching are also able to restore the shadow brightness, but not as effective as the Linear-Correlation method.

ACKNOWLEDGMENT

The authors would like to thank all the staffs of the CSE department of SNGCE and friends for their valuable suggestions and feedback. We also like to thank all the authors of the papers which we have referred to get a clear idea regarding our work.

REFERENCES

1. P.M. Dare, Shadow analysis in high-resolution satellite imagery of urban areas, Photogramm. Eng. Remote Sens., vol. 71, no. 2, pp. 169 177,2005.

2. E. Salvador, A. Cavallaro, and T. Ebrahimi, Shadow identification and classification using invariant color models, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process.,2001, vol. 3, pp. 15451548.

3. H. Y. Yu, J. G. Sun, L. N. Liu, Y. H. Wang, and Y. D. Wang, MSER based shadow detection in high resolution remote sensing image, in Proc. ICMLC, 2010, pp. 780783.

4. A. Makarau, R. Richter, R. Müller, and P. Reinartz, Adaptive shadow detection using a blackbody radiator model, IEEE Trans. Geosci. Remote Sens., vol. 49, no. 6, pp. 2049 2059, Jun. 2011..

5. J. Su, X. Lin, and D. Liu, An automatic shadow detection and compensation method for remote sensed color images, in Proc. 8th Int. Conf. Signal Process., Nov. 2006, vol. 2, pp.14.

6. D. Cai, M. Li, Z. Bao, Z. Chen,W.Wei, and H. Zhang, Study on shadow detection method on high resolution remote sensing image based on HIS space transformation and NDVI index, in Proc. 18th Int. Conf. Geoinf., Jun. 2010, pp. 14.

7. H. Ma, Q. Qin, and X. Shen, Shadow segmentation and compensation in high resolution satellite images, in Proc. IEEE IGARSS, Jul. 2008, vol. 2,pp. 10361039.

8. Luca Lorenzi, Farid Melgani, Grégoire Mercier, "A Complete Processing Chain for Shadow Detection and Reconstruction in VHR Images, "IEEE Trans. Geosci. Remoe Sens., vol. 50, pp. 3440- 3452

9. P. Soille, Morphological Image Analysis. New York: Springer-Verlag, 1999.

10. P. Sarabandi, F. Yamazaki, M. Matsuoka, and A. Kiremidjian, Shadow detection and radiometric restoration in satellite high resolution images, in Proc. IEEE IGARSS, Sep. 2004, vol. 6, pp. 37443747.