Enhancement of Hidden Objects Due to Shadow of Obstacles in Urban Areas

Enhancement of Hidden Objects Due to Shadow of Obstacles in Urban Areas

ISSN: 2278-0181

NCEASE-2015 Conference Proceedings

1. S. Ilakkiya, 2.D. Santhanalakshmi, 3.D. Sarala

St.Annes College Of Engg And Tech

Abstract -In accordance with the characteristics of urban high- resolution color remote sensing images, we put forward an object- oriented shadow detection and removal method. In this method, shadow features are taken into consideration during image segmentation, and then, according to the statistical features of the images, suspected shadows are extracted. Furthermore, some dark objects which could be mistaken for shadows are ruled out according to object properties and spatial relationship between objects. For shadow removal, innerouter outline profile line (IOOPL) matching is used. First, the IOOPLs are obtained with respect to the boundary lines of shadows. Shadow removal is then performed according to the homogeneous sections attained through IOOPL similarity matching. Experiments show that the new method can accurately detect shadows from urban high- resolution remote sensing images and can effectively restore shadows with a rate of over 85%.

INTRODUCTION

IN the last ten or more years, with the availability of high- spatial-resolution satellites such as IKONOS, Quick Bird, Gooey, and Resource 3 for the observation of Earth and the rapid development of some aerial platforms such as airships and unmanned aerial vehicles, there has been an increasing need to analyze high-resolution images for different applications. In urban areas, surface features are quite complex, with a great variety of objects and shadows formed by elevated objects such as high buildings, bridges, and trees. Although shadows themselves can be regarded as a type of useful information in 3-D reconstruction, building position recognition, and height estimation .they can also interfere with the process- ing and application of high-resolution remote sensing images. gray scale, brightness, saturation, and texture. An improved algorithm exists that combines the two methods. First, the shadow areas are estimated according to the space coordinates of buildings calculated from digital surface models and the altitude and azimuth of the sun. Then, to accurately identify a shadow, the threshold value is obtained from the estimated grayscale value of the shadow areas. However, information such Scheme was proposed to detect shadows . To avoid the false

shadows of dark objects such as vegetation and moist soil, normalized difference vegetation index the normalized saturationvalue difference index, and the size and shape of the shadow area are considered the method used by Makarau scheme was proposed to detect shadows . To avoid the false shadows of dark objects such as vegetation and moist soil, the normalized difference vegetation index the normalized saturationvalue..

II. SHADOW DETECTION

Shadows are created because the light source has been blocked by something. There are two types of shadows: the

self-shadow and the cast shadow. A self-shadow is the shadow on a subject on the side that is not directly facing the light source. A cast shadow is the shadow of a subject falling on the surface of another subject because the former subject has blocked the light source. A cast shadow consists of two parts: the umbra and the penumbra. The umbra is created because the direct light has been completely blocked, while the penumbra is created by something partly blocking the direct light.

1. Image Segmentation Considering Shadow Features

   Images with higher resolution contain richer spatial infor- mation. The spectral differences of neighboring pixels within an object increase gradually. Pixel-based methods may pay too much attention to the details of an object when processing high- resolution images, making it difficult to obtain overall structural information about the object. In order to use spatial information to detect shadows, image segmentation is needed. We adopt convexity model (CM) constraints for segmentation [27], [28].

   Traditional image segmentation methods are likely to result in insufficient segmentation, which makes it difficult to separate shadows from dark objects. The CM constraints can improve the situation to a certain degree. To make a further distinction between shadows and dark objects, color factor and shape factor have been added to the segmentation criteria. The parameters of each object have been recorded, including grayscale average, variance, area, and perimeter. The segmentation scale could be set empirically for better and less time-consuming results, or it could be adaptively estimated according to data such as resolution.

2. Detection of Suspected SHADOW AREAS.

   For shadow detection, a properly set threshold can separate shadow from nonshadow without too many pixels being mis- classified [3]. Researchers have used several different methods to find the threshold to accurately separate shadow and non-

   shadow areas. Bimodal histogram splitting provides a feasible way to find the threshold for shadow detection, and the mean of the two peaks is adopted as the threshold [3]. In our work, we attain the threshold according to the histogram of the original image and then find the suspected shadow objects by comparing the threshold and grayscale average of each object obtained in segmentation. We chose the grayscale value with the minimum frequency in the neighborhood of the mean of the two peaks as the threshold, as shown in

   Gq = 1 (Gm + Gs) (1)

   2

   h(T ) = Min {h(Gq ), h(Gq + )} . (2)

   In the equations, Gm is the average grayscale value of an image; Gs stands for the left peak of the shadow in the histogram; T is the threshold; represents the neighborhood of T , where T [Gq , Gq + ]; and h(I) is the frequency of I, where I = 0, 1,.. . , 255.

   By conducting a large number of experiments, we have found that the average of the grayscale values can be used to replace the right peak. To simplify the operation, when the left peak is not obvious, Gs can be replaced by half of the grayscale average. Meanwhile, to avoid the influence of abnor- mal information, 2% of the pixels on the left and right sides of the histogram are not included. In addition, atmospheric molecules scatter the blue wavelength most among the visible rays (Rayleigh scattering). So for the same object, when in shadow and nonshadow, its grayscale difference at the red and green wavebands is more noticeable than at the blue wave- band. Thus, we retrieve a suspected shadow with the threshold method at the red and green wavebands. Specifically, an object

   II. SHADOW REMOVAL

   1

   ISSN: 2278-0181

   NCEASE-2015 Conference Proceedings

   is determined to be a suspected shadow if its grayscale average

   is less than the thresholds in both red and green wavebands.

3. Elimination of false shadow

Dark objects may be included in the suspected shadows, so more accurate shadow detection results are needed to eliminate these dark objects. Rayleigh scattering results in a smaller grayscale difference between a shadow area and a nonshadow area in the blue (B) waveband than in the red (R) and green

(G) wavebands. After the elimination of false shadows from vegetation, spatial information of objects, i.e., geometrical characteristics and the spatial relationship between objects, is used to rule out other dark objects from the suspected shadows. Lakes, ponds, and rivers all have specific areas, shapes, and other geometrical characteristics. Most bodies of water can be ruled ou due to the area and shape of the suspected shadows of the object that they produce. However, the aforementioned method still cannot separate shadows from some other dark objects. Spatial relationship features are used to rule out dark objects in the suspected shadows. Dark objects are substantive objects, while shadows are created by taller objects which block the light sources and may be linked together with the objects that result in the shadows. An obscured area (i.e., a shadow) forms a darker area in an image. The object blocking the light forms a lighter area in an image. At the same time, the sun has a definite altitude angle, and a shadow boundary reflects the boundary of a building and the position of a light source. Buildings, trees, and telegraph poles are the main objects creating shadows in urban remote sensing images. Their shadow boundaries usually have a certain direction. To retrieve shadows using spatial rela- tionships, the linear boundaries of suspected shadows are first analyzed to predict the probable trend of a shadow, according to which the approximate position of a large object is predicted. To determine whether it is a shadow, the proximity of a dark object to a light object within this azimuth is measured. An average spectral difference can be used to decide whether there are light objects linked around a shadow.

4. Diagram of IOOPL and IOOPL matching. (a) IOOPL at the red waveband. (b) Matching result after Gaussian smoothing for IOOPL, where the red background shows the parts that do not match.

removal method based on IOOPL matching. There is a large probability that both shadow and nonshadow areas in close range on both sides of the shadow boundary belong to the same type of object. The inner and outer outlines can be obtained

To recover the shadow areas in an image, we use a shadow by contracting the shadow boundary inward and expanding it

outward, respectively. Then, the inner and outer outline profile lines are generated along the inner and outer outline lines to determine the radiation features of the same type of object on both sides. As shown in Fig. 3, R is the vector line of the shadow boundary obtained from shadow detection, R1 is the outer outline in the nonshadow area after expanding R outward, and R2 is the inner outline in the shadow area after contracting R inward. There is a one-to-one correspondence between nodes on R1 and R2 . When the correlation between R1 and R2 is close enough, there is a large probability that this location belongs to the same type of object. The grayscale value of the corresponding nodes along R1 and R2 at each waveband is collected to obtain the IOOPL. The outer profile lines (OPLs) in the shadow area are marked as inner OPLs; OPLs in the nonshadow area are marked as outer OPLs (Fig. 3).

The objects on both sides of the shadow boundary linked with a building forming a shadow are usually not homogeneous, and the corresponding inner and outer outline profile line sections

1. EXPERIMENTAL ANALYSIS

   A. Comparative Analysis of Shadow Detection

   To validate that our method works, the following experiment was performed. The datum used in this experiment is a Quick- Bird image of Kunming, China. Each step of this method is described herein, and the steps and corresponding results of each step are given [from Fig. 6(a)(e)]. Moreover, to com- pare with our method, we determined the pixel-level thresh- old shadow detection result with manually selected proper threshold according to the image grayscale scale histogramIt can be seen from the segmentation result [Fig. 6(b)] that segmentation that considers shadow features can effectively segment shadows and dark objects such as vegetation and bodies of water into different subjects This means that, in the following process, the problem of shadow and dark objects being segmented as a whole subject can be avoided. The results, shown in Fig. 6(c), show the retrieval of a rough shadow with the threshold, which indicates that vegetation, rivers, dark moist soil, and true shadows can be detected. Comparing images (c)and (f) in Fig. 6, one can see that the shadow area detected with morphological characteristics of objects, the rivers in image (d). , an object-level shadow detection method based on spectral features and spatial features can accurately and effectively detect shadows in an urban high-resolution remote sensing images with true shadow. therefore using spatial relative.

   ISSN: 2278-0181

   are not reliable. In additionN, CthEeAaSbEno-2rm01a5l CseocntfieornesnocenPthroecienendeinr gs and outer outlines that cannot represent homogeneous objects need to be ruled out. Consequently, similarity matching needs to be applied to the IOOPL section by section to rule out the two kinds of nonhomogeneous sections mentioned previously. The parameters for shadow removal are obtained by analyzing the grayscale distribution characteristics of the inner and outer homogeneous IOOPL sections.

   1. IOOPL

      Matching

      IOOPL matching is a process of obtaining homogeneous sections by conducting similarity matching to the IOOPL sec- tion by section. During the process, Gaussian smoothing is performed to simplify the view of IOOPLTo rule out the nonhomogeneous sections, the IOOPL is divided into average sections with the same standard, and then, the similarity of each line pair is calculated section by section with (4). If the correlation coefficient is large, it means that the shade and light fluctuation features of the IOOPL line pair at this section are consistent. If consistent, then this line pair belongs to the same type of object, with different illumina- tions, and thus is considered to be matching. If the correlation coefficient is small, then some abnormal parts representing some different types of objects exist in this section; therefore, these parts should be ruled out.The sections that have failed the matching are indicated in red. If more accurate matching is needed, the two sections adjacent to the section with the smallest correlation coefficient can be detected.

      Fig. 9. Examples of shadow removal. (a) Remote sensing image in which a shadow has been detected. (b) Inner and outer outline lines generated by a shadow boundary; the red line is the inner outline line, and the blue line is the outer outline line. (c) Result of RRN after IOOPL matching. (d) Result of boundary treatment for (c). (e) Result of RRN skipping IOOPL matching.

   2. Analysis of Shadow Removal Experiments

   To verify our shadow removal method, the following exper- iment was performed. The results of each step of the method are shown in Fig. 9(a)(d). Furthermore, to illustrate the neces- sity of IOOPL matching, we present a shadow removal result .

   8 ISSN: 2278-0181

   NCEASE-2015 Conference Proceedings

   Fig. 10. Comparison charts of IOOPL after Gaussian smoothing of the following wavebands: (a) red, (b) green, and (c) blue (the light gray parts represent fail matching).

   IOOPL matching did not fully solve color cast as there is still a little red in the bottom left part in Fig. 9(c). According to the contrast between Fig. 9(d) and (f), we conclude that the Plus method could restore the background radiation characteristic with poor contrast between the object and the background while our approach could restore both the background color and the contrast. Furthermore, we choose the samples with same scene in the nonshadow area, shadow area, and shadow- removed area to analyze, as shown in Fig. 11. Table I shows the sample information, which verifies the effectiveness of our approach numerically. In Table I, there is a tremendous difference between the nonshadow and shadow regions of the same scenes in spectral consistency according to the average value and standard deviation. After applying our approach, the average value and standard deviation of the shadow-removed region are close to that of the nonshadow region. Therefore, we could obtain the deshaded data which meet the needs of both vision and spectral consisency through the presented approach. As shown in Fig. 12, both relative radiometric normalization (RRN) and PF based on IOOPL matching could effectively remove the shadow. Similar to the former experiment, the Plus method can only restore the entire radiant luminance of the shadow but barely identify the objects in shadow. By comparing Fig. 12(c) and (d), we see that the results of RRN are clearer than the PF results. The PF result is better than the RRN result Of the defective shadow detection most shadow have been effectively removed.refering to the statistics of the supervised classification results the propotion of the shadow has been significantly reduced from 45.1% to 6.05%by the RRN method and form 45.1% to2.88%.the shadow removal rate could reach 86.2% by the RRN and 93.63% by the PF method in this proc.

   IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING

   at the whole radiant brightness in subjective sensation, although

   it is a little bluish.

   Both RRN and PF could restore the shadow area in a visual sense, as shown in Fig. 12. To better analyze and compare the RRN and PF techniques, we classified the original image and shadow removal results with the same training samples. Comparing Fig. 13(a) with Fig. 12(a), the shadow area marked in red was correctly classified with the exception of some cars in dull color. In the classification results [Fig. 13(b) and (c)], although some trees and shadows from cars still exist because

   Fig. 14. Examples of shadow boundary after shadow removal.

2. BOUNDARY PROCESSING AFTER SHADOW REMOVAL

   In the aforementioned experiment, there are several instances of the shadow boundary, shown in Fig. 14.

   Because of penumbra and diffuse reflection, an exact bound- ary does not exist in the shadow area, so the boundary may be presented in two cases: a gradually changing boundary and a boundary with a darker color compared to the shadow area. Fig. 15(a) and (e) shows the grayscale cross section of these two cases. In Fig. 15, points A and B are the starting point and ending point of the shadow boundary, respectively. Point C is next to point A, which belongs to the shadow region, while point D is next to point B, which belongs to the nonshadow region. After boundary treatment, points A1, B1, C 1, and D1 correspond to points A, B, C , and D, obtained with the segmentation method may be near point A or B.

   Shadow boundaries after shadow removal may appear in different cases, as explained previously. However, we can see from Fig. 15 that the points C1 and D1 remain the correct gray scale after boundary treatment. Therefore, we first get the shadow boundaries according to the shadow detection result and then obtain points C and D. Finally, the points between points C and D are filled with incremental gray scale according to the grayscale values of C and D. To find points C and D, we obtain the points at a certain distance from the boundaries on both sides, and the distance will range from 3 pixels to 5 pixels based on image resolution.

3. CONCLUSION

We have put forward a systematic and effective method for shadow detection and removal in a single urban high-resolution remote sensing image. In order to get a shadow detection result, image segmentation considering shadows is applied first. Then, suspected shadows are selected through spectral features and spatial information of objects, and false shadows are ruled out. The subsequent shadow detection experiments compared tra- ditional image segmentation and the segmentation considering shadows, as well as results from traditional pixel-level threshold detection and object-oriented detection. Meanwhile, they also show the effects of different steps with the proposed method. For shadow removal, after the homogeneous sections have been obtained by IOOPL matching, we put forward two strategies: relative radiation correction for the objects one at a time, and removal of all shadows directly after PF is applied to all the homogeneous sections and correction parameters are obtained. Both strategies were implemented in high-resolution images, and their performances were compared in experiments. The experimental results revealed the following.

1. The shadow detection method proposed in this paper can stably and accurately identify shadows. Threshold selection and false shadow removal can be conducted in simple but effective ways to ensure shadow detection accuracy.

   ISSN: 2278-0181

2. Compared with pixNelC-lEevAeSlEd-e2t0e1c5tiCono,nftehreenocbejePcrto-ocereiednintegds shadow detection method proposed in this paper can make full use of the spatial information of an image and can effectively rule out speckles and false shadows in the detection result. However, it is difficult to segment the small size shadows into an independent object, which will cause errors.

3. The shadow removal method based on IOOPL matching can effectively restore the information in a shadow area. The homogeneous sections obtained by IOOPL matching can show the radiation gray scale of the same object in a shadow area and a nonshadow area. The parameters calculated by using the radiation difference between inner and outer homogeneous sections can retrieve a shadow very effectively.

4. The two shadow removal strategies (RRN and PF) are both suitable for high-resolution urban remote sensing images. Moreover, there are advantages to each strategy: RRN can restore the texture details well while PF has a more stable background radiance.

Further improvements are needed in the following ways.

1. Although image segmentation considering shadows can have better segmentation results, insufficient segmenta- tion still exists. For example, a black car and its shadow cannot be separated. Also, parts of the shadow from low trees cannot be separated from the leaves.

2. Because of the filming environment or some other rea- sons, obvious color cast can be seen in some parts of a shadow area. IOOPL matching could relieve this case to a certain extent but not completely resolve the problem.

   ACKNOWLEDGMENT

   The authors would like to thank the anonymous reviewers for their comments. Their insightful suggestions have significantly improved this paper.

   REFERENCES

   1. T. Kim, T. Javzandulam, and T.-Y. Lee, Semiautomatic reconstruction of building height and footprints .

   2. S. Ji and X. Yuan, A method for shadow detection and change detection of man-made objects, J. Remote Sens.

   3. P. M. Dare, Shadow analysis in high-resolution satellite imagery of urban areas,

   4. Y. Li, P. Gong, and T. Sacagawea, Integrated shadow removal based on photogrammetric and image analysis,

   5. W. Zhou, G. Huang, A. Troy, and M. L. Cadenzas, Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas:

   6. J. Yoon, C. Koch, and T. J. Ellis, Shadow Flash: An approach for shadow

      removal in an active illumination environment, .

   7. R. B. Irvin and D. M. McKeon, Jr, Methods for exploiting the relation- ship between buildings