New Method for Large-Area Satellite Image Quality Enhancement with Local Aerial Multi-Point Calibration

New Method for Large-Area Satellite Image Quality Enhancement with Local Aerial Multi-Point Calibration

Jepsy D' cruz J

M-Tech Signal Processing Marian Engineering College Trivandrum, India

Reshmalakshmi C

Asst. Professor, ECE Marian Engineering College Trivandrum, India

Abstract — Satellite images are more sensitive to atmospheric effects compared with aerial images. Atmospheric effects on aerial images are even negligible in fine weather. Given that aerial remote sensing has high spatial resolution and geometric fidelity, more spatial details can be recorded in aerial images. However, the scan bandwidth of aerial images is limited compared with that of satellite images. Thus, taking high-quality aerial images of a neighborhood as reference can provide prior knowledge for point spread function (PSF) estimation and for the quality enhancement. The least square method and interpolation are used for the PSF estimation of spatial variation, and then total variation minimization is used for recovery.

Index Terms–Image restoration, least squares, space-variant point-spread functions, total variation minimization.

INTRODUCTION

Aerial images acquired usually consist of far fewer "shots" that taken by aircraft. The extra distance means that more area can be covered in one pass at the deficit of detail. Capture data in strips similar to a continual video of the area and allow a larger amount of data to be acquired per digital file. Some features of images are Speed, Level of detail, Location, types of data, etc., for the same area, multi-temporal, multi-sensor, multi-platform, or multi-resolution images can be acquired, thus diversifying information expression. The platform for aerial photography is comparatively lower with higher ground resolution and has better flexibility. Thus, aerial images can provide a more precise geometrical description of objects compared with satellite images. A frame perspective camera or a charge-coupled device camera is often used as an airborne sensor.

The geometric distortion of images captured by these kinds of sensors is greatly affected by flight altitude and is less affected, even negligibly, by the atmosphere [1]. The principal objective of image enhancement is to process a given image so that the result is more suitable than the original image for a specific application. It accentuates or sharpens image features such as edges, boundaries, or contrast to make a graphic display more helpful for display and analysis. Image enhancement methods can be based on either spatial or frequency domain techniques.

The PSFs of images have to be extracted for restoration by establishing image degradation models. Ground target measurement, edge extraction in characteristic object images, and model estimation all require improvement. Although PSF can generally be estimated by using a blind deconvolution algorithm, PSF estimation and image deconvolution are highly constrained for a single degraded image [9]. However, the PSFs of large-area satellite images often vary with space, thus hindering the accurate realization of restoration by using only the PSF method. Studies on the restoration of this kind of image have been conducted repeatedly. The iterative estimation of PSF and the original images depends on a great deal of prior knowledge such as target strength and non-negative PSF values.

After considering that the PSF of each point in a satellite image varies with space, prior information is provided by multiple high-quality aerial images, and the advantages of multi-source remote sensing are complemented. Moreover, weighted coefficients are constructed in accordance with the distance between image blocks. Interpolation is used to solve for the PSF estimation values of other image blocks; after which total variation minimization is used to restore a large-area satellite image block by block. Blocking the satellite images with the overlapping area enabled the gradient splicing of image blocks.

1. METHOD FOR IMAGE QUALITY ENHANCEMENT This paper uses aerial images as reference to

   provide prior knowledge for the PSF estimation of satellite images. Total variation minimization and deconvolution are employed to determine the estimation values of the original aerial images. Image pre-processing refers to the registration of aerial and satellite images. Interpolation is used to solve for the PSF estimation values of other image blocks. Each procedure of the method will be introduced in the following sections. The steps of the method are shown in Fig 1.

   Image Enhancement

   Image Blocking (With Overlapped Pixel between Adjacent Blocks)

   Psfs Estimation of Aerial Based Images

   Restoration of Each Image Blocking Total Variation Minimization for Restoration

   Gradient Stitching Of Satellite Image

   Fig 1 Procedures

   Given that these kinds of images generally have different radiation characteristics, relative correction can be performed on them by using methods such as histogram matching so that better results can be obtained. Each procedure of the method will be introduced in the following sections.

2. PSF ESTIMATION OF LOCAL AERIAL IMAGES AS REFERENCE

   The image degradation process can be described as a two dimensional and a linear-shift-invariant system f(x,y) is the original image, and T stands for the degradation system. After being processed by T and after superposition with noise, f(x,y) is changed into the observed image g(x,y), h(x,y) stands for the response of T to the pulse function, that is, PSF. The degradation process can be expressed as the Convolution of the original input image f(x,y) and the system PSF

   g(x,y) = f(x,y) * h(x,y) + n(x,y) (1)

   Based on the methods the PSF blind estimation of the satellite images to be restored is converted through deconvolution with the aid of high-quality aerial images of the same scene. gois assumed to be the reference image, g is the image to be restored, and f is the ideal original image. The following equations can be obtained from the image degradation model

   g = f * h + n1 (2) go= f * ho + n2 (3)

   ho and h stand for the degradation functions of the two images, n1 and n2 are assumed to be Gaussian white noise of zero mean.

   observed image h can be estimated by using high-quality images of the same scene (no requirement on noise) go as reference and by using the known observed image g for processing.

   Other than Gaussian noise, intensity variation of images due to camera is set to a long or short exposure time, the image is blurred due to phase variation, change of scale, using SIFT algorithm of comparison of key points of the original image with key points of the distorted image , and can eliminate these distortion,examples are given for SIFT based image enhancement

   g = go * h + n3 (4)

   Where n3 is a Gaussian noise of zero mean.Gaussian noise always remains Gaussian when it passes linear system. In unknown situations, each pixel noise (n1 and n2) can be specified by using image analysis and some other methods. Eq. (4) shows that the PSF of the

   Fig.2. Estimation of image blocks with reference images-a) selected image, b)key points of the selected image, c)intensity change in image, d) key points of intensity change image, e) zoomed image ,f) ) key points of zoomed image g)angle shifted mage, h) ) key points of angle shifted image

   From the example its clear that the SIFT algorithm for image distorsion are removed and increases effiency.If we estimate PSF with regularization method, we have to deal with an iterative optimization procedure, owing to the restrictions on h such as non-negative, normalized and finite supported [2] that is what we try to avoid, considering the large data size of remote sensing images. An iterative estimation can be done in with the least square criterion is estimated to minimize the influence of noise. Then h satisfies the following equation:

   argh min J(h) = || g-Fgo|| (6)

   The aperiodic matrix deconvolution model proposed by Zou and Unbenhauen [3] is adopted here. Given that Fgois an aperiodic convolution kernel matrix generated by go and that this has a full column rank, a unique solution to the aforementioned least square problem exists, which is

   Thus, our constrained minimization problem is:

   1. Minimize dx dy

   2. The second constraint uses a priori information that the standard deviation of the noise n(x, y).

   In the result sequence PSF to be estimated. Go(u,v) and G(u,v)are the Fourier transform of the reference image go and the observed image g, respectively. The pre-defined PSF (salt & pepper noise) is added to the clean aerial image in Fig. 2(a) to synthesize the noisy aerial image (b), and the restored image (c) synthesized the blur aerial image (d) is very close the clean aerial image,

   h = ( FgoFgo)¯ ¹ Fgo g (7)

   Aperiodic convolution kernel matrix is complex. The direct use of this matrix to calculate certain existing difficulties results in several difficulties. However, under the premise of its unspecific form, which is based on Fourier transform, equations can be easily calculated using iterative methods.

   Fgo Fgoh Fgo g = 0 (8)

   Thus, Fgo Fgoh Fgo g = 0 becomes the gradient of target function J(h), and Fgo Fgois its Hess matrix. This equation can be solved by using conventional numerical methods.

   The conjugate gradient method has a quadratic termination property. This property indicates that the minimal point can be obtained after finite iterations.

   In practice, to estimate a true signal in noise, the most frequently used methods are based on the least squares criteria [4]. The rationale comes from the statistical argument that the least squares estimation is the best over an entire ensemble of all possible pictures. This procedure is L2 norm dependent. However it has been conjectured in ref. [5] that the proper norm for images is the total variation (TV) norm and not the L 2 norm. TV norms are essentially L1 norms of derivatives; hence L1 estimation procedures are more appropriate for the subject of image estimation (restoration).

   Fig.3. PSF estimation and restoration of satellite image.(a) The noisy aerial image (b)restored image (c)blur aerial image (d) clean aerial image

   Which proves that the PSF estimation algorithm is effective h is the PSF estimation.

   h =

   0.0082 0.0119 0.02230.0325 0.0368 0.0325

   0.0223 0.0119 0.0050 0.0253 0.0368 0.0417

   0.0368 0.0253 0.0135 0.0056 0.0050 0.0119

   0.0223 0.0325 0.0368 0.0325 0.0223 0.0119

   0.0034 0.0082 0.0153 0.0223 0.0253 0.0223

   0.0018 0.0044 0.0082 0.0119 0.0135 0.0119

3. INTERPOLATION FOR SOLVING THE PSF OF SATELLITE IMAGE BLOCKS

   The moment satellite images are acquired, atmospheric interference can be considered stable and continuously varieties with space. Consequently, different ground regions in an image are closely related with the PSFs of the corresponding image blocks. A smaller distance results in stronger regional correlation. Only a few regions in a satellite image can be found in the corresponding aerial images. The PSF of the rest of the regions that have no reference images can be obtained by interpolation.

   Total variation regularization, one of the most effective regularization methods, was presented by Rudin, Osher, and Fatemi in [6]. This method depends on the description of piecewise smooth images and on the balancing capability among algorithm complexities [6]. Total variation regularization can restrain noise and does not compulsively smooth the solution Thus; the detailed information of the edges of the solution is retained. After PSF estimation, total variation minimization is then used to improve satellite image quality, which is the essence of image restoration. By using the total variation regularization method to restore an image, the algorithm can be transformed to calculate the minimum of

   J(u) = ½ || Ku g ||² + | | (9)

   g is the observed image, K is the fuzzy operator u, is the original image, is the regularization constant that controls the balance among total variation regularization items, || Ku g ||² and is the data approximation item. A small positive integer is constructed for numerical regulation to avoid the | u| = 0.The minimization of J(u) through the variation method is equivalent to solving the following Euler- Lagrange equation by using the Neumann boundary condition.

   After being processed block by block, the blocks of a satellite image cannot be directly joined to create a smooth image because an obvious boundary exists between adjacent image blocks. To avoid this condition, an overlap is maintained between adjacent image blocks during blocking.

   After an image is processed by using the method in this paper, the grey means of the overlaps become almost the same. Grey scaling is no longer necessary, which is one of the advantages of this new method.

   Superposition calculation is directly conducted on the overlaps by using the weighted coefficients formed by the distance between the pixels of the overlaps and the boundary of the image block. Graded stitching can be realized, and a fragmented sense of the visual can be removed from the stitched image. The grey continuity between images blocks can obviously be improved after the image is processed.

   The overlapping pixels in the 1-D direction are taken as an example. The two adjacent image blocks are assumed to be X and Y, as shown in Fig. 3. After these blocks are stitched, we obtain image block Z that has a graded stitching edge. If a pixel K of the overlap is located in row d of block Y in accordance with the weighted coefficient formed by the distance from the edge, the following equation can be used to obtain the grey value of the corresponding pixel after stitching

   Fig.4. overlaps of image blocks (shadow area) one dimensional diagram (L is the width).

   Zk = Xk(1 L-d/L) + Yk. d/L (10)

4. EXPERIMENTS AND ANALYSIS

   This paper adopts two groups of aerial images and the corresponding regional satellite experiments. Aerial image is sampled to satellite-image resolution because of the relatively high resolution of aerial images. The use of SIFT algorithm which involves a large number of corresponding matching points. Aerial and satellite image sub pixel registration is obtained by combining digital differential rectification with tiny facet primitive. Registration precision should at least be controlled in less than one pixel to improve PSF estimation accuracy.

   Histogram matching methods are also needed to provide the two images with relative radiometric correction, and

   thus obtain betterrestoration effect. In order to prove the validity of this method, we take comparing experiments with the same image data based on the work of Wang et al. [7]

   Experiment

   We used a set of high-quality aerial images of the Huangguoshu Airport as reference. Blur estimation and enhancement processing were conducted on large-area satellite images of the Huangguoshu Airport taken in 2005. Fig. 4 shows the satellite images of the area. A certain degree of blur in details such as roads and airfield runways can be seen in the satellite images. The aerial images and satellite images were registered and resampled to have the

   same resolution. Using the total variation minimization method based on the three PSF estimates. The image blocks have visually improved after being processed, with richer and more detailed spatial information as well as clearer edges

   The mean can reflect the overall brightness level of animage. The average gradient can generally reflect the clarity of the detailed information of an image. The entropy reflects the richness of image information from the perspective of the information theory. Detail energy can reflect image grey fluctuations of the overall detail areas. Fig. 6 shows the spot images of the area. A certain degree of blur can be observed in the details, such as roads and buildings, of the satellite images. The processed image shows a minimal difference in mean compared with the original image. Thus, image processing based on PSF has not damaged the overall brightness distribution of the original image

   Consequently, these indices are used to evaluate the quality of the image blocks after being objectively processed, as shown in Table I. shows the objective evaluation of the quality of image blocks after being objectively processed.p and p which are PSF estimates of fig 5(a), b(b) were still solved using the least square algorithm

   Fig.5. Blur estimation and restoration of satellite image blocks with reference images,(a) to (b) a set of aerial reference images after resampling, (c)gradient switched satellite image blocks corresponding to (a) to (b) reference aerial images.

   p =

   0.0082 0.0119 0.0223 0.0325 0.0368 0.0325

   0.0223 0.0119 0.0050 0.0253 0.0368 0.0417

   0.0368 0.0253 0.0135 0.0056 0.0050 0.0119

   0.0223 0.0325 0.0368 0.0325 0.0223 0.0119

   0.0034 0.0082 0.0153 0.0223 0.0253 0.0223

   0.0018 0.0044 0.0082 0.0119 0.0135 0.0119

   p =

   0.0008 0.0018 0.0044 0.0082 0.0119 0.0135

   0.0153 0.0223 0.0253 0.0223 0.0153 0.0082

   0.0325 0.0223 0.0119 0.0050 0.0056 0.0135

   0.0253 0.0368 0.0417 0.0368 0.0253 0.0135

   0.0056 0.0050 0.0119 0.0223 0.0325 0.0368

   0.0325 0.0223 0.0119 0.0050 0.0034 0.0082

   Average gradient and detail energy have both significantly improved, increasing to 1.3 times their original value, when overall brightness distribution of the original image is well maintained.

   TABLE I

   QUALITY ASSESSMENT OF IMAGE BLOCKS AFTER RESTORATION BASED ON PSF

   Image	Mean	Variance	Gradient	Entropy	Energy
   Aerial Image 1	108.4	5900	37.90	5.66	2090
   Aerial Image 2	102.1	6410	19.61	7.13	2320
   Stitched Image	155.5	3640	12.53	4.60	1620

5. CONCLUSION

A set of high-quality aerial images of local areas are used to estimate the PSFs of spatial change in a large-area image and to restore the image block by block. This method does not require the sampling of imaging information of the ground targets and natural objects. Only registration and several pre-processing steps are required. Consequently, the method has promising applications in automatic operation. The method has also combined the advantages of aerial images into the process satellite image reconstruction. This combination is beneficial to the comprehensive use of multi-source remote-sensing images. Experiments prove that satisfactory restoration results can be achieved by using this method. After processing, the quality of a large-area image can be significantly improved, with clearer information on the details of edges, thus increasing the subsequent application value of the image.

REFERENCES

1. Li S and Zhu C. 2005.DMC satellite image MTF analysis and restoration method research. Journal of remote sensing, 9(4): 475 479

2. L.Yuan, J. Sun, L. Quan, and H.-Y. Shum, Image deblurring with blurred/noisy image pairs, ACM Trans. Graphics, vol. 26, 2007, Article NO.1.

3. M. Y. Zou and R. Unbenhauen,On thecomputational model of a kind of deconvolution problems, IEEE Trans. Image Process., vol. 4, pp. 14641467, 1995

4. L. Bar, N. Sochen, and N. Kiryati, Restoration of images with piecewise space-variant blur, in Proc. 1st Int. Conf. Scale Space Methods and Variational Methods in Computer Vision (SSVM 2007), 2007, pp. 533544

5. L. Rudin, Images, numerical analysis of singularities and shock filters, Caltech, C.S. Dept. Report #TR: 5250:87 (1987).

6. L. I. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms,Physica D: Nonlinear Phenomena, vol. 60, pp. 259268, 1992.

7. T. H. Wang, S. Y. Li, and X. Z. Li, Blur estimation and restoration of remote sensing images using a reference image, J. Remote Sens., vol. 14, pp. 460467, 2010.

8. J. Li, Y. Q. Zhou, and D. R. Li, High-resolution aerial imagery for improving the spatial resolution of spaceborne images, Pattern Recognition and Artificial Intelligence, vol. 12, pp. 461466, 1999.

9. H. J. Trussell and B.R. Hunt, Image restoration of space variant blurs by sectioned methods, IEEE Trans. Acoustics, Speech Signal Process, vol. 26, pp.608609, 1978.

10. H. J. Trussell and B. R. Hunt, Sectioned methods for image restoration, IEEE Trans. Acoustics, Speech Signal Process, vol. 26, pp. 157164, 1978.

11. T. P. Costello and W. B. Mikhael, Efficient restoration of space- variant blurs from physical optics by sectioning with modified wiener filtering, Digital Signal Process, vol. 13, pp. l22, 2003

12. B.R. Hunt, The application of constrained least squares estimation to image restoration by digital computer,IEEE Trans. Comput. 22 (1973) 805.