Fog Removal in Images based on Turbulence Mitigation and Contrast Enhancement

Fog Removal in Images based on Turbulence Mitigation and Contrast Enhancement

G. Vignesp, A. Jothilingam2, S. Kalaimagal3, Sabique P. V4

Madras Institute of Technology Anna University, Chennai, India

Abstract In a real time world photo graphic and video surveillance were playing a vital role. Humans are using this photo for video surveillance either for security or for entertainment purpose. The general problems were occurred while imaging the atmosphere is most appearance of fog and the appearance of atmospheric turbulence in the images. There were many researchers have provided a recovery for either the fog removal or turbulence mitigation from the image. Eventually, they failed to recover both the problem at the same time. This paper provided an analysis that incorporates both model such as Fog removal and Turbulence mitigation. This paper suggest CETM (Contrast Enhancement Turbulence Mitigation) method which is more efficient so that it can operate in a fractions of second for the real-time applications while imaging.

Keywords Single Image Defogging ,Image alignment, Dehazing, Debluring Turbulace Mitigatin,Contrast enhancement

1. INTRODUCTION

   An image defined in the real world is considered to be a function of two real variables such as a (x,y) where a is the amplitude (e.g. brightness) of the image at the real coordinate position (x,y )In a highly advanced image processing scheme it should be likely to apply specific image processing operations to selected regions. To suppress motion blur one fragment of an image (region) might be administered while another part might be processed to improve color rendition.

   Vision is the most advanced of our sense, so its not surprising that Figure play the single most major role in human insight. However, unlike humans, who are limited to the visual band of the electromagnetic (EM) spectrum ,imaging machine cover almost the entire EM spectrum, ranging from gamma to radio waves .They can operate on generated by source that humans are not accustomed to associating .These include ultrasound electron microscopy and computer generated. Thus digital image processing encompasses a wide and varied field of application.

   A common problem in the atmosphere is fog and atmospheric turbulence. Over the years, many researchers have suggested perception into the physics of either the turbulence or fog turbulence but not both. Most recently, researchers have proposed methods to remove fog in fast enough for real-time processing. Additionally, methods have been proposed by other researchers that address the atmospheric turbulence problem. This work, provide an analysis that incorporates both physics model. i) Fog and ii) Turbulence

   To observe the contrast enhancement (fog removal) can affect image alignment and image averaging. This paper

   proposed, a new joint contrast enhancement and turbulence mitigation (CETM) method that utilizes estimation from the contrast enhancement algorithm to improve the turbulence removal algorithm. This paper contribute a new turbulence mitigation object metric that measure temporal consistency. Finally, designed the CETM to be efficient such that it can operate in fraction of a second for near real-time applications. Imaging system positioned near the ocean often suffer in performance, perceptually and objectively, because of atmospheric turbulence, fog ,sun-glare, camera movement from wind battering and several other adverse weather condition. The environment itself will have fog or mist, airstream and heat that reasons eddy currents which is perceived as turbulence in an imaging system.

2. EXISTING METHOD

   Already in existing methods there are several approaches are available to remove fog substance and some more methods are available to remove turbulence from the image. But it can remove either turbulence or fog, but not able to remove both at a time. Atmospheric turbulence caused by variation of refractive index along the optical transmission path can toughly distress the performance of larger-distance imaging systems. It produces geometric distortion, motion blur (if not sufficiently short the exposure time), and occasionally out-of- focus blur when the turbulence is violent.

3. OBJECTIVE

   The general problems were occurred while imaging in the atmosphere is the appearance of fog and also the appearance of atmospheric noise and turbulent in the images. This article work is carried out for both models such as fog removal and Turbulence mitigation and also to verify that contrast enhancement and turbulence mitigation algorithm is more efficient than all other existing algorithm. Fog removal can be operate in fractions of second for the real-time applications.

4. PROPOSED METHOD

   1. Turbulence Mitigation and Contrast Enhancement

      Atmospheric turbulence can severely degrade the quality of Figure produced by long range observation systems, rendering the Figure unsuitable for vision applications such as surveillance or scene assessment. The major visual special effects affected by atmospheric turbulence are geometric distortion and space-time-varying blur. The distortion is primarily generated by optical turbulence and scattering and absorption by particulates aerosols, for example, diffuse light

      will also cause blur. Several approaches have been used to restore Figure, including adaptive optics techniques and pure image processing-based methods. Due to random fluctuations in turbulence, calculating a sensible approximation of atmospheric modulation transfer function (MTF) is extremely difficult. However, this function is of dangerous significance for optics-based restoration approaches. Therefore, this article, focused only on image processing to manage the degradation instigated by turbulence. Assuming that the scene and the image sensor are both static, the mathematical model has been used to interpret imaging process.

      Contrast enhancements improve the perceptibility of objects in the background by improving the brightness variance between objects and their backgrounds. Contrast enhancements are classically executed as a contrast stretch trailed by a tonal enhancement, although these could both be performed in single step. A contrast stretch improves the brightness differences uniformly across the dynamic range of the image, although tonal enhancements advance the intensity differences in the shadow (dark), midtone (gray), or highlight (bright) regions at the expense of the brightness differences in the other regions.

   2. Recommended Block Diagram With Description

      The function of the image denoise block is the process of removing noise from the image. Symmetric filtering method is used here to remove the unwanted noise in the input image. The Defogging block is used to remove fog effectively and efficiently for each frame. Fast Fourier transform defogging method is used here for the defogging process. Turbulence in the specified image has been mitigated by means of the turbulence mitigation metric block. This turbulence problem can be initiated due to bad weather or disturbed atmospheric situations. In this process, different Figure is matched with one another by means of Image Alignment block. The method of matching one image called template with another image is called Image alignment. There are numerous applications for image alignment like tracking objects on video, motion analysis, and many other tasks of computer vision. To remove the blur caused by the atmosphere and image alignment errors, Image Deblur block is preferred. De-convolution method is used for deblurring here.

      TM metric

      however, reducing image noise can improve the image alignment performance. Additionally, it is observed that removing noise before fog removal is an important approach in order to improve transmission and air light estimation, therefore this work proposed hoe to remove noise, enhance the contrast and then estimate the global motion.

      In order to diminish the complication of our procedure such that the processing speed is near real-time, a two dimensional median filter is preferred for each color channel for fast single image denoising.

      1. Median Filter

   A median filter belongs to the class of nonlinear filters distinct the mean filter. The median filter also tracks the moving window method analogous to the mean filter. It is necessary to scan a 3× 3, 5× 5, or 7× 7 grain of pixels over pixel matrix of the entire image. The computed median value should be placed in the center pixel value in the window. Median filtering can be done by sorting out all the pixel values from the surrounding neighborhood into numerical order and then substituting the pixel being deliberated with the middle pixel value. Note that the median value must be inscribed to a distinct array or buffer so that the outcomes are not corrupted as the process is performed. The median is supplementary robust likened to the mean. Thus, a single very unrepresentative pixel in a neighborhood will not affect the median value meaningfully. Subsequently the median value must really be the value of one of the pixels in the neighborhood, the median filter do not generate new impractical pixel values when the filter straddles an edge. For this reason the median filter is considerable superior at conserving piercing edges than the mean filter. These advantages aid median filters in denoising uniform noise as well from an image.

   Sample image

   Image

   Denoise Defog

   Image

   Alignment Deblur

   output image

   Fig 2. Input and output of median filter

   Fig 1. Proposed block diagram

5. METHDOLOGY

   1. Removal of Noise From Turbulance

      It is shown that contrast enhancement does not improve the image alignment performance when image noise is present. The result in terms of error is the same if enhancement is before or after global tracking. The analysis does show,

      As mentioned earlier, the image moon.tif is corrupted with salt and pepper noise with the imnoise () function after loading the image using imread (). Fig 2 is the image corrupted with salt and pepper noise and is given to the function medfilt 2 () for median filtering. The window specified is of size 3×3.Fig 2 is the output after median filtering. It can be observed that the edges are preserved and the quality of denoising is much better compared to the input image

   2. Defogging of Image

   A defogger, demister, or defroster is a system to clear condensation and thaw frost from the windshield, back glass, and/or side windows of a motor vehicle. Lets see the benefits and applications of image defogging. To remove fog effectively and efficiently for each frame and have an estimate of transmission and used the Locally Adaptive Wiener Defogging method. The Wiener Defogging method is 50 to 100 times faster than existing methods and can operate at real time speeds for frames of sizes 720 × 480 and larger. This paper proposed a modification to the Wiener Defog method in order to automate the defogging process. The window size must be sufficiently large to estimate a smooth transmission map that best reflects the scene depth and also suggest a method for automatically selecting || based on the statistics of the frames. Given a foggy color image iR, denote the Number of color channels with .

   I (x) = t (x) o (x) + (1 t (x)) a + w(x). (1) The main idea in estimating the transmission with an adaptive local Wiener filter in is that the observation model is composed of the veiling v= (1 t) and texture ntxt. The observation is the dark pixel measurement for each pixel is

   given by,

   D(X) =min ic (X) (2)

   Where Ic(x) is the nth color channel of the image i(x). (For grey Figure, d(x) = i (x) R). The dark pixel measure takes advantage of the atmospheric dichromatic model by assuming

   at least one color component is possibly dark therefore exposing the atmospheric veiling. Thus the model at each pixel is

   1. Image Deblurring

      Where deblurring is used to remove the blurriness from the image. Image deblurring can be formulated as the process of inverting image blurring. This section first introduces a model of image degradation, and then presents problem definitions of image deblurring with their basic solution strategies and associated difficulties

   2. Turbulance Mitigation Metric

      In order to illustrate that the turbulence has been mitigated in time with our proposed method, article a Turbulence Mitigation Metric (TMM). The goal in turbulence mitigation is to not only recover the true structure (phase) of the scene but also to enforce temporal consistency in each color channel. Instead of using a subjective measure by displaying multiple frames of a video sequence, this work developed an objective measure. Our goal in designing the TMM is to make it an objective measure of turbulence mitigation, simple to construct, and easily accessible such that it can be used in any other turbulence mitigation method. One approach to developing a TMM is to utilize the motion estimated from optical flow and global estimations. This approach, however, is complex given that not every method uses optical flow and it required an input image sequence and output image sequence.

      D(x) = v(x) + ntxt (3) Where v is t min ic (X) true veiling and n txt is the textured noise. The goal in the Wiener Defog method is to filter out the texture from the observation d by treating the texture ntxt as noise but preserve edges from depth discontinuities. For Figure with a large amount of texture (forest floor with leaves), the size of window must be sufficiently large to filter out the texture variations. The choice of arises from the local

      moment estimators,

      1

      Fig 3. Example for Weiner deconvolution method

      1. Blurred image b) Deblurred image

6. PROPOSED ALGORITHM

   The following algorithm is to be implemented as per the

   . () = |()| () ()

   (4)

   requirement of fog removal

   Where (X) is the number of pixels within the local sample window positional at pixel location x.

   C. Image Aligment

   This work employ the symmetric filter, Fast Fourier transform (FFT) to estimate the global motion parameter g. This filter is fast and efficient and works well with turbulence video because the phase correlation update averages out the zero mean phase perturbations induced by turbulence.

   The local motion induced by turbulence must also be compensated in order to have a sharper image after frame averaging and also enhanced before optical flow in order to improve the motion estimation. Enhancing before motion compensation reduces the number of possible motion vectors for each pixel. For video coding, intra-frame coding is used more when enhancing first therefore details are preserved whereas inter-frame coding is used more when there is low contrast which results in loss of details.

   Step 1: START

   Step 2: Read the input image

   Step 3: Compute air light value from the image with fog .if He method followed, following steps are taken

   Step 3a: Read the R, G, B components Step 3b: Divide each by 255

   Step 3c: Find min & max value of three values

   Step 3d: Find difference between min & max values Step 3e: Assign V component as the max value

   Step 3f: Assign S component as the difference divided by the V component value.

   Step 3g: Compute H component according to weather max value is R, G, B.

   Step 3h: Apply symmetric filter on the HSV image. Step 3i: Find all pixels greater than the largest value.

   <>Step 3j: Take maximum of these values as air light values.

   Step 4: if air light computation is by OUR method the following steps are taken.

   Step 4a: Apply symmetric filter on the image

   Step 4b: Return the highest value of the image as air light value.

   Step 5: If no method is specified take the first pixel of the image as air light value

   Step 6: Derive the boundary condition.

   Step 7: using the air light value. Calculate the optimal t value using following steps

   Step 7a: parameter values are set accordingly (including scaling parameter)

   Step 7b: Iterate through to obtain optimal transmission value.

   Step 8: Divide the R, G and B component respectively to obtain the R, G, B components of input image by the optimal t value. .

   Step 9: add the air light components respectively to obtain the R, G, B components of the dehazed image.

   Step 10: Display the input image without fog Step 11: Stop.

7. HARDWARE AND SOFTWARE

   Our experiments was executed in a 64-bit machine with Intel(R) Core(TM) i7-4500U CPU @ 3.0GHz with 8GB of RAM (critical for large values) and NVIDIA 4 GB Graphics cards. Software were used Matlab and Open CV for fast processing.

8. RESULTS AND DISCUSSION

1. Experiment with Real Time Image

   To demonstrate the proposed method along with the TMM, Contrast Enhancement Turbulence Mitigation (CETM) algorithm method is applied to a simulated foggy and turbulent image sequence. The PSNR and TMM values are measured with different K values and plot the results in the following figure. The minimum frames to average Kmin = 44 was estimated. The TMM measure starts low and on average increases after each new image. The TMM is 0 at frame K = 10 because the sample size in time was set to 10. The highest TMM is with K = 90 and lowest with K = 10. What is interesting is that there is a significant improvement from 10 to 44 frames but not much improvement when the number of frames to average is almost doubled from 44 to 90. An identical survey can be made with the PSNR values in the figure. The PSNR is increased dramatically at the beginning and the performance reaches a limit at each K value. The performance of the method at each stage of the algorithm is

   Illustrated in Fig 4, Fig 5, Fig 6, and Fig 7. The same simulated sequence is used and each measurement is with K =

   44. The PSNR value of the contrast enhanced sequence is very low because noise is introduced after enhancement. The optical flow slightly improves the result and a dramatic improvement occurs with frame averaging. Without optical flow the frame averaging performs about 1 dB lower than with optical flow.

2. Simulation Results

   The following figure Depict the simulated output of both RGB and for Gray scale image.

   1. Simulation Results for removal fog from image

      (a)

      (b)

      Fig 4.Example 1 of Proposed Defogging algorithm for color image (a) Foggy image (b) defogging Image

      (a)

      (b)

      Fig 5.Example 2 of Proposed Defogging algorithm for color image (a) Foggy image (b) defogging Image

   2. Simulation Results for turbulence image

      (a)

      (b)

      Fig 6.Example 2 of Turbulence Removal Algorithm for Color Image (a)

      Turbulent image (b) Turbulent Mitigated image

   3. Simulation Results for removal fog from image

(a)

(b)

Fig 7. Fog and Turbulence Removal from Color Image

(a) Image with Fog and Turbulence (b) Defogged and Turbulent Mitigated Image

VIII. CONCLUSION AND FUTURE WORK

Many researchers have provided a method to remove either the fog or turbulence from the images but not both at the same time. Many researchers have proposed methods to remove fog in images fast enough for real-time processing. Additionally, some more methods have been proposed by other researchers that address the atmospheric turbulence problem. This work analyzed fog and turbulence of image by observing how contrast enhancements (fog removal) can affect image alignment and image averaging. A new joint contrast enhancement and turbulence mitigation (CETM) method that utilizes estimations from the contrast enhancement algorithm to improve the turbulence removal algorithm.

The main goal of my work was to develop a joint turbulence mitigation and fog removal method that can recover the object image fast enough for near real-time performance.

Future of this project it is proposed to apply fog removal algorithm in traffic monitoring cameras for real time surveillance of vehicles. This is extremely useful in monitoring systems during early morning hours as conventional cameras would give shady images (image with high noise).

REFERENCES

1. Tripathi S.A. and Mukhopadhyay K. (2015), Single image fog removal using anisotropic diffusion, The Institution of Engineering and Technology 2012 IET Image Process., 2012, IEEE,Vol. 6, pp. 56- 64.

2. Xiaoqiang Ji ,Yuping F. Gang L. Ming D. And Chuanli Y (2010), Real- Time Defogging Processing of Aerial Images Proc. IEEE, Vol. 209, pp. 107-114.

3. Bo Y, Lei H and Changping L. (2009) Adaptive Defogging of a Single Image. In Proc of the Second International Symposium on Computational Intelligence and Design. Published in Computer Society. Vol. 1, pp. 56-59.

4. Weijiang F. Naiyang G. Xiang Z. Xuhui H. Zhigang L. (2014), Single image defogging with single and multiple hybrid scattering model IEEE International conference on security, Pattern Analysis, and Cybernetics. Vol. 209, pp. 107-114.

5. Inhye Y. Jaehwan J. Jinhee L., and Joonki P. (2010), Weighted Image Defogging Method Using Statistical RGB Channel Feature Extraction , ISOCC International conference. Vol. 20, pp. 34-35.

6. Lark K.C. Jaehee, and Alan .C. Bovik. (2014), Referenceless perceptual image defogging. SSIAI. Vol. 24, pp. 3888-3901.

7. Caraffa and Jean P.T. (2013), Markov Random Field Model for Single Image Defogging Published in IEEE Intelligent Vehicles Symposium (IV). Vol. 24, pp. 994-999.

8. Xiaochen Hu. Li Zhuo and Li Zhuo. (2014), A Moving Objects Based Real-time Defogging Method for Traffic Monitoring Videos, Proc of the 19th International Conference on Digital Signal Processing, IEEE. Vol. 109, pp. 1-6.

9. Inhye Y. Seonyung K. Donggyun K. (2012), Adaptive Defogging with Color Correction in the HSV Color Space for Consumer Surveillance System. IEEE Transactions on Consumer Electronics, vol. 58,pp. 111- 116

10. Xinwei L. Jon Y.H. (2013), Fog Removal Algorithms Survey and Perceptual Evaluation. Visual Information Processing, 4th European Workshop University of Paris. Vol. 33, pp. 118-123.

11. Ashish S, and Yogadhar P. (2014), Real time Fog technique with improved Quality through FET. IEEE (computer society) Fourth international conference on communication systems and network technologies. Vol. 26, pp. 814-818.

12. Apurva K. Philip J T and Sahoo S.K. (2014), Single Image Fog Removal Using Gamma transformation and median filtering. Annual IEEE India Conference. Vol. 44, pp. 113-118.

13. Abhishek K T and Sudipta M. (2012), Single Image Fog Removal Using Bilateral Filter, Signal Processing, Computing and Control, IEEE International Conference. Vol. 33, pp. 447-449.

14. Kaiming H , Jian S and Xiaoou T (2009), Single image haze removal using dark channel prior Computer Vision and Pattern Recognition, (2009). IEEE Conference. Vol. 46, pp. 662-669.

15. WANG L J and ZHU R, ( 2013 ), Image Defogging Algorithm of Single Color Image Based on Wavelet Transform and Histogram Equalization, Applied Mathematical Sciences, Vol. 7, no. 79, 3913 – 3921

16. Raanan F. (2008) Single Image Dehazing, Published in Journal (ACM Transactions on Graphics). Vol. 24, pp. 207-214.

17. Jing Yu, Chuangbai Xiao, and Dapeng Li (2010), Physics-based Fast Single Image Fog Removal, Signal Processing IEEE 10th Internaional Conference. Vol. 55, pp. 307-314.

18. Xinwei L, Jon Y H (2013), Fog Removal Algorithms Survey And Perceptual Evaluation, Published in Visual Information Processing, 4th European Workshop. Vol. 219, pp. 107-114.

19. Kaiming H, Jian S Xiaoou T (2009), Single image haze removal using dark channel prior, Computer Vision and Pattern Recognition,.IEEE Conference, Vol. 222, pp. 502-514.