Restoration of Hazy Videos using Dark Channel Approach and Guided Filtering

Restoration of Hazy Videos using Dark Channel Approach and Guided Filtering

Shari Thomas, Lintu Liz Thomas, Mathews M, Tonu James

Department of Computer Engineering College of Engineering

Chengannur, India

Abstract Restoration of weather degraded videos, especially hazy videos is important in computer vision/graphics. A method for de hazing of videos using dark channel method along with the use of guided filter is proposed. On the hazy video first dark channel approach is applied for transmission estimation and then for better enhancement guided filter is applied. Guided filter rely on a guidance or reference image. It has a better filtering capability and fast running time and gives us a clear haze free video. The proposed method will successfully be applied to videos; hence we can use this algorithm in even real time video applications.

KeywordsGuided filter, Dark channel, Transmission estimation, Haze removal, Video processing

1. INTRODUCTION

   The images or videos taken in bad weather conditions may degraded due to the presence of the turbid medium (e.g., particles and water droplets) in the atmosphere. They reduce the visibility of the scenes and lower the reliability of outdoor surveillance systems. So in bad weather the effect of such phenomenon needs to be removed, to recover clear haze free images or videos. Therefore, removing haze from frames of hazy videos is an important and widely demanded topic in computer vision and computer graphics areas. For this purpose we propose the method with, dark channel [2] calculation combined with guided filtering. In this method first the hazy video is divided into frames. The number of frames depends on the size of the video. Then each frame can be considered as similar to a single hazy image, and on each of those hazy images our method is applied.

   Considering the case of hazy and haze free images; in haze free images, in almost all small image blocks of any of the color (RGB) channels, there may be some pixels that have very low intensity, approximate to zero. These dark pixels can be due to shadows, colourfulness, geometry, or other factors.While taking the same hazy image, the intensity of such pixels in that channel is mainly contributed by atmospheric light. Therefore by combining this dark channel[2] approach with an image model we can calculate the amount of transmission (amount of light reaches camera without attenuation divided by the amount of light reflected from the scene point) and recover haze free image. The clarity

   of haze free image can be enhanced by removing noise from image by using filters. Here we are using a guided filter. Guided filter works with a reference or guidance image. Here we are using initial haze free image, obtained from dark channel method as our input image and an image model as guidance image. It results in a better haze free image. By applying this method repeatedly on each frame in hazy video, we get all frames as haze free and finally combining these haze free frames we get our final haze free video.

   Most of the de hazing methods proposed so far are for individual images. But many photographic and real-time videos also need to be de hazed. So, the method proposed here plays a good role in such applications. Fig. 1 shows an representationof such a video dehazing.

   Fig. 1: (a) Hazy video (b)Haze free video

   With the target of haze removal, multiple image haze removal methods are first introduced. In the multiple image haze removal method take two or more images of the same scene. This strategy increases the number of known variables, at the same time it brings more unknown so special settings must be needed to avoid too many unknown being introduced. The dichromatic method[7], takes multiple images of the same scene, under different atmospheric conditions . The main problem of this method is that weather may remain unchanged in several minutes or even hours. Polarization based method[8] is another multiple image haze removal technique. Its main problem is its settings that is, Capturing two strictly aligned polarized images is trouble.. To overcome the limitation of multiple image haze removal another strategy, single image haze removal[1,2,6] introduced. It is based on some assumption or knowledge. This strategy become more

   popular since, it requires only one image. Besides of quality and speed our new method, combining dark channel with guided filtering [9,10,11] in video dehazing is introduced. So, it is practically applicable to real time video applications.

2. BASIC METHODS

   Here we explain how our proposed method works. We know video can divided into frames and each frame is equivalent to a single image. So our hazy video is first divided into hazy frames and upon each such frame our method for haze removal is applied to obtain haze free frames. Finally by combining all these frames a haze free video is obtained.

   In our proposed haze removal method each frame undergoes two phases; transmission estimation and guided filtering[9,10,11]. It is depicted in fig. 2.

   has the high value in the dark channel is taken as the constant atmospheric light . while taking the concept of dark channel it is clear that in almost all local region which do not cover sky has very low intensity value and its tends zero. Therefore dark channel of image J becomes zero:

   J 0 (3)

   C. Transmission Estimation

   Transmission is the part of light reaches camera without attenuation from the part of light reflected from the scene point. Since it is a fraction its, value ranges between 0 and 1. The value of 0 means its completely haze and nontransparent, 1 means no haze and completely clear, value in between 0 and 1 means semitransparent. Using the above described equations 1 and 3 and by above calculated we can easily estimate the transmission value as follows; normalize haze image equation

   1 by :

   I c (x) Ac

   t(x)

   J c (x) Ac

   1 t(x)

   (4)

   Applying dark channel method on the above equation 4, then it becomes:

   Fig 2 : General overview of proposed method

   I c (x) J c (x)

   (5)

   Dark channel Ac Dark channel Ac t(x) (1 t(x))

   The method need to apply on each frame is explained in detail below:

   1. Haze image equation

      Degraded hazy image can be defined by the following

      By equating equation 3, on equation 4 it will give final transmission estimation [2] as follows:

      I c (x)

      t(x) 1 w * Dark channel Ac (6)

      equation:

      I x

      J xt x

      A1 t x

      (1)

      We can optionally introduce a constant parameter

      w(0 w 1) , its value depends on the application to keep a

      Where I(x) is an observed intensity, J(x) is the actual intensity reflected from the scene point, t(x) transmission or transparency [6] of haze and A denotes the atmospheric light [1].

   2. Atmospheric light

   Atmospheric light[1] is considered as the color of the

   very small amount of haze since if we remove the haze thoroughly, the image may seem unnatural.

   1. Haze Free Image

      From the above calculated parameters actual image intensity J can be calculated as:

      I c (x) Ac c

      atmosphere, horizon or sky. The dark channel of a hazy image

      J (x)

      t(x)

      A

      (7)

      can be calculated as follows:

      Dark channel x minc{r,g,b} (min

      y x

      I c (y))

      (2)

   2. Guided Filtering

   On the above obtained haze free image J, apply an image filtr here. We are using a guided filter so we get a better

   Where I c color hazy input image, x is local patch centered at x.

   The most brightest local region of a dark channel is

   considered as atmospheric light, since it appears as most opaque. In haze free image, most of the local area, which do not cover sky has very low intensity value in at least one of the color channel red, green and blue. Since the most opaque region gives the estimation of atmospheric light, region that

   restored image. Let q be the output of guided filtering that is our required restored image, and I and J be the two images used for filtering. Guided filtering works on input image I under the guidance of another reference image J. Here we are taking initial hazy image I as input image that is filtered under the guidance of image J, that is obtained as the haze free image in above section. Based on the application both input image and guidance image[9] can be same.

   The output image can be represented as a linear transform of b) Calculate mean value of input image I

   guidance image J as:

   meanI

   1 I

   |w| i

   qi ak Ji bk , i wk

   (8)

   iwk

   Here ( ak , bk ) are linear coefficients assumed to be

   1. Calculate correlation of guidance image J

      constant in window wk , considering square window of radius

      corr 1 (J .*J )

      r. We model the q by subtracting noise components n from p by:

      J | w |

      i i

      iwk

      qi Ii ni

      (9)

   2. Calculate correlation of input image I and guidance image J

   We aim a solution that minimizes the difference between q

   corr 1 (I .*J )

   and p and at the same time maintaining linear model. So our needs is to minimize the following cost function:

   IJ | w |

   i i

   iwk

   2.a) Calculate variance of guidance image J

   2

   E(a , b ) (a J b I ) a 2 (10)

   varJ corrJ meanJ .*meanJ

   k k k i k i k iwk

   Here , is a regularization parameter avoiding large a .

   b) Calculate the covariance between input image I and Guidance image J

   k cov corr mean .*mean

   Its solution is given by the appropriate calculation of

   a and b as follows:

   IJ IJ I J

   1. Calculate the parameters a and b

      k k a

      cov

      . / (var )

      a 1 J I p

      k 2 | w | i i k k

      (11)

      k IJ J

      k iwk

      bk pk ak k

      (12)

      bK meanI ak .*meanJ

   2. Calculate mean of a and b

      Here,

      and 2 are the mean and variance of J in window a) mean

      1 a

      k

      i

      k k a

      |w| kw

      wk ; |w | is the number of pixels in the window wk . On

      averaging final filtering output is given by:

      b) mean

      1 b

      qi ai Ji bi

      i

      b

      k

      (13)

      |w| kw

      Here, a

      1 a

      and b

      1 b

   3. Final filtering output q

   q mean .* J mean

   i | w |

   k

   kwk

   i | w |

   k a b

   kwk

   The above discussed guided filtering process can be concluded in the following steps:

   Algorithm

   Input: filtering input image I, guidance image J, regularization , radius r.

   Output: filtering output q

   1.a) Calculate mean value of guidance image J

   |w|

   1

   From the above all operations, we will got a collection of haze free frames , that is, the frames obtained from the hazy video are turned to haze free and clear frames. By combining all those frames back we get final video as haze free one.

3. RESULTS AND DISCUSSION

   meanJ

   Ji iwk

   In this section, we are discussing about the experimental results and its comparison with various methods. The fig. 3 shows input hazy image and its dark channel obtained. From

   the dark channel the most opaque region is taken as atmospheric light. From which we can easily estimate transmission and recover haze free image.

   Fig. 3: (a)Input hazy image (b)minimum of RGB valued

   (c) Dark channel

   The difference of hazy and haze free image can be cleared by its appearance. That is, haze free images are more cleared haze are removed. So, it can be used in various applications. Such a hazy and haze free image is shown in fig. 4.

   Fig. 5. Shows, the output obtained by various filters such as imfilter, hybrid median filter and guided filter when applied on our hazy image. They show the improvement by various filters. The input hazy image becomes clearer by the effect of each filter, and the effects by different filters are different. The differences in effect can be analyzed by its appearance and also by visibility metric value. Analysis of visibility metric values are shown in table also:

   Fig. 5: Haze free image of different filters

   Table 1: comparison of visibility Metric value for different methods

   Methods	Visibility Metric Value
   Previous method	17.9988
   HM filter	21.9353
   IM filter	20.3109
   Guided filter	30.3657

   From the table 1 and from fig. 4 it is clear that the guided filter has higher visibility value so, it provides better enhancement and results. Due to its better enhancing property, guided filter is combined with dark channel method for video dehazing. The resultant dehazed video representation is given in fig. 6.

   Fig. 6: Dehazing Video

4. CONCLUSION

This paper proposed a method for dehazing video. This method have better advantages over existing methods. Existing methods mainly focus on image dehazing . But here our proposed method explaines how can we perform video dehazing.

In addition to the video dehazing mainly highlighted in this paper, it shows the quality of restored image using Visibility Metric Value. For better enhancement, we use guided filter in this method. From other existing filters it has better restoration capacity( better noise removal ) and requires only less computational time. So, our proposed method provides better restoration in hazy video.

ACKNOWLEDGMENT

We would like to take this opportunity to express our gratitude to all those who have guided in the successful completion of this endeavor. We express our sincere thanks to

REFERENCES

1. Soowoong Jeong and Sangkeun Lee,"The Single Image Dehazing based on Effcient Transmission Estimation", 2013 IEEE International Conference on Consumer Electronics(ICCE).

2. K. He, J. Sun, and X. Tang,"Single Image Haze Removal Using Dark Channel Prior", Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1956-1963, 2009.

3. J. Tarel and N. Hautiere,\Fast Visibility Restoration from a Single Color or Gray level image", Proc. IEEE International Conference on Computer Vision, pp. 2201- 2208, 2009.

4. R. Fattal,\Single Image Dehazing", ACM Transactions on Graphics, vol. 27, pp.1-9, 2008.

5. R.C. Gonzalez and R.E. Woods," Digital Image Processing", second ed. Prentice Hall, 2002.

6. HE, Kaiming,\Single Image Haze Removal Using Dark Channel Prior", A Thesis Submitted in Partial Ful_lment of the Requirements for the Degree of Doctor of Philosophy in Information Engineering.

7. S.G. Narasimhan and S.K. Nayar, \Vision and the Atmosphere" Intl J. Computer Vision, vol. 48, pp. 233-254, 2002.

8. Y.Y. Schechner, S.G. Narasimhan, and S.K. Nayar, \Instant Dehazing of Images Using Polarization" Proc IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 325-332, 2001.

9. Kaiming He, Jian Sun, and Xiaoou Tang, \Guided Image Filtering" IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 35, No. 6, June 2013.

10. Jiahao Pang, Oscar C. Au and Zheng Guo, \Improved Single Image Dehazing Using Guided Filter" APSIPA ASC 2011 Xian.

11. K. He, J. Sun, and X. Tang, "Guided image filltering," in Proc. Europ. Conf. on Comp. Vis., Sep. 2010.