De-Hazing of Single Underwater Image by Multiscale Fusion

De-Hazing of Single Underwater Image by Multiscale Fusion

Kavya N.K1 , Akhila S 2 Sowmya Sunkara3

1,2,3 Department of Electronics and Communication Engineering,

B.M.S College of Engineering, Bangalore India

Abstract Underwater images are prone to hazy appearance due to absorption and scattering effect. An effective method to make an underwater image haze free has been built based on the fusion principles. The grey world algorithm along with the red channel compensation has been employed to make the image haze free. A high resolution image is got constructing the pyramids of the images and employing the multiscale fusion. The resultant de-hazed images quality is tested using Mean squared error (MSE), Peak Signal to Noise Ratio (PSNR), Normalized Cross Correlation (NCC) and Normalized Absolute Error (NAE).

KeywordsHaze, grey world, Gaussian pyramid, Laplacian pyramid, multiscale fusion.

1. INTRODUCTION

   Underwater photography faces numerous challenges, and they are not as easy as land photography. Just above the water, the camera magnification remains same as our eyes magnification. But the same doesnt hold good for underwater photography. For e.g. the huge fish that is photographed under the water seem really small in the photo. To put it up, in general the divers mask magnifies everything under the water by 25%, hence what is seen or captured is not what is actually got in the concerned photo. The focal point also changes because of the mask distortion and light refraction. In general, the focal point will always be four feet in front of the camera or around twice the length of our arms.

   There are many constraints to be taken care in underwater images as compared to land images, as they suffer from poor visibility due to absorption and scattering. Absorption reduces the energy of light and on the other hand scattering changes the direction of propagating light. These results in degradation of the image quality giving a misty appearance to the distant objects. The wavelength gets cut in accordance with the depth and this in turn fades the color of the objects captured in underwater environment. When the light propagates, due to the change in the medium that takes place, scattering effects comes into picture, which in-turn causes the blurring effect on the captured underwater image. The wavelength absorption results in color reduction of the captured underwater images, which is not usually the case with land images. High dimensional imaging is also affected by the sediments present in the water. Underwater images most often are captured with artificial lightening and because of the non-uniform lightening, the images suffer from vignette effect i.e., darkening towards the corners of the frame. Added to these, on a sunny day, flickering effects will be seen. While considering light effect, one of the major problematic challenge is color absorption. Some photos display no hues but blue or green. This happens because the hues of yellow, orange and red are

   emitted by the shorter wavelengths of light, which are quickly absorbed in water and they are also absent at depths below 10 feet. On the other hand, blue and green are emitted significantly from longer wavelengths, which thereby reaches further depth. On reaching a depth of 80 feet, blue will be the only color left, before everything turns black. These color absorption challenge can be overcome by using color- correcting filters. The second option will be to use the strobe to overcome the color absorption in underwater photography. The strobes will have adjustable settings that will allow one to control the light output for each image, giving an opportunity to create the perfect light output for each capture. Along with these, images loose much of their appeal without the contrast. Contrast is the difference between the photos lightest element and its darkest element. It provides a sense of depth and also contributes in making the colors look brighter. In order to maintain a good contrast, it is necessary to make sure photographer is close to the objects or positioning should be done in such a way that the light objects are just posed with dark backgrounds and vice -versa.

2. BACKGROUND KNOWLEDGE

   1. Effect of light in underwater

      One of the major problem that is encountered in underwater is the light levels. It progressively becomes dimmer, as we dive down. And in addition to this, the spectrum of light narrows as the red light gets diminished. In order to compensate for the changes in color spectrum, many photographers use strobe flashers to improve the quality of lighting. Refraction, the phenomenon, where in the light bends when it passes from one medium to another, occurs when light travelling in underwater passes through the air and glass of the camera held to capture the underwater environment. This causes the objects intended to capture closer and larger by 25 percent than their real position. If this is not taken care of, it will result in capturing blurry image. Water, by its nature is a natural light filter, absorbs considerable amount of light as it passes deeper through it. In general, for every 10 meters of depth, half the light is absorbed. So, if we consider a depth of 10 meters, therell be only 50 percent of the light that is seen at the surface and 25 percent of the light at a depth of 20 meters and so on. On all this, the amount of available light also doesnt remain the same, it will change depending on the time in a day. There is a general acceptance that the best time to shoot the underwater environment are between 10AM to 2PM,as this is the time when the sun is directly overhead and the water surface reflects the least amount of its light. The availability of light also depends on the

      weather conditions. In stormy weather conditions, there will be significant impact on a choppy water. Along with this the water clarity also has a significant effect on the amount of light that is got on the subjects or objects focused. There will be requirement of a strobe, if planned to shoot below 10 meters. If chosen to shoot in natural light, then staying within 0-10 meters is recommended.

   2. Survey

   In the past few years there has been several types of hardware and software solutions that the researchers have worked upon to overcome the hazy effect encountered in the captured underwater images. Hardware solutions are employed to overcome the poor visibility in the underwater images. Each of them uses the hardware to reject the backscattered light, enhance the intensity and contrast of the detected object in the image. The incorporation of hardware equipment in order to dehaze the underwater image leads to complexity, makes the technique expensive and power consuming. With all this in mind several software algorithms have been proposed in order to make the underwater image haze free.

   One traditional and commonly used method to dehaze the image is the histogram equalization [3]. The paper describes the utilization of adaptive histogram equalization to defog the captured image. Histogram equalization is a technique for adjusting image intensities to enhance the image contrast. It is the redistribution of the intensities of the image. In the mentioned paper, mean enhancement and adaptive histogram equalization are employed for defogging. YCbCr model is employed, where luminance image is used to remove the fog and show a clear picture. On the other hand, two more images are used to preserve the color. And finally, all the three images are merged to obtain the final fog free image. This method employed gives good result with respect topreserving the edges and also provides good hue preservation. But on analysis the results seem to be poor, by failing to remove the color shift and gives bluish appearance.

   The gray world algorithm [4], is a classic auto white balance algorithm. The gray world algorithm assumes that the mean value of the average reflection of the natural scene in standard light is a constant value, which is approximately gray. The gray world algorithm assumes that the average reflectance in the scene with rich color changes is achromatic. That is, the average of three-color channels Ravg, Gavg, Bavg should be equal. In the physical sense, the gray world algorithm assumes that the mean value of the average reflection of the natural scene in standard light is a constant value, which is approximately gray. The employment of histogram equalization has resulted in giving bluish appearance to the image. The results showed better edge preservation and with reduced noise. On employing grey world algorithm, bluish tone encountered with histogram equalization was removable but this algorithm creates red artifacts.

   The work in [6], comes with the observations as the green channel is well preserved in underwater compared to the red and blue ones, light with longer wavelength that is red will be lost first. Therefore, red attenuation has to be compensated, which here is carried out by adding a fraction of green channel is to the red channel. Compensation is

   made proportional to the mean red and mean green values, with an assumption in grey world algorithm that all the channels have same mean before attenuation. The authors have made an observation, that in order to avoid saturation of red channel during the application of gray world algorithm to dehaze the underwater image, the information in the green channel must not be transferred to the regions where the red channel information is significant. Thus, they have concluded that following this method will help avoid the reddish appearance introduced in the gray world algorithm.

   Multiscale fusion principles [9], is employed for increasing the visibility. Set of weight maps for the hazy image are defined. These weight maps are normalized. The weight maps calculated and normalized are multiplied with the input image. Different weight maps such as the saliency (measures the saliency level in the considered image), Laplacian (speaks about the global contrast), saturation (It takes care of adoption to the chromatic information) and so on, according to the need of the image can be calculated. Image fusion finds its application in image computing, multispectral video enhancement, defogging and HDR imaging.

   The authors in the paper [12], have surveyed the quality assessment methods for signal processing. The have explored the both subjective and objective methods of quality assessment. They have spoken open about the types of subjective and objective assessment methods and the advantages of one over the other.

3. METHODOLOGY

   1. WORK FLOW

      1. As an initial step of the process an image from the underwater environment of width 355 and height 1884 is considered.

      2. On the considered image histogram equalization is applied in order to re-distribute the intensity of the image.

      3. For the histogram equalized image, gray world algorithm technique is implemented on the image. Grey world algorithm takes the average of all the channels. The color of the light source is computed. Dividing the average obtained by the color light source calculated, the grey world output will be obtained.

      4. The image obtained on application of gray world algorithm color cast which is usually seen in the underwater image, for which red channel has to be compensated.

      5. This color cast is compensated by adding part of the well-preserved green channel to the red channel.

      6. Two versions of the so obtained compensated image is derived.

      7. Gamma correction is applied on the white balanced image to correct the global contrast.

      8. On applying un-sharp masking principle on the white balanced image, sharpened version of the grey world output will be obtained.

      9. Gaussian pyramid of the gamma corrected image is built and Laplacian pyramid of the sharpened image is constructed.

      10. Multiscale fusion of the constructed Laplacian pyramid of sharpened image and Gaussian pyramid of the gamma corrected image is obtained. Thus, a dehazed high resolution underwater image is obtained.

      11. The MSE (Mean Squared Error), PSNR (Peak Signal to Noise Ratio), Normalized Cross Correlation (NCC) and Normalized absolute Error (NAE) is measured at each level of dehazing. Thus, the results are compared.

          INITIAL IMAGE

          INITIAL IMAGE

   2. Gray world algorithm

   The gray world algorithm is a classic auto white balance algorithm. The gray world algorithm assumes that the mean value of the average reflection of the natural scene in standard light is a constant value, which is approximately gray. The gray world algorithm assumes that the average reflectance in the scene with rich color changes is achromatic. That is, the average of three-color channels Ravg, Gavg, Bavg should be equal. In the physical sense, the gray world algorithm assumes that the mean value of the average reflection of the natural scene in standard light is a constant value, which is approximately gray. But the average value of three channels is not equal in non-standard light source. That is, the means of three channels are greater or less than the grey value. Thereby a scaling value is computed to overcome this drawback.

   HISTOGRAM EQUALIZATION

   HISTOGRAM EQUALIZATION

   The scaling value is computed as:

   GRAY WORLD IMAGE

   GRAY WORLD IMAGE

   Scaling (1)

   RED CHANNEL COMPENSATION

   RED CHANNEL COMPENSATION

   This scaling value is used to scale the entire image linearly. That is as follows:

   Gray world image= (2)

   GAMMA CORRECTED IMAGE

   GAUSSIAN PYRAMID OF GAMMA CORRECTED IMAGE

   SHARPENED VERSION

   LAPLACIAN PYRAMID OF SHARPENED IMAGE

   On application of the classic white balanced, gray world algorithm, there is a better visibility in the so obtained image as compared to the hazy image that had been considered initially. But on application the gray world algorithm, though the image gave a better quality, the red artifacts has come into picture. Now there is a necessity to compensate for this red artifact that has been encountered. Since red color being the longer wavelength, gets lost first while travelling through the water.

   Based on the observation on underwater environment [6], following equation can be employed to overcome the red artifacts created.

   MULTISCALE FUSION OF

   GAUSSIAN AND LAPLACIAN PYRAMID

   HR IMAGE

   HR IMAGE

   COMPARISION THROUGH QUALITY METRICES

   COMPARISION THROUGH QUALITY METRICES

   Figure 1: Proposed system

   Irc(x) = Ir(x) +. (Iag Iar). (1 Ir(x)). Ig(x) (3) Where, Irc is compensated red channel at every pixel location (x), Ir and Ig are red and green color channels. Iag, Iar are mean values of Ig and Ir respectively.

4. RESULTS AND DISCUSSION

A. RESULTS

The figures below shows the initial underwater hazy image considered and the modifications done over the image to arrive at the final de-hazed image with high resolution.

Input image Hist plot before Heq

Gray world output Gamma corrected image

Sharpened image

Histogram equalized image Hist plot after Heq

Red Channel compensated image Gaussian pyramid of the gamma corrected image

Laplacian pyramid of sharpened image

Final dehazed image

Four quality matrices are considered in order to measure the quality of the dehazed image obtained at each step.

Mean squared error (MSE): In mean squared error full reference quality measure of the image, the error signal is computed by finding the differnce between test and the reference image, then the average of the error signal is computed. MSE is computed as below:

(4)

Peak Signal to Noise Ratio (PSNR):

PSNR, is the ratio between the maximum power of the signal considered to the power of the distorted signal and it is represented in decibels. PSNR is computed as below:

(5)

Normalized Cross Correlation (NCC):

Normalized Cross Correlation speaks about the comparison of the processed image and that of the reference image. An eventual decrease in the value of NCC tells that the image quality has become good.

(6)

Normalized Absolute Error (NAE):

Normalized Absolute Error is one of the full referenced quality image metrics, whose higher value indicated that the image is of good quality.

NAE can be defined as:

(7)

The MSE value as been decreasing as proceeded with the dehazing as can be observed from the table 1. The PSNR value as increased with progress. NCC and NAE have been decreasing with the progress. From all the above observation and the learning about the metrics from the previous chapter, it can be concluded that the image obtained is thus of good quality.

Table 1: Quality comparison

. CONCLUSION

An underwater hazy image is considered and a series of steps

are involved to obtain a high resolution dehazy image. Multiscale fusion of the two Grey world versions obtained are employed in order to accomplish the above mentioned result. The results at each stage are measured for quality using MSE, PSNR, NCC and NAE.

REFERENCES

1. M. D. Kocak, F. R. Dalgleish, M. F. Caimi, and Y. Y. Schechner, A focus on recent developments and trends in underwater imaging, Marine Technol. Soc. J., vol. 42, no. 1, pp. 5267, 2013.

2. Y. Y. Schechner and Y. Averbuch, Regularized image recovery in scattering media, IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 9, pp. 16551660, Sep. 2007

3. Ashok Shrivastava, Dr. Sanjay Jain, Single image dehazing based on one dimensional linear filtering and adoptive histogram equalization method, IEEE conference . Vol. 50, pp 88 -97, 2016.

4. C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, Enhancing Underwater images and videos by fusion, in Proc. IEEE CVPR, pp. 8188, jun 2012.

5. K. He, J. Sun, and X. Tang, Single image haze removal using dark channel prior, IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 23412353, Dec. 2016.

6. C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, Color balance and fusion for underwater image enhancement, IEEE Trans,vol 27, jan 2018.

7. C. O. Ancuti and C. Ancuti, Single image dehazing by multi-scale fusion, IEEE Trans. Image Process., vol. 22, no. 8, pp. 32713282, Aug. 2013.

8. C. O. Ancuti, C. Ancuti, and P. Bekaert, Effective single image dehazing by fusion, in Proc. IEEE ICIP, pp. 35413544, Sept 2014.

9. R. Achantay, S. Hemamiz, F. Estraday, and S. Susstrunk, Frequencytuned salient region detection, in Proc. IEEE CVPR, pp. 15971604, Jun. 2009.

10. P. Burt and T. Adelson, The laplacian pyramid as a compact image code, IEEE Trans. Commun., vol. COM-31, no. 4, pp. 532 540.