Color Constancy Using Standard Deviation of Color Channels

Color Constancy Using Standard Deviation of Color Channels

1 Iris Paul,

PG Scholar CSE, Ilahia College of Engineering & Technology, Ernakulam, Kerala

2Sreeja. B. P,

Faculty of IT, Karpagam College of Engineering, Coimbatore, TN, India.

3Saratha Devi. G,

Faculty of IT, Karpagam College of Engineering, Coimbatore, TN, India.

Abstract

Color constancy algorithms are generally based on the simplifying assumption that the spectral distribution of a light source is uniform across scenes. However, in reality, this assumption is often violated due to the presence of multiple light sources. In this paper, we will address more realistic scenarios where the uniform light-source assumption is too restrictive. First, a methodology is proposed to extend existing algorithms by applying color constancy locally to image patches, rather than globally to the en-tire image. After local illuminant estimation, these estimates are combined into more robust estimations, and a local correction is applied based on a modified diagonal model Quantitative and qualitative experiments on spectral and real images show that the proposed methodology reduces the influence of two light sources simultaneously present in one scene.

1. Introduction

   Color constancy is the color of the object remains constant under varying illuminant conditions. For example a green apple for instance looks green to us at midday, when the main illumination is white sunlight, and also at sunset, when the main illumination is red. Color of the light source can change the object colors that is light source has the significant influence in the object colors. Therefore, the same object, taken by the same camera but under different illumination, may vary in its measured color values. This may negatively

   effects the digital images and many applications like object recognition, tracking ect.

   The aim of color constancy is to correct for the effect of the illuminant color, either by computing invariant features, or by transforming the input image such that the effects of the color of the light source are removed. Number of color constancy algorithms is proposed. These color constancy algorithms are based on the assumption that the light source across the scene is uniform

   1. A Basic Color Processing

      1.1.1 Raw

      The vast majority of color image sensors use a Bayer pattern of red, green and blue filter to make it a color sensor. So each pixel is just able to detect one color, it sees red, green or blue. As green is the most important color for the luminance information, we have two green pixel for one blue and one red pixel

      1. Demosaicing

         To get the red, green and blue information per pixel, the important step of demosaicing interpolates the missing information. This is a very vital part of image quality and therefore a well kept secret of every manufacturer what they do in detail. As the different filter results in a different sensitivity to light and a low signal strength, the noise level can be very different. In the demosaicing process, the noise is spread over the neighbors, the noise in the different color channels is correlated to each other.

      2. White Balance

         The sensitivity of the different color channels can be very different in a digital camera. To get correct colors as they appear to the human visual system, the camera to control the gain of the different channels differently. After white balancing, neutral areas in the image appear neutral and have nearly the same digital value for red, green and blue. COLOR CORRECTION MATRIX (CCM)

         Each camera has an individual spectral sensitivity. So the RGB output of a camera is specific to that camera. To get consistent results for all cameras, this RGB_camera has to be transformed into a defined and know colorspace. In most cases this is sRGB, but it can be any other color space. To transform the values from RGB_camera to e.g. sRGB, one has to apply a 3×3 color correction matrix (CCM) to the data.

      3. Gamma

         Until now, the image data is still linear. So doubling the light intensity will double the digital value in the image, not matter if checked in the dark areas or in the bright areas if the image. To get a correct representation on output devices, images normally have a gamma function applied. This tonal curve is applied to the image at the end of image processing, as from now on the image data is non- linear. The goal of computational color constancy is to find a nontrivial illuminant invariant description of a scene from an image taken under unknown lighting conditions. This is often broken into two steps. The first step is to estimate illuminant parameters, and then a second step uses those parameters to compute illumination independent surface descriptors. These descriptors can be quite abstract, but here we simplify matters by specifying that the illumination invariant description is an image of the scene as if it were taken under a known, standard, canonical, light [8]. The choice of the canonical illuminant is somewhat arbitrary. For image reproduction applications it makes most sense to use an illuminant for which the camera is balanced, and this is the choice we have used To the extent that this same scaling works for the other, nonwhite patches, we say that the diagonal model holds.

2. Proposed Work

   The proposed work focuses on the sampling of patches of the image. There are five steps involved in color constancy for multiple light sources[1].They are the sampling of image, using grid based sampling or Dense sampling then Key-point based sampling and the segmentation based sampling. Grid-

   based sampling has the advantage that the patches contain varied amount of information, whereas patches that are selected using segmentation will generally contain similar colors (and hence less variation). Finally, key-point sampling is specifically suited for edge-based color constancy methods as the key points are located around edges and junctions. In this paper, key points are located using the Enhanced Harris detector at multiple scales. using a similar patch size as the grid-based sampling, whereas segmentation is performed using the graph cut method. After sampling of patches we estimate the illumination from each patch and then combine the estimates to get more robust estimations. Then local correction is made using the modified diagonal model. In this paper we propose to use only the Harris Detector for Key-point sampling so that we get more number of Key- points based on edges and functions.

   Fig 2: Five steps involved in color constancy

   1. Sampling

      The first step is to sample patches from the image. Key-point based sampling using Harris function so we get the number of patches, then from each patch illuminant estimation is computed. It is assumed that the color of the light source is uniform over each patch .Therefore, the patch size should be limited.

      The first step is to sample patches from the image. For each patch, estimation is computed of the light source valid for that patch. It is assumed that the color of the light source is uniform over each patch, which is a reasonable assumption in practice. Therefore, the patch size should be limited but of sufficient size to extract enough image properties to accurately esti-mate the light source. Different sampling strategies can be used, e.g., dense sampling, interest points, and segmentation. Dense or grid- based sampling and segmentation-based sampling ensure that the union of all patches covers the full image. Further-mre, segmentation-based sampling can result in boundaries between segments that naturally fol-lows the boundary between light sources (as most segmentation algorithms are sensitive to

      changes in the illuminant). Grid-based sampling has the advantage that the patches contain varied amount of information, whereas patches that are selected using segmentation will generally contain similar colors (and hence less variation). Finally, key-point sampling is specifically suited for edge-based color constancy methods as the key points are located around edges and junctions. In this paper, keyoints are located using the Harris detector using a similar patch size as the grid-based sampling,

   2. Illuminant estimation.

      The illuminant for each patch is assumed to be spectrally uniform, traditional color constancy methods is ap-plied on every patch to estimate the local illuminant. The framework proposed in allows for systematically generating color constancy as follows:

      1

      illuminant classification, where every pixel is assigned to one of the estimated light sources. After back-projection, a pixelwise illuminant estimate is obtained.

      2.5. Color correction

      Finally, using the pixelwise estimates, the output image is constructed. Trans-forming the input image so that it appears to be taken under a white light source is an instantiation of chromatic adaptation, Many methods to do this exist, but all assume that the color of the light source in the input image is known. Since the focus in this paper is to estimate the illuminant, the diagonal model, or von Kries model is used.

      p

      p

      The proposed system also have five steps involved but gives more attention to key-point based sampling where we identified more interest points using the Harris function that is the cornerness(R)

      n I ( X ) p

      c

      c

      c,

      X

      dx

      kLn, p,

      Cornerness® = det( ( X ,I ,D ))

      trace2(µ(X, I ,D ))

      Where Ln, p, is used to denote different instantiations of the framework.

      1. Combination of estimates.

         Since there is only a lim-ited amount of information available when using a relatively small patch for the estimation of the light source, this may introduce estimation errors. To overcome this lack of information, patches that are taken from parts of the image that are illuminated by the same light source are combined to form a larger patch.They form as clustures Since there is only a lim-ited amount of information available when using a relatively small patch for the estimation of the light source, this may introduce estimation errors. To overcome this lack of information, patches that are taken from parts of the image that are illumi-nated by the same light source are combined to form a larger patch (and consequently result in a more ac-curate estimate). Patches that are illuminated by the same light source are likely to vote for the same il-luminant, as illustrated. Assuming that the number of clusters is

         These are the invariant features of the image with respect to rotation and illuminance.These points are more than enough to make the image color constant.Our proposed method doesnt support Harris Detector with multiscale point detection by applying the laplacian-of-Guassian,since only limited points are obtained, which will not be enough to get the accurate color constancy of an image.

         Images in the proposed method are divided into patches, which are assumed to be small enough such that it is consistent with the uniform spectral assumption. For each patch, illuminant estimation is obtained by using a standard color constancy algorithm (based on a uniform light source). For simplicity, although other color constancy methods can be used, we focus on the five instantiations , which include pixel and derivative-based methods. Multiple light-source estimates can be simultaneously taken into account, but in this paper, the focus is on single estimates per patch. Since the used algorithms merely estimate the chromaticity of the light source, every estimate is normalized for intensity.

         known the chromaticities can be grouped together using any clustering algorithm.

         r R

         R G B

      2. Back-projection

      All the clusters. After the different estimates

      g G

      R G B

      are grouped together into groups, the result can be back-projected onto the original image to identify the locations in the image that are illuminated by each of the estimated light sources. This results in an

      The illuminant over each patch is represented by a 1 2 vector. The theoretical situation could occur where the illuminant estimate result in a black light source.i e R=G=B=0.

      1. Key-point Based Sampling

         In the case of key-point based sampling we are detecting the points in the object which do not vary under different lluminations.To get the key- points most algorithms uses the Harris Detector with multi-scale values. Harris Detector uses the scale- adapted Harris function

         Cornerness® = det( ( X ,I ,D ))

         trace2(µ(X, I ,D ))

         to localize points in scale-space. It then selects the points for which the Laplacian-of-Gaussian,

         LoGX , n L X , L X ,

         LoGX , n L X , L X ,

         2

         n

         n

         xx

         xx

         n yy n

         and attains a maximum over scale. The algorithm consists of two steps:

         1. Multi-scale point detection

            We first build a scale space representation with the Harris function for pre-selected scales, where scale factor between successive levels is set. At each level of the representation we extract the interest points by detecting the local maxima in the 8- neighborhood of a point x. A threshold is used to reject the maxima of small cornerness, as they are less stable under variations in imaging conditions.

         2. Iterative selection of the scale and the location

            For each point we then apply an iterative algorithm that simultaneously detects the location and the scale of interest points. The extrema over scale of the Laplacian-of-Gaussian are used to select the scale of interest points. We reject the points for which the Laplacian-of-Gaussian response attains no extremum and for which the response is below a threshold. Given an initial point x with scale, the iteration steps are: Find the local extremum over scale of the LoG for the point, otherwise reject the point. Detect the spatial location of a maximum of the Harris measure nearest to x for the selected scale Go to Step 1 if not in the investigated range of scales or spatial location.

            These are the two steps involved in the Key- point based sampling. but they simply cannot sample densely enough to produce leading-edge classification results. Performance almost always increases with the number of patches sampled. So we will only get limited patches so the result may vary or it is very difficult to get the accurate result. So grid based samplings were always preferred.

            The proposed method uses this scale adapted Harris Detector. Here we go through the multi-scale point detection only i e the cornerness(R) which will give the invariant features of the image such as the

            points which will not vary with change in rotation, change in illumination, change in viewing angle, and also those points will be easy to extract ,Guassian filter is applied to remove the high frequency noises in the image the Laplacian-of-Gaussian is not used so that we get more key-points as compared to that of the other. The cornerness is only used to get the key- points. Performance normally increases with large number of samples, by using this we get more sampled patches and the performance can be increased.

            Cornerness® = det( ( X ,I ,D ))

            trace2(µ(X, I ,D ))

            Cornrness (R) are the invariant to image rotation. The value of that points does not vary with different lights. so here we get large number of points and this help us to get more accurate result. Now these patches are considered independently and estimating the illumination of each patch. Each patch is assumed to have uniform light source. After estimating the illuminance these estimates are combined to form more robust estimations.The estimates with similar light source are combined to form the clusters.Assuming that the number of clusters is known, the chromaticities can be grouped together using any clustering algorithm. After the different estimates are grouped together into groups, the result can be back-projected onto the original image to identify the locations in the image that are illuminated by each of the estimated light sources. This results in an illuminant classification, where every pixel is assigned to one of the estimated light sources. After back-projection, a pixelwise illuminant estimate is obtained. Finally, using the pixelwise estimates, the output image is constructed. Trans- forming the input image so that it appears to be taken under a white light source is an instantiation of chromatic adaptation. Many methods to do this exist and but all assume that the color of the light source in the input image is known. Here we estimate the illuminant, the diagonal model, or von Kries.The aim of diagonal model or von kries model is that to transform the input images, taken under an unknown light source, into colors as if they appear under a canonical light source

            Ic = u,cIu

            Where Iu is the image taken under an unknown light source.Ic is the image transformed, which appears as if it is taken under the canonical illuminant. u,c is the mapping diagonal matrix. Thus we get a color constant image now we are identifying the source camera for the color constant image. The first step involves the feature extraction of the color constant image then comes the feature vector classification

            and finally SVM classifier is used and the source camera is identified.

            All features come from image quality metrics (IQM) of the re-balanced images. The under-lying philosophy is, for re-balanced image, the less change from re-balancing, the better quality it is , compared to the original image. This quality metrics thus could be used to identify the Mean Absolute Error (MAE), Mean Square Error (MSE), Normalized MSE, Maximum Difference, Structural Content ect and again the Feature selection is used to reduce the noise in the features and to eliminate outliers. For simplicity and computational reasons, we use the sequential backward feature se-lection (SBS) algorithm. This method attempts to optimize some criterion by removing features from an initial candidate feature set. In our implementation, the ensemble of features of 17 camera models is used as initial candidate set, and the SVM classifier accuracy is used as optimization criterion. Since SBS eliminates four features from the initial set of features, thus in all experiments we use a feature vector of dimension 404.

            We use support vector machine (SVM) of the RBF kernel to test the effectiveness of our proposed features, with C = 27, = 2 7.5 we mainly focus on multiple class identification.Furthermore, we observe that the methods that are based on the same assumption trend to produce similar results. For example, methods based on the gray-world assumption may receive the same illuminant estimation, thus more likely gives similar results. Finally, we we observe that methods based on different assumptions tends to give much larger color changes. There are two questions that we must address to solid our theory. First, is the AWB performed at the end of imaging pipeline? The answer is NO. Since at least, the JPEG compression happens after it. But the lucky thing is that, from analysis of imaging pipeline, it is reasonable to believe that some major operations in DIP happen ahead of white- balance, including infrared rejection, gamma correc- tion, demosaicking, lens aberration, antialiasing, etc. Also, our experiments show that for many images, we

            can find the AWB algorithm that has little effect when performed again. (Mean Square Error < 0.5). We attribute this different to the image quality degradation due to JPEG compression. What is more, sometimes, for a high quality images (compression quality 98%), we can find the exact AWB algorithm that will be side effect free. From above, we can reasonably assume that white-balance is performed near the end of imaging pipeline, thus the proposed method does not suffer the side effects from other processes applied inside DIP. The second question is, how could we assure that we have find the right AWB method? Since most digital cam-eras include multiple AWB methods designed for different lighting

            conditions. And what is worse, the actual method that is used inside each camera is unknown. SVM classification is performed to identify the source camera of the given image.

      2. SVM Classifier

      We use support vector machine (SVM) of the RBF kernel to test the effectiveness of our proposed features, with C =2 7 , = 2 7.5 .All algorithms studied here make some assumptions about the statistics of the reflectances to be encountered, and most make assumptions about the illuminants that will be encountered. The gray world algorithms make assumptions about the stability of the expected value of scene averages; SCALE-BY- MAX makes a similar assumption about the maximum in each channel; the gamut mapping algorithms make assumptions about the ranges of expected reflectances and (for some variants) illuminants. Each method for choosing the solution makes additional assumptions. The neural net method and color by correlation methods go further and model the occurrence distributions. As assumptions get stronger, the prospect for success increases. It remains an open question to the extent that vulnerability to failures of the assumptions also increases. We seek algorithms which can exploit reasonable assumptions, preferably backed by empirical studies, but which are not overly sensitive to common failures of these assumptions. Our experiments indicate that the methods which emphasize the use of input data statistics, specifically Color by Correlation and the neural net algorithm, are potentially the most effective at estimating the chromaticity of the scene illuminant. Some of the 3-D variants of Forsyths gamut-mapping method also do well, and these algorithms have the advantage that they are able to also estimate the illuminant brightness.

      When specularities are present, these methods do chromaticity is of interest, full color algorithms should be considered. Our detailed study of the effect of specularities on algorithms showed that their effect is significantly algorithm dependent. We also found that the effect of subsequent clipping of specular values is again algorithm dependent. Specularities are very common and are often clipped in standard cameras, especially when the aperture is automatically controlled.

3. Results and Analysis

   1. Select the image from the dataset

   2. Key-point based Sampling

      Using Harris Detector we sample the input image. Harris Detector algorithm gives invariant features of the image. The Gaussian filtering and Laplasian of Gaussian is applied.

      Key-point sampling using only Harris Function

      This is enhanced Harris Detector which gives more number of patches. This finds the cornerness

   3. Color constant Image(Gray World)

      This gives a color constant image, where we displayed gray world algorithm. All other color constancy methods are also needed for the identification of camera so we just showed all other methods too.

   4. Source camera prediction

      The camera identified as AGFA-DC-504-0

4. Conclusion

   Practically almost all digital cameras on market use JPEG compression to stor the images. When someone modifies an image afterwards, the result is saved with JPEG compression again, resulting in double JPEG compression. To test whether our proposed method would be resistant to double JPEG compression, we compressed the original JPEG image again with 75% quality metric. To analyze the robustness of the proposed approach, we used the same experimental setup as the first experiment in section. Training is performed on 60% of the original images; while testing is performed using the rest images that are double JPEG compressed. Experiment shows the average prediction accuracy is 89.90%. Compared with the 99.14% for original accuracy, the performance decreased a lot. When only 5 cameras are used, the prediction accuracy is still 98.40%, while its original accuracy is 99.24%. This experiment proved that double JPEG compression disturbs the consistency of image quality characters. When the number of camera devices grows larger, especially with cameras of close models, prediction accuracy decreases a lot. we propose a novel method to identify the source camera by using the AWB residue pattern. Experimental results on a large-scale data set show the proposed method is very effective. Moreover, the prediction accuracy almost does not degrade as the number of different cam-eras increases, demonstrating the scalability of the proposed method.

   Finally, we show that even for different devices of the same model and brand, the proposed method is still able to distinguish among them. Although we only do the source camera identification, the same idea could be applied to various applications, including but not limited to copy-move detection and steganalysis, as well as reverse engineering.

5. References

[1].Arjan Gijsenij, Member, IEEE, Rui Lu, and Theo Gevers, Member, IEEE color constancy for multiple light sources.

[2].Zhonghai Deng,Arjan Gijsenij,Jingyuan Zhang Source Camera Identification Using Auto-White Balance Approximation 2011 IEEE International Conference on Computer Vision 978-1-4577-1102-2/11/$26.00 c 2011 IEEE

[3]. J. van de Weijer, T. Gevers, and A. Gijsenij, Edge- based color con-stancy,IEEE Trans. Image Process., vol. 16, no. 9, pp. 22072214, Sep. 2007.

[4]. Arjan Gijsenij and Theo Geveers Color Constancy by Derivative-based Gamut Mapping

[5]. A. Gijsenij and T. Gevers, Color constancy using natural image statis-tics, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 18.

[6]. J. van de Weijer, T. Gevers, and A. Gijsenij, Edge- based color con-stancy,IEEE Trans. Image Process., vol. 16, no. 9, pp. 22072214,Sep. 2007

[7]. Yongjian Hu ,Chang-Tsun Li an improved algorithm for camera model identification using inter-channel demosaicking traces 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing

[8].K. San Choi, E. Lam, and K. Wong. Source camera identifi-cation using footprints from lens aberration. Digital Photog-raphy II SPIE, 6069(1):172179, 2006

[9]. J. Adams, K. Parulski, and K. Spaulding. Color processing in digital cameras. 18(6):2030, 1998.

1. B. Gunturk, J. Glotzbach, Y. Altunbasak, R. Schafer, and R. Mersereau. Demosaicking: Color filter array interpola-tion in single chip digital cameras. IEEE Signal Processing Magazine, 22(1):4454, 2005.

2. E. Land.The retinex theory of color vision. Scientific Amer-ican, 237(6):108128, December 1977.

3. T. V. Lanh, K.-S. Chong, S. Emmanuel, and M. Kankan-halli.A survey on digital camera image forensic

methods. In Proc. IEEE International Conference on Multimedia and Expo, pages 1619, 25 July 2007.