Prediction Based CFA Image Compression and Demosaicking Using Lagrange Interpolation.

Prediction Based CFA Image Compression and Demosaicking Using Lagrange Interpolation.

Kailash babu B.S Mrs.Asha N

4th Sem M.tech, Asst. Prof, Dept of CSE

APSCE, Bangalore APSCE, Bangalore

ABSTRACT

In most digital cameras, Bayer CFA images are captured and demosaicing is generally carried out before compression. Recently, it was found that compression-first schemes outperform the conventional demosaicing-first schemes in terms of output image quality. An efficient prediction-based lossless compression scheme for Bayer CFA images is proposed in this paper. It exploits a context matching technique to rank the neighboring pixels when predicting a pixel, an adaptive color difference estimation scheme to remove the color spectral redundancy when handling red and blue samples. CFA image compression is carried out first and in the later stage same compressed CFA image is interpolated using Lagrange interpolation

.Simulation results show that the proposed compression and interpolation scheme can achieve a better compression performance than the existing interpolation algorithm.

INTRODUCTION

To reduce cost, most digital cameras use a single image sensor to capture color images. A Bayer color filter array (CFA) as shown in Figure 1, is usually coated over the sensor in these cameras to record only one of the three color components at each pixel location. The resultant image is referred to as a CFA image in this paper hereafter. In general, a CFA image is first interpolated via a demosaicing process to form a full color image before being compressed for storage. Figure 2a shows the workflow of this imaging chain. Recently, some reports indicated that such a demosaicing-first processing sequence was inefficient in a way that the demosaicing process always introduced some redundancy which should eventually be removed in the following compression step. As a result, an alternative processing sequence which carries out compression before demosaicing as shown in Figure 2b has been proposed lately. Under this new strategy, digital cameras can have a simpler design and lower power consumption as computationally heavy processes like demosaicing can be carried out in an offline

powerful personal computer. This motivates the demand of CFA image compression schemes. There are two categories of CFA image compression schemes: lossy and lossless. Lossy schemes compress a CFA image by discarding its visually redundant information. These schemes usually yield a higher compression ratio as compared with the lossless schemes. Schemes presented in are some examples of this approach. In these schemes, different lossy compression techniques such as discrete cosine transform, vector quantization sub-band coding with symmetric short kernel filters , transform followed by JPEG or JPEG 2000 and low-pass filtering followed by JPEG-LS or JPEG 2000 (lossless mode) are used to reduce data redundancy. In some high-end photography applications such as commercial poster production, original CFA images are required for producing high quality full color images directly. In such cases, lossless compression of CFA images is necessary. Some lossless image compression schemes like JPEG-LS and JPEG2000 can be used to en- code a CFA image but only a fair performance can be attained. Recently, an advanced lossless CFA image compression scheme (LCMI) was proposed. In this scheme, the mosaic data is de-correlated by the Mallat wavelet packet transform, and the coefficients are then compressed by Rice code.

In this paper, a prediction-based lossless CFA compression scheme as shown in Figure 3 is proposed. It divides a CFA image into 2 sub-images: a green sub-image which contains all green samples of the CFA image and a non-green sub-image which holds the red and the blue samples. The green sub-image is coded first and the non- green sub-image follows based on the green sub-image as a reference. To reduce the spectral redundancy, the non- green sub-image is processed in the color difference domain whereas the green sub-image is processed in the intensity domain as a reference for the color difference content of the non-green sub-image. Both sub-images are processed in raster scan sequence with our proposed context matching based prediction technique to remove the spatial dependency. The prediction residue planes of the

two sub-images are then entropy encoded sequentially with our proposed realization scheme of adaptive Rice code.

Experimental results show that the proposed compression scheme can effectively and efficiently reduce the redundancy in both spatial and color spectral domains. As compared with the existing lossless CFA image coding schemes such as [10-12], the proposed scheme provides the best compression performance in our simulation study.

CONTEXT MATCHING BASED PREDICTION

The proposed prediction technique handles the green plane and the non-green plane separately in a raster scan manner. It weights the neighboring samples such that the one has higher context similarity to that of the current sample contributes more to the current prediction. Accordingly, this prediction technique is referred to as context matching based prediction (CMBP) in this paper. The green plane (green sub-image) is handled first as a CFA image contains double number of green samples to that of red/blue samples and the correlation among green samples can be exploited easily as compared with that among red or blue samples. Accordingly, the green plane can be used as a good reference to estimate the color difference of a red or blue sample when handling the non-green plane (non-green sub- image).

Prediction on the green plane

As the green plane is raster scanned during the prediction and all prediction errors are recorded, all processed green

samples are known and can be exploited in the prediction of the pixels which have not yet been processed. Assume that we are now processing a particular green sample g(i,j) as shown in Figure 4a. The four nearest processed neighboring green samples of g(i,j) form a candidate set g(i,j)={g(i,j-2), g(i-1,j-1), g(i-2,j), g(i-1,j+1)}. The candidates are ranked by comparing their support regions (i.e. context) with that of g(i,j). The support region of a green sample at position (p,q), Sg(p,q), is defined as shown in Figure 5a. In formulation, we have Sg(p,q)={(p,q-2), (p- 1,q-1), (p-2,q), (p-1,q+1)}. The matching extent of the support region of g(i,j) and the support region of g(m,n) for g(m,n)g(i,j) is then measured by

Though a higher order distance such as Euclidian distance can be used instead of eqn.(1) to achieve a better matching performance, we found in our simulations that the improvement was not significant enough to for its high realization complexity.

Figure 4. Positions of the pixels included in the candidate set of (a) a green sample and (b) a red/blue

Sample

Figure 5. The support region of (a) a green sample and (b) a red/blue sample

Let g(mk,nk)g(i,j) for k=1,2,3,4 be the 4 ranked candidates of sample g(i,j) such that

D(Sg (i,j) , Sg (mu ,nu ) ) <= D(Sg (i,j) , Sg (mv ,nv ) ) for 1u<v4. The value of g(i,j) can then be predicted with a prediction filter as

where wk for k=1,2,3,4 are normalized weighting coefficients such that

Let Dir(i,j){W, NW, N, NE} be a direction vector associated with sample g(i,j). It is defined as the direction pointed from sample g(i,j) to g(i,j)s 1st ranked candidate g(m1,n1). Figure 6 shows its all possible values. This definition applies to all green samples in the green sub- image. As an example, Figure 7 shows the direction map o a testing image shown in Figure 8. If the direction of g(i,j) is identical to the directions of all green samples in Sg(i,j), pixel (i,j) will be considered in a homogenous region and g (i, j) will then be estimated to be g(m1,n1) directly. In formulation, we have

Figure 6. The four possible directions associated with a green pixel

Which implies {w1, w2, w3, w4} = {1,0,0,0}. Otherwise, g(i,j) is considered to be in a heterogeneous region and a pre-defined prediction filter is used to estimate g(i,j) with eqn.(2) instead.

In our study, wk are obtained by quantizing the training result derived by linear regression with a set of training images covering half of the test images shown in Figure 8. They are quantized to reduce the realization effort of eqn.(2). After all, when g(i,j) is not in a homogeneous region, the coefficients of the prediction filter used to obtain the result presented in this paper are given by {w1, w2, w3, w4}={5/8, 2/8, 1/8, 0}, which allows the realization of eqn.(2) to be achieved with only shift and addition operations as follows.

The prediction error is determined with g(i, j) g(i, j) . Figure 9 summaries how to generate the prediction residue of the green plane of a CFA image.

In CMBP, a green sample is classified according to the homogeneity of its local region to improve the prediction performance. Figure 10 shows the effect of this classification step. By comparing Figures 10a and 10b, one can see that the approach with classification can handle the edge regions more effectively and more edge details can be eliminated in the corresponding prediction residue planes. Another supporting observation is the stronger decorrelation power of the approach using classification. Figure 11 shows the correlation among prediction residues in the green plane of testing image 8 under the two different conditions. The correlation of the residues

obtained with region classification is lower, which implies that the approach is more effective in data compression. Besides, the entropy of the prediction residues obtained with region classification is also lower. As far as testing image 8 is concerned, their zero-order entropy values are, respectively, 6.195 and 6.039 bpp.

Prediction on the non-green plane

of all green samples in Sc(i,j) are identical, pixel (i,j) can also be considered as in a homogenous region.

Prediction based CFA image compression

As for the case when the sample being processed is a red or blue sample in the non-green plane, the prediction is carried out in the color difference domain instead of the green intensity domain as in the green plane. This is done to remove the inter-channel redundancy. Since the non-green plane is processed after the green plane, all green samples in a CFA image are known and can be exploited when

Image compression

Figure 7 CFA

processing the non-green plane. Besides, as the non-green plane is raster scanned in the prediction, the color difference values of all processed non-green samples in the CFA image should also be known and hence can be exploited when predicting the color difference of a particular non-green sample.

Let d(p,q) be the green-red (or green-blue) color difference value of a non-green sample c(p,q). Its determination will be discussed in detail in Section 3. For any non-green sample c(i,j), its candidate set is c(i,j)={d(i,j-2), d(i-2,j-2), d(i-2,j), d(i-2,j+2)}, and its support region (context) is defined as Sc(i,j)={(i,j-1),(i-1,j), (i,j+1), (i+1,j)}. Figure 4b and Figure 5b, show, respectively, the positions of the pixels involved in the definition of c(i,j) and Sc(i,j). The prediction for a non-green sample is carried out in the color difference domain. Specifically, the predicted color difference value of sample c(i,j) is given by

where wk and d(mk,nk) are, respectively, the kth predictor coefficient and the kth ranked candidate in c(i,j) such that ( S c(i,j ) , Sc(mu ,nu )) <= ( S c(i,j ) , Sc(mu ,nv ),S for 1u<v

4, where

In the prediction carried out in the green plane, region homogeneity is exploited to simplify the prediction filter and improve the prediction result. Theoretically, similar idea can be adopted in handling a non-green sample by considering the direction information of its neighboring samples. For any non-green sample c(i,j), if the directions

Most of the earlier methods were dealing with normal image compression, but CFA image compression is the new technique .the compression of a image results in loss of data and lead to low quality of image and leads to zipper effect and many other effects, in the proposed system we carry out a CFA image compression and later the same CFA is interpolated using lagrange interpolation this method results in better quality and low memory of a high resolution Image.

Lagrange interpolation

INPUT: vector x; vector y = f(x); a point to evaluate z OUTPUT: P z Lagrange polynomial P(x) evaluated at z Step 1 Initialize variables. Set P z equal zero. Set n to the number of pairs of points (x; y). Set

L to be the all ones vector of length n.

Step 2 For i = 1 to n do . . .

Step 3 For j = 1 to n do Step 4.

Step 4 If i 6= j then Li = (z xj)=(xi xj) Li

Step 5 P z = Li yi + P z

Step 6 Output P z. Stop

The two missing colors are estimated using the green and blue measurements made by neighboring pixels. This process is called interpolation. Each pixel will have three values the actual value of the color it measures through its filter, as well as two interpolated values for the two missing colors. The interpolation is applied to each and every pixel to obtain a full color image. The color interpolation process is known as demosaicing Lagrange interpolation results in better quality picture with reduced image size Overcomes fringe effects and false colors. Physical size of a digital camera can be reduced by using a single sensor.

EXPERIMENTAL RESULTS

A set of 6, 500 × 500 (or 512 × 768) Kodak color images is used to train the statistical model parameters of the proposed algorithm. Another set of 18 different Kodak color images is used to evaluate the performance of the algorithm. For performance comparison

For performance comparison we used different

interpolation methods such as bilinear interpolation, gradient-based interpolation

Fig. 8 shows the demosaic results. Fig. 8(a) shows a color image input Fig. 8(b) shows compressed CFA image 8(c) shows the interpolated image generated using lagrange interpolation. The proposed algorithm has the best overall PSNR average, outperforming the nearest method(bilinear interpolation) by 0.16 dB andgradient based method by

0.68 dB.

demosaicing solutions in terms of objective PSNR comparison. The performance of the proposed algorithm might be improved by using adaptive size lters in green channel inter- polation stage instead of a xed lter. Introducing a gentle post processing step might also improve its results. However, additional computational cost and quality tradeoff is always a concern with post processing steps.

Original image

CFA image

Demosaick image

Figure 8 a )shows a color image input, b) compressed CFA image and c) image generated using lagrange interpolation

CONCLUSION

In this paper, we presented an easy to implement, noniterative, prediction based compression and interpolation demosaicing algorithm. Experimental results show that proposed algorithm outperforms other avail- able

.

REFERENCES

1. B. E. Bayer, Color Imaging Array, U.S. Patent No. 3,971,065, July 20, 1976.

2. Y. T. Tsai, Color image compression for single-chip cameras, IEEE Trans. Electron

   Devices, vol. 38, pp. 12261232, May 1991.

3. S.-Y. Lee and A. Ortega, A novel approach of image compression in digital cameras with a

   Bayer color filter array, Proc. IEEE Intl. Conf. Image Proc., vol. 3, pp. 482485, Oct. 2001.

4. C.C. Koh, J. Mukherjee, and S. K. Mitra, New efficient methods of image compression in

5. N. Zhang and X. Wu, Lossless compression of color mosaic images, IEEE Trans. Image

   Proc., vol. 16, pp. 13791388, June 2006.

6. J. Shapiro, Embedded image coding using zerotrees of wavelet coefficients, IEEE Trans.

   Signal Proc., vol. 41, pp. 34453462, Dec. 1993.

7. D. S. Taubman and M. W. Marcellin, JPEG2000: Image Compression Fundamentals,

   Standards and Practice. Boston, MA: Kluwer Academic Publishers, 2002.

8. P. Schelkens, A. Skodras, and T. Ebrahimi (eds.), The JPEG 2000 Suite, Chichester, UK:

   Wiley, 2009.

9. S. Srinivasan, C. Tu, S. L. Regunathan, and G. J. Sullivan, HD Photo: A New Image Coding

   Technology for Digital Photography, Proc. SPIE Appl. Digital Image Proc. XXX, vol. 6696,

   paper 6696-90, sequence number 6696 0A, pp. 119, Aug.

   2007.

10. F. Dufaux, G. J. Sullivan, and T. Ebrahimi, The JPEG XR Image Coding Standard, IEEE

Signal Proc. Magazine, Standards in a Nutshell series, vol. 26, pp. 195199, Nov. 2009