Comparison Of Image Quality Metrics

Comparison Of Image Quality Metrics

K.Silpa#1, Dr.S.Aruna Mastani*2

Vol. 1 Issue 4, June – 2012

#1 M.Tech (DECS,)Department of ECE, JNTU College of Engineering, Anantapur, Andhra Pradesh, India

*2 Assistant Professor, Department of ECE, JNTU College of Engineering, Anantapur, Andhra Pradesh, India

Abstract: Generally quality metrics are used to measure the quality of improvement in the images after they are processed and compared with the original and other different alternatives methods. Measurement of image quality is very crucial to many image processing applications. Compression is one of the applications where it is required to monitor the quality of decompressed / decoded image. JPEG compression is the lossy compression which is most prevalent technique for image codecs. But it suffers from blocking artifacts Here in this paper Various objective evaluation algorithms for measuring image quality like MSE, PSNR, SSIM and PSNR-B are simulated and compared w.r.t. JPEG compression application. Different deblocking filters are used to reduce blocking artifacts and deblocked images are compared through various quality metrics. As the degree of blocking depends on the quantization step, the quality metrics are also simulated and compared by varying the quantization step size. We discussed a new concept called Modified PSNR-B which is under review process that gives even better results compared to the existing PSNR-B which includes the blocking effect factor (BEF).

Keywords— Blocking artifacts, Deblocked images, Image quality, MSE, PSNR, SSIM, PSNR-B and Quantization

1. Introduction

   Digital images are subject to a wide variety of distortions during acquisition, processing, compression, storage, transmission and reproduction, any of which may result in a degradation of visual quality. So, measurement of image quality is very important to numerous image processing applications. JPEG compression is the most popular image compression standard among all the members of lossy compression standards family. JPEG image coding is based on block based discrete cosine transform. BDCT coding has been successfully used in image and video compression applications due to its energy compacting property and relative ease of implementation. Blocking effects are common in block-based image and video compression systems. Blocking artifacts are more serious at low bit rates, where network bandwidths are limited. Significant research has been done on blocking artifact reduction [7][13]. In order to achieve high compression rates using BTC (Block Transform Coding) with visually acceptable results, a procedure known as deblocking is done in order to eliminate blocking artifacts. A deblocking filter can improve image quality in some aspects, but can reduce image quality in other regards.

   1. Need of Quality Measure:

      Basically, quality assessment algorithms are needed for mainly three types of applications:

      1. For optimization purpose, where one maximize quality at a given cost.

      2. For comparative analysis between different alternatives.

      3. For quality monitoring in real time applications.

2. Existing Quality Metrics

   1. Estimation of Quality Metrics:

      To Measure the quality degradation of an available distorted image with reference to the original image, a class of quality assessment metrics called full reference (FR) are considered. Full reference metrics perform distortion measures having full access to the original image. The quality assessment metrics are estimated as follows

      1. PSNR : Peak Signal-to-Noise Ratio (PSNR) and mean Square error are most widely used full reference (FR) QA metrics [2], [13].As before X is the reference image and Y is the test image. The error signal between X and Y is assumed as e. Then

         (2.1.1)

         (2.1.2)

         Where N represent Number of pixels in an image. However, The PSNR does not correlate well with perceived visual Quality [14], [15]-[18].

      2. SSIM: The Structural similarity (SSIM) metric aims to measure quality by capturing the similarity of images [2]. Three aspects of similarity: Luminance, contrast and structure is determined and their product is measured. Luminance comparison function l(X,Y) for reference image X and test image Y is defined as below

      (2.1.3)

      Where µx and µy are the mean values of X and Y respectively and C1 is the stabilization constant.

      Similarly the contrast comparison function c(X, Y) is defined as

      (2.1.4)

      Where the standard deviation of X and Y are represented as x and y and C2 is the stabilization constant.

      The structure comparison function s(X, Y) is defined as

      (2.1.5)

      Where xy represents correlation between X and Y and C3 is a constant that provides stability. By combining the three comparison functions, The SSIM index is obtained as below

      (3.1.2)

      Vol. 1 Issue 4, June – 2012

      (3.1.3)

      (2.1.6)

      and the parameters are set as and C3=C2/2 From the above parameters the SSIM index can be defined as

      (2.1.7)

      Symmetric Gaussian weighting functions are used to estimate local SSIM statics. The mean SSIM index pools the

      (3.1.4)

      Where be the number of pixel

      pairs in and respectively and B is the block size.

      Fig. 2 shows a simple example for illustration of pixel blocks with , , and B=4 . The thick lines represent the block boundaries. In this example , , , . The sets of pixel pairs in this example are

      spatial SSIM values to evaluate overall image quality [2].

      {(y , y

      ), (y , y ),.. (y , y )}

      25 33

      26 34

      32 40

      (2.1.8)

      Where and are image patches covered by the jth window and the number of local windows over the image are represented by M.

3. New Approach of PSNR including blocking effect factor:

   3.1) PSNR-B: PSNR-B is a new quality metric which includes ordinary PSNR by blocking effect factor is considered. PSNR-B correlates well with subjective quality when compared to PSNR. Consider an image that contains integer number of blocks such that the horizontal and vertical dimensions of the image are divisible by block dimension and the blocking artifacts occur along the horizontal and vertical dimensions.

   {y1, y9), (y9, y17), (y17, y25),.. (y56,y64)}

   {(y4, y5), (y12, y13),..(y60,y61)}

   =(y1,y2),(y2,y3),(y3,y4),(y5,y6),.(y63,y64)} (3.1.5)

   Then we define the mean boundary pixel squared difference ( and the mean nonboundary pixel squared difference ( for image y to be

   (3.1.6)

   (3.1.7)

   Blocking artifacts will become more visible as the quantization step size increases; mean boundary pixel squared difference will increase relative to mean non boundary pixel square difference. The blocking effect factor is given by

   Y1	Y9	Y17	Y25	Y33	Y41	Y49	Y57
   Y2	Y10	Y18	Y26	Y34	Y42	Y50	Y58
   Y3	Y11	Y19	Y27	Y35	Y43	Y51	Y59
   Y4	Y12	Y20	Y28	Y36	Y44	Y52	Y60
   Y5	Y13	Y21	Y2	Y37	Y45	Y53	Y61
   Y6	Y14	Y22	Y30	Y38	Y46	Y54	Y62
   Y7	Y15	Y23	Y31	Y39	Y47	Y55	Y63
   Y8	Y16	Y24	Y32	Y40	Y48	Y56	Y64

   Where

   – ] (3.1.8)

   )

   Fig1: Example for illustration of pixel blocks

   Let and be the horizontal and vertical dimensions of the image I. Let be the set of horizontal neighboring pixel pairs in I. Let be the set of horizontal neighboring pixel pairs that lie across a block boundary. Let be the set of Horizontal neighboring pixel pairs, not lying across a block boundary, i.e.

   , . Similarly, let be the set of vertical neighboring pixel pairs, and be the set of vertical neighboring pixel pairs lying across block boundaries. Let be the set of vertical neighboring pixel pairs not lying across block boundaries

   i.e. .

   A decoded image may contain multiple block sizes like 16×16 macro block sizes and 4×4 transform blocks, both contributing to blocking effects. Then the blocking effect factor for kth block is given by

   – ] (3.1.10)

   For overall block sizes BEF is given by

   (3.1.11)

   The mean square error including blocking effects for reference image X and test image Y is defined as follows,

   (3.1.12)

   Finally the proposed PSNR-B is given as,

   (3.1.1)

   (3.1.13)

   The MSE term in (2.1.1) measures the distortion between the reference image and the test image, while the BEF term in (3.1.8) specifically measures the amount of blocking artifacts just using the test image. These no-reference quality indices claim to be efficient for measuring the amount of blockiness, but may not be efficient for measuring image quality relative to full-reference quality assessment. We argue that the combination of MSE and BEF is an effective measurement for quality assessment considering both the distortions from the original image and the blocking effects in the test image. The PSNR-B is attractive since it is specific for assessing image quality, specifically the severity of blocking artifacts.

   A new approach of PSNR-B is introduced which gives better results compared to well known blockiness specific index. In this method, a set of diagonal neighboring pixel pairs which are not lying across block boundaries are considered. Simulation results shows that the modified PSNR-B gives better results compared to well known blockiness specific indices.

4. Quantization step size and Deblocking:

   Compression and Deblocking operations are performed on original images. JPEG Compression is used to compress the image in which quantization step size is a key factor but information is lost. The amount of compression and the quality can be controlled by the quantization step. As quantization step increases, the quality of the image degrades due to the increase in compression ratio. The trade off exists between compression ratio and deblocked images. The input image is divided into L×L blocks in block transform coding in which each block is transformed independently in to transform coefficients.

   Therefore an input image block b is transformed into a DCT coefficient block is given by

   (4.1)

   Where T is the transform matrix and is the transpose matrix of T. The transform coefficients are then quantized using a scalar quantizer Q

   rate is not increased. However, it results in blurred images. More sophisticated methods involve iterativVeol.m1eIsthsuoed4s, Jsuunceh- 2012

   as projection on convex sets [3, 4] and constrained least squares [4, 5].In this paper deblocking algorithms including low pass filtering, median filtering and projection on to convex sets have used. The efficiency of these algorithms will be studied after performing simulations on compressed and deblocked images. After performing simulations on compressed and deblocked images, various objective quality metrics are applied for assessing the image quality. Among all the quality metrics, PSNR-B produced better results compared to other well known blockiness specific indices.

5. Simulation Results:

   Simulations are performed using Matlab software which possesses excellent graphics and matrix handling capabilities. Matlab has a separate toolbox for image processing applications, which provided simpler solutions for many of the problems encountered in this research. In this paper image quality assessment is done by objective measurement in which evaluations are automatic and mathematical defined algorithms. A new approach of PSNR-B and well known objective evaluation algorithms for measuring image quality such as MSE, PSNR, Structural Similarity Index Metric (SSIM) have used.

   Original image Compressed Image Deblocked images (a) LPF

   (b) 3×3 filter (c) 7×7 filter (d) POCS filter

   PSNR comparision

   60

   1

   No filter

   ssim comparision

   No filter

   50

   (4.2)

   —>PSNR

   The quantized coefficients are stored or transmitted to 40

   POCS

   3×3 fil 7×7 fil

   0.9 POCS

   3×3 fil

   0.8 7×7 fil

   0.7

   0.6

   by

   decoder. Therefore the output of the decoder is then given 30

   20

   (4.3)

   10

   0.5

   —>ssim

   0.4

   0.3

   0.2

   0.1

   X: 10

   X: 10

   Y: 17.6

   Y: 0.4076

   Quantization step is represented by . The SSIM index captures the similarity of reference and test images. As the quantization step size becomes larger, the structural

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   PSNRB comparision

   60

   No filter

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   PSNRB comparision

   60

   No filter

   differences between reference and test image will generally 50

   increase. Hence, the SSIM index and PSNR are

   40

   —>PSNRB

   monotonically decreasing functions of the quantization step

   size .To remove blocking artifacts, several deblocking 30

   techniques have been proposed in the literature as post 20

   10

   process mechanisms after JPEG compression. If deblocking is viewed as an estimation problem, the simplest solution is

   POCS

   3×3 fil

   X: 10

   Y: 25.84

   7×7 fil

   50

   40

   —>PSNRB

   X: 10

   30 Y: 27.62

   20

   10

   POCS

   3×3 fil

   7×7 fil

   probably just to low pass the blocky JPEG compressed

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   image. The advantage of low pass filtering technique is that no additional information is needed and as a result, the bit

   Figure 2: Comparison of quality metrics for cameraman image (a) PSNR (b) SSIM (c) PSNR-B(d)modified PSNR-B

   PSNR comparision

   ssim comparision

   ISSN: 2278-0181

   40 1

   Vol. 1 Issue 4N,o fJilteur ne – 2012

   No filter POCS

   3×3 fil

   7×7 fil

   35 0.9 POCS

   3×3 fil

   1. 7×7 fil

      30

      0.7

      —>PSNR

      25 0.6

      Original image Compressed Image Deblocked images (a) LPF

      20

      X: 10

      Y: 15.43

      15

      10

      5

      0

      0 10 20 30 40 50 60 70 80 90 100

      quantisation step size

      PSNRB comparision

      0.5

      —>ssim

      0.4

      0.3

      0.2

      0.1

      0

      X: 10

      Y: 0.4498

      0 10 20 30 40 50 60 70 80 90 100

      quantisation step size

      B

      PSNR comparision

      No filter

      POCS

      3×3 fil

      7×7 fil

      70 0

      No filter

      POCS

      60 3×3 fil 70

      7×7 fil

      50 60

      —>PSNRB

      50

      X: 10

      Y: 30.55

      —>PSNRB

      40

      X: 10

      40 Y: 37.03

      30

      30

      (b) 3×3 filter (c) 7×7 filter (d) POCS filter

      20

      10

      0

      0 10 20 30 40 50 60 70 80 90 100

      quantisation step size

      20

      10

      0

      0 10 20 30 40 50 60 70 80 90 100

      quantisation step size

      PSNR comparision

      60

      1

      No filter

      ssim comparision

      No filter

      50 X: 0

      POCS

      3×3 fil

   2. POCS

   3×3 fil

   Figure 4: Comparison of quality metrics for Lena image

   Y: 45.19 7×7 fil

   40

   1. 7×7 fil

      0.7

      0.6

      1. PSNR (b) SSIM (c) PSNR-B (d) modified PSNR-B

   —>PSNR

   30

   20

   10

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   PSNRB comparision

   70

   No filter

   0.5

   —>ssim

   0.4

   0.3

   0.2

   0.1

   0

   80

   X: 40

   Y: 0.1732

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   PSNRB comparision

   No filter

   5.1 Comparison of quality metrics: Consider a sample image cameraman as shown in the above figure. Simulations are performed on these image and quality metrics are estimated. Quantization step sizes of 10, 20, 30, 40, 50, and 100 are used in the simulations to analyze the effects of quantization step size

   POCS

   60 3×3 fil

   7×7 fil

   50

   —>PSNRB

   40

   30

   20

   10

   X: 100

   Y: 2.701

   70 POCS

   3×3 fil

   7×7 fil

   60

   —>PSNRB

   50

   40

   30

   20

   10

   X: 100

   A.PSNR Analysis:

   Figure 3 shows that when the quantization step size was large ( 20), the 3×3 filter and POCS methods resulted in higher PSNR than the no filter and 7×7 filter case on the image. All the deblocking methods produced lower PSNR when the quantization step size was small

   Y: 7.24

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   0

   0 10 20 30 40 50 60 70 80 90 100

   quantisation step size

   ( 20).

   Figure 3: Comparison of quality metrics for leopard image

   1. PSNR (b) SSIM (c) PSNR-B (d) modified PSNR-B

      Original image Compressed Image Deblocked images (a) LPF

   2. 3×3 filter (c) 7×7 filter (d) POCS filter

   B.SSIM Analysis:

   Figure 4 shows that when the quantization step was large ( 20), on the image, all the filtered methods resulted in larger SSIM values. The 3×3 and 7×7 low pass filters resulted in lower SSIM values than the no filter case when the quantization step size was small ( 30).

   C.PSNR-B Analysis:

   For large quantization steps, the PSNR-B values improved for the cameraman image by employing low pass filtering methods. The POCS resulted in improved PSNR-B values compared to the no filtered case, even at small quantization step size.

   D .New approach of PSNR-B Analysis: (modified PSNR-B)

   For large quantization steps, the PSNR-B values improved for the cameraman image by employing low pass filtering methods. The POCS resulted in improved PSNR-B values compared to the no filtered case, even at small quantization step size. Modified PSNR-B produced better results compared to the existing well known quality indices.

   Cameraman Image
   Filtering methods	SSIM	PSNR	PSNR-B (H&V Pixel pairs)	Modified PSNR-B (Diagonal Pixel pairs)
   No filter	0.4551	20.01	20.21	27.61
   POCS	0.4454	17.6	25.84	27.62
   3×3 filter	0.4076	13.81	16.88	25.11
   7×7 filter	0.3242	10.51	9.57	14.54
   Leopard Image
   Filtering methods	SSIM	PSNR	PSNR-B (H&V Pixel pairs)	Modified PSNR-B (Diagonal Pixel pairs)
   No filter	0.4846	25.22	14.47	15.48
   POCS	0.4319	22.6	22.84	39.82
   3×3 filter	0.4054	19.91	14.47	8.010
   7×7 filter	0.3173	16.59	9.636	8.008
   Lena Image
   Filtering methods	SSIM	PSNR	PSNR-B (H&V Pixel pairs)	Modified PSNR-B (Diagonal Pixel pairs)
   No filter	0.4935	21.65	30.55	12.99
   POCS	0.4498	15.43	30.55	37.03
   3×3 filter	0.4117	18.52	27.77	33.67
   7×7 filter	0.327	15.43	16.08	12.99

   References

   ISSN: 2278-0181

   Vol. 1 Issue 4, June – 2012

   Table 5.1: Comparison of Quality metrics for Cameraman, Leopard and Lena images (at quantization step size=10)

6. Conclusion:

Image quality assessment plays an important role in various image processing applications. Experimental results indicate that MSE and PSNR are very simple, easy to implement and have low computational complexities. But these methods do not show good results. MSE and PSNR are acceptable for image similarity measure only when the images differ by simply increasing distortion of a certain type. But they fail to capture image quality when they are used to measure across distortion types. SSIM is widely used method for measurement of image quality. It works accurately can measure better across distortion types as compared to MSE and PSNR, but fails in case of highly blurred image. Standard and natural images were tested by these quality metrics. Those sample images are shown in above figure. We have found that new approach of PSNR-B is the better quality metric for JPEG compression which shows better performance than the other well known quality metrics .This Analysis will brings out a new trend in the quality metrics of the image and proves to be efficient than the conventional metrics.

For future work, we look forward to new problems in this direction of inquiry. Quality studies of PSNR-B and perceptually proven index SSIM in conjunction are of considerable value, not only for studying deblocking operations, but also for other image improvement applications, such as restoration, denoising, enhancement, and so on.

1. Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, Projection-based spatially adaptive reconstruction of block- transform compressed images, IEEE Trans. Image Process., vol. 4, no. 7, pp. 896908, Jul. 1995.

2. Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images, IEEE Trans. Circuits Syst. Video Technol., vol. 3, no. 6, pp. 421 432, Dec. 1993.

3. H. Paek, R.-C. Kim, and S. U. Lee, On the POCS-based post processing technique to reduce the blocking artifacts in transform coded images, IEEE Trans.Circuits Syst. Video Technol., vol. 8, no. 3, pp. 358367, 1998.

4. S. H. Park and D.S. Kim, Theory of projection onto narrow quantization constraint set and its applications, IEEE Trans. Image Process., vol. 8, no. 10, pp. 13611373, Oct. 1999.

5. A. Zakhor, Iterative procedure for reduction of blocking effects in transform image coding, IEEE Trans. Circuits Syst. Video Technol., vol. 2, no. 1, pp. 9195, Mar. 1992.

6. S. Liu and A. C. Bovik, Efficient DCT-domain blind measurement and reduction of blocking artifacts, IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 12, pp. 11391149, Dec. 2002.

7. Z.Wang and A. C. Bovik, Blind measurement of blocking artifacts in images, in Proc. IEEE Int. Conf. Image Process., Vancouver, Canada, Oct. 2000, pp. 981 984.

8. Y.Yang, N.P.Galatsanos, and A.K.Katsaggelos, Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images, IEEE Trans. Circuits Syst. Video Technol., vol.3, no.6, pp.421- 432, Dec.1993.

9. Z.Wang, A.C.Bovik, and E.P.Simoncelli, Multi-scale structural similarity for image quality assessment, in Proc. Y.Jeong, I. IEEE Asilomar Conf.Signal Syst. Comput.,No v.2003.

10. Kim, and H.Kang, Practical projection based postprocessing of block coded images with fast convergence rate, IEEE Trans. Circuits Syst. Video Technol., vol.10,no.4 , pp.617-623,Jun.2000.

11. P. List, A. Joch, J. Laimena, J. Bjøntegaard, and M. Karczewicz, Adaptive deblocking filter, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 614619, Jul. 2003.

12. A. M. Eskicioglu and P. S. Fisher, Image quality measures and their performance, IEEE Trans. Communications, vol. 43, pp. 29592965, Dec. 1995.

13. G. Zhai, W. Zhang, X. Yang, W. Lin, and Y. Xu, No- reference noticeable blockiness estimation in images, Signal Process. Image Commun., vol. 23, pp. 417432, 2008.