 Open Access
 Total Downloads : 4266
 Authors : K.Silpa, Dr.S.Aruna Mastani
 Paper ID : IJERTV1IS4105
 Volume & Issue : Volume 01, Issue 04 (June 2012)
 Published (First Online): 30062012
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
Comparison Of Image Quality Metrics
K.Silpa#1, Dr.S.Aruna Mastani*2
Vol. 1 Issue 4, June – 2012
#1 M.Tech (DECS,)Department of ECE, JNTU College of Engineering, Anantapur, Andhra Pradesh, India
*2 Assistant Professor, Department of ECE, JNTU College of Engineering, Anantapur, Andhra Pradesh, India
Abstract: Generally quality metrics are used to measure the quality of improvement in the images after they are processed and compared with the original and other different alternatives methods. Measurement of image quality is very crucial to many image processing applications. Compression is one of the applications where it is required to monitor the quality of decompressed / decoded image. JPEG compression is the lossy compression which is most prevalent technique for image codecs. But it suffers from blocking artifacts Here in this paper Various objective evaluation algorithms for measuring image quality like MSE, PSNR, SSIM and PSNRB are simulated and compared w.r.t. JPEG compression application. Different deblocking filters are used to reduce blocking artifacts and deblocked images are compared through various quality metrics. As the degree of blocking depends on the quantization step, the quality metrics are also simulated and compared by varying the quantization step size. We discussed a new concept called Modified PSNRB which is under review process that gives even better results compared to the existing PSNRB which includes the blocking effect factor (BEF).
Keywords— Blocking artifacts, Deblocked images, Image quality, MSE, PSNR, SSIM, PSNRB and Quantization

Introduction
Digital images are subject to a wide variety of distortions during acquisition, processing, compression, storage, transmission and reproduction, any of which may result in a degradation of visual quality. So, measurement of image quality is very important to numerous image processing applications. JPEG compression is the most popular image compression standard among all the members of lossy compression standards family. JPEG image coding is based on block based discrete cosine transform. BDCT coding has been successfully used in image and video compression applications due to its energy compacting property and relative ease of implementation. Blocking effects are common in blockbased image and video compression systems. Blocking artifacts are more serious at low bit rates, where network bandwidths are limited. Significant research has been done on blocking artifact reduction [7][13]. In order to achieve high compression rates using BTC (Block Transform Coding) with visually acceptable results, a procedure known as deblocking is done in order to eliminate blocking artifacts. A deblocking filter can improve image quality in some aspects, but can reduce image quality in other regards.

Need of Quality Measure:
Basically, quality assessment algorithms are needed for mainly three types of applications:

For optimization purpose, where one maximize quality at a given cost.

For comparative analysis between different alternatives.

For quality monitoring in real time applications.



Existing Quality Metrics

Estimation of Quality Metrics:
To Measure the quality degradation of an available distorted image with reference to the original image, a class of quality assessment metrics called full reference (FR) are considered. Full reference metrics perform distortion measures having full access to the original image. The quality assessment metrics are estimated as follows

PSNR : Peak SignaltoNoise Ratio (PSNR) and mean Square error are most widely used full reference (FR) QA metrics [2], [13].As before X is the reference image and Y is the test image. The error signal between X and Y is assumed as e. Then
(2.1.1)
(2.1.2)
Where N represent Number of pixels in an image. However, The PSNR does not correlate well with perceived visual Quality [14], [15][18].

SSIM: The Structural similarity (SSIM) metric aims to measure quality by capturing the similarity of images [2]. Three aspects of similarity: Luminance, contrast and structure is determined and their product is measured. Luminance comparison function l(X,Y) for reference image X and test image Y is defined as below
(2.1.3)
Where Âµx and Âµy are the mean values of X and Y respectively and C1 is the stabilization constant.
Similarly the contrast comparison function c(X, Y) is defined as
(2.1.4)
Where the standard deviation of X and Y are represented as x and y and C2 is the stabilization constant.
The structure comparison function s(X, Y) is defined as
(2.1.5)
Where xy represents correlation between X and Y and C3 is a constant that provides stability. By combining the three comparison functions, The SSIM index is obtained as below
(3.1.2)
Vol. 1 Issue 4, June – 2012
(3.1.3)
(2.1.6)
and the parameters are set as and C3=C2/2 From the above parameters the SSIM index can be defined as
(2.1.7)
Symmetric Gaussian weighting functions are used to estimate local SSIM statics. The mean SSIM index pools the
(3.1.4)
Where be the number of pixel
pairs in and respectively and B is the block size.
Fig. 2 shows a simple example for illustration of pixel blocks with , , and B=4 . The thick lines represent the block boundaries. In this example , , , . The sets of pixel pairs in this example are
spatial SSIM values to evaluate overall image quality [2].
{(y , y
), (y , y ),.. (y , y )}
25 33
26 34
32 40
(2.1.8)
Where and are image patches covered by the jth window and the number of local windows over the image are represented by M.



New Approach of PSNR including blocking effect factor:
3.1) PSNRB: PSNRB is a new quality metric which includes ordinary PSNR by blocking effect factor is considered. PSNRB correlates well with subjective quality when compared to PSNR. Consider an image that contains integer number of blocks such that the horizontal and vertical dimensions of the image are divisible by block dimension and the blocking artifacts occur along the horizontal and vertical dimensions.
{y1, y9), (y9, y17), (y17, y25),.. (y56,y64)}
{(y4, y5), (y12, y13),..(y60,y61)}
=(y1,y2),(y2,y3),(y3,y4),(y5,y6),.(y63,y64)} (3.1.5)
Then we define the mean boundary pixel squared difference ( and the mean nonboundary pixel squared difference ( for image y to be
(3.1.6)
(3.1.7)
Blocking artifacts will become more visible as the quantization step size increases; mean boundary pixel squared difference will increase relative to mean non boundary pixel square difference. The blocking effect factor is given by
Y1
Y9
Y17
Y25
Y33
Y41
Y49
Y57
Y2
Y10
Y18
Y26
Y34
Y42
Y50
Y58
Y3
Y11
Y19
Y27
Y35
Y43
Y51
Y59
Y4
Y12
Y20
Y28
Y36
Y44
Y52
Y60
Y5
Y13
Y21
Y2
Y37
Y45
Y53
Y61
Y6
Y14
Y22
Y30
Y38
Y46
Y54
Y62
Y7
Y15
Y23
Y31
Y39
Y47
Y55
Y63
Y8
Y16
Y24
Y32
Y40
Y48
Y56
Y64
Where
– ] (3.1.8)
)
Fig1: Example for illustration of pixel blocks
Let and be the horizontal and vertical dimensions of the image I. Let be the set of horizontal neighboring pixel pairs in I. Let be the set of horizontal neighboring pixel pairs that lie across a block boundary. Let be the set of Horizontal neighboring pixel pairs, not lying across a block boundary, i.e.
, . Similarly, let be the set of vertical neighboring pixel pairs, and be the set of vertical neighboring pixel pairs lying across block boundaries. Let be the set of vertical neighboring pixel pairs not lying across block boundaries
i.e. .
A decoded image may contain multiple block sizes like 16Ã—16 macro block sizes and 4Ã—4 transform blocks, both contributing to blocking effects. Then the blocking effect factor for kth block is given by
– ] (3.1.10)
For overall block sizes BEF is given by
(3.1.11)
The mean square error including blocking effects for reference image X and test image Y is defined as follows,
(3.1.12)
Finally the proposed PSNRB is given as,
(3.1.1)
(3.1.13)
The MSE term in (2.1.1) measures the distortion between the reference image and the test image, while the BEF term in (3.1.8) specifically measures the amount of blocking artifacts just using the test image. These noreference quality indices claim to be efficient for measuring the amount of blockiness, but may not be efficient for measuring image quality relative to fullreference quality assessment. We argue that the combination of MSE and BEF is an effective measurement for quality assessment considering both the distortions from the original image and the blocking effects in the test image. The PSNRB is attractive since it is specific for assessing image quality, specifically the severity of blocking artifacts.
A new approach of PSNRB is introduced which gives better results compared to well known blockiness specific index. In this method, a set of diagonal neighboring pixel pairs which are not lying across block boundaries are considered. Simulation results shows that the modified PSNRB gives better results compared to well known blockiness specific indices.

Quantization step size and Deblocking:
Compression and Deblocking operations are performed on original images. JPEG Compression is used to compress the image in which quantization step size is a key factor but information is lost. The amount of compression and the quality can be controlled by the quantization step. As quantization step increases, the quality of the image degrades due to the increase in compression ratio. The trade off exists between compression ratio and deblocked images. The input image is divided into LÃ—L blocks in block transform coding in which each block is transformed independently in to transform coefficients.
Therefore an input image block b is transformed into a DCT coefficient block is given by
(4.1)
Where T is the transform matrix and is the transpose matrix of T. The transform coefficients are then quantized using a scalar quantizer Q
rate is not increased. However, it results in blurred images. More sophisticated methods involve iterativVeol.m1eIsthsuoed4s, Jsuunceh 2012
as projection on convex sets [3, 4] and constrained least squares [4, 5].In this paper deblocking algorithms including low pass filtering, median filtering and projection on to convex sets have used. The efficiency of these algorithms will be studied after performing simulations on compressed and deblocked images. After performing simulations on compressed and deblocked images, various objective quality metrics are applied for assessing the image quality. Among all the quality metrics, PSNRB produced better results compared to other well known blockiness specific indices.

Simulation Results:
Simulations are performed using Matlab software which possesses excellent graphics and matrix handling capabilities. Matlab has a separate toolbox for image processing applications, which provided simpler solutions for many of the problems encountered in this research. In this paper image quality assessment is done by objective measurement in which evaluations are automatic and mathematical defined algorithms. A new approach of PSNRB and well known objective evaluation algorithms for measuring image quality such as MSE, PSNR, Structural Similarity Index Metric (SSIM) have used.
Original image Compressed Image Deblocked images (a) LPF
(b) 3×3 filter (c) 7×7 filter (d) POCS filter
PSNR comparision
60
1
No filter
ssim comparision
No filter
50
(4.2)
—>PSNR
The quantized coefficients are stored or transmitted to 40
POCS
3×3 fil 7×7 fil
0.9 POCS
3×3 fil
0.8 7×7 fil
0.7
0.6
by
decoder. Therefore the output of the decoder is then given 30
20
(4.3)
10
0.5
—>ssim
0.4
0.3
0.2
0.1
X: 10
X: 10
Y: 17.6
Y: 0.4076
Quantization step is represented by . The SSIM index captures the similarity of reference and test images. As the quantization step size becomes larger, the structural
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
PSNRB comparision
60
No filter
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
PSNRB comparision
60
No filter
differences between reference and test image will generally 50
increase. Hence, the SSIM index and PSNR are
40
—>PSNRB
monotonically decreasing functions of the quantization step
size .To remove blocking artifacts, several deblocking 30
techniques have been proposed in the literature as post 20
10
process mechanisms after JPEG compression. If deblocking is viewed as an estimation problem, the simplest solution is
POCS
3×3 fil
X: 10
Y: 25.84
7×7 fil
50
40
—>PSNRB
X: 10
30 Y: 27.62
20
10
POCS
3×3 fil
7×7 fil
probably just to low pass the blocky JPEG compressed
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
image. The advantage of low pass filtering technique is that no additional information is needed and as a result, the bit
Figure 2: Comparison of quality metrics for cameraman image (a) PSNR (b) SSIM (c) PSNRB(d)modified PSNRB
PSNR comparision
ssim comparision
ISSN: 22780181
40 1
Vol. 1 Issue 4N,o fJilteur ne – 2012
No filter POCS
3×3 fil
7×7 fil
35 0.9 POCS
3×3 fil

7×7 fil
30
0.7
—>PSNR
25 0.6
Original image Compressed Image Deblocked images (a) LPF
20
X: 10
Y: 15.43
15
10
5
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
PSNRB comparision
0.5
—>ssim
0.4
0.3
0.2
0.1
0
X: 10
Y: 0.4498
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
B
PSNR comparision
No filter
POCS
3×3 fil
7×7 fil
70 0
No filter
POCS
60 3×3 fil 70
7×7 fil
50 60
—>PSNRB
50
X: 10
Y: 30.55
—>PSNRB
40
X: 10
40 Y: 37.03
30
30
(b) 3×3 filter (c) 7×7 filter (d) POCS filter
20
10
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
20
10
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
PSNR comparision
60
1
No filter
ssim comparision
No filter
50 X: 0
POCS
3×3 fil

POCS
3×3 fil
Figure 4: Comparison of quality metrics for Lena image
Y: 45.19 7×7 fil
40

7×7 fil
0.7
0.6

PSNR (b) SSIM (c) PSNRB (d) modified PSNRB

—>PSNR
30
20
10
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
PSNRB comparision
70
No filter
0.5
—>ssim
0.4
0.3
0.2
0.1
0
80
X: 40
Y: 0.1732
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
PSNRB comparision
No filter
5.1 Comparison of quality metrics: Consider a sample image cameraman as shown in the above figure. Simulations are performed on these image and quality metrics are estimated. Quantization step sizes of 10, 20, 30, 40, 50, and 100 are used in the simulations to analyze the effects of quantization step size
POCS
60 3×3 fil
7×7 fil
50
—>PSNRB
40
30
20
10
X: 100
Y: 2.701
70 POCS
3×3 fil
7×7 fil
60
—>PSNRB
50
40
30
20
10
X: 100
A.PSNR Analysis:
Figure 3 shows that when the quantization step size was large ( 20), the 3Ã—3 filter and POCS methods resulted in higher PSNR than the no filter and 7×7 filter case on the image. All the deblocking methods produced lower PSNR when the quantization step size was small
Y: 7.24
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
0
0 10 20 30 40 50 60 70 80 90 100
quantisation step size
( 20).
Figure 3: Comparison of quality metrics for leopard image

PSNR (b) SSIM (c) PSNRB (d) modified PSNRB
Original image Compressed Image Deblocked images (a) LPF

3×3 filter (c) 7×7 filter (d) POCS filter
B.SSIM Analysis:
Figure 4 shows that when the quantization step was large ( 20), on the image, all the filtered methods resulted in larger SSIM values. The 3Ã—3 and 7Ã—7 low pass filters resulted in lower SSIM values than the no filter case when the quantization step size was small ( 30).
C.PSNRB Analysis:
For large quantization steps, the PSNRB values improved for the cameraman image by employing low pass filtering methods. The POCS resulted in improved PSNRB values compared to the no filtered case, even at small quantization step size.
D .New approach of PSNRB Analysis: (modified PSNRB)
For large quantization steps, the PSNRB values improved for the cameraman image by employing low pass filtering methods. The POCS resulted in improved PSNRB values compared to the no filtered case, even at small quantization step size. Modified PSNRB produced better results compared to the existing well known quality indices.
Cameraman Image
Filtering methods
SSIM
PSNR
PSNRB
(H&V Pixel pairs)
Modified PSNRB
(Diagonal Pixel pairs)
No filter
0.4551
20.01
20.21
27.61
POCS
0.4454
17.6
25.84
27.62
3×3 filter
0.4076
13.81
16.88
25.11
7×7 filter
0.3242
10.51
9.57
14.54
Leopard Image
Filtering methods
SSIM
PSNR
PSNRB
(H&V Pixel pairs)
Modified PSNRB
(Diagonal Pixel pairs)
No filter
0.4846
25.22
14.47
15.48
POCS
0.4319
22.6
22.84
39.82
3×3 filter
0.4054
19.91
14.47
8.010
7×7 filter
0.3173
16.59
9.636
8.008
Lena Image
Filtering methods
SSIM
PSNR
PSNRB
(H&V Pixel pairs)
Modified PSNRB
(Diagonal Pixel pairs)
No filter
0.4935
21.65
30.55
12.99
POCS
0.4498
15.43
30.55
37.03
3×3 filter
0.4117
18.52
27.77
33.67
7×7 filter
0.327
15.43
16.08
12.99
References
ISSN: 22780181
Vol. 1 Issue 4, June – 2012
Table 5.1: Comparison of Quality metrics for Cameraman, Leopard and Lena images (at quantization step size=10)


Conclusion:
Image quality assessment plays an important role in various image processing applications. Experimental results indicate that MSE and PSNR are very simple, easy to implement and have low computational complexities. But these methods do not show good results. MSE and PSNR are acceptable for image similarity measure only when the images differ by simply increasing distortion of a certain type. But they fail to capture image quality when they are used to measure across distortion types. SSIM is widely used method for measurement of image quality. It works accurately can measure better across distortion types as compared to MSE and PSNR, but fails in case of highly blurred image. Standard and natural images were tested by these quality metrics. Those sample images are shown in above figure. We have found that new approach of PSNRB is the better quality metric for JPEG compression which shows better performance than the other well known quality metrics .This Analysis will brings out a new trend in the quality metrics of the image and proves to be efficient than the conventional metrics.
For future work, we look forward to new problems in this direction of inquiry. Quality studies of PSNRB and perceptually proven index SSIM in conjunction are of considerable value, not only for studying deblocking operations, but also for other image improvement applications, such as restoration, denoising, enhancement, and so on.

Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, Projectionbased spatially adaptive reconstruction of block transform compressed images, IEEE Trans. Image Process., vol. 4, no. 7, pp. 896908, Jul. 1995.

Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images, IEEE Trans. Circuits Syst. Video Technol., vol. 3, no. 6, pp. 421 432, Dec. 1993.

H. Paek, R.C. Kim, and S. U. Lee, On the POCSbased post processing technique to reduce the blocking artifacts in transform coded images, IEEE Trans.Circuits Syst. Video Technol., vol. 8, no. 3, pp. 358367, 1998.

S. H. Park and D.S. Kim, Theory of projection onto narrow quantization constraint set and its applications, IEEE Trans. Image Process., vol. 8, no. 10, pp. 13611373, Oct. 1999.

A. Zakhor, Iterative procedure for reduction of blocking effects in transform image coding, IEEE Trans. Circuits Syst. Video Technol., vol. 2, no. 1, pp. 9195, Mar. 1992.

S. Liu and A. C. Bovik, Efficient DCTdomain blind measurement and reduction of blocking artifacts, IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 12, pp. 11391149, Dec. 2002.

Z.Wang and A. C. Bovik, Blind measurement of blocking artifacts in images, in Proc. IEEE Int. Conf. Image Process., Vancouver, Canada, Oct. 2000, pp. 981 984.

Y.Yang, N.P.Galatsanos, and A.K.Katsaggelos, Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images, IEEE Trans. Circuits Syst. Video Technol., vol.3, no.6, pp.421 432, Dec.1993.

Z.Wang, A.C.Bovik, and E.P.Simoncelli, Multiscale structural similarity for image quality assessment, in Proc. Y.Jeong, I. IEEE Asilomar Conf.Signal Syst. Comput.,No v.2003.

Kim, and H.Kang, Practical projection based postprocessing of block coded images with fast convergence rate, IEEE Trans. Circuits Syst. Video Technol., vol.10,no.4 , pp.617623,Jun.2000.

P. List, A. Joch, J. Laimena, J. BjÃ¸ntegaard, and M. Karczewicz, Adaptive deblocking filter, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 614619, Jul. 2003.

A. M. Eskicioglu and P. S. Fisher, Image quality measures and their performance, IEEE Trans. Communications, vol. 43, pp. 29592965, Dec. 1995.

G. Zhai, W. Zhang, X. Yang, W. Lin, and Y. Xu, No reference noticeable blockiness estimation in images, Signal Process. Image Commun., vol. 23, pp. 417432, 2008.