 Open Access
 Total Downloads : 569
 Authors : Nirali Harsoda, Namarata Joshi
 Paper ID : IJERTV3IS21063
 Volume & Issue : Volume 03, Issue 02 (February 2014)
 Published (First Online): 04032014
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
A Review on Image Super Resolution Techniques
Nirali Harsoda
Department of Electronics and Communication R.K.University
Rajkot,india
Namarata Joshi
Department of Electronics and Communication R.K.University
Rajkot,india
Abstract Enhancing image quality or Images with high resolution (HR) has always been a continuous ongoing process in image Technology.In order to restore an image into a HR image correctly,it is necessary to infer high frequency components of a low resolution image.In some applications it becomes essential to extract the useful information from the images.For example in video surveillance, forensic investigation, in medical diagnosis and even the satellite images and pattern recognition in computer vision.During this process, enlarging the image beyond a certain limitresults in a blurred image with no peculiar information.Hardware limitations of sensors is one of the main cause behind this problem which includes sampling rates of ChargedCoupled Devices(CCD). Also, the main idea behind achieving the HR images is not to tamper with the observable quality of image.Many super resolution techniques have been proposed to overcome the hardware limitations in order to achieve the best results .In this electronic document a comprehensive review of all wellknown super resolution techniques has been presented.
Keywords restoration;Super resolution ;iterative back projection;POCS;interpolation.

INTRODUCTION
The goal of super resolution techniques is to reconstruct a high resolution image from a single or multiple low resolution images. Multiple low resolution images of the same scene can be obtained by using either single sensor or many sensors as depicted in fig.1.[1].HR images contains high pixel density and thus they offer improved details of the image that be precious in various applications. As the pixel density increases, image quality degrades due to shot noise[3].Another approach is to increase chip size,this in turn increases the capacitance i.e. higher charge transfer rate and also the size of device. Therefore, limits exists on prevailing sensors and optics processors[3].This can be overcome by increasing spatial resolution and this can be achieved using various signal processing techniques.
SR techniques consists the main features of image interpolation and image restoration. Image interpolation changes the dimension of an image and image restoration recovers a degraded image. Thus, image SR is a technique that restores the degraded image and also increases the size of the image[1].
Figure.1.(a)HR image obtained from four different cameras.
Figure.1.(b)Multiple frames obtained by a single camera
The SR image reconstruction consists of three stages. By means of image registration, all pixels from available LR images can be mapped back onto the reference image. Then via interpolation a uniformly spaced up sampled image is obtained. In the end, image restoration is applied to the up sampled image to remove the effect of sensorPSF blur and noise [4].
To represent the SR technique, a commonly used model describes the relationship between original HR and a set of LR images. Let us consider the desired HR image represented as
x of size 11 Ã— 22.
It is assumed that x is to be sampled at or above the Nyquist rate from a continuous scene to be bandlimited. Let the parameters L1 and L2 represent the down sampling factors for the horizontal and vertical directions in the observed model.
Thus,each observed LR image is of size 1 Ã—
2.Let denotes the kth LR image represented as
=[,1, ,2, . . ,] (1) Where, k=1,2,3p and m= 1 Ã— 2.
The observation model is mathematically represented as:
= + (2) Here, is a MÃ— decimating subsampling matrix.
is blur matrix of size Ã— .is warp matrix of size
Ã— and is white Gaussian noise being encountered in the observation model.
The aim of image SR reconstruction is to estimate the HR imagex from the LR images .

INTERPOLATION
This technique is based on the nonuniform sampling theory. Clark et.al.[5] was the first one to introduce this idea. Estimation of HR image using interpolation follows 3 steps: Registration, Interpolation and restoration.
In the registration step the relative motion information i.e. relative shifts is estimated between LR images in comparision with reference LR images with subpixel accuracy[2]. Then the direct or iterative reconstruction procedure is followed to produce uniformly spaced sampling points[6][9]. Next step is to apply nonuniform interpolation because the registered HR image does not always match up to a uniformly spaced grid so nonuniform interpolation results a uniformly spaced HR image[2].In the restoration step noise and blurring effect in HR image is removed.
This technique has advantage of less computational complexity and also realtime applications are possible. However there are some limitations like errors occurred in the interpolation step are ignored by restoration step, so optimally of the whole reconstruction algorithm is not guaranteed [2].

ITERATIVE BACK PROJECTION(IBP)
Irani and Peleg [10] formulated the iterative backprojection SR reconstruction approach. In this approach, the HR image is estimated by back projecting the error difference between simulated LR image and observed LR images as shown in Figure 2. This process is repeated iteratively to minimize the energy of error[3].This method uses multiple simulated LR image of similar scene to find corresponding HR image.
(0) is the high frequency data of the image X(0) that is obtained from the interpolation of initial LR image[11][12]. This technique is very easy to understand but SR reconstruction is not unique due to ill posed nature of inverse problem.
It should be noted that the selection of backprojection matrix affects the resolution. Afterwards, extended version of the approach was proposed by Van Cittert and TikhonovMiller [13].The results of this comprehensive work are compared in [14]. In [14], the author has also discussed the approach based on Adaptation of conjugate gradient that fastlyconverges the iterations. The original backprojection method is suffering from chessboard effect or ringing effect, especially at edges.

PROJECTION ONTO CONVEX SETS (POCS)
The projection onto a convex set (POCS) approach was originally developed by D.C.youla [16] and H.Webb[17] for image restoration problems, and has been widely employed indeveloping image superresolution construction[18,19,20]. The method of POCS requires definition of closed convex sets within a welldefined vector space that contain the actual SR image [19]. An estimate of the SR image is then defined as a point in the intersection of these constraint sets, and is determined by projecting an arbitrary initial estimate onto the constraint sets[19].
A convex set is defined which represents certain tight constraints on the solution image. The POCS yields progressively improved super resoluted construction with assured convergence [15].
The image formation model is expressed by
1, 2, =
(1,2) 1, 2 (1 1, 22, ) +v(1, 2)
(4)
Where (1, 2, ) is a linear blur mapping of high resolution source image 1, 2, to the kth measured image
1, 2, at a lower resolution and v(1, 2) is an additive white Gaussian noise[15].
A closed, convex constraint sets, one for each pixel within the LR image sequence (1,2, ) is defined as follows:
= { 1, 2, : (1, 2, ) 0(1, 2, )}
(5)
Where,
1, 2, = 1, 2,
1, 2, (1 1, 22, )
(1,2)
(6)
is the residual associated with an arbitrary member , f, of the constraint set bounded in magnitude by 0. The projection of an estimate 1, 2, onto is defined by [16][22]:
1, 2, 1, 2, =
Fig.2.Pictorial representation of IBP
( 0)(1, 2,)
2 ( , ,)
,
> 0
Mathematically the SR steps according to IBP are written as:
1 2 1 2
( +1) = ( ) +
+ (0) (3)
1, 2, +
0 , 0
Where,
( + )( , ,)
( +1)is estimated HR image of n+1th iteration;
( )is estimated HR image of nth iteration;
is error correction;
0 1 2
1 2 2 (1, 2,)
, < 0
(7)
Where function is meant to be 1, 2, .Note that we allow the blurring functionh to be slightly different at each frame depending on the small motion between neighboring frames which usually is only a small fraction of a pixel. Theamplitude constraint should always be satisfied, i.e.,
0 1, 2, 255.
The estimation , , of the high resolution image

S. C. Park, M. K. Park and M.G.Kang, SuperResolution Image Reconstruction: A Technical Overview, IEEE signal processing magazine, pp 2136, May 2003.

S.Chaudhuri, superresolution imaging, Kluwer Academic Publishers, 1/e, 2002.

N.Nguyen and P.Milanfar, An efficient waveletbased algorithm for image superresolution,in Proc.Int.Conf.Imageprocessing,vol 2,2000,pp351 354.
1 2 [6] J.J.Cark,M.R.Palmer and P.D.Laurence, A transformation method for
f(n1,n2;k) is obtained from images gd(m1,m2,q) (q = k1, k, k+1) through the projections in the backward and forward directions[15].
An application of Waveletbased POCS Super resolution for Cardiovascular MRI Image Enhancement is discuss in [15]. Xia Su et al. had combined sparse signal representation with the projection onto convex sets in [22]. Human face image superresolution techniques have a variety of application such as surveillance, recognition, transmission [23].
As performance criteria, Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR) and Mean Absolute Error MAE) are calculated. The mathematical equations for MÃ—N image analysis are as given below.
[ , , ]2
the reconstruction of functions from nonuniformly spaced samples,IEEE Trans.Acoust.,Speech,SignalProcessing,vol.ASSP 33,pp.11511165,1985.

S.P.Kim and N.K.Bose, Reconstruction of 2D banlimited discrete signals from nonuniform samples,Proc,Inst,Elec.Eng.,vol. 137,pt.F,pp,197204,June 1990.

A.Papoulis,Generalized sampling theorem,IEEE Trans,Circuits,Sys.,vol.24.pp.652654,Feb.1977.

J.L.Brown, Multichannel sampling of low pass signals,IEEE Trans,Circuits,Sys.,vol.CAS28,pp.101 106,Feb.1981.

M.Irani and S.Peleg, Improving resolution by image registration,CVGIP:Graphical Models and Image Proc.,vol.53,pp.231 239,May,1991.

R.R.Makwana, N.D.Mehta, Single Image Super Resolution VIA Iterative Back Projection Based Canny Edge Detection and a Gabor Filter Prior, International Journal of Soft Computing and Engineering (IJSCE)ISSN:22312307,Vol3,pp.379 384 Issue
=
Ã—
255 Ã—255
(8)
1, March 2013.

M.N.Bareja and C.K.Modi, An Effective Iterative Back Projection based Single Image Super Resolution Approach,2012 International
SNR = 1010
(9)
Conference on Communication Systems and Network Technologies.IEEE,pp.9599.

Marcia L. S. Aguena, Nelson D. A. Mascarenhas, Generalization of Iterative RestorationTechniques for SuperResolution,SIBGRAPI Conference on Graphics, patterns and Images, pp. 258265,


CONCLUSION
Image super resolution concept and overview of techniques has been discussed here.SR has more optimized the image acquisition process and provided improved content visualization and object recognition.IBP and POCS uses sequence of LR images to extract a SR image.The advantage of interpolation approach is that it takes relatively low computational load and makes realtime applications possible.However,interpolation is not guaranteed, since restoration step ignores the errors that occurs in interpolation step.Projection onto convex sets (POCS) is simple and allows a convenient inclusion of priori information. POCS has disadvantage of nonuniqueness of solution, slow convergence and high computational cost.
Iterative Back Projection (IBP) is understood intuitively and easily. However, this method has no unique solution due to the illposed nature of the inverse problem and it has some difficulty in choosing the .In contrast to POCS it is difficult to apply a priori constraints.
REFERENCES

A.J.Shah,S.B.Gupta, Image super resolution A Survey , 1st international conference on emerginf technology and trends in electronics,communication and networking,IEEE ,2012.

S.S.Qureshi, X.M.Li, T.Ahmad, Investigating Image Super Resolution Techniques:What to Choose?ICACT 2012
Aug.2011.

S. P. Kim and W. Su, Recursive highresolution reconstruction of blurred multiframe images, IEEE Transactions on Image Processing,vol. 2, pp. 534539, 1993.

J. T. Hsu, C. C. Yen, C. C. Li, M. Sun, B. Tian and M. Kaygusuz, Application of Waveletbased POCS Superresolution for CardiovascularMRI Image Enhancement, Proceedings of the Third International Conference on Image and Graphics (ICIG04) IEEE,2004.

D.C. Youla, "Generalized image restoration by the method of alternating orthogonal projections," IEEE Trans.Circuits Syst., vol. CAS25, pp. 694702, 1978.

D.C. Youla and H. Webb, "Image Restoration by the Method of Convex Projections: Part 1Theory," IEEE Trans.Medical Imaging, vol. MI 1, no. 2, pp. 8194, October 1982.

M.K. Ozkan, A.M. Tekalp, and M.I. Sezan, "Pocs BasedRestoration of SpaceVarying Blurred Images," IEEETransactions on Image Processing, vol. 3, pp. 450454,1994.

A.J. Patti, M.I. Sezan, and A.M. Tekalp, "Superresolutionvideo reconstruction with arbitrary sampling lattices and nonzero aperture time," IEEE Transactions on ImageProcessing, vol. 6, pp. 1064 1076, 1997.

M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Transactions onImage Processing, vol. 6, pp. 16461658, 1997.

W.K. Pratt, Digital image processing, 2nd ed. New YorkChichester: Wiley, 1991.

X.q.Su,S.Li, Multiframe Image SuperresolutionReconstruction based on Sparse Representation and POCS, International Journal of Digital Content Technology and its Applications. Volume 5, Number 8, pp 127 135, Aug. 2011.

Hong Yu, Ma Xiang, Huang Hua, Qi Chun, Face Image Superresolution Through POCS and Residue Compensation,VIE, pp 494497, 2008.