 Open Access
 Total Downloads : 12
 Authors : Amruta Sonavale, Shreya Prabhu K
 Paper ID : IJERTCONV3IS19044
 Volume & Issue : ICESMART – 2015 (Volume 3 – Issue 19)
 Published (First Online): 24042018
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
Design & Analysis of Blind Super Resolution via Sparse Representation
Amruta Sonavale#1
# M Tech Student,ECE,MITE Modabidri, Karnataka, India
Shreya Prabhu K*2
*Assistant Professor,ECE,MITE Moodabidri, Karnataka, India
AbstractThis paper presents a new approach to singleimage super resolution, based on sparse signal representation. Research on image statistics suggests that image patches can be well represented as a sparse linear combination of elements from an appropriately chosen overcomplete dictionary. Inspired by this observation, we seek a sparse representation for each patch of the lowresolution input, and then use the coefficients of this representation to generate the highresolution output. Theoretical results from compressed sensing suggest that under mild condi tins, the sparse representation can be correctly recovered from the down sampled signals. By jointly training two dictionaries for the low and highresolution image patches, we can enforce the similarity of sparse representations between the low resolution and high resolution image patch pair with respect to their own dictionaries. Therefore, the sparse representation of a low resolution image patch can be applied with the high resolution image patch dictionary to generate a high resolution image patch. The learned dictionary pair is a more compact representation of the patch pairs, compared to previous approaches, which simply sample a large amount of image patch pairs [1], reducing the computational cost substantially. The effectiveness of such a sparsity prior is demonstrated for both general image super resolution and the special case of face hallucination. In both cases, our algorithm generates highresolution images that are competitive or even superior in quality to images produced by other similar SR methods. In addition, the local sparse modeling of our approach is naturally robust to noise, and therefore the proposed algorithm can handle superresolution with noisy inputs in a more unified framework.
Keywords: Super resolution, sparsity, image processing, sparse coding, image restoration.

INTRODUCTION
The quality of images can be significantly affected by the situation when capturing. The goal of superresolution (SR) image reconstruction is to promote the space resolution of captured original images through software. After the first effective approach proposed by Tsai to solve this illposed problem, there are diverse methods to handle SR problem which can be classified into two groups the multiple frames based approaches and single frame based approaches. For multiple frames based approaches, spatial domain and frequency domain are two major directions and the former one is more popular these years as it can be applied to more general images with motion blur and noise to produce higher quality result. Some popular approaches belonging to spatial ones are projectiononto convex sets (POCS), iterative back
projection (IBP), maximum a posteriori estimation (MAP). However, the deficiencies of these methods are in that they all need priori information of the point spread function (PSF) which is always difficult to get in practical situation. Therefore, to solve this problem, the blind SR method is proposed which combines the image registration, image restoration and PSF estimation into one framework. The method uses partial differential equation (PDE) as the regularization term of the high resolution image to preserve the edges while suppressing noise. However, the parameter of the regularization term which is significant in determining the smoothness of the image should be adjusted manually, which increases the in definability in generating high quality images with detailed texture. The method we propose here is a self adaptive blind super resolution image reconstruction approach based on multiple frames. In the project, PDE framework and eigenvectorbased alternating minimization (EVAM) constraint are used as the regularization term. In addition, we also design a novel image quality assessment method without reference image for adaptively choosing parameter of this blind superresolution algorithm.
Superresolution (SR) image reconstruction is currently a very active area of research, as it offers the promise of overcoming some of the inherent resolution limitations of lowcost imaging sensors (e.g. cell phone or surveillance cameras) allowing better utilization of the growing capability of highresolution displays (e.g. highdefinition LCDs). Such resolutionenhancing technology may also prove to be essential in medical imaging and satellite imaging where diagnosis or analysis from lowquality images can be extremely difficult. Conventional approaches to generating a superresolution image normally require as input multiple lowresolution images of the same scene, which are aligned with subpixel accuracy. The SR task is cast as the inverse problem of recovering the original highresolution image by fusing the lowresolution images, based on reasonable assumptions or prior knowledge about the observation model that maps the highresolution image to the lowresolution ones. The fundamental reconstruction constraint for SR is that the recovered image, after applying the same generation model, should reproduce the observed low resolution images. However, SR image reconstruction is generally a severely ill posed problem because of the insufficient number of low resolution images, illconditioned registration and unknown blurring operators and the solution from the reconstruction constraint is not unique. Various regularization methods have
been proposed to further stabilize the inversion of this ill posed problem. However, the performance of these reconstructionbased superresolution algorithms degrades rapidly when the desired magnification factor is large or the number of available input images is small. In these cases, the result may be overly smooth, lacking important high frequency details. Another class of SR approach is based on interpolation. While simple interpolation methods such as Bilinear or Bicubic interpolation tend to generate overly smooth images with ringing and jagged artifacts, interpolation by exploiting the natural image priors will generally produce more favorable results. It represented the local image patches using the background/foreground descriptors and reconstructed the sharp discontinuity between the two. Explored the gradient profile prior for local image structures and applied it to superresolution. Such approaches are effective in preserving the edges in the zoomed image. However, they are limited in modeling the visual complexity of the real images. For natural images with fine textures or smooth shading, these approaches tend to produce watercolor like artifacts. A third category of SR approach is based on machine learning techniques, which attempt to capture the occurrence prior between lowresolution and highresolution image patches. Proposed an examplebased learning strategy that applies to generic images where the lowresolution to highresolution prediction is learned via a Markov Random Field (MRF) solved by belief propagation. Here it extends this approach by using the Primal Sketch priors to enhance blurred edges, ridges and corners. Nevertheless, the above methods typically require enormous databases of millions of high resolution and lowresolution patch pairs, and are therefore computationally intensive. Here it adopts the philosophy of Locally Linear Embedding (LLE) from manifold learning, assuming similarity between the two manifolds in the high resolution and the lowresolution patch spaces. Thir algorithm maps the local geometry of the lowresolution patch space to the highresolution one, generating highresolution patch as a linear combination of neighbors. Using this strategy, more patch patterns can be represented using a smaller training database. However, using a fixed number K neighbors for reconstruction often results in blurring effects, due to over or underfitting. In our previous work we proposed a method for adaptively choosing the most relevant reconstruction neighbors based on sparse coding, avoiding over or under fitting of and producing superior results.
However, sparse coding over a large sampled image patch database directly is too timeconsuming. While the mentioned approaches above were proposed for generic image super resolution, specific image priors can be incorporated when tailored to SR applications for specific domains such as human faces. This face hallucination problem was addressed in the pioneering work of Baker and Canada. However, the gradient pyramidbased prediction introduced in does not directly model the face prior, and the pixels are predicted individually, causing discontinuities and artifacts. It proposed a twostep statistical approach integrating the global PCA model and a local patch model. Although the algorithm yields good results, the holistic PCA model tends to yield results like the mean face and the probabilistic local patch model is complicated and computationally demanding. Wei Liu
proposed a new approach based on Tensor Patches and residue compensation. While this algorithm adds more details to the face, it also introduces more artifacts. This project focuses on the problem of recovering the super resolution version of a given lowresolution image.

METHODOLOGY

SelfAdaptive Blind SuperResolution ImageReconstruction

Methemathical Model:
SR can be seen as the opposite procedure of observation model in imaging system. The low resolution images are captured through the process of blurring, distortion, down sampling as well as system noise. The blurring warping model is an appropriate mathematical model in SR can be represented as
Lk = D[Mk(Bk * H)] + Nk (1)
Where Lk is the captured frame Kth low resolution image with the size of m Ã— n, H is the corresponding high resolution image,Bk is the PSF of the Kth frame which convolves with the high resolution image and determines the type of blur. Mk is the motion and distortion operator,D is the downsampling operator, and Nk denotes the system noise.

Approach Pipeline:This approach consists of two main stages. First, PDE is used as the regularization term of high resolution image, which contains Lorentzian function as spread coefficient. In another aspect, it uses EVAM constraint as the regularization term of the extended PSF. Alternating minimization algorithm is used to minimize the cost function. Second, we propose a no reference image quality assessment method which considers the effects of blurring and ringing to guide the choice of parameters in the regularization term. In this way it is possible to preserve the details of the image since the parameter significantly determines the smoothness of the resulting image.


Image Resolution Vai Sparse Representation
To solve these illposed and illconditioned proplem two constraints are modeled in this session those are

Reconstruction Constraint: The observed low resolution image Y is a blurred and down sampled version of the high resolution image X:
= (2)
Here, H represents a blurring filter, and S the down sampling operator. Superresolution remains extremely illposed, since for a given lowresolution input Y , infinitely many high resolution images X satisfy the above reconstruction constraint. We further regularize the problem via the following prior on small patches x of X.

Sparsity Prior: The patches x of the highresolution image X can be represented as a sparse linear combination in a dictionary Dh trained from highresolution patches sampled from training images:
For some for some (3)
The sparse representation will be recovered by representing patches y of the input image Y, with respect to a low resolution dictionary Dl cotrained with Dh. It applies this approach to both generic images and face images. For generic image superresolution, we divide the problem into two steps. First, as suggested by the sparsity prior, we find the sparse representation for each local patch, respecting spatial compatibility between neighbors. Next, using the result from this local sparse representation, we further regularize and refine the entire image using the reconstruction constraint. In this strategy, a local model from the sparsity prior is used to recover lost highfrequency for local details. The global model



ALGORITHMS
Algorithm 1: (SuperResolution via Sparse Representation).
1: Input: training dictionaries Dh and Dl, a lowresolution image Y.
2: For each 3 Ã— 3 patch y of Y, taken starting from the upper left corner with 1 pixel overlap in each direction Compute the mean pixel value m of patch y.

Solve the optimization problem with and y that is defined in:
2 + 1 (4)
2
from the reconstruction constraint is then applied to remove
possible artifacts from the first step and make the image more consistent and natural. The face images differ from the generic images in that the face images have more regular structure and thus reconstruction constraints in the face subspace can be more effective. For face image superresolution, we reverse the above two steps to make better use of the global face structure as a regularize. We first find a suitable subspace for human faces, and apply the reconstruction constraints to recover a medium resolution image. We then recover the local details using the sparsity prior for image patches.

Generic Image SuperResolution from Sparsity

Local Model from Sparse Representastion: Similar to the patchbased methods mentioned previously, our algorithm tries to infer the high resolution image patch for each low resolution image patch from the input. For this local model, we have two dictionaries Dh and Dl, which are trained to have the same sparse representations for each highresolution and lowresolution image patch pair. We subtract the mean pixel value for each patch, so that the dictionary represents image textures rather than absolute intensities. In the recovery process, the mean value for each highresolution image patch is then predicted by its lowresolution version. For each input lowresolution patch y, we find a sparse representation with respect to Dl. The corresponding highresolution patch bases Dh will be combined according to these coefficients to generate the output highresolution patch x.

Learning the dictionary pairs: This section will focus on learning a more compact dictionary pair for speeding up the computation.The superresolution problem using sparse prior which states that each pair of high and lowresolution image patches have the same sparse representations with respect to the two dictionaries Dh and Dl.A straightforward


Generate the highresolution patch x = Dh. Put the patch x
+ m
into a highresolution image X0. 3: End.
4: Using gradient descent, find the closest image to X0 which satisfies the reconstruction constraint:
2 2
2 2
= 2 + 02 (5)
5: Output: superresolution image
The entire superresolution process is summarized as Algorithm 1.
Face image resolution enhancement is usually desirable in many surveillance scenarios, where there is always a large distance between the camera and the objects (people) of interest. Unlike the generic image superresolution discussed earlier, face images are more reglar in structure and thus should be easier to handle. Indeed, for face super resolution, we can deal with lower resolution input images. The basic idea is first to use the face prior to zoom the input to a reasonable medium resolution, and then to employ the local sparsity prior model to recover details. To be precise, the solution is also approached in two steps: 1) global model: use reconstruction constraint to recover a medium highresolution face image, but the solution is searched only in the face subspace; and 2) local model: use the local sparse model to recover the image details.
Algorithm 2: (Face Hallucination via Sparse Representation). 1: Input: sparse basis matrix U, training dictionaries Dh and Dl, a
lowresolution aligned face image Y.
2: Find a smooth highresolution face X from the subspace spanned by U through:

Solve the optimization problem in:

2 + 2 . 0 (6)
way to obtain two such dictionaries is to sample image patch
pairs directly, which preserves the correspondence between the high resolution and low resolution patch items
2
^ =
However,such a strategy will result in large dictionaries and hence expensive computation.

Single Dictionary Training: Sparse coding is the problem of finding sparse representations of the signals with respect to an over complete dictionary D. The dictionary is usually learned from a set of training examples X = {x1, x2, …, xt}. Generally, it is hard to learn a compact dictionary which
3: For each patch y of X, taken starting from the upperleft corner
with 1 pixel overlap in each direction, Compute and record the mean pixel value of y as m. Solve the optimization problem with D and y defined.
n~ ~2 + 1 (7)
guarantees that sparse representation can be recovered 2
minimization. Fortunately, many sparse coding algorithms
proposed.

Generate the highresolution patch x = Dh + m. Put the patch
x into a highresolution image
4: Output: superresolution face . For estimated the high resolution face, and is a parameter used to balance the reconstruction fidelity and the penalty of the prior term. In this project, we simply use a generic image prior requiring that the solution be smooth. Let denote a matrix performing highpass filtering. The final formulation is:
= 2 + 2 . 0 (8)

SelfAdaptive Blind SuperResolution Image Reconstruction GUI
2
The medium highresolution image X is approximated by .
The prior term in suppresses the high frequency components, resulting in oversmoothness in the solution image. We rectify this using the local patch model based on sparse representation mentioned. The complete framework of our algorithm is summarized as Algorithm 2.


EXPERIMENTAL RESULTS
The main GUI window is designed to have two buttons along With a button to exit the GUI. The two buttons are to select different methods to implement blind resolution. The button that says Self a daptive blind super resolution performs super resolution on the image. The next button Sparse representation performs the sparse representation for single image. Hence we can compare the results of both the methods on same GUI. We can use the button Exit to exit from the GUI window.
Fig 1:The main GUI of Analysis of Super Resolution
To select image which was already stored in image file, by click on the brows button, that will show the registered images which are stored in image file and results with a title of original image that is show in below gui.
However, SR image reconstruction method generally contains illposed problem because of the insufficient number of low resolution images and it also contain illconditioned problem due to the registration and unknown blurring operators, and the solution from the reconstruction constraint is not unique. To overcome these problems Sparse Representation method is proposed.

Image SuperResolution via Sparse Representation GUI


CONCLUSION
The superresolution reconstruction problem is an inverse problem, dealing with the recovery of a single highresolution image from a set of low quality images. In its general form, the super resolution problem may consist of images with arbitrary geometric warp, space variant blur and colored noise.
Self Adaptive Blind Super resolution uses Lorentzian function as spread coefficient and partial differential function as regularization term of resulting image. A generalized version of the eigenvectorbased alternating minimization (EVAM) constraint is used to regularize PSF and estimate resulting image and PSF simultaneously. In addition, in order to achieve selfadaptive regularization terms parameter choosing, we also present a new robust noreference image quality assessment method which provides blurring and ringing effect assessment value as feedback.
Sparse Representation method presented a novel approach toward single image superresolution based on sparse representations in terms of coupled dictionaries jointly trained from high and lowresolution image patch pairs. The compatibilities among adjacent patches are enforced both locally and globally. Experimental results demonstrate the effectiveness of the sparsity as a prior for patchbased super resolution both for generic and face images.
REFERENCES

J. Yang, J. Wright, T. Huang, and Y. Ma, Image superresolution as sparse representation of raw image patches, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 18.

M. Irani and S. Peleg, Motion analysis for image enhancement: resolution occlusion and transparency, Journal of Visual Communication and Image Representation, vol. 4, no. 4, pp.324335, 1993.

R. C. Hardie, K. J. Barnard, and E. A. Armstrong Joint map registration and high resolution image estimation using a sequence of undersampled images, IEEE Transactions on Image Processing, vol. 6, pp. 1621 1633, 1997.

S. Baker and T. Kanade, Limits on superresolution and how to break them, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 11671183, Sep. 2002.

S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, Fast and robust multi frame super resolution, IEEE Transactions on Image Processing, vol. 13, pp. 13271344, 2004.

M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Transactions on Image Processing, vol.15, pp, 37363745, 2006.

J. Mairal, G. Sapiro, and M. Elad, Learning multiscale sparse representations for image and video restoration Multiscale Modeling andSimulation, vol. 7, pp. 214241, 2008. Email:julien.mairal@inria.fr

Jianchao Yang and Thomas Huang Image superresolution: Historical overview and future challenges University of Illinois at Urbana Champaign.

Jianchao Yang,J. Wright , Yi. Ma Image superresolution as sparse representation slide, SIAM imaging science,july 7,2008.

Assaf Zomet and Shmuel Peleg super resolution from multiple images having arbitrary mutual motion

Wagner, Waagen and Cassabaum SuperResolution slide.