Face Recognition using Combined Features of DRLBP and SIFT

Face Recognition using Combined Features of DRLBP and SIFT

Reema Shahul

Department of Electronics and Communication Engineering Ilahia College of Engineering and Technology

Ernakulam, India

Bipin P. R

Department of Electronics and Communication Engineering Assoc Prof Ilahia College of Engg and Technology Ernakulam, India

Abstract The detection of human face from images plays a vital role in, cognitive science and forensic science and computer vision. This paper proposes a novel method of classifying the human face using Artificial Neural Network. The computational and mathematical models like Scale Invariant Feature transform (SIFT) and Dominant Rotated Local Binary Pattern (DRLBP) is used here for classifying human face. The face image is preprocessed at first and then the face features are extracted using SIFT algorithm. The detection of human faces is done using Back Propagation Network (BPN). The experimental results shows that our proposed methodology results in better performance in terms of accuracy.

Keywords Feature extraction, DRLBP, SIFT, Back Propogation Network

I. INTRODUCTION

Face recognition system can be of great help in conveying peoples identity, authentication for banking and security purposes, forensic sciences, identification for law enforcement and surveillance. Face recognition systems comprises of acquisition, feature extraction, recognition. The main face detection methods include knowledge based methods, featured- based methods, template matching methods. This paper addresses two goals of recognition i.e, texture and face feature extraction. There is an active area in pattern recognition and machine learning for Robust subspace learning [1]. Many algorithms have been proposed to deal with the effectiveness of feature design and extraction [2] however, the performance of many existing methods is still highly sensitive to variations of imaging conditions, such as outdoor illumination, exaggerated expression, and continuous occlusion. Appearance-based subspace learning is one of the simplest approach for feature extraction, and many methods are usually based on linear correlation of pixel intensities. Eigenface [3] uses eigen system of pixel intensities to estimate the lower rank linear subspace of a set of training face images by minimizing the 2 distance metric. The solution enjoys optimality properties when noise is independent identically distributed Gaussian only. Fisherface

1. will suffer more due to the estimation of inverse within-class

   covariance matrix [5], thus the performance will degenerate rapidly in the cases of occlusion and small sample size. Laplacian faces [6] refer to another appearance-based approach which uses a locality preserving subspace and seeks to obtain the intrinsic geometry and local structure of the data. This achieves lower error rates. A fundamental problem of appearance-based methods for face recognition, however, is that they are sensitive to imaging conditions [7]. The estimated subspace will be biased since data corrupted by illumination

   changes, occlusions, and inaccurate alignment, thus much of the efforts concentrate on removing/shrinking the noise components. Local feature descriptors [8] have certain merits as they are more stable to local changes. Gradient-based methods have been used for texture description and image classification due to its robustness to local variations and efficiency for computation. Vu and Caplier [9] proposed to enhance the face recognition performance by optimizing the patterns of oriented edge magnitudes descriptor. The ability to deal with discontinuity, caused by occlusion and expression variations, is not taken into account. To address these problems, an enhanced IGO descriptor has been proposed in [10]. The section II describes the proposed system. The section III includes the experimental results and discussions. The paper is concluded in the section IV.

   1. PROPOSED SYSTEM

The main aim of our proposed system is to recognize the face image by comparing the input image with the database image. This is done with the help of feature extraction using features of DRLBP and SIFT algorithms. The input image is preprocessed to remove the unwanted distortions. In this stage resizing and refiltering of the image takes place. The features of the preprocessed image is extracted using algorithms like Discriminative Robust Local Binary Pattern (DRLBP) and Scale Invariant Feature Transform (SIFT). The texture features are extracted using DRLBP whereas the SIFT differentiates the face features. The same features from the database images are also extracted. Both the features of input images and the database images are compared using the Back Propagation Network (BPN). If the match occurs while comparing the image is authenticated else unauthenticated. Figure 2.1 illustrates the block diagram of the proposed system.

INPUT IMAGE

PREPROCESSING

DRLBP

HISTOGRAM

& GLCM

+

SIFT

COMBINED FEATURES

DATABASE IMAGE

FEATURES

BPN

AUTHENTICATE/ UNAUTHENTICATE

Fig. 2.1. Face Recognition System using combined features of DRLBP & SIFT

1. Discriminative Robust Local Binary Pattern (DRLBP)

   Different faces does not have the same shapes and colours. The features like iris, eye brows and lips are in different colours. The Robust Local Binary Pattern (RLPB) is a powerful tool in describing the colour textures, but failed to identify edge or shape information within the face image. Since RLBP uses only texture information, the uneven illumination and weak contrast local patterns and similarly strong local pattern cannot be discriminated by RLBP. In order to overcome this limitation, DRLBP is used in the proposed system where edge and texture information are integrated to retain local structure that RLBP misinterpret and named as discriminative robust local binary pattern (DRLBP).

   I=1

   DRLBP = I=9 w(x, y) RLBP(x, y)

   where w(x, y) is calculated by gradient operator by finding square root of magnitude in x direction and magnitude in y direction. The DRLBP contains both texture information and edge information which is desirable for distinguishing the faces clearly. The central value is taken as threshold. If the other values of the pixel is more than that of the centre threshold binary 1 is taken else less, binary 0 is chosen.

2. Scale Invarant Feature Transform (SIFT)

   The SIFT algorithm consists of four computational stages: (i)scale-space extrema detection, (ii) removal of unreliable keypoints, (iii) orientation assignment, and (iv) keypoint descriptor calculation.

   1. Scale-Space Extrema Detecton:

The keypoints are identified in the scale space in the first stage by looking for image locations that represent maxima or minima of the difference-of-Gaussian function. The scale space of an image is defined as a function L(x,y,), that is produced from the convolution of a variable-scale Gaussian, G(x,y,), with the input image, I(x,y):

L(x,y,)=G(x,y,)I(x,y) (1)

(x2+y2

keypoints. The first criterion evaluates the value of |D(x,y,)| at each candidate keypoint. If the value is below some threshold, the structure has low contrast, the keypoint is removed. The second criterion evaluates the ratio of principal curvatures of each candidate keypoint to search for poorly defined peaks in the difference of Gaussian function. The principal curvature across the edge will be much larger than the principal curvature along it for keypoints with high edge reponses. The ratio of principal curvatures of each candidate keypoint is checked to remove the unstable keypoints based on the second criterion. If the ratio is below some threshold, the keypoint is kept, otherwise it is removed.

1. Orientation Assignment.

   An orientation is assigned to each keypoint by building a histogram of gradient orientations (x,y) weighted by the gradient magnitudes m(x,y) from the keypoints neighborhood:

   m(x,y)=

   ( L(x + 1, y) L(x 1, y))2 + (L (x, y + 1) L(x, y 1))2 (4)

   (x,y)=tanh

   (L(x, y + 1) L(x, y 1))(L(x + 1, y) L(x 1, y)) (5)

   where L is a Gaussian smoothed image with a closest scale to that of a keypoint. By assigning a consistent orientation to each keypoint, the keypoint descriptor can be represented relative to this orientation and, therefore, invariance to image rotation is achieved.

2. Keypoint descriptor calculation.

At each image point of the 16×16 keypoint neighborhood, the gradient magnitude and orientations are computed to create the keypoint descriptor. This neighborhood is weighted by a Gaussian window and then accumulated into orientation histograms summarizing the contents over subregions of the neighborhood of size 4 × 4 with the length of each arrow corresponding to the sum of the gradient magnitudes

2

e

with G(x, y, ) =

1

2

22

(2)

near that direction within the region. Each histogram contains 8 bins, therefore each keypoint descriptor features 4 × 4 × 8 = 128 elements. The coordinates of the gradient orientations and

where denotes the standard deviation of the Gaussian G(x,y,)

The difference-of-Gaussian function D(x,y,) can be computed from the difference of Gaussians of two scales that are separated by a factor k

D(x,y,)=(G(x,y,k)G(x,y,))I(x,y)=L(x,y,k)L(x,y,) (3)

Based on the comparison of the sample point and its eight neighbors in the current image as well as the nine neighbors in the scale above and below, local maxima and minima of D(x,y,) are computed. A candidate keypoint is selected if the pixel represents a local maximum or minimum.

(ii) Removal of unreliable keypoints.

The final keypoints are selected based on measures of the stability. Low contrast points (sensitive to noise) and poorly localized points along edges (unstable) are discarded during this stage. Two criteria are used for the detection of unreliable

descriptor are rotated relative to the keypoint orientation to achieve orientation invariance and the descriptor is normalized to enhance invariance to changes in illumination.

Fig. 2.2. 2 × 2 sub regions are computed from an 8 × 8 neighborhood

D. Matching

Each keypoint descriptor extracted from the test image is matched to the database of descriptors extracted from all

training images when using the SIFT algorithm for object recognition. By identifying its nearest neighbor (closest descriptor) in the database of keypoint descriptors from the training images the best match for each descriptor is found. A subsequent threshold is used, to discard keypoints whose descriptors do not have any good match in the training database, which rejects matches that are too ambiguous. If the distance ratio between the closest neighbor and the second-closest neighbor, (i.e., the closest neighbor that is known to come from a different object than the first) is below some threshold, than the match is kept, otherwise the match is rejected and the keypoint is removed. The object in the database with the largest number of matching points is considered the matched object, and is used for the classification of the object in the test image.

C. Back Propagaton Network (BPN)

Neural networks are predictive models which are based on the action of biological neurons. BPN is a type of neural network. The Back Propogation Algorithm is worked by considering a network with a single real input x and network function F. The derivative F(x) is computed in two phases.

Feed-forward: the input x is fed into the network. The primitive functions at the nodes and their derivatives are evaluated at each node. The derivatives are stored.

Back propagation: The constant 1 is fed into the output unit and the network is run backwards. Incoming information to a node is added and the result is multiplied by the value stored in the left part of the unit. The result is transmitted to the left of the unit. The result collected at the input unit is the derivative of the network function with respect to x.

The back propagation algorithm is used to compute the necessary corrections, after choosing the weights of the network randomly. The algorithm can be decomposed in the following four steps:

1. Feed-forward computation

2. Back propagation to the output layer

3. Back propagation to the hidden layer

4. Weight updates

   The architecture of BPN is shown in fig below as

   graph is plotted and thereafter the features are extracted followed by the generation of different DRLBP patterns. The face and landmark detection is done. The shape features are extracted using the SIFT algorithm in which the keypoints (edges, prominent corners) are localized and finally indicates whether the person is known or unknown. Fig 3.1 shows the simulated outputs of a known person for the input face image that represents fig 3.1 (a). Similarly fig 3.2 shows the simulated outputs of an unknown person for the input face image as shown in fig 3.2 (a).

   1. (b)

(c) (d)

(e) (f)

follows

Fig. 2.3. Architecture of BPN

1. RESULTS AND DISCUSSIONS

   The face recognition using combined features of DRLBP & SIFT features can be obtained by making experiments on data samples. For this purpose, the features are extracted for input images and database images, thereby comparison is made in order to obtain the match between the images. Consider an input image fig 3.1 (a). The face is chosen and the DRLBP pattern is generated. Then the DRLBP pattern

   (g) (h)

   Fig. 3.1. Non real time outputs; (a) Input image (b) Face detection (c) DRLBP pattern (d) DRLBP pattern graph (e) Feature extraction using DRLBP (f) Face and landmark detection (g) Keypoint localisation using SIFT (h) Comparison of image (known person)

   (a) (b)

   ( c ) (d)

   (e) (f)

   (g) (h)

   Fig. 3.2. Non real time outputs; (a) Input image (b) Face detection (c) DRLBP pattern (d) DRLBP pattern graph (e) Feature extraction using DRLBP (f) Face and landmark detection (g) Keypoint localisation using SIFT (h) Comparison of image (unknown person).

   Table 1. Recognition Accuracy (%) in terms of RLBP & LDA v/s DRLBP & SIFT

   No of Samples Recognition Accuracy (%)

   	RLBP & LDA	DRLBP & SIFT
   1	25	36
   2	42	52
   3	54	61
   4	63	70
   5	74	80

   6 85 88

   Fig. 3.3. Performance Evaluation of RLBP & LDA v/s DRLBP & SIFT

   The performance evaluation is done by taking 6 input data samples of a persons face image. The recognition accuracy for RLBP & LDA v/s DRLBP & SIFT in terms of the input data samples of the input face images are shown in table 3.1. Fig 3.3 clearly shows that the recognition accuracy obtained by extracting the features using DRLBP & SIFT is more than the features extracted using RLBP & LDA. The accuracy is 88% by using the DRLBP and SIFT algorithms while it is 85% for RLBP and LDA for 6 data samples. As the number of sample increases the recognition accuracy also increases.

   The proposed system is implemented in real time with the help of webcam. The input image and dtabase image are compared using the DRLBP and SIFT algorithms. The simulation results are shown in fig 3.4. The image is captured at first as shown in fig 3.4 (a). 6 data samples are taken and DRLBP patterns are generated fig 3.4 (b) for each sample in order to extract the texture features. The texture features (patterns) are extracted from both the images. Similarly the face features such as difference in shape for eyes, nose are detected. The DRLBP pattern is generated for the images. The keypoints are localized by the SIFT algorithm as in fig 3.4 (d).

   (a) (b) (c)

   (d) (e)

   Fig. 3.4. Real time outputs; a) Captured face image (b) DRLBP pattern c) Detected face d) Keypoints localisation e) Comparison (Match found).

2. CONCLUSION

An enhanced face recognition system based on Discriminant Robust Local Binary Pattern (DRLBP) And Scale Invariant Feature Transform (SIFT) is presented. Both the texture and face feature extraction is made and matching has been done. The appearance based methods using LDA is less accurate of feature description because of whole image consideration. It doesnt provide optimal results. The human face is classified using the Back Propagation Network (BPN). The experimental results proves the better performance of the DRLBP and SIFT algorithms than compared with the existing method that uses RLBP & LDA. The proposed system is implemented in real time.

ACKNOWLEDGMENT

The author wish to thank Bipin P. R (Associate Professor, Ilahia College of Engineering and Technology, Muvattupuzha, Ernakulam, Kerala), for the valuable guidance and assistance throughout the work.

REFERENCES

1. S. C. Yan et al., Graph embedding and extensions: A general framework for dimensionality reduction, IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 4051, Jan. 2007.

2. H. C. Chi, M. A. Tahir, J. Kittler, and M. Pietikainen, Multiscale local phase quantization for robust component- based face recognition using kernel fusion of multiple descriptors, IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 5, pp. 11641177, May 2013.

3. M. Turk and A. Pentland, Eigenfaces for recognition, J. Cogn. Neurosci., vol. 3, no. 1, pp. 7186, 1991.

4. P.N.Belhumeur,J.P.Hespanha, andD.Kriegman,Eigenface vs. Fisherfaces: Recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711720, Jul. 1997.

5. D.-Q. Dai and P. C. Yuen, Face recognition by regularized discriminant analysis, IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 4, pp. 10801085, Aug. 2007.

6. X. F. He, S. C. Yan, Y. X. Hu, P. Niyogi, and H. J. Zhang, Face recognition using Laplacianfaces, IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 3, pp. 328340, Mar. 2005.

7. C. J. Liu and H. Wechsler, Gabor feature based classification using the enhanced Fisher linear discriminant model for face recognition, IEEE Trans. Image Process., vol. 11, no. 4, pp. 467476, Apr. 2002.

8. Z. Lei, S. Liao, M. Pietikainen, and S. Z. Li, Face recognition by exploring information jointly in space, scale and orientation, IEEE Trans. Image Process., vol. 20, no. 1, pp. 247256, Jan. 2011

9. N.-S. Vu and A. Caplier, Enhanced patterns of oriented edge magnitudes for face recognition and image matching, IEEE Trans. Image Process., vol. 21, no. 3, pp. 13521365, Mar. 2012.

10. Chuan-Xian Ren, Zhen Lei, Dao-Qing Dai, and Stan Z. Li, Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition, IEEE Trans. Cybernetics., vol. 46, Oct. 2015.