An Efficient Hybrid Technique Of Feature Extraction For Facial Expression Recognition Using Adaboost Classifier

An Efficient Hybrid Technique Of Feature Extraction For Facial Expression Recognition Using Adaboost Classifier

ShilpaChoudhary*, KamleshLakhwani1 ,Shubhlakshmi Agrwal2

1 Suresh GyanVihar, University, Jaipur, India

2Swami keshwanand Institute of Technology, Jaipur, India

Abstract

Facial Expression Recognition is widely used for designing of human-machine interface. The research issue of Facial Expression Recognition is to select the features which are required to represent a Facial Expression. In this paper we proposed a hybrid method of feature extraction using Discrete Cosine Transform, Wavelet Transform, Gabor Filter and Gaussian distribution to select the distinguished feature for improving the recognition rate of facial expression. JAFFE dataset are used for recognition of different seven expressions: anger, disgust, fear, happiness, sadness, surprise, neutral in Experiments and the result of proposed hybrid technique is compared with results of individual Feature Extraction Techniques as DCT based technique, Wavelet transform based technique, Gabor filter based technique & Gaussian Derivatives based technique which shows that Recognition Rate can be improved by combining distinguished optimum features of DCT, Gabor Filter, Wavelet Transform and Gaussian Distribution in a feature vector for facial expression recognition.

Keywords- Hybrid, Facial Expression Recognition, Gesture, DCT, Wavelet, Gabor Filter, Gaussian distribution.

1. Introduction

Human face is a very useful and powerful source of communicative information about human behavior. Facial expression provides

Sensitive cues about emotional response and plays a major role in human interaction and nonverbal communication [1]. It can complement verbal communication, or can convey complete thoughts by itself. Thus, to make use of the information afforded by facial expressions, automated reliable, valid, and efficient methods of measurement are critical [2]. Facial expressions have been studied by cognitive psychologists, social psychologist, neurophysiologists, cognitive scientist and computer scientists. Computer vision based approaches to facial expression analysis discriminate among a small set of emotions. This focus follows from the work of Darwin [3] and more Ekman [4], who proposed seven basic expressions: neutral, anger, fear, disgust, happiness, sadness, and surprise. The Facial expressions are the facial changes of a persons internal emotional states, intension, or social communications. The various approaches of facial expression recognition is categorized into two categories, namely holistic based facial expression recognition and feature based facial expression recognition. Discrete transform is used for reduction of data redundancy as the primary step of holistic approaches [5]. Discrete cosine transform has strong data decorrelation and there are fast algorithms for DCT [6]. These properties make DCT useful in facial expression recognition in the area of pattern recognition [7]. Ramasubramanian and Venkatesh used a combination of the DCT, PCA and the characteristics of the Human Visual System for encoding and recognition of faces [8]. In [9] To decrease the effect of illumination variation the first three low

frequency coefficients or the DC have been truncated. In facial Expression recognition Different Channels of Gabor Filter have different Distribution and reasonable combination of these features can improve the performance of facial Expression Recognition [10]. Gabor Filter is based on on spatial locality, scale and orientation on facial images. These images are most suitable for Facial Expression Recognition and Face Recognition because these are robust to variations, expression and scale [11]. Feature Extraction is mostly concentrate on facial expression information regions, so the mouth, eye and eyebrow regions are segmented from the images then low dimensional features are extracted using Wavelet Transform [12]. The Wavelet transform decompose the signal into high frequency sub-band (detailed components) and sub-band with low frequency (approximate components). Approximate components are consistent with characteristics of a signal and Detail components are related with noise and disturbance in a signal. The multivariate Gaussian distribution is the generalization of the well-shaped normal density to multiple dimensions.

In the Proposed work, A Hybrid Method based on DCT, Gabor Filter, Wavelet Transform and Gaussian distribution are proposed for feature extraction and Experiments results show that proposed hybrid technique have higher recognition rate than each technique based on DCT, Gabor Filter method, Wavelet Transform method and Gaussian distribution based method individually.

1. Feature Extraction

   2.1 Discrete Cosine Transform

   The 2D Discrete cosine transform of an M*N image is defined as following:

   , = ()() = 0 1 = 0 1, [2 + 12]2 + 12 (1)

   Similarly, the inverse transformation is defined as:

   fx, y = x = 0M 1y =

   0N 1uvFu, vcos [2x + 1u2M] cos2y + 1v2N (2)

   where u=0,1,2,M-1 and v

   =0,1,2,N-1.In both equations

   (4) and (5) (u) and v is defined as:

   uand v=1M

   foru,v=02M for u,v0

   The DCT is applied on M*N size image and DCT coefficient matrix of size M*N is achieved [10]. In DCT matrix each element represents frequency of an image. Low frequency components are at the top left corner of the matrix contains useful information or pattern about the image and high frequency components are at the bottom right corner of the matrix which represents the redundancy & noise in image. To select the coefficient, static Coefficient selection approach is used for optimum features selection. In this approach the most prominent coefficients are selected from a DC coefficient [13] using zigzag manner diagonally as Figure 1.

   Fig. 1. Zigzag scan of DCT coefficients in 8×8 pixels image

   1. Wavelet Transform

      Wavelet transformation is a powerful signal analysis tool, widely used for feature extraction, compression and de-noising. It is useful in face detection because we want to focus on a localized area of the image and find whether it contains a face. Wavelet transform represents the signals with small waves of limited durations, which are called wavelets. It provides examination of the signal both in frequency and time domains. If (t) L2(R), the basic wavelet, (t) is defined as

      C= -|(w)|2w dw

      Where (w) is basic wavelets Fourier Transform and w is circular frequency. The wavelet transform decompose the signal into sub-band with high frequency called detailed components and sub-band with low frequency called approximate components. Approximate component are consistent with characteristics of a signal and Detail components are related with noise and disturbance in a image [14]. The two dimensional wavelet transform is performed by applying the one dimensional wavelet transform to the rows and columns of the input image block, consecutively. The scaling component can be decomposed further to obtain higher order wavelet transform [15].

   2. Gabor Filter

      The Gabor filter is a complex exponential modulated by a Gaussian function In the spatial domain. A Gabor filter can be represented by the following equation:

      x, y, , =12sxsye-

      12(x12Sx2+y12Sy2) ej2x1

      where (x,y) is the pixel position in the spatial domain, is the wavelength (a reciprocal of frequency) in pixels, is the orientation of a Gabor filter, and sx,sy are the standard dviation along the x and y directions respectively. The parameters x1 and y1 are given as

      x1 = x cos + y sin y1 = – x sin +y cos

      The amplitude and phases of Gabor filter bank both provide valuable cues about specific pattern present in images. The amplitude contains directional frequency spectrum information and a phase contains information about the location of edges and image details [16]. Gabor filters with different frequencies and orientations are very effective in capturing local Information present in images [17].The Gabor features are calculated by convolution of input image with Gabor filter bank. I(x, y) is a grey- scale face image of size a* b pixels. The feature extraction procedure can then be defined as a filtering operation of the given face image I(x, y) with the Gabor filter u,v(x, y) of size u and orientation v.

      Gu,v(x,y) = I (x,y) * (x,y)

      In Gabor feature extraction approach we use Holistic approach in which the features are

      extracted from the whole image. Gabor filters are applied on images to extract features aligned at particular orientation. The orientation and frequency are the most important parameter of Gabor filter. Certain features that share the similar orientation and frequency can be selected and used to differentiate between different facial expressions depicted in image [24].The Gabor feature representation |o(x;y)|m,n of an image I(x,y), for x=1,2,…..N, y=1,2,….M, m=1,2…mL , n=1,2,….No, is calculated as the convolution of the input image I(x,y) with Gabor filter bank function (x,y, m,n).The convolution operation is performed separately for real and imaginary part.

      Re(O(x,y))m ,n = I ( x, y ) * Re((x,y,m

      ,n)) Im(O(x,y))m ,n = I ( x, y ) * Im((x,y,m ,n))

      This is followed by the amplitude calculation as follows:

      O(x,y)m ,n =((Re(O(x,y))m ,n )2+ (Im(O(x,y))m ,n )2 )1/2

   3. Gaussian Distribution

The multivariate Gaussian distribution is the generalization of the well-shaped normal density to multiple dimensions. The univariate normal distribution, with mean and variance 2, has the probability density function as following.

() = 122 ( )222

Where

A d-dimensional Gaussian distribution is given as following:

p(x) = 12d2 12 e(-12 (x-)T-1(x-))

Vector and covariance matrix is represented by and respectively. The random vector x satisfies the Gaussian distribution with mean and is notated by x N (, ). The contours of constant density for the d- dimensional Gaussian distribution are ellipsoids defined by the following equation:

(x-)T-1(x-) = c2

The ellipsoids are centered at and have axes iei , where ei is an eigenvector of and i is the corresponding eigen value.

1. Proposed Work

   Concepts of Proposed work: Each feature extraction technique can extract only a limited own features with redundant information so that a limited recognition rate can be achieved. To obtain more optimum feature, different feature extraction techniques are combined into a single combined techniques and advantage of different techniques also merged in proposed technique because each technique have own different advantage. But combining the features from different techniques may be increase the redundant feature also so only optimum distinguished feature must be combined for higher recognition rate.

   1. Algorithm of Proposed hybrid feature extraction techniques based on DCT, Wavelet, Gabor Filter & Gaussian derivatives

      1. The input image I is converted into gray scaled image Ig.

      2. The Ig is transformed using Discrete Cosine Transform. The some desired DCT coefficient values are kept in the feature vector Fdct1 at the zigzag positions from DC coefficient as explained in above section II.

      3. Then mouth region with nose region is extracted from Ig and Discrete Cosine Transform is applied on this sub image. Some desired DCT coefficient values are kept in the feature vector Fdct2 at the zigzag positions from DC coefficient.

      4. The two eyes region with forehead region is extracted and Discrete Cosine Transform is applied sub image. Some desired DCT coefficient values are kept in the feature vector Fdct3 at the zigzag positions from DC coefficient. Feature vector Fdct1, Fdct2, Fdct3 is merged in single feature vector Fdct.

      5. Wavelet decomposition is applied on Ig at level 3 and Approximation coefficients of LL sub band are selected and Add these features to feature vector Fw1.

      6. Wavelet decomposition is applied on Ig at level 4 and Approximation coefficients of LL sub band are selected and Add these features to feature vector Fw2.

      7. Extract the sub image of mouth region with nose region from the Ig and step 5 and

         6 is applied on this sub image. Add this feature to feature vector Fw3.

      8. Extract the sub image of eye region with forehead region from the Ig and step 5 and 6 is applied on this sub image. Add these feature to feature vector Fw4. Feature vector Fw1, Fw2, Fw3, Fw4 is merged in single feature vector Fw.

      9. The Gabor features are calculated by convolution of image Ig with Gabor filter bank using 3 different scale and 5 different orientation and down sampling by factor 2. These features are put in Feature Vector Fg.

      10. Filters the data in image Ig with the 2D FIR using an appropriate mask(2 D vector) for convolution. The mask is generated from a 2D Gaussian function with SIGMA which is specified by parameters DX, DY where Dx& DY specify the number of differentiations along x direction & y direction respectively. By default the size of the mask is 8xSIGMA,8xSIGMA. Five dimensional feature vector is computed at each pixel by convolution with the first derivative (Gx,Gy) of Gaussian in x and y direction and second derivative (Gxx,Gxy,Gyy) is used. This is Gaussian feature vector Fgf.

      11. Finally feature vector Fdct,Fw, Fg, Fgf are added in a single feature vector of F. this is our final feature vector.

4 Experiments & Results

The simulation of proposed work is implemented in MATLAB and JAFFE dataset is used for evaluation of proposed algorithm for facial expression recognition. The JAFFE dataset (Lyons et al., 1998; Zhang et al., 1998) used in experiment contains 213 images posed by 10 female. Among 213 images 140 (70 %) are training image and 73 (30%) are testing image. The images were taken from 10 Japanese female models. Each image has a resolution of 256

256 pixels. The depth of each pixel is 8. The images in the database are grayscale images in the tiff file format. The number of images corresponding to each of the 7 categories of expression (neutral, happiness, sadness, surprise, anger, disgust and fear) is almost the same 3 or 4.

The multiclass AdaBoost classifier is applied for classification of facial expressions. Facial expression recognition using DCT method is implemented as

section 2.1 and from step 1 to step 4 of proposed algorithm in section 3.1 or combined feature vector Fdct. Facial expression recognition based Wavelet Transform is implemented as Section 2.2 and step from 5 to step 8 of section 3.1 or using feature vector Fw. Facial expression recognition based Gabor Filter is implemented as mentioned in section 2.3 and step 9 of section 3.1 or using feature vector Fg mentioned in section 3.1. Facial expression recognition based Gaussian Distribution is implemented as mentioned in section 2.4 and step 9 of section 3.1 or using feature vector Fgf. Facial expression recognition using proposed work is implemented as mentioned in section 3 or using feature vector F mentioned in section

3.1. Result of facial expression recognition obtained from above feature extraction techniques on JAFFE dataset are shown in Table I. Confusion table of recognition of expression using proposed hybrid method is shown in Table II and respective graph is shown i figure 3. Comparative Graph of correct classification of each expression based on DCT technique, Wavelet Transform technique, Gabor Filter method, Gaussian Distribution method are shown in Figure 4.

Fig. 2. Sample image of JAFFE dataset

Table 1. Comparison of recognition rate for different technique on JAFEE dataset using Adaboost Classifier

Table 2. Confusion table for proposed hybrid technique on JAFFE dataset

Fig. 3. Graph of percentage correct classification using proposed hybrid technique for different expression

Fig. 4.Comparison of proposed hybrid method with different technique ofcorrectclassification of each expression on JAFFE dataset using Adaboost Classifier.

5. Conclusion

A scheme of combined feature extraction technique using DCT, Wavelet transform, Gabor Filter and Gaussian Distribution are proposed for facial expression recognition. Experimental results shows that proposed hybrid techniques have 94.1% recognition rate while facial expression recognition based on DCT method, Wavelet Transform method, Gabor Filter method, Gaussian Distribution method presented in section II & same ways as mentioned in section III have 76.1%, 64.9%, 67.9%, 63.7%

recognition rate respectively. So Proposed technique can extract more distinguished information about the expression because each technique can extract only a limited features with redundant information so that a limited recognition rate is obtained but hybrid technique of different feature extraction techniques extract different features which increase the distinguished information for different expression. The results shown in the confusion tables of proposed hybrid technique shows that expression Happy is most highly confused with disgust and surprise while Natural & Surprise expressions have no confusion in recognition & have 100% correct recognized.

References

1. J.J.J.Lien., Automatic recognition of facial expressions using hidden markov models and estimation of expression intensity. The Robotics Institute Carnegie Mellon University Ph. D. dissertation, 1998.

2. Zhiliang Wang. Artificial psychology – a most accessible science research to human brain, International Journal of University of Science and Technology Beijing, Vol.22, No.5 pp. 478- 481, 2000.

3. C. Darwin., The expression of emotions in man and animals, John Murray, reprinted by University of Chicago Press, Chicago, 1965.

4. P. Ekman, W. V. Friesen., Constants across cultures in the face and emotion, International Journal of Personality Social Psychology, vol.17, no.2, pp. 124-129, 1971.

5. P.J. Phillips W. Zhao, R.Chellappa and A. Rosenfeld. Face recognition: A literature survey. volume 35, pages 399458. ACM Computing Surveys, 2003.

6. C L. Wilson R. Chellappa and S.Sirohey. Human and machine recognition of faces : A survey. In Proceeding of IEEE, volume 83, pages 705740. IEEE, 1995.

7. Z.M. Hafed and M.D Levine. Face recognition using discrete cosine transform. Journal of Computer Vision, 43(3):167188, 2001.

8. D. Ramasubramanian and Y.V.Venkatesh. Encoding and recognition of faces based on human visual model and dct. volume 34, pages 24472458, 2001.

9. W. Chen M. J. Er and S. Wu. High speed face recognition based on discrete cosine transform and rbf neural networks. volume 16, pages 679 691, 2005.

10. WeiFeng Liu and ZengFu Wang. Facial expression recognition based on fusion of

    multiple gabor features. In IEEE International conference on Pattern Recognition. IEEE, 2006.

11. W.L. Woo R.M. Mutelo and S.S. Dlay. Two dimensional principal component analysis of gabor features for face representation and recognition.InCSNDSP Proceeding, pages 457

    460. IEEE, 2008.

12. Fenguin Chen Zhiliang Wang ZhengguangXuJianng Xiao and Guojiang Wang. Facial expression recognition using wavelet transform and neural network ensemble. In Internation Symposium on Intelligent Information Technology Application, volume 24, pages 871 875. IEEE, 2008.

13. A. AghagolzadehS.Dabbaghchian and M.S. Moin. Feature extraction using discrete cosine transform for face recognition. IEEE, 2007.

14. ZhengguangXuFenguin Chen, Zhiliang Wang and Donglin Wang. Research on a method of facial expression recognition. In IEEE International conference on Electronic Measurement and instruments, pages 225229. IEEE, 2009.

15. Nan Liu and Han Wang , Feature selection in frequency Domain and its application to face Recognition, IEEE Transactions on Feature selection techniques,vol.37, no.4.pp.23-32,2008.

16. Zheng, D., Zhao,Y., Wang, J, Features Extraction Using A Gabor Filter Family, Proceedings of the sixth LASTED International conference, Signal and Image processing, Hawaii, USA ,2004.

17. S.Dabbaghchian, A. Aghagolzadeh,and M.S. Moin , Feature Extraction using Discrete Cosine Transform for Face Recognition ,IEEE Transaction on Neural Network, vol.18, no.5, pp.1-4, 2007.