Facial expression recognition using 2D gauss hermite moments of images

Abstract

Automatic recognition of facial expressions can be an important component of natural human-machine interfaces; in behavioural sciences and in clinical practices. Expression recognition can be considered to consist of deformations of facial parts and their spatial relations or changes in the pigmentation of the face. The challenge of such recognition lies in classifying expressions both in the case of posed and spontaneous forms; where the former is an intentional expression and the latter is natural. Two major approaches of facial expression recognition include the holistic and landmark. This thesis deals with holistic based feature extraction since such approach considers the entire face images instead of selecting a few interested areas using computationally expensive algorithms for locating landmarks. Commonly used feature extraction techniques in holistic expression recognition methods include the principal component analysis (PCA), the linear discriminant analysis (LDA) and their variants, independent component analysis and even using the orthogonal moments. However, most of the existing approaches fail to consider either the local inherent spatial changes of the facial expression e.g., PCA or LDA. Although orthogonal moments carry local information of facial regions, the previously proposed methods select higher order moments heuristically without any justification. In this thesis, the Gauss-Hermite Moments (GHMs) are used for developing a holistic facial expression recognition algorithm since the GHMs are widely used in visual signal processing. Based on a novel concept of scattering ratio, moments are selected having higher discrimination power of expression in the GHM subspace. Further, due to the existence of significant correlations among certain expressions in the case of spontaneous form as compared to posed form, the GHM features are projected to a new expression subspace where the information are de-correlated using the PCA. Finally, these feature vectors are used to recognize the expressions using the well known support vector machine classifier. Experiments are carried out using two exhaustive databases, namely, the Cohn Kanade and Facial Recognition Grand Challenge, the former representing posed expressions while the latter spontaneous expressions. Experimental results on mutually exclusive subjects reveal that the proposed method can provide the recognition accuracies of at least 7 % and 4 % higher than the existing methods for posed and spontaneous expressions, respectively.