Radial Basis Function Neural Network Trained with Variant Spread Learning

Radial Basis Function Neural Network Trained with Variant Spread Learning

Kumud Arora#, Dr. V. P. Vishwakarma*, Dr. Poonam Garg##

#Inderprastha Engineering College,Ghaziabad

*GGSIP University,Dwarka, NewDelhi

##IMT,Ghaziabad

Abstract– The face database contains images taken at various instants of the same person and it is very difficult to obtain nearly same images for the various instances. This fact leads to misclassification of input face images in automatic face recognition. To attenuate the image variations up to certain extent, image data is transformed from spatial domain to transformed domain. In this paper an effort is made to improve the accuracy of face recognition by estimating the value of spread parameter of the Gaussian kernel activation function on the basis of transformed facial data fed into RBF neural network architecture. Pre-processing of the high dimension input facial data is done by using the PCA (Principal Component Analysis). Simulation results show strong generalization ability with the Principal components of image and achieve excellent performance with high recognition rate. The accuracy achieved is compared with other state of art face recognition algorithms. The effect of selecting dimension reduction factor (sub-dimension) on classification accuracy is also studied.

Keywords: Radial Basis Function Neural Network (RBFNN), Principal Component Analysis (PCA).

1. INTRODUCTION

   Humans have the strong perception system that enables them to recognise the faces of the persons as a routine task with high degree of accuracy. However, in case of automatic face recognition, the accurate recognition is heavily dependent upon the accuracy with which the features are extracted for face pattern representation and the classifier used to distinguish between faces [1]. Both the extraction and classification steps require the processing of high dimension of input data. For e.g.: if the database contains thirty persons and there are ten instances of face image each of 512×512, then automatic face identification requires processing of data of dimension 512x512x30x10 = 78643200. Due to the high dimension of input facial data, the size of input data is reduced first to improve the processing speed thereby improving the performance of recognition system. Approaches based on Principal component analysis (PCA) [4], Linear Discriminant Analysis (LDA) and Discrete Cosine Transformation (DCT)

   [9] are among the few which are commonly used in reducing dimensionality of data with reasonable computational complexity/time. Eigenface[4], a classical method belonging to appearance based algorithms for face recognition uses principal component analysis (PCA) for reducing the dimensions of input facial data. Face space constructed by PCA with reduced dimensions however dont use the face class information. As compared to PCA, employing DCT has

   several advantages. DCT is data independent and can be implemented using a fast algorithm. In this paper PCA is used first to reduce the dimensionality of data and then the data is transformed by employing DCT II kernel of DCT for transforming the image representation domain. The Transformed DCT coefficient vectors are fed into radial basis neural network classifier. Radial basis function (RBF) networks, a local network that is trained in a supervised manner [2] offer several advantages compared to other neural network architectures due to the universal function approximating capabilities. The main input of this paper is the learning of spread parameter value from the transformed input data during the learning process of RBF based network architecture design. The spread parameter being the determinant of width of the kernel activation function, determines the accuracy of networks classification. An attempt is made to improve the recognition accuracy using spread parameter learned from the input data by RBF neural network. The detail of feature selection and dimensionality reduction process followed is represented in Section 2. Section 3 illustrates the approach proposed to estimate the activation value of the Gaussian activation function of Radial basis function neural network on the basis of instance information and Section 4 shows the experimental results and their interpretation obtained by testing the random input data to the trained RBFNN. Paper concludes with the conclusion and future work in section 5.

2. FEATURE SELECTION & FEATURE

   CLASSIFICATION

   For accurate pattern classification, the selection of feature space representing the pattern play very crucial role. The selection of appropriate feature space is desired as all the extracted features may not contribute to the classification positively. The presence of redundant features in pattern affects adversely the accuracy of classification [11]. For reliable recognition, it is desirable to extract features space robustly from the training images even though some sort of distortion or deformation (up to certain extent) is present.

   1. FEATURE SELECTION

      Within the last several years, Discrete Cosine Transform (DCT) has been used for feature extraction step in various studies of face recognition [9, 10, and 12]. The feature selection process using DCT may involve either taking the

      transformation of the image as a whole or performing transformation block wise. The relevant coefficients are then separated. Mathematically, the 2D-DCT of face image with size dimensions [R, C] having rows and columns, is given by:

      F(u,v) =

      2D-DCT when applied block wise with the blocks of size 16×16 result in 256 coefficients per block. The first coefficient of the matrix is marked as the DC coefficient that represents the average intensity of an image, while the rest are the AC coefficients corresponding to high frequency components of the image. The computational load of the DCT is reduced by choosing feature vector from the prominent coefficients of the transformed block after performing zigzag scanning. The coefficients now obtained can effectively represent the face image.

   2. CLASSIFICATION USING RBFNN

   In different fields of application including pattern recognition, identification and classification, neural networks have been trained to perform complex functions. Neural networks have the advantage that before testing with unknown instances, they can be trained to capture knowledge about the variation of face patterns through training images and thereby achieving good generalization. For face recognition, which is inherently a non linear classification problem, recently good recognition results have been obtained by employing RBF neural network with Gaussian radial basis function as activation function [13, 14]. RBFN performs a nonlinear mapping from the input space (x1, x2,xm) to the hidden space, followed by a linear mapping from the hidden space to the output space [5].

   The parameters of feature classifier are adjusted by using training sets. After doing this, for a given test image, face image is classified as either as one corresponding to the subject already existing in the database or to the unknown subject.

3. PROPOSED INSTANCE BASED SPREAD ESTIMATION OF RBF NEURAL NETWORKS

   The approach proposed here use RBF neural network for input face recognition when DCT coefficients are fed as input features. The value of RBF kernels activation function spread factor is estimated from the input face instances.

   1. ARCHITECTURE OF THE PROPOSED RBF NN

      The feed forward structure of RBFNN comprises of single hidden layer of locally tued units interconnected fully to an output layer. The proposed architecture of RBF NN is shown in the following figure and its details are described afterwards:

      Fig.1

      • The input neurons number in input layer is set equal to 92×112 in case of AT&T database and 175×220 for Yale database i.e. it was set equal to that of dimension of input space.

      • The output neuron number in output layer is set equal to 40 for AT&T database, 15 for Yale database i.e. the number of subjects/persons to be classified

      • The hidden units (x neurons) are interconnected in such a way that all the units simultaneously receive the m-dimensional input vector from y neurons where x<=y.

      • The hidden layer neurons are excited by kernel function. These functions are characterized by center and width of the receptive field of associated kernel function. Here, the RBF neuron activation is computed by Gaussian distribution function:

        G . (4),

        where k=1,2..,m ; j=1,2,..,t and the is the width of the receptive field of the jth hidden layer unit . The ontrols the width of the bell curve.

        d (xk , cj) = 1/2 . (5) represents Euclidean distance between the centre and the network input feature vector, where Xk is the kth input vector ,Cj is the center of the jth hidden layer and m is the dimension of the input vector. This distance is calculated at every hidden layer node.

      • This distance is then passed to the radial basis function thereby performing a fixed nonlinear transformation of mapping the input space onto a new space.

      • The output of RBF NN for kth input is defined by

        , where is the weight between the ith hidden layer unit and the kth neuron of the output layer.

      • o(Xi) is the positive bias having value unity and wio the weight to the kth neuron of the output layer from the bias neuron respectively.

   In this paper the, value of spread is estimated from the sample variance of the ith input variable estimated from the available training input data i.e. :

   Spread (Width) = 2 ..(7) , where i=1,2,m ; m is the

   value (in our case it is 85% ) after that error rate increases. The error rates shown are the average values of 20 runs with random training inputs.

   i i

   dimensionality of the input space and i are the positive input scale factors.

   The smaller the value of particular i , the more sensitive the overall network output is to the associated input dimension. Conversely, the larger the value of i,, the less relevant the associated input dimension is to explaining the variation of overall network output with respect to the changes in the input[5].

   Error Rate Vs Number of DCT Coefficients Retained

   14

   Yale Database

   AT&T Database

   13

   12

   11

   10

   9

   8

   0.65 0.7 0.75 0.8 0.85 0.9 0.95 1

   Fig. 2

   Hence i is to rank the relative significance of the input data. Here we heuristically determine the value of i = 22 there by re-defining

   Spread (Width) = (8) , where dmax is the maximum distance between the chosen centers and m is the number of centers.

4. EXPERIMENTAL RESULTS & ANALYSIS

   In order to evaluate the proposed scheme we tested the proposed approach on the above mentioned three benchmark face databases. For each database, in the following experiments, five images of each individual from AT&T database are selected randomly for training set and the five face images for the test set; five images of each individual from Yale database are selected randomly for training set and the six face images for the test set.

   1. Experiment with variation of truncation of high frequency Coefficients

      The discrete cosine transform (DCT), converts a N ×N matrix of gray levels (representing pixel values) to an N ×N matrix of coefficients, which tells how much spatial frequency is present in the image. In natural images the high frequencies tend to have very small DCT coefficients, which may be omitted to reduce the processing requirements and also save the storage space. Instead of doing a DCT on the whole image, we have applied DCT on sub blocks of image with the block size of 16 x 16. From the figure Fig.3, it can be observed that error rate in face recognition decreases with the increase in number of coefficients retained up to threshold

      (Fig. 3)

   2. Variation of size of Training length

      In this experiment, error rate is studied versus the number of images used for training. From the given graph it can be observed that the error rate in correct classification drops rapidly with the increase in number of images. Error rate shown is the average of 20 runs with random sampling of training inputs using RBF classifier.

      Recognition performance Vs Number of Training Images

      60

      Yale Database ATT DataBase

      50

      Percentage Error Rate

      40

      30

      20

      10

      0

      1 2 3 4 5 6 7 8 9

      Number of images per subject, used for training

      (Fig. 4)

   3. Experiment with variation of sub-space Dimensionality

      The effect of number of dimensions to be retained in Principal Component Analysis on the error rate is studied in this paper. PCA projections are optimal for reconstruction from a low dimensional basis. In general, the performance of the Eigen face method varies with the number of principal components. Thus, we compared the performance of PCA components with the recognition performance by computing error rate in recognition. In case of Yale database images

      improvement is more significant as the images contains lots of intensity variations. Following are the average values for the twenty iterations carried with random training and testing set of images. It is quite evident from Fig.5 & Fig.6 that error rate improves significantly when instance based spread values are computed for Radial Basis Neural network rather than nearest neighbour classifier, which is used for image classification.

      average of twenty iterations for every reduced dimensionality rate.

      Sub Space Dimension	AT&T Database		Yale Database
      	PCA+NN	PCA+RBF	PCA+NN	PCA+RB F
      50	6.6250	5.2500	24.4000	6.5333
      55	6.8500	5.4250	26.1333	8.2000
      60	6.3750	5.4750	24.7333	7.1333
      65	6.7000	5.5750	27.0667	8.8000
      70	6.7750	5.3000	25.4667	7.2000
      75	6.7000	5.0500	23.9333	7.4000
      80	7.1500	4.6250	23.9333	6.0667
      85	7.1250	6.1750	26.2667	9.0667
      90	8.1000	6.3000	24.5333	6.7333
      95	6.9250	6.4000	26.5333	7.0667
      100	6.2500	4.9250	23.4667	5.0667
      105	7.0250	5.6000	25.2000	8.4000
      110	7.2000	5.8500	23.6667	7.2000

      Retained Sub Space Dimensionality Vs Error Rate(%)

      30

      25

      Error Rate(%)

      20 Error Rate with PCA and Narest Neighbour Classifier

      Error Rate with PCA and variable spread based RBF Classifier

      15

      10

      5

      50 60 70 80 90 100 110

      Sub Space Dimensionality Retained

      Fig.5 (Error rate for Yale Database)

      8.5

      8

      7.5

      Error Rate(%)

      7

      6.5

      6

      5.5

      5

      4.5

      Retained Sub Space Dimensionality Vs Error Rate(%) Error Rate with PCA and Nearest Neighbour Classifier

      Error Rate with PCA and variable spread based RBF Classifier

      Table 1

   4. Comparison with other Approaches

   The superiority of our approach is highlighted by the results obtained with same training & testing set with different approaches. The superiority over the other approaches lies in the design of the RBF networks for face recognition. The following

   table 2 highlights the error rate obtained for AT&T and Yale database with the a) Transforming the images by Discrete Cosine Transformation(DCT) & then classifying the transformed images with RBFNN b) Directly classifying images with RBFNN without any pre processing c) Reducing images to low dimensionality by PCA and then classifying it with Nearest neighbour classification d) Reducing images to low dimensionality by PCA ,then transforming the reduced image dimensions by DCT and then classifying it with RBFNN e) Reducing images to low dimensionality by PCA ,then transforming the reduced image dimensions by

   50 60 70 80 90 100 110

   Sub Space Dimensionality Retained

   Fig.6 (Error rate for AT&T Database)

   The Table 1 values shows the error rate computed after the images of the indicated databases are reduced to lower dimensions by PCA and then classified by the two approaches: nearest neighbour (Euclidean distance) classifier and Radial basis neural network classifier with variable spread calculated by the equation no.9 (mentioned above).The table shows the

   DCT and then classifying it with Nearest neighbour classification

   Table 2

   Database	Error Rate achieved by the approach
   	DCT & RBF	Image Intensity values & RBFNN	PCA & Nearest Neighbour Classifier	PCA & RBFNN	PCA & DCT & RBFNN	PCA & DCT & Nearest Neighbour Classifier
   AT&T	10.400	7.0250	7	4.675	11.0750	8.4250
   Yale	7.5333	8.0000	24.1333	7.333	7.533	23.400

5. CONCLUSIONS AND FUTURE WORK

Face recognition using Radial Basis Function Neural Network Trained with Variant Spread Learning has been described. The approach has been tested on AT&T database & Yale Database after normalization. The performance of the method has been evaluated with experiments. The experimental results when compared with the recent approaches [12, 14] achieve the better performance both in terms of rates of classification and learning efficiency. The minimum error rate has been found to be 0.6% in one of the twenty experimental runs. In the future work we will try to use hybrid learning process with the fuzzy C means clustering algorithm to reduce further the dimensionality of input data.

REFERENCES

1. Stan Z. Li, Anil K. Jain, Handbook of Face Recognition, Springer,

   New York, 2005

2. Broomhead, D., Lowe, D., Multivariable Functional Interpolation and Adaptive Networks, Complex Systems Vol. 2, pp: 321-355, 1998

3. T. Mikami, M. Wada, Example-based Face Detection Using Independent Component Analysis and RBF Network, SICE Annual Conference, August 4-6, 2003.

4. A. Pentland, B. Moghaddam, T. Starner, View-based and modular eigenspaces for face recognition, in Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition, pp: 8491, 1994

5. Simon Haykin, Neural Networks: a comprehensive foundation, Prentice-Hall, Inc., Upper Saddle River, New Jersey (1999).

6. Zhang Xiaozheng, Gao Yongsheng, Face recognition across pose: A review, Pattern Recognition, Vol. 42, No. 11, pp: 2876 2896, 2009

7. T.K.Kim,J.Kittler, Design and fusion of pose-invariant face- identification experts, IEEE Transsctions on Circuits Syst.VideoTechnol. , Vol. 16, No. 9, pp: 10961106, 2006

1. M.J. Er, W. Chen, S. Wu., High Speed Face Recognition Based on Discrete Cosine Transform and RBF Neural Networks, IEEE Transactions on Neural Networks, Vol. 16, No. 3, pp:679-691, 2005

2. C.Podilchuk, Xiao Zhang, Face Recognition using DCT-Based Feature Vectors in Proceedings of the Acoustics, Speech, and Signal Processing, ICASSP96, pp: 2144-2147

3. Z. H. Sun, G. Bebis, and R. Miller, Object detection using feature subset selection, Pattern Recognition, Vol. 37, pp: 2165-2176, 2004.

4. J. Jiang, G. Feng, Robustness Analysis on Facial Image Description in DCT Domain, IEEE Electronics Letters, Vol. 43, No. 24, 2007

5. Yee Wan Wong, Kah Phooi Seng, and Li-Minn Ang, Radial Basis Function Neural Network With Incremental Learning for Face Recognition, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 41, NO. 4, AUGUST 2011

6. Guo Wei Yang, Wenyi Cao, Fengchang Xu, Radial Basis Neural Network with Gabor Wavelet Transform for Face Recognition, J.

   Applied Mechanics and Materials, Vol.: 321-324, pp: 1055-1060, 2013

7. Wei, Chih-Chiang, RBF Neural Networks Combined with Principal Component Analysis Applied to Quantitative Precipitation Forecast for a Reservoir Watershed during Typhoon Periods , J. Hydrometeor, Vol.13, No. 2, pp:722734, 2012