Multimodal Multi-Algorithmic Biometric Fusion for Reduced Data Set

Multimodal Multi-Algorithmic Biometric Fusion for Reduced Data Set

Fathima Nazimudeen

Department of Electronics and Communication College of Engineering, Trivandrum

Kerala, India

Smitha Satheesh

Department of Electronics and Communication College of Engineering, Trivandrum

Kerala, India

AbstractBiometrics are now replacing todays authentication systems due to its accuracy. This paper introduces a new multimodal biometric system which uses multiple algorithmic approach for the reduced data set. Multimodal biometric systems can overcome the drawbacks of unimodal systems. Two modalities used in this approach are face and fingerprint. Multi algorithmic approach used for face uses two algorithms PCA (Principal Component Analysis), followed by LDA (Linear Discriminant Analysis). And Fingerprint recognition is done using crossing number method. These two modalities are finally fused at a matching score level using a sum rule to improve the accuracy. Experimental results shows that the proposed system has better results compared to the unimodal face and fingerprint recognition systems. This work also compares the PCA and LDA approach with the combined multi algorithmic approach.

KeywordsCrossing Number, LDA, Multimodal Biometrics, PCA.

1. INTRODUCTION

   In this present world, with an increase in number of threats, it is must for us to increase the security measures. Traditional authentication systems which uses the things that we carry or remember are more vulnerable to the atacks [9]. Biometrics provides a change in the way how we are identified and thereby increases the security. So conventional systems are now replaced by biometric systems [2] due to its universality, uniqueness, permeance, accuracy and so on. But Unimodal biometric systems have certain drawbacks like intra-class variation, inter-class similarity, spoofing, noisy data etc [1]. Multimodal biometric systems can overcome the limitations of unimodal systems as it combines more than one biometric to increase the accuracy.

   The various biometric modalities used can be combined at different levels like feature level, matching score level, decision level etc. This process of combining various biometric modalities is referred to as biometric fusion. In feature level fusion, features are extracted from each biometric trait and a composite feature vector is created. This composite feature vector is then used for further classification. In matching score level fusion, each biometric trait is processed separately and a score is generated from each trait. This score is then used for classification. In decision level fusion each trait is indepen- dently processed and classified as accept/reject. The output result from each trait is then finally used for classification.

   Multi-algorithmic approach can also be used along with multimodal biometrics to furthur improve the performance.

   Proposed method uses face and fingerprint biometrics and multi-algorithmic approach for face. Multi-algorithmic ap- proach used for face uses two algorithms PCA(principal component analysis), followed by LDA(linear discriminant analysis).

   PCA is the most powerfull algorithm widely used for face recognition. Its main goal is dimensionality reduction by finding the direction of maximum variance [7]. Eigen faces are the features used in PCA for face recognition. But PCA is an unsupervised algorithm as it ignores class labels. Compared to PCA, LDA is a supervised algorithm [16]and it can be used for multi class classification work. LDA is also a dimensionality reduction technique. Its aim is to maximize the linear discriminant criteria. PCA finds the axes with maximum variance while LDA finds the axes for better class seperability.

   Fingerprints are the most oldest and widely used biometric due to its accuracy and ease of use. Various techniques are available for fingerprint matching like minutiae based technique, pattern matching or ridge feature based techniques, correlation based technique and image based techniques. Minutiae based techniques are widely used as minutiae contain much of the individual information, storage efficient and also relatively stable. For the minutiae feature extraction different methods are available [3], [16], [17], like chain code based method, run representation based method, crossing number method and so on, Crossing number method which works on thinned binarized images is widely adopted due to its simplicity and computational efficiency.

   The remainder of this paper is organized as follows. In Section II,we present a survey on multimodal biometric systems. Section III describes the proposed work in detail. In Section IV, we present and analyze the experimental results. Finally, we summarize this paper in section V.

2. RELATED WORKS

   A multimodal biometric system is developed using fingerprint and iris biometric in [13]. The system fuses fingerprint and iris at feature level, even though their features at image level are incompatible and non-homogeneous. The system provides single feature vector obtained by fusing fingerprint and iris image and extracting a unique textural pattern from fused image by efficient wavelet transform. Matching is carried using Hamming distance. Here independent databases are used for face and iris images and each fingerprint ia assigned a corresponding iris image.

   A multimodal biometric system in [14] combines palmprint and fingerprint in feature level. Palm print and finger print images were fused using wavelet based image fusion techniques with min-min approximation. Features were extracted using Discrete Cosine Transform (DCT) and feature reduction achieved using Information Gain (IG). In their work also independent databases are used for palmprint and fingerprint and they are combined by assigning a fingerprint image to each palmprint image. Their shows that multi modal

   eigenfaces can safely represent the whole original training set, as they depict major features that make up the data set. In the classification phase, an input face image is projected to the same Eigenspace and is then classified by using Euclidian distance classifier. Major steps in PCA algorithm is asfollows:

   • Obtain face images and represent each image as a column i of a single matrix

   • Compute average face vector

   biometrics are more efficient than conventional palm print

   = 1

   (1)

   based methods.

   The multimodal biometric system in [15] combines face and fingerprint biometrics in matching score level. They used the gray-level co-occurrence matrix (GLCM) as an effective method for extracting the texture features in the face recognition and crossing number method is used for fingerprint feature extraction. For matching process they used correlatiion coefficient as the similarity measure. A multimodal biometric system is developed by combining face and fingerprint biometric by score level fusion. According to \cite{19} face recognition is done using PCA and fingerprint recognition is done using minutiae matching and gabor filtering.

3. PROPOSED METHOD

   Proposed method uses face and fingerprint biometrics and multi-algorithmic approach for face. Overall proposed System is divided into four modules

   1. Face feature extraction and score generation

   2. Fingerprint feature extraction and score generation

   3. Fusion module

   4. Decision module

   In module 1 face features are extracted and a score is generated. Multi-algorithmic approach used for face uses two algorithms- Principal Component Analysis (PCA)and Linear Discriminant Analysis (LDA).In module 2 fingerprint features are extracted and a matching score is generated. Crossing number method is used for fingerprint feature extraction. The features extracted are minutiae points namely ridge ending and ridge bifurcation and then generates a score. In fusion module, scores generated from different modules are combined using sum rule and make decision in the decision module.

   1. Face feature extraction and score generation

      Face feature extraction using PCA: First face images are projected to low dimensional feature space using Principal Component Analysis or PCA. PCA is a dimensionality reduction technique that is widely used in face recognition. It is a procedure for identifying a smaller no: of uncorrelated variables, called principal components, from large set of data. It requires two stages: training stage and classification stage. In training stage, an Eigenspace is created from the training samples by using PCA. Then training face images are mapped to the Eigenspace for classification. PCA uses orthogonal transformation to convert a set of N face images into a set of K uncorrelated variables called eigenfaces. Eigenfaces are computed from the covariance matrix which is constructed by using training data set . . Consider to be the matrix obtained

      =1

      Where N is the total no: of face images in the training data set.

      • Subtract mean face from each 1D face vector

        = (2)

      • Compute the co-variance matrix, C

        = (3)

      • Compute eigen values and eigen vectors of C

      • Calculate projected images from k large eigen values by multiplying with eigen values

        So images are projected to a low dimensional feature space created by PCA. Now apply the second algorithm linear discriminant analysis (LDA) to the images projected in this low dimensional feature space.

        Face Feature Extraction using LDA: Linear Discrimi- nant Analysis (LDA) is a dimensionality reduction technique widely used for face recognition[5]. It overcomes the draw- backs of the PCA technique by applying the linear discrim- inant criterion. This criterion tries to maximize the ratio of the determinant of the between-class scatter matrix of the projected samples to the determinant of the within class scat- ter matrix of the projected samples[12]. Linear discriminant groups the images of same class and separates images of different classes of the face images. Class seperation in LDA is shown in Fig 1. Within class scatter matrix (Sw) and the between class scatter matrix (Sb) is defined as follows:

        Figure 1: Class separation in LDA

        • Within class scatter matrix

          by subtracting mean from each face image. Computing

          =

          ( )(

          (4)

          eigenfaces of covariance matrix T is not practical as T is too large. So instead, we compute the eigenfaces of T . K

          =1

          =1

          )

        • Between class scatter matrix

        The fingerprint image can be taken either using off-line

        =1

        =1

        =

        ( )( )

        (5)

        methods such as through inked impression on paper or on- line through a live capture device consisting of an optical,

        Where c is the number of classes, M is the mean of all classes, N is the total no: of images in training set. The Sw matrix shows how face images are closely distributed with- in classes and Sb matrix represents how classes are separated from each other. Then the subspace for LDA is spanned by a set of vectors W = [W1, W2,., Wd], satisfying

        capacitive, ultrasound or thermal sensor. The performance of fingerprint feature extraction and matching mainly depends on the quality of the input fingerprint image. The robustness of the fingerprint recognition system can be improved by incorporating an enhancement stage prior to feature extraction. The first step in image enhancement is to

        = = | |

        ||

        (6)

        make fingerprint images more clearer for furthur operations.

        For that Histogram equilisation and Fourier transform is

        = (7)

        In this proposed work multi-algorithmic approach is used for face. So first PCA is used to project images to low di- mensional space and then LDA is applied to images projected on this low dimensional feature space. Euclidean distance classifier is used to classify the image into different classes. Image with minimum Euclidian distance is the best match and the normalized minimum euclidian distance is used to compute the score from face module. Score from face module is calculated using the equation 8

        = 1 ( ) (8)

   2. Fingerprint Feature Extraction and Score Generation

      Fingerprint is one of the most unique feature of human body. It has been primarily used for authentication[4]. Fingerprint recognition is a method of authenticating an individual by comparing two fingerprints. Identification is

      used. The enhanced image is then binarized and thinned to obtain a skelton image to be used for furthur operations. Next step is the feature extraction using crossing number method. This method uses the skeleton image where the ridge flow pattern is eight- connected. This method uses 3×3 neighbourhood of the ridge pixel [8] and extracts two types of minutiae features, ridge endings and ridge bifurcations. It is done by examining the local neighbourhood[4] of each ridge pixel in the skeleton image using a window of size 3×3. For a pixel P, its eight neighbouring pixels are scanned in anti-clockwise direction as shown

      P5	P4	P3
      P6	P	P2
      P7	P8	P1

      The Crossing Number (CN) for the given ridge pixel P is given by:

      based on fingerprint features called minutiae. They are the

      = 1 8 |

      |

      (9)

      point of interest in fingerprints. Minutiae include

      2 =1

      +1

      • Ridge ending : Abrupt end of ridge

      • Bifurcation : Bifurcation is a single edge that divides into two.

      • Short ridge or independent ridge : A ridge that com- mences, travels a short distance and then ends

      • Island : A single small ridge inside a short ridge or ridge ending that is not connected to all other ridges

      • Ridge enclosure : A single ridge that bifurcates and reunites shortly afterward to continue as a single ridge

      • Spur : A bifurcation with a short ridge branching o a longer ridge

      • Crossover or bridge : A short ridge that runs between two parallel ridges

      • Delta : A Y-shaped ridge meeting

      • Core : A U-turn in the ridge pattern

        The most important minutiae features are ridge ending and ridge bifurcation which is shown in Fig 2. Proposed method uses Rutowitz Crossing Number (CN) method for fingerprint feature extraction.

        Figure 2: Ridge ending and bifurcation

        Figure 4: Ridge ending with crossing no:1 and bifurcation with crossing no:3

        Crossing number is defined as the half the sum of difference between adjacent pairs of pixels in the eight connected neighbourhood [6]. So the fingerprint skeleton image is first scanned and then all the pixels are labelled by the properties of corresponding crossing number. Using the CN values, each pixel is identified as ridge ending, bifurcation or non-minutiae point as shown in table I.

        TABLE I: Crossing number and its properties

        /table>

        Next step is fingerprint matching. The most important stage of a fingerprint verification system is the matching process. Matching algorithm determines whether two fingerprints are from same fingerprint or not. Matching is

        done by computing the similarity score. The score is obtained using the equation 10.

        Where FS is the final score, x1 and x2 are the scores obtained from face and fingerprint modules respectively. w1 and w2

        2

        2

        =

        (10)

        are the user defined weights which is decided during

        m = No: of matching minu

        enrollment. If there is any repeated pi then score is computed

        tiae NI = No: of minutiae in input

        NT = No: of minutiae in template

        Images with this score higher than a particular threshold are considered as matched images. Then final decision is made by comparing with result from face recognition.

        Fusion Module

        Output from the two modalities are combined using sum rule by taking matching score. Scores are taken from each modalities. In the case of face recognition, normalized minimum euclidean distance is used to compute the score. Score from face module is calculated using the equation 8. In the case of fingerprint recognition matching score is calculated using equation 10. From the face recognition module, five face images close to the input image is taken based on euclidean distance. Let it be p=[ p1 p2 p3 p4 p5]. It indicates 5 persons or even 5 images of same person as 9 images per person is used for training. Similarly from the

        only for its first occurance. Final decision is made on the basis of this final score.

        Decision Module

        Final score is used to decide whether the person is authen- ticated or not. The person common in both face and fingerprint output is selected as the authenticating person. If there are more than one person that is common in both face and fingerprint selected images, then the person with highest score is authenticated as the finally selected person.

        RESULTS AND PERFORMANCE ANALYSIS

        1. Dataset

           This section shows the results of the proposed method and its comparison with unimodal biometric systems. The data set used in this project work uses face and fingerprint of 40 persons. The proposed framework is evaluated using standard databases and a database created by us. A well known standard ORL database is used to evaluate the performance of face recognition algorithms and to compare the results. In the case of fingerprint recognition, two standard databases are used: FVC2002 and FVC2004. As both databases are independent each fingerprint was assigned a corresponding face image. The database thus formed had fingerprint and face from 40 persons with each having single fingerprint image and nine face images in the training set. The proposed work is implemented using MATLAB2015.

        2. Results and comparison with unimodal systems

           Performance metrices used to evaluate the work are True acceptance rate (TAR), false acceptance rate (FAR) and false rejection rate (FRR). True acceptance rate is the probability that the system correctly authorizes an authorized person. False acceptance rate is the probability that the system incorrectly authorizes a non-authorized person. False rejection rate is the probability that the system incorrectly rejects access to an authorized person. They are calculated using equations 12, 13, 14

           :

           =

           :

           (12)

           Figure 3: Fingerprint recognition and score generation

           fingerprint recognition module, 5 fingerprint images close to input fingerprint is taken based on the similarity score. It indicates 5 persons selected by fingerprint recognition module. Let it be q=[ q1 q2 q3 q4 q5]. Then check whether any pi is equal to qj. If equal then find the final score using the equation 11

           :

           = (13)

           :

           :

           =

           :

           (14)

           = 11 + 22 (11)

           Face Recognition is done using the well known ORL database. It consist of a set of 400 face images of 40 persons with 10 images per person. All images are gray-scale and

           normalized to a resolution of 112 x 92 pixel. For evaluating face recognition algorithm, experiment is conducted by reducing the number of images in the training data set. Table II shows the comparison of results of proposed method with that of face recognition in terms of false acceptance rate.

           TABLE 2: Comparison with face recognition in terms of FAR(%)

        Crossing Number	PROPERTY
        0	Isolated Point
        1	Ridge Ending
        2	Continuing Ridge
        3	Ridge Bifurcation

        No: of training images per person	PCA	PCA+LDA	Proposed Method
        9	0.5	0.25	0
        8	1.25	0.25	0.25
        7	2.25	0.75	0
        6	3.5	1.5	0
        5	11	5.75	0.5

        For face, first 9 face images per person is used for training and last image is used for testing and obtained 0.5% FAR for PCA, 0.25% FAR for PCA+LDA and 0% FAR for proposed method. Then the no: of face images per person in training set is reduced from 9 to 8 and obtained 1.25% FAR for PCA, 0.25% FAR for PCA+LDA and 0.25%FAR for proposed method. This process is repeated until no: of images in training set becomes 5 and observed that

        1. PCA+LDA has low FAR than single PCA method which means multi-algorithm approach for face has improved results than using a single algorithm.

        2. Proposed method has very low FAR than unimodal face recognition system.

        3. Also (from Fig 5) as no: of images in the training set reduces, TAR reduces and FAR increases. But even in the reduced no: of training images, proposed method has greater true acceptance rate and low FAR compared to PCA and PCA+LDA methods.

   Figure 5: Comparison of proposed method with face recognition (%)

   Table III shows the results of proposed method in detail for the reducing no: of images in the training dataset. It

   shows that for the given dataset with 9 training face images per person it has 100% TAR , 0% FAR and 2.5% FRR.

   Performance of fingerprint recognition is evaluated using fingerprint of 40 persons with one training fingerprint image per person. For that 2 databases FVC2002 and FVC2004 are used. It gives 8 different poses of 10 distinct fingers and images are taken using optical sensor. In addition to this 10 imposter images of 10 different individuals are also used to analyze false acceptance rate. Table IV shows the performace of fingerprint recognition algorithm

   TABLE III: Results of proposed method

   No: of training images per person	TAR	FAR	FRR
   9	100	0	2.5
   8	99.75	0.25	2.25
   7	100	0	2.5
   6	100	0	2.75
   5	99.5	0.5	4.5

   TABLE IV : Performance of fingerprint recognition algorithm

   Results shows that as threshold increases FAR reduces but FRR increases. So an optimim threshold is selected by trial and error method. Table V shows the comparison of results of proposed method with that of fingerprint part. It shows that proposed method has improved results than using only fingerprint as biometric.

   Table VII shows the overall performance analysis. From the table we can infer that multi-algorithm used for face reduces FAR by .25% and increases TAR about 0.7% for 9 training face images per person. Also by combining both face and fingerprint biometrics, TAR increases and FAR reduces to a great extend.

   TABLE V: Comparison with fingerprint recognition

   Threshold	FRR	FAR
   0.42	0	0.2
   0.43	0.05	0.2
   0.45	0.075	0.2
   0.46	0.075	0.1
   0.5	0.1	0

   Method	TAR	FAR	FRR
   Fingerprint	85	12.5	2.5
   Proposed Method	100	0	2.5

   Figure 6: Performance analysis (%)

   Table VI shows some figures of proposed work. Consider example of person 40. If unimodal face recognition

   system is used then output will be 15th person which is incorrectly recognized. If unimodal fingerprint recogniton system is used, then output will be correctly recognized as 40th person. The proposed multimodal system always recogizes correctly as 40th person.

   TABLE VI: Some figures

   person	face output	fingerprint output	output
   11	11, 11, 11, 11, 14	11, 18, 35, 15, 6	11
   5	8, 5, 5, 5, 5	5, 18, 10, 36, 15	5
   14	14, 14, 14, 14, 14	1, 14, 15, 13, 36	14
   28	32, 32, 32, 28, 26	10, 1, 28, 7, 25	28
   25	26, 25, 25, 25, 25	15, 1, 10, 8, 25	25
   30	30, 30, 30, 30, 30	10, 7, 6, 30, 35	30
   31	31, 31, 31, 31, 32	7, 31, 33, 8, 22	31
   40	15, 30, 15, 15, 40	40, 33, 17, 20, 21	40
   1	17, 1, 17, 17, 1	1, 15, 23, 7, 40	1

   Lets consider the 28th person. Unimodal face recognition system incorrectly recognized it as 32nd person and unimodal fingerprint recognition system incorrrectly recognized it as 10th person. But the proposed multimodal system correctly recognized it as 28th person.

   TABLE VII: Performance Analysis

   Method	TAR	FAR
   PCA	99.5	0.5
   PCA+LDA	99.75	0.25
   Fingerprint	85	15
   Proposed Method	100	0

4. CONCLUSION

Multimodal biometrics can overcomes the drawbacks of unimodal biometrics like intra-class variation, inter-class similarities, and spoofing. Proposed method combines multi- modal biometrics and multi-algorithmic approach to reduce the issues with unimodal biometrics and to increase security. Two algorithms, PCA and LDA were used for processing face and crossing number based method is used for processing fingerprint. We compare the results of face recognition algo- rithms and fingerprint recognition algorithm with the proposed method in terms of false acceptance rate, false rejection rate and true acceptance rate by reducing the number of face images per person in the training data set. The experimental results shows that proposed method performs better than unimodal face and fingerprint recognition systems. Also the multi-algorithmic PCA+LDA method used in face recognition has improved results than that of using single PCA algorithm. Proposed method reduces false rejection rate and false accep- tance rate and improves true acceptance rate as compared to unimodal biometric systems but with increased computational time.

REFERENCES

1. P. S. Sanjekar 1and J. B. Patil, An overview of multimodal biometrics, An International Journal (SIPIJ) Vol.4, No.1, February 2013.

2. A. Mishra, Multimodal Biometrics it is: Need for Future System, Inter- national Journal ofComputer Application, vol. 3, no. 4, pp. 28- 33, June 2010.

3. Roli Bansal, Priti Sehgal and Punam Bedi, Minutiae extraction from fingerprint images-a review, International journal of computer science issues, Vol.8,Issue 5,September 2011

4. Atul S. Chaudari, Girish K. Patnaik, Sandip S. Patil, Implementation of Minutiae Based Fingerprint Identification System Using Crossing Number Concept, Informatica Economic vol. 18, no. 1/2014

5. Suman Kumar and Kumar Rahul,Face Recognition by Linear Discrim- inant Analysis ,ISSN International Journal of Communication Network Security,vol.2,2013.

6. Sunny Arief Sudiro, Michel Paindavoine, Tb. Maulana Kusuma, Simple Fingerprint Minutiae Extraction Algorithm Using Crossing Number On Valley Structure, IEEE 2007

7. Liton Chandra Paul , Abdulla Al Sumam,Face Recognition Using Prin- cipal Component Analysis Method, International Journal of Advanced Research in Computer Engineering Technology (IJARCET) Volume 1, Issue 9, November 2012.

8. Sandip Pawar and Ajit Ghodke, A Survey of Minutiae Extraction from Various Fingerprint Images, International Journal of Advanced Research in Computer Science and Software Engineering, Volume 6,

   Issue 6, June 2016

9. A. Ross and A. Jain, Information Fusion in Biometrics, Journal of Pattern Recognition Letters, vol. 24, pp. 2115-2125, 2003.

10. Manal Abdullah, Majda Wazzan and Sahar Bo-saeed, OPTIMIZING FACE RECOGNITION USING PCA, International Journal of Artificial Intelligence Applications (IJAIA), Vol.3, No.2, March 2012.

11. Kevin W. Bowyer,1 K. I. Chang,1 P. Yan,1 P. J. Flynn,1 E. Hansley,2

12. Steven Fernandes and Josemin Bala, Performance Analysis of PCA- based and LDAbased Algorithms for Face Recognition, International Journal of Signal Processing Systems Vol. 1, No. 1 June 2013.

[13] S. M. Rajbhoj and P. B. Mane,An Approach of Combining Iris and Fingerprint Biometric At Image Level in Multimodal Biometrics Sys- tem,International Journal of Soft Computing and Engineering (IJSCE)

[14] Krishneswari, K. and Arumugam ,Multimodal Biometrics using Feature Fusion,Journal of Computer Science 8 (3): 431-435, 2012 ISSN 1549- 3636 2012 Science Publications.

[15] M. E. ElAlami, A. E. Amin and A. E. El-Alfi,A Personal Iden- tification Framework based on Facial Image and Fingerprint Fusion Biometric,International Journal of Computer Applications (0975 8887) Volume 51 No.7, August 2012.

[16] S. K. Hese and M. R. Banwaskar,Performance Evaluation of PCA and LDA for Face Recognition,International Journal of Advanced Research in Electronics and Communication Engineering (IJARECE) Volume 2, Issue 2, February 2013.

1. Zhixin Shi and Venu Govindaraju, A chaincode based scheme for fin- gerprint feature extraction, Pattern Recognition vol.27,2006, pp.462468.

2. Hui-Yeoun Hwang, Jung-Hwan Shin, Sung-I1 chien and Joon-Jae Lee, Run Representation Based Minutiae Extraction in Fingerprint Image, lAPR Workshop on Machine Vision Applications, Dec. 11 13, 2002, Nara- ken New Public Hall. Nara, Japan.

3. Rupali L Telgad, Dr. P D Deshmukh, Almas M N Siddiqui, Combi- nation approach to score level fusion for multimodal biometric system by using face and fingerprint, IEEE international conference on Recent advances and innovation in engineering, May 2014