Facial Recognition Attendance System using Machine Learning and Deep Learning

Facial Recognition Attendance System using Machine Learning and Deep Learning

Shashank Joshi

Computer Science Vishwakarma Institute of Technology,

Pune, India

Prof. Sandeep Shinde

Computer Science Vishwakarma Institute of Technology,

Pune, India

Prerna Shinde

Computer Science Vishwakarma Institute of Technology,

Pune, India

Neha Sagar

Computer Science Vishwakarma Institute of Technology,

Pune, India

Sairam Rathod

Computer Science Vishwakarma Institute of Technology,

Pune, India

Abstract – The old method of marking attendance involves the lecturer providing an attendance sheet to the students for their signature or the teacher calling out students name individually to mark them present. This old manual method is pretty hectic for teachers and students too. Since, after taking the signed attendance sheet from students, teachers have to manually keep track of every student in the logbook which turned out to a lot of time wastage, missing out students presenteeism or students giving proxies for the absentees due to which logbooks can be easily manipulated or prone to errors also, wastage of pen and paper. To avoid this problem, we have developed a system which will monitor the attendance of students by identifying their faces via their facial features. While developing this system we have used a web Camera to capture multiple live images of students for face recognition, Viola-Jones Algorithm to achieve face detection which uses Haar Cascade classifier, Pre- processing which converts the image in greyscale, LBPH algorithm and deep learning algorithms like CNN (Convolutional neural networks) for feature extraction and last but not the least the input faces are then matched with the trained images in the database and once they match, the student will be marked as present and the ones who didnt match weremarked as absent in the class. Accuracy of 85% and 95% was obtained by testing the model with ten different faces with different facial expressions, angle and lighting conditions for LBPH algorithm and CNN (Convolutional neural networks) respectively.

Keywords Face recognition, Face detection, feature extraction, LBPH algorithm, Viola-Jones algorithm, Haar Cascade classifier, CNN (Convolutional neural networks), Deep Learning.

1. INTRODUCTION

   Traditionally marking attendance involves students sitting in a classroom and the teacher calling out the names of the students individually to mark their attendance. The attendance is usually marked with a pen and then stored in a logbook. The traditional attendance system has a lot of

   disadvantages it is hard to keep records, it is time- consuming, error prone, wastage of resources.

   Another system that is around is a biometrics system. A biometric system has three phases registration phase, storage, and recognition phase. The registration phase involves capturing specific triats. The triat is saved in the form of a graph or code. The recognition phase extracts biometric triat and compares it with the database to see if matches or not.

   Face recognition has numerous advantages over biometrics as they require less action from users and multiple attendances can be marked at a time.

   The main moto of the paper is to implement an attendance system using facial recognition. Face recognition involves image capture through a web camera, Face detection using viola jones algorithm which uses haar cascade classifier, pre-processing, storing the image in the database, feature extraction through LBPH algorithm and CNN (Convolutional neural networks) then comparing it with the input given by the user and if it matches attendance will be marked if it doesnt attendance wouldnt be marked.

   Hence, our proposed system aims to mark attendance automatically by means of face recognition. The teacher can monitor attendance easily.

2. LITERATURE SURVEY

   Poornima, Sripriya, Vijayalakshmi, Vishnupriya proposed a work of monitoring attendance using Facial Recognition in which Audio was generated when the student was absent, and also gender classification was implemented. For audio output, they used speech API and used facial distance measure as a progenitor to achieve the gender

   classification. They have used the PCA algorithm for it [1]. Kennedy Okokpujie, Etinosa Noma- Osaghae, Samuel John, Kalu-Anyah Grace, Imhade Okokpujie proposed a system that marks the attendance of students and sends notifications on cell phone using facial recognition technique and GSM. Software used by

   them were PostgreSQL, SDK, and fisher face. [2]. Jeevan. M, Yashwanth Gowda. S, Sindhu. J developed a system that recognizes the faces of people moving in a cluster. For face detection, they have used viola- jones and for feature extraction, theyhave used PCA [3]. Shivam Singh, Prof. S. Graceline Jasmine proposed a system that recognizes the faces of people in motion in this paper they have used algorithms like KLT, Viola-Jones for face detection [4]. A Madhu Sudhan, Sudhir Kumar. Sharma developed a system of marking students attendance automatically by using face recognition algorithms like LBP, SVM [5]. Mrs. Madhuram.M, B. Prithvi Kumar, Lakshman

   Sridhar, Nishanth Prem, Venkatesh Prasad designed a system that recognizes a persons face by means of an IP camera and image set algorithm using open CV [6]. SudhaNarang, Kriti Jain, MeghaSaxena, AashnaArora proposed a model for monitoring students attendance using OpenCV also, they have compared different face recognition algorithms in this paper [7].

3. PROPOSED MODEL

   Fig.1:Proposed Model

   This face recognition system consists of mainly two parts. The first section is while enrolling a student into a course his/her face will be captured and stored in the database. In the second section, while a student is entering the class his/her face will be captured through a camera in the classroom and then recognized, and accordingly, the student will be marked absent or present.

   1. Enrollment

      1. Image Capture- Multiple live images

         will be captured through the webcam.

         We have included/captured several images of each student with variations like students wearing spectacles or not, students with beard/moustache/clean shaved, etc.

      2. Face Detection-Viola-Jones Algorithm is used for face detection. Below is an explanation of how the algorithm exactly works.

         Viola-Jones Algorithm- Given a picture, the algorithmic rule looks into several smaller regions and tries to search a face by finding specific features in each region. This algorithmic rule works on a grayscale image, it must check many different positions and scales because an image can contain many faces of various sizes. This algorithm uses Haar-like features to detect faces.

         It has four main steps, that we shall discuss within the sections to follow:

         1. Selecting Haar-like features-Haar-like features are used in recognizing an object. Human faces have some properties like the eye region is dark as compared to its neighbour pixels, and the nose region is brighter than the eye region. In the dark region, the total sum of pixels will be smaller than the total sum of pixels in the light region.

            There are three types of Haar-like features that are used in this algorithm.

            1. Edge features

            2. Line-features

            3. Four-sided features

            Fig.2:Haar-like features

         2. Creating an integral image-To calculate a value for each feature, we need to perform computations on all the pixels within that specific feaure. Actually, these calculations can be very large, so the integral image allows us to perform the calculations easily. An integral image, the value of each square is the total sum of all pixels to above of it plus, the sum of all to the left side of it plus, the pixel at the square itself.

         3. Running AdaBoost training-The range offeatures present in the detector is approximately 160,000, but only a some of those features play a vital role in detecting a face. So, we use this algorithm to identify the best of all of the available features.

            So, when were training the AdaBoost to identify important features, the algorithmic rule sets a minimum boundary to inform whether some feature can be classified as useful or not when we have trained the algorithm according to the input.

         4. Creating classifier cascades-The cascade classifier eliminates all the non-faces instead of wasting time in computations. In this way, speed can be increased. At every stage, the classifier eliminates the non-face.

         Fig.3:Face Detection

      3. Pre-Processing- In this, the image is converted into greyscale. Then these detected images are stored in the data base.

         Fig.4:Image after Pre-processing

   2. Training

      The images undergo feature extraction using the LBPH algorithm and Convolutional neural networks (CNN). Students are given unique roll no. Based on the facial features and unique roll no the model is trained.

      Fig.5:Training Database

   3. Recognition

   1. Image Capture The real-time images of the students are taken as input through a web camera.

   2. Face Detection After the image is pre- processed it is then, treated with the Voila-Jones algorithm for the process of face detection.

   3. Pre-processing This process mainly involves converting the coloured image into grayscale. This is achieved by passing it through the Gaussian Filter and Median Filter. In image processing, Gaussian Filter is known as linear filter wherein, the Median filter uses the non-linear filter technique. The two of them can be used for unsharp masking. Where Gaussian filter blurs edges of the image and also reduces the contrast wherein, Median filter reduces the noise present in an image by keeping edges relatively sharp.

   4. Feature Extraction The images undergo feature extraction using the LBPH algorithm as it least affected by light variation. The idea with LBPH is not to look at the image as a whole, but instead, try to find its local structure by comparing each pixel to the neighbouring pixels.

      In LBPH we tend to compare each picture element(pixel) of the image with its neighbouring pixel. Center pixel is thresholding picture element(pixel) and its value may be an edge value. The neighbouring pixel values are compared with the edge value. If the value is more than edge value then it is set to one (1) and if it's less than the edge value then it is set to zero (0). This value is regenerated to binary by concatenating all the neighbour pixels. The pixels will concate while moving in a clockwise direction or anticlockwise direction.In this algorithm we tend to concate pixels in clockwise direction and convert it to binary.

      From obtaining value in binary, we tend to convert it into decimal. Then each picture pixel of image assigns a brand- new value. From the new value of the image, we make a histogram. All the local histogram is concatenating to get a feature vector. This is how Local Binary Pattern Histograms (LBPH) works.

      Fig.6:LBPH Architecture

      Feature extraction can also be done by deep learning .In deep learning Convolutional neural networks(CNN) is used.

      Convolutional neural consists of convolutional layers,

      pooling layers, fully connected layers .The model consists of 5 convolutional layers with max pooling and 3 fully connected layers. Max pooling is done to reduce dimensionality.This architecture is also known as Alexnet. It reduces the number of parameters and computations.

      Relu activation function is used in each layer except the output one. Using non linerarity of Relu CNN can be trained six times faster than functions like tanh or Sigmoid. To prevent overfitting data augmentation is done. Crops sized 227×227 from inside the

      256×256 image boundary are used for data augmentation. Dropout is done to prevent overfitting. A neuron is dropped with a probabitlity of 0.5. Doing this a more robust model is created. Alexnet accepts image of size [227x227x3] as input. For face recognition 2000 images per person are taken for higher accuracy . All the images are associated with the roll no of the student. The roll and name are in the hashmap form in the code. So the model can print the roll no as well as name in the excel sheet when the input is given and. Accuracy of 95% was achieved by using alexnet.

      Layer	Size of feature map
      Input	227x227x3
      Conv 1	55 x 55 x 96
      Max Pool 1	27 x 27 x 96
      Conv 2	27 x 27 x 256
      Max Pool 2	13 x 13 x 256
      Conv 3	13 x 13 x 384
      Conv 4	13 x 13 x 384
      Conv 5	13 x 13 x 256
      Max Pool 3	6x6x256
      Dropout 1	6x6x256

      Table.1:CNN Architecture

   5. Matching and Attendance The inputface images are compared with all face images stored in the training dataset, one by one. The images of the face of students were matched with the images available in the training dataset, once they match the student will be marked as present and the ones who didnt match were markedas absent in the class.

   Fig.7:Face Recognition

   Fig.8:Face Detection

   Fig.9:Image After Pre- processing

   Fig.10:Training Database

   Fig.11:Face Recognition

4. FUTURE SCOPE

   1. Frauds in ATM can be prevented. The facial recognition system will become more robust in the future as continuous improvement is being made in it. It will only authorize customers the entry.

   2. It can likewise be utilized during exams, for example, Civil Services Exam, SSC, IIT, MBBS, and others to distinguish the competitors.

   3. Duplicate electors are being reported all over the world. To stop this, it will only allow authorized voters with id to vote.

   4. This framework will widely be used in government and corporate offices all over the world.

5. RESULT

First step of Face recognition model involves image capture through a web camera. Second step involves Face detection using viola jones algorithm which uses haar cascade classifier. Third step involves pre- processing, this process mainly involves converting the coloured image into grayscale. This is achieved by passing image through the Gaussian Filter and Median Filter. Fourth step involves storing the image in the database. Then feature extraction through LBPH Convolutional neural networks (CNN). Then comparing it with the input givenby the user and if it matches attendance will be marked in the csv file, if it doesnt attendance will not be marked. GUI is also developed using tkinter in python.

S / N	Name	Corre ctly Identif ied	Incorrectl y Identified	Accur acy
1	Shashank	9	1	90
2	Sairam	8	2	80
3	Rohan	9	1	90
4	Prerna	8	2	80
5	Neha	10	0	100
6	Bobby	7	3	70
7	Abhi	9	1	90
8	Pratik	8	2	80
9	Rahul	9	1	90
10	Yash	8	2	80

S / N	Name	Corre ctly Identif ied	Incorrectly Identified	Accur acy
1	Shashank	10	0	100
2	Sairam	8	2	80
3	Rohan	9	1	90
4	Prerna	10	0	100
5	Neha	10	0	100
6	Bobby	10	0	100
7	Abhi	9	1	90
8	Pratik	10	0	100
9	Rahul	9	1	90
10	Yash	10	0	100

Table.2: Results obtained by different facial expressions, angle and lighting conditions by LBPH.

Table.3: Results obtained by differentfacial expressions, angle and lighting conditions by Convolutional Neural Network(CNN).

Accuracy=average of accuracy obtainedby testing with ten faces. For LBPH=850/10=85% For CNN= 950/10=95%

Roll Number	Student Name
1	Rohan
2	Shashank
3	Sairam
4	Prerna
5	Neha

Table.4:CSV file of attendance

Fig.12: GUI for face recognition VI.CONCLUSION

This system works very well as compared to

other systems which uses eigen faces, fisher faces, as we have used the most efficient algorithms here, that is Viola- Jones Algorithm used for face detection and the LBPH algorithm and Convolutional Neural Network (CNN) for feature extraction.

Accuracy of 85% is obtained by using LBPH algorithm. Accuracy of 95% is obtained by using CNN. This system also has confidence added to it which when only above 70% marks the attendance otherwise it doesnt mark the attendance Also, this system can recognize multiple faces, so that even the whole class can give attendance at thesame time. An image on a cell phone or a photocopy wont work while marking attendance, that is a student must physically be present for his/her attendance to be marked. In this way, proxy attendance using pictures on mobile phones can be avoided.

So, every institute or workplace must implement a facial recognition attendance system as the fingerprint system cant mark multiple attendances at a time and people may end up waiting in queues

VII. REFERENCES

[1] Poornima, Sripriya, Vijayalakshmi, Vishnupriya Attendance Monitoring System using Facial Recognition with Audio Output and Gender Classification, IEEE International Conference on Computer, Communication, and Signal Processing (ICCCSP- 2017)

[2] Kennedy Okokpujie, Etinosa Noma- Osaghae, Samuel John, Kalu- Anyah Grace, Imhade Okokpujie (2017) A Face Recognition Attendance System with GSM Notification, IEEE 3rd International Conference on Electro- Technology for National Development (NIGERCON)

[3] Jeevan. M, Yashwanth Gowda. S, Sindhu. J Face Recognition System, International Journal of Engineering Research & Technology (IJERT) ISSN: 2278-0181 NCESC – 2018 Conference Proceedings

[4] Shivam Singh, Prof. S. Graceline Jasmine Face Recognition System, International Journal of Engineering Research & Technology (IJERT)

ISSN: 2278-0181 Vol. 8 Issue 05, May-2019

[5] A Madhu Sudhan, Sudhir Kumar. Sharma Webcam Based Computerized Attendance System using Face Recognition Algorithms, International Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249 8958, Volume- 9 Issue-

3,February 2020

[6] Mrs. Madhuram.M, B. Prithvi Kumar, Lakshman Sridhar, Nishanth Prem, Venkatesh Prasad Face Detection and Recognition Using OpenCV, International Research Journal of Engineering and Technology (IRJET)

[7] SudhaNarang, Kriti Jain, MeghaSaxena, AashnaArora Comparison of Face Recognition Algorithms Using Opencv for Attendance System, International Journal of Scientific and Research Publications, Volume 8, Issue 2, February 2018