Online Attendance using Facial Recognition

Online Attendance using Facial Recognition

Sharanya T

Student, Dept. of CSE Sir MVIT

Bangalore, India

Sucharith P Student, Dept. of CSE

Sir MVIT

Bangalore, India

Ms. Dhivya V

Asst. Professor,

Dept. of CSE, Sir Mvit-Bangalore

Ujwal Kasturi Student, Dept. of CSE Sir MVIT

Bangalore, India

Trisheeka Mahesh Student, Dept. of CSE Sir MVIT

Bangalore, India

Abstract – In the modern world, education system is advancing day-by-day due to the introduction of concept of smart classroom. However, the attendance system still remains primitive, where the teacher/lecturer calls the name of students to mark their attendance. The automatic attendance management will replace the manual method, which takes a lot of time, that is, it is very time consuming and is difficult to maintain. There are many biometric processes, among which face recognition is the best method. In this project, we are going to describe the attendance without human interference. In this method the camera is fixed within the classroom and it'll capture the image, the faces are detected and then it's recognized with the database and finally the attendance is marked. It also proposes a single image-based face liveness detection method for discriminating 2-D paper masks from the live faces. Still images taken from live faces and 2-D paper masks were found with the differences in terms of shape and detailedness. In order to effectively employ such differences, we exploit frequency and texture information using various algorithms. We will be trying to improve the accuracy to a great extent and thus generate the final attendance report after updation in the database.

1. INTRODUCTION

   1. History

      During 1964 and 1965, Bledsoe, alongside Helen Chan and Charles Bisson, worked on using the pc to acknowledge human faces (Bledsoe 1966a,

      1966b; Bledsoe and Chan 1965). He was pleased with this work, but because the funding was provided by an unnamed intelligence that didn't allow much publicity, little of the work was published. Based on the available references, it had been revealed that the Bledsoe's initial approach involved the manual marking of varied landmarks on the face like center of the eye, mouth, etc., and these were mathematically rotated by computer to satisfy the pose variation. The distances between landmarks were also automatically computed and compared between images for identification.

      Given an outsized database of images (in effect, a book of mug shots) and a photograph, the matter was to pick from the database some set of records such that one among the image records matched the photograph. The success of this

      method was measured in terms of the ratio of the solution list to the amount of records within the database.

   2. Overview

   One of the foremost reminiscences everyone has about college is the morning roll call that the lecturers would in person call upon our names, and we tend to reply in affirmation to prove our attendance. Its a long and tedious routine in educational institutions and several people have manipulated the manual attendance system. Attendance being a very important side of administration might usually become a time constraint, repetitive job, loaning itself to inaccuracies. Organizations need to keep a track of individuals inside the organization like staff and students to maximize their performance. Managing students attendance at lecture periods has become a tough challenge. The ability to work out the attendance proportion becomes a significant task as manual computation produces errors, and wastes a great deal of our time. The basis of developing an automatic attendance management system is to computerize the standard method of taking attendance. In Face Detection and Recognition Systems, the flow of process starts by being able to detect and recognize frontal faces from an input device, i.e, camera. In todays world, it has been proven that students engage better during lectures only when there is effective classroom control. The need for high level student engagement is very important. Students need to be continuously engaged during lectures and one of the ways is to recognise and address them by their names. Therefore, a system like this may improve classroom control.

   Maintaining the attendance is very important in all the institutes for checking the performance of students. Some institutes are taking attendance using the old file or paper based manual approach and a few have adopted methods of automatic attendance using some biometric techniques. But in these methods students have to wait for long time in making a queue at time they enter the class room. Many biometric systems are available but the key authentications are same in all the techniques.

   Every biometric system consists of enrolment process during which unique features of an individual is

   stored within the database, then there are processes of identification and verification. These two processes compare the biometric feature of a person with previously stored template captured at the time of enrolment. Biometric templates are often of the many types like Fingerprints, Eye Iris, Face, Hand Geometry, Signature, Gait and voice.

   Feature extractions in general is done by collecting a large set of faces and adopt certain machine learning algorithms to learn a face model to perform classification. Face recognition is the process to detect the faces in the frame and extract feature from the detected faces and compare it with the faces stored in the database. The face which best matches is recognized and the attendance of the recognized person is updated. This image will act as input to the system. A database of faces must be created for the aim of comparison. By enrolling the students to that database, we can maintain the information of the students like roll number, student's name, and his or her photo for the identification. Many biometric systems are available but the key authentications are same in all the techniques. Every biometric system consists of enrolment process during which unique features of an individual is stored within the database, then there are processes of identification and verification.

2. LITERATURE REVIEW

   In the paper, Face Recognition based Attendance System [5], the task of the proposed system was to capture the face of each student and to store it in the database for their attendance. Capture video: The Camera is fixed at a specific distance inside a classroom to capture videos of the frontal images of the entire students of the class. Separate as frames from the video: The captured video needs to be converted into frames per second for easier detection and recognition of the students face to generate the attendance database network: CNN (Convolution Neural Network). It can be implemented in larger areas like in a seminar hall where it helps in sensing the presence of many people. Sometimes the poor lighting condition of the classroom may indirectly degrade system performance, this can be overcome the quality of the video or by using some algorithms.

   According to the paper, Attendance Management Using Facial Recogniton [6], the proposed system strives to outgrow the constraints of the existing systems and provides features such as detection of faces, extraction of features, detection of extracted features and analysis of students attendance. The systems correctness in detecting and recognizing faces will be more due to use of larger number of features (shape, color, LBP, wavelet, Auto correlation, etc.) of the face. It mainy used algorithms like Eigen face, line edge map, Histogram of oriented gradients (HOG) for detecting faces. Eigen Face: This algorithm extracts the necessary information from an image and encodes it efficiently. Line Edge Map: Line matching is done to map the most prominent features of the face. Histogram of oriented gradients: Applied to detect objects. Images are

   converted to greyscale and every pixel is assigned an integer gradient: Applied to detect objects. Images are converted to greyscale and every pixel is assigned an integer.

   In reference to, Automatic Student Attendance Management System using Facial Recognition [4], the proposed system describes the algorithms used in the automated attendance system such as Viola & Jones algorithm for face detection and face recognition by MSE. The model converts the RGB image to YCbCr image to detect features from the face using Viola & Jones algorithm. MSE is calculated using both the live image and referenced image pixels, if the MSE is greater than the threshold, then the face will be detected. This system has high efficiency rate and is faster and provides ease of access for the user.

   In the paper Face Liveness Detection Based on Texture and Frequency Analyses by Gahyun Kim, Sungmin Eum, Jae Kyu Suhr, Dong Ik Kim, Kang Ryong Park and Jaihee Kim, the proposed system is a single imagebased fake face detection method based on frequency and texture analyses for discriminating 2-D paper masks from the live faces. For the frequency analysis, they carried out power spectrum based method which exploits not only the low frequency information but also the information residing in the high frequency regions. Moreover, the Local Binary Pattern (LBP) based description method has been employed for analyzing the textures on the given facial images. In addition, the fused information of the specific decision values from the frequency based classifier and the texture based classifier has also been utilized for detecting fake faces. Frequency-based feature extraction: The given facial image is transformed into the frequency domain by using the 2-D discrete Fourier transform. Texture-based feature extraction: For analyzing the texture characteristics of the images taken from the live faces and masks, this paper utilizes the LBP. Fusion-based feature extraction: This paper utilizes Support Vector Machine classifier to learn liveness detectors with the feature vectors generated by power spectrum-based and LBP-based methods.

   According to the authors of Automated Attendance Management System Using Facial recognition, the objective of the system proposed is to automate the attendance system that is reliable, practical and, eradicates disruption and loss of time in the obsolete system of attendance. The process starts with training the system with face of students/employees for whom the attendance has to be marked in the near future. Different faces are assigned different names in the system. The duration of class decides the threshold time limit within which attendance will be marked for the student. Once the threshold limit is crossed is crossed the student will not be marked absent in the system for that particular period. Algorithms such as PCA, LDA and LBPH can be used in varying light scenarios, as light plays an important role in image processing. It used algorithms like Eigen face, line edge map, Histogram of oriented gradients (HOG) for detecting faces.

3. HAAR CASCADES IN OPENCV

   OpenCV was started at Intel in 1999 by Gary Bradski for the purposes of accelerating research in computer vision and commercial applications of computer

   vision in the world and, creating a demand for ever more powerful computers by such kind of applications. Intels open-source computer-vision library can maximally simplify computer-vision programming. It includes advanced capabilities such as, face detection, face tracking, face recognition, Kalman filtering, and a variety of artificial intelligence (AI) methods in ready-to-use form. In addition, it provides many basic computer-vision algorithms through its lower-level APIs.

   OpenCV uses a type of face detector algorithm or a classifier called a Haar Cascade classifier. Given an image, which can come from a file or from a live video input, the face detector examines each image location and classifies it as Face or Not Face. The classification improvises a fixed scale for a face, say 75×75 pixels. Since faces in an image might be smaller or larger than this, the classifier runs over the image several number of times, to search for faces across a range of scales.

   A Haar wavelet is a mathematical function that produces square-shaped waves with a beginning and an end, and is used to create box shaped patterns to recognise signals with sudden transformations. To analyse an image or a set of images using Haar cascades, a scale is selected smaller than the target image. It is then placed on the image, and the mean of the values of pixels in each section is taken. If the difference between two values pass a certain threshold, it is considered a match. Face detection on a human face is performed by matching a combination of different Haar-like features also known as digital image features.

   Fig. 1. Rectangular Hair-like features

4. CONVOLUTIONAL NEURAL NETWORKS

   A Convolutional Neural Network (CNN) is a Deep Learning algorithm which can take an input image or a set of images, assign importance (learnable weights) to various aspects in the images and be able to differentiate one from the other. The pre-processing required in this algorithm is much lesser compared to other classification algorithms. Although, the primitive method filters are hand-engineered, with enough training, these have the ability to learn the features of images. The architecture of a CNN is similar to that of the connectivity pattern of Neurons in the Human Brain and was inspired by the organization of the Visual Cortex. The different layers of the CNN can be briefed as follows,

   • Convolution is the first layer to extract both small and high level features from an input image. It preserves the relationship between the pixels by learning image features using small squares of the input data. Convolution is a mathematical operation that takes two inputs such as image matrix and a filter (kernel). At the end of the convolution process, there is a featured matrix which has lesser parameters (dimensions) than the actual image as well as more clear features than the actual one. The featured matrix is now worked upon further.

   • The pooling layer is solely to decrease the computational power required to process the data. It is done by decreasing the dimensions of the featured matrix even more. In this layer, the dominant features are extracted from a restricted amount of neighbourhood. There are two types of pooling techniques – AVERAGE-pooling and MAX-pooling. The difference between these two is, in AVERAGE-pooling, the average of all the values of pooling region is taken and in MAX-pooling, the maximum amongst all the values lying inside the pooling region is considered. So, after pooling layer, there exists a matrix containing the main features of the image and this matrix has even lesser dimensions, which will help a lot in the next step.

   • Now that the input image is covered into a suitable form for the Multi-Level fully connected architecture, the image is flattened into one column vector. The flattened output is fed to a feed-forward neural network and back-propagation applied to every iteration of training. Over a series of epochs, the model can differentiate between dominating and certain low-level features in images in order to classify them.

     Fig.2. Representation of a CNN

5. PROPOSED SYSTEM

1. Face Detection

   In this section, we implement the use of Haar- cascades which is included in the OpenCV library.

   Face detection uses some classifiers, which are algorithms that detects if sections of an iage is either a face(1) or not a face(0). Classifiers have been trained to detect faces using thousands to millions of images in order to get more accuracy. OpenCV uses two types of classifiers, Local Binary Pattern (LBP) and Haar Cascades (Haar Classifier). We plan to implement the latter classifier which uses integral image concepts to compute the features (characteristics) detected. Our Haar Cascades use the Adaboost learning algorithm which selects a small number of important features from a large set to give an efficient result.

2. Face Recognition

In this section, we implement a deep learning algorithm known as the Convolutional Neural Network. This algorithm is used within the Keras library as a sequential model to perform image preprocessing, face recognition and liveness detection. The Keras library can be defined as a high-level neural networks API, written in Python which is capable of running on top of TensorFlow, CNTK, or Theano. Keras is an API designed for human beings, and not machines. Keras follows the best practices for reducing cognitive load on systems as it offers consistent and simple APIs. It also minimizes the number of user actions required for common use cases, and it provides clear feedback upon user error. The Convolutional Neural Network here, is comprised of many layers to perform image pre-processing before it is presented as an input to the feed- forward network. Each of the input images will pass through a series of convolution layers with filters (Kernals), Pooling, fully connected layers (FC) and then apply Softmax function to classify live faces with probabilistic values between 0 and 1.

Fig. 3. Activity diagram of the System

1. IMPLEMENTATION DETAILS

   VI. SYSTEM ANALYSIS

   1. System Description

      The automated online attendance system uses face recognition technology for its core purpose. It provides online access for two user roles : Student and Administrator roles respectively.

      Administrator Role : The administrators role is deemed as the most fundamental since he/she will either be aggregating the entire process or monitoring in depth. The administrator must first register his/her subject if there isnt one in order for the students to enrol. If the subject already exists, the administrator can choose the subject for which attendance should be marked.

      Student Role : The students primary role is to provide attendance by interacting with the system. However, the student must be enrolled for the system to recognise his/her face. This is done initially with the student following through the register process. After the registration process is complete, the student will now be able to provide attendance for many of the various subjects.

   2. System Design

   The workflow of this proposed methodology can be summarised and graphically represented through the activity diagram below:

   To achieve our set objectives, the components were developed as a window service application using Machine Learning on a Python framework. The system is designed to shift compute intensive tasks like face detection and recognition to the on-premise local server which uses TensorFlow framework to further work upon, since image processing could be very compute intensive especially when large data is involved. This enables client local server to be any commodity hardware when implementing the system. The window service component is a lightweight application that runs on the background of the on-premise local server and a front end interface using a GUI component. It is responsible for listening to attendance request, starting the camera and continuous acquisition of images from anywhere. The service sends all acquired images to the local server for storage through web service mechanism where they are processed and updated in a mutable excel sheet. The GUI aspect of the system is developed to provide an intuitive and simple self- service interface for users to interact with the system. It performs all the tasks of image storage, image processing (face enrolment, face detection, face recognition) and attendance management. It also enables users perform all administrative tasks. One of the key features of this system is, it does not allow any form of presentation attacks such as identity spoofing, that could be forced upon the interface. This represents the significance of liveness detection of the automated attendance system.

2. ANALYSIS AND RESULTS

   The online attendance system and its machine learning models were designed on the Visual Studio Code IDE. The source code was developed using Python for back- end and server side frameworks, and Tkinter GUI for the front-end user interface. The implementational workflow of this system can be summarised as follows,

   A machine learning model was designed and developed using the OpenCV, TensorFlow and Keras libraries to perform facial recognition and liveness detection. This was followed by implementing a dynamically run source code to create data sets upon which the model was trained.

   The datasets were created when the user or the student takes a picture of himself while following the enrolment process. The model was then trained on this picture. The model was further tested on a live feed which was a dynamic dataset captured during attendance process.

   A mutable excel sheet was created in order to store student and attendance details. The source code could dynamically interact and manipulate this worksheet using functionality from the openpyxl library.

   To provide a simple and suitable user interface, the Tkinter GUI model which belongs to the Python UI modules was implemented. This provided an interactive window for the user to perform the various actions required for managing and maintaining attendance.

   Fig. 4. Recognizing the face and displaying USN.

   Fig. 5. Identifying a spoofed face.

   Fig. 6. Updating attendance

   Table I. Test Cases

3. CONCLUSION

   In this approach, a face recognition based automated student attendance system is thoroughly described. The proposed approach provides a method to identify the individuals by comparing their input image obtained from recording video frame with respect to train image during the respective semesters registration process. This proposed approach is able to detect and localize faces from an input facial images present in the database, which is obtained from the recording video frame. Besides, it provides a method in pre-processing stage to reduce the illumination effect and enhance the image contrast. From the challenges we faced, it can be concluded that the use of ordinary generic cameras for video replay attacks in a non- intrusive technique may lead us to some cost-effective face anti-spoofing systems. There is a need for designing,

   developing and deploying non-intrusive methods without using extra devices. As a conclusion, this proposed system replaces the manual system with an automated system which is fast, efficient, cost and time saving as it replaces the stationary material such as bulky registers and the paper work.

4. FUTURE ENHANCEMENT

   In the future this system can be implemented as a part of the smart classroom objective. However, the efficiency could be improved by integrating high speed computers with a good RAM and here are some of the ways in which the entire system can be improved:

   1. Hardware Enhancement

      Usage of high speed storage drivers and extended RAM could fetch us faster training time and turn-around time for requests made to the TensorFlow engine.

   2. Camera Enhancement

      Usage of a good quality video camera capable of capturing live feed in 720p resolution or 4K quality can help the built model to extract high level features. The software however, is another aspect yet the most significant one to be considered for improvement. Some of the improvements that can be done to enhance the qualities of the Neural Network are:

      • Separatin person and pose information based on unsupervised clustering: Since a single image is not enough to form the decision boundaries which are representative of diferent variations, one of the most common approaches is enlarging the available dataset. This can be done by generating variations of the frontal-view face and these virtual images are used to appropriately form the decision boundaries corresponding to the different variations. This brings about better results compared to those of training with single frontal-view images.

      • Neural network structure inspired by neortex functional model: The goal of this paper is to introduce and implement a novel neural network which can recognize input face images with various variations properly, similar to human brain function. This model should be adaptive and bidirectional like Koerner model of neocortex in which recognition is not performed in a single stage, but via different phases and in each phase, the recognition results of previous stages are used for model adaptation.

      • Algorithm Tuning: Some of the factors to be considered in algorithm tuning are, Diagnostics, Weight Initialization, Learning Rate, Activation Functions, Network Topology, Batches and Epochs, Regularization, Optimization and Loss, Early Stopping, and preventing underfitting and overfitting of data.

      Using these methods, the number of variant and learning parameters in recognition system can be increased. With these properties, which are required for designing a robust

      recognition system, we strive towards developing a system to perform face recognition just like the human brain.

5. REFERENCES

1. Khem Puthea, Rudy Hartanto and Risanuri Hidayat, A Review Paper on Attendance Marking System based on Face Recognition, 5th Int'l Conference on Image Processing, Communication, Engineering and Technology (IPCET-17) Oct. 11-12, 2017.

2. Khan, Mubashir, Apeksha M. Gopale, Obaid Chowdhary, Ali Punjani and Yasir Choudhary. Automated attendance management system using face recognition. (2017).

3. Venkata Kalyan Polamarasetty, Muralidhar Reddy Reddem, Dheeraj Ravi, Mahith Sai Madala, Attendance System based on Face Recognition, International Research Journal of Engineering and Technology (IRJET), Volume: 05, Issue:04, April-2018.

4. Sathyanarayana, Ramya, Ruchitha, and Shwetha H S, Automatic Student Attendance Management System Using Facial Recognition, International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE) ISSN: 0976-1353 Volume 25 Issue 6 MAY 2018.

5. Nandhini R, Duraimurugan N, S.P.Chokkalingam, Face Recognition Based Attendance System, International Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249 8958, Volume-8, Issue-3S, February 2019.

6. Rajath S Bharadwaj, Tejus S Rao, Vinay T R, Attendance Management Using Facial Recognition, International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN: 2278-3075, Volume-8 Issue-6, April 2019.