Student attendance system with drowsiness detection using Smart Camera

Student attendance system with drowsiness detection using Smart Camera

ICCIDT – 2023 Conference Proceedings

Chemmangat Hari Manoj Kumar

Department of Computer Science & Engineering, Albertian Institute of Science and Technology, India Email: chemmangathari@gmail.com

Ajeesha T.A

Department of Computer Science & Engineering, Albertian Institute of Science and Technology, India Email: ajeeshata4@gmail.com

Neha Ann Shibu

Department of Computer Science & Engineering, Albertian Institute of Science and Technology, India

Email: nehaannshibu10@gmail.com

Saranya Shaji

Department of Computer Science & Engineering, Albertian Institute of Science and Technology, India Email: saranyashaji@aisat.ac.in

Roshan Jishad Mecheril

Department of Computer Science & Engineering, Albertian Institute of Science and Technology, India Email: roshanjishad@gmail.com

AbstractAttending a class is an individuals first step to being ready to gain new knowledge and insights about various things that may be beneficial in ones life. But for younger generations attending a class doesnt seem to be much fun as they lack the motivation since the actual report shows their attendance comes up monthly or quarterly[1] A smart automatic attendance management system is being utilized to resolve this problem. Utilizing this framework, not only solves the problem of lack of motivation but also helps in reducing fraudulent cases such as calling for proxies thus generating the problem of students being marked present even though they are not physically present can easily be solved. This model will be an effective way to manage student attendance. Although there are many other automated techniques and systems being used in the industry like a biometric system that scans fingerprints one individual at a time, the other system available is similar to our system where the camera will be placed in front of the door and will recognize and mark attendance as each individual passes by. But when we look at these methods, there are still some procedures that need to be done for taking attendance in the fingerprint attendance system, each individual has to go to the fingerprint-scanning machine and mark the attendance by punching their finger on the machine, although attendance will become more secure when compared with the manual attendance system it does no or less noticeable changes to the time consumption. Our system, on the other hand, aims to make sure that attendance is being taken in the background with it being completely unnoticeable while a teacher/lecturer is taking a class. Our system aims to automate the attendance system completely by taking care of generating reports and we also introduce drowsiness/distraction detection in the class. Although detecting drowsiness or distraction[2] is one of our important features, our motive is not to use it for punishing the students, but to check the interactiveness of the class, so as to provide the teachers with constructive feedback. The extensiveness of the application for our system is very large and widespread as it can be used in exam halls, conferences, workshops, and also in online meetings.

KeywordsManual, Fingerprints, Biometrics, Automated, Attendance, Snapshot

1. INTRODUCTION

   The attendance system is of utmost importance nowadays. There are several reasons why a student attendance system may be necessary:

   1. To track student attendance.

   2. To monitor students.

   3. To ensure safety.

   4. To meet legal requirements.

   The method currently employed by institutions requires the faculty to pass an attendance sheet, call the students' names or roll numbers, and mark the students' attendance, sometimes disrupting the classroom's disciplined environment. This sheet then proceeds to the admin department where it is updated on an Excel sheet.This process is quite hectic and time-consuming. Since manual work is included in the attendance system there is a high chance that false attendance will be marked no matter how thoroughly one takes the attendance. The disadvantage of the manual system is that:

   1. The chances of proxy attendance are high and are difficult for teachers.

   2. It is difficult to search for particular data from this system (especially if the data we are asking for is from very long ago).

   3. Because manual computation is time-consuming and prone to error, it becomes difficult to calculate the attendance rate.

   4. The attendance sheet could be stolen or misplaced and so impersonation is easily possible.

      In this project, we have proposed an efficient Student Attendance System with drowsiness detection to take and maintain the attendance of students for all educational institutes. This proposed system along with taking, viewing attendance, and making requests to teachers so that unmarked attendance can be marked and has an additional feature of detecting

      drowsiness and generating corresponding reports. An important point to note here is that drowsiness detection is not intended to be used for punishing the students, this is recorded so that the department and the tutors can understand how interactive their class is. Our plans to execute an "Automated Attendance System Based on Facial Recognition", which imbibes broad applications, were previously outlined. Face identification is a feature of the application that saves time and removes the possibility of proxy attendance. The teachers need not know or interfere in the attendance process since the system takes up the task of doing

2. RELATED WORK

1. Attendance System Based on Face Recognition System Using CNN-PCA Method and Real-time Camera

   The study of human facial recognition is one advancement in computer vision. The human face recognition system is used as an attendance system in one of its applications.

   In order to detect and identify faces as a person's identity and to store them in a face database, the attendance system employs faces as objects. The face identification of the object faces captured by the camera is accomplished through the process of comparing face image data captured by the camera with face images that have been saved in the face database. In this research, a hybrid feature extraction technique called CNN- PCA (Convolutional Neural Network – Principal Component Analysis) is used for the face recognition-based attendance system. The goal of this combination of techniques is to create a feature extraction technique that is more precise. This camera is very effective and efficient in further improving the accuracy of user data for the facial recognition-based attendance system. This camera's facial recognition-based attendance system uses highly accurate data processing, enabling the creation of a system that is dependable and potent enough to recognize human faces in real-time.

2. IAAS: IoT-Based Automatic Attendance System with Photo Face Recognition in Smart Campus

   Students' student identification (ID) cards or mobile apps are used in the current attendance checking system. The process of verifying attendance may be more convenient for both students and teachers if it is handled automatically by an attendance- checking system as opposed to manually. The said article suggests an

   IAAS is a face recognition-based device for tracking attendance. Students' images are captured by a device (such as a smartphone or tablet PC) and processed by a facial recognition system. This system verfies who is present in the class and notifies an attendance tracking system via email of their presence.

3. Automatic attendance management system using face detection

   In this approach, a cameIrCa CisIfDixTe-d2in02t3heCcolnafsesrreonocme ,Pwrohciecehdwiniglsl take pictures, identify any features, identify them in the database, and then record attendance. Parents are informed of a student's absence if their attendance is noted as missing. There are several ways to compare features. One of the techniques is the Eigenface. A collection of Eigenvectors called Eigenfaces is applied to face recognition issues in computer vision. The manual method of managing attendance, which requires a lot of time and is challenging to keep, will be replaced by automatic attendance management.

4. Real-Time Eye Monitoring System Using CNN for Drowsiness and Attentiveness System

   Eye movements or a sense of drowsiness can have a variety of negative impacts on a person and even trigger disastrous events. Eye Monitoring Device for Attentiveness and Slumber A detection system is created to track a person's level of alertness and sleepiness. Many motorists experience sleepiness while operating automobiles, trucks, and auto rickshaws.

5. Smart Attendance Monitoring System (SAMS): A Face Recognition-Based Attendance System for Classroom Environment

   Students' frequent attendance in class is a key component of performance evaluation and quality control in the current educational system. Most organizations currently use insecure and time-consuming traditional methods like calling names or signing documents. The automatic attendance management system is discussed in this piece for convenience or data accuracy. The system is created by combining widely available parts to create a portable device for managing the attendance of the pupils using face recognition technology.

6. Vision-Face Recognition Attendance Monitoring System for Surveillance using Deep Learning Technology and Computer Vision

Artificial neural networks can now be trained over many billion images and used instantly to identify and recognise faces with a fair amount of ease and flexibility. This idea is used to create a real-time attendance monitoring system that can be prototyped and put into operation.

One of the main uses for this ground-breaking technique is face attendance using a single snap mode on smartphones for university classes. It can also be used for real-time facial recognition surveillance of lab spaces and workplaces, which can serve as the first line of defense against intruders gaining access

III.METHODOLOGY

Before using a picture, face recognition works by entering it into a database. After being processed, the picture becomes an analogue and is stored in the database. Face analysis is a method

that maps out facial characteristics like the size of the forehead, the distance between the brows, the length of the nose, and the size of the lips. The system records and stores these characteristics. The camera recognises the picture after the database has preserved it. It must be used properly for it to work effectively. The algorithm must be able to recognise the people being studied. It should only be applied to people who are aware that they are in the directory beforehand. People who are not listed in the database shouldn't use the system.

Our system runs using Python where the camera opens up in a GUI (Graphical User Interface). For testing purposes, we use a webcam where we test whether it is able to detect and recognize faces. After recognising the faces it will be marked into a CSV file and then a webpage will be used to display the attendance of the individual students provided they log in. We are using Python since its gaining more popularity with each day passing by and is also one of the most powerful languages

in the market. GUI is only being used for displaying the webcam and the contents that it is capturing.

Reasons for choosing Python:

1. Readable and Maintainable Code.

2. Compatible with major platforms and systems.

3. Powerful and extensive standard library.

4. Modern language and OOP.

5. Versatility, reliability, efficiency, and speed.

Python

Python is a widely used, high-performance, high-performance language. Developed by Guido van Rossum in 1991 and developed by the Python Software Foundation. Its syntax enables programmers to convey their ideas in a few lines of code and its design places a strong emphasis on the readability of the code. Python is a programming language that makes it possible to complete tasks more quickly and incorporate systems more successfully. Python has two main iterations: Python 2 and Python 3.

Below are some of the features of Python that we are using in our project:

1. Face Detection

2. Face Recognition

3. Image Enhancement

4. Writing to a file

5. Drowsiness Detection

6. Model Training

7. Attendance Marking

8. Displaying the attendance

   Before going into detail about the specifics, the image thats being used to detect and recognize the faces is given below.

   ICCIDT – 2023 Conference Proceedings

   Fig -1: Classroom Image

   Above is the image on which we are going to run our program. Initially, we should know which students have enrolled for the class and to store that information we create a database where a photo is taken of the

   student and it is saved in the dataset folder along with the name. While doing so, we must make sure that we are including all the students/individuals in the class, if not done then the program wont recognize the entity even if his/her face is clearly visible.

   Below is the dataset that was registered in our database folder so as to define the students list in the class. The image was taken in such a way that their face was clearly visible while taking the photo.

   Fig -2: Classroom Dataset

   1. Importing Image from Google Drive

      Before starting the whole process of recognizing and marking the attendance of the students, there needs to be a group photo that the tutor needs to take. For this purpose, there is a simple front end that is integrated with Google Drive API, through which teachers can easily upload images by just choosing and uploading the required image (In our case- class photo). This front end was made with the help of google script through which we can add Google Drive API and also add a html file for serving the purpose of uploading class group photos. After

      writing the script we can deploy the same onto the web and this can be accessed by anyone (in our case- tutors).

      After uploading the photo to google drive, the next task is to download the image from the drive link and is done with the help of python libraries. We set the desired path, since everything from this point of time will be based on this group photo.

   2. Cropping faces from the group photo

      Once the group photo has been downloaded to the system, the system with the help of a haar cascade classifier detects faces and makes a bounding box around it. After making the bounding box around the face, the face is then extracted to a different folder. It uses haarcascade_frontalface_default.xml for detecting faces efficiently. This is a pre-trained model that can efficiently detect a face in an image. The bounding box is initially on the edges of the face, to have more space we extend the bounding box in such a way that some of the extra part of the image is also included so as a face with a background can be the output.

      Fig -3: Cropped Faces

      As we can see here, the faces are clearly visible and have been accurately cropped for each individual. The point here to note is that some of the background has also been included so as to simplify the task of face recognition.

   3. Face Recognition

      p>The third task is to recognize the face which has been detected and put it in a bounding box by the face detection phase. Face recognition is carried out by various technologies, in our software it is carried out by OpenCV which uses various other algorithms and is mostly done by FaceNet Algorithm[3] which is based on CNN (Convolution Neural Networks). The program is run in such a way that the cropped faces are given as input one by one to the software and the program checks if the faces match with the dataset or not. The system is also able to detect individuals where the face is only partially visible or is of low quality (i.e. pixelated).

      In our sample classroom snapshot, there are 3 rows and almost all faces are fairly visible, but the third row has a noticeable drop in image quality, even then the system is able to recognize the individual with accuracy. Below is a sample snapshot of the

      same where a students fIaCcCeIhDaTs -lo2w023imCaognefeqreunaclietyPraoncdeeidsinngost completely facing the camera, even then the system is able to detect and recognize the student. In case of low lighting we can use algorithms that are more effective by using image enhancement techniques[4].

      Fig -3(a): Similar face recognition test

      Fig -3(b): Different face recognition test

      In the above figure we used the encodings that the facenet algorithm uses and tried to compare the face in the database with the face that was cropped. The results were accurate enough. There is a face encoding vector that comes as output for each face and then it compares it with the face that has been cropped. If the distance is less than the threshold value then it recognizes the face.

   4. Marking Attendance

      Once it recognizes the faces from the cropped faces, it then compares the recognized face names with the one in the dataset and creates two CSV files namely attendance.csv and absentees.csv, in which it places the face that has a match with the database into attendance.csv, and the face which does not have any match will be put into absentees.csv.

      D.Drowsiness Detection

      Drowsiness detection is done with the help of a pretrained model that uses shape_predictor_68_face_landmarks to detect and mark 68 landmarks on a human face. This is done so as to know some vital points on the human face that can help one to detect various features of the human face. Since our aim is to

      detect whether a person is drowsy or not, we are focusing on the landmark situated in the eyes and around it. If we get some landmark points in the eyes then by various means drowsiness detection can be done easily.

      Fig -4:Marking landmarks on the face

      In the above example we can see some dots placed on the face, and each dot has unique characteristics and can be called individually.

      Fig -5: Checking for drowsiness

      A person is checked for drowsiness or not by checking how much the human eye is open, we can set our own threshold to determine when the system should say that someone is drowsy.

      E.Model Training

      Our system is being tested on only a small group of people, but as our project has a positive scope to extend this to a full-fledged classroom, model training will prove to be useful. With the number of faces improving the system might sometimes get confused and produce an undesirable result (i.e. recognizing a face wrongly).Here instead of feeding the dataset with a single photo of an individual, we provide 10 to 15 images of an individual and place them under a folder which will be a sub- folder in our data set. Then we run the training program where the program makes sure that it takes and stores the feature information of an individual up to a such level that it can recognize the individual with higher accuracy even in a pixelated image.

      Model training is an optional feature that we want to use when the sample classroom starts to accommodate more students so as to increase the efficiency of the system (i.e. being able to detect and recognize all faces).

      ICCIDT – 2023 Conference Proceedings

      1. RESULTS AND ANALYSIS

         Successfully identified various methods to improve the efficiency of the current system and thus leading to a huge path for future scope.

         Below we have discussed how effective our system is with minor additions to the current systems. We have compared our system with the current single-face attendance system.

         Table -1: Comparison of our system with single face attendance system.

         Features	Single Face	Our System
         Time Consumption	High	Low
         Drowsiness detection	No	Yes
         Report Generation	No	Yes
         Efficient	High	Very High
         Accuracy	Very High	High
         Economic	No	Yes
         Installation	Complex	Easy

         Now we compare our system with the manual attendance system and observe how efficient it makes our current manual attendance system.

         Table -2: Comparison of manual attendance system vs our system

         Features	Manual Attendance	Our System
         Time Consumption	Very High	Low
         Efficient	Intermediate	High
         Economic	Yes	No
         Accuracy	Intermediate	High
         Feasibility	Very Low	Very High
         Workload	High	Very Low

         As we can see from the above two tables, our system is more efficient than attendance in almost every aspect and highly efficient in terms of time consumption and efficiency when compared with a single face recognition system. Although the accuracy is more in single-face recognition when the overall aspect is looked upon our system has the upper hand

      2. CONCLUSION

The Smart Attendance System can be used in a full-fledged classroom where many individuals are present and any additional feature can just be easily integrated into the system at the users convenience. The camera that is being used can also act as a surveillance camera as well as equipment to take attendance thus making it economical and open to a wide variety of uses which also includes using the same camera for conducting online meetings.

With the correct integration and implementation of software and hardware, this system can be scaled up for highly populated classes.

References

1. Dassanayake, Seneth & Wanniarachchi, Ashen. (2021). Challenges of Manual Attendance System Towards Student Motivation.

   ICCIDT – 2023 Conference Proceedings

2. Khan, Rafflesia. (2020). Human Distraction Detection from Real Time Video Data using Artificial Intelligence and Deep Learning Techniques.

3. F. Cahyono, W. Wirawan and R. Fuad Rachmadi, "Face Recognition System using Facenet Algorithm for Employee Presence," 2020 4th International Conference on Vocational Education and Training (ICOVET), Malang, Indonesia, 2020, pp. 57-62, doi: 10.1109/ICOVET50258.2020.9229888.

4. Kamenetsky, S. Y. Yiu and M. Hole, "Image Enhancement for Face Recognition in Adverse Environments," 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, ACT, Australia, 2018, pp. 1-6, doi: 10.1109/DICTA.2018.8615793.