Development of A Simple and Low Cost Unified Smart Video Surveillance System for Monitoring Hostel Student Attendance

Development of A Simple and Low Cost Unified Smart Video Surveillance System for Monitoring Hostel Student Attendance

Dhanshri Mali1, Prof. R. T. Patil2

1,2 Department of Electronics and Telecommunication, Rajarambapu Institute of Technology,

Rajaramnagar, 415409, India

Dr. Nagaraj Dharwadkar3

3 Department of Computer Science Engineering, Rajarambapu Institute of Technology, Rajaramnagar, 415409, India

Abstract:- Personal security and surveillance is a hot topic in the field of research and computer vision. The CCTV is used for surveillance and security purpose. The crime detection is possible with surveillance but if the face recognition is integrated with surveillance the performance of existing surveillance and security system can be enhanced. The smart surveillance system using face recognition can be used for a verity of application such as attendance system for students and employees at private and government offices, research labs, ATM etc. In this paper, one application of smart surveillance called hostel attendance system is demonstrated.

Keywords: CCTV, Face detection, face recognition, face landmark detection, feature extraction.

1. INTRODUCTION

   Now a days maintaining attendance is the challenging and important task. Every organization has its own attendance system like pen and paper or register based attendance, some have biometric attendance system like RFID, fingerprint scanner system. These systems are time-consuming and require students to stand in a waiting queue. Every system has to follow two processes as enrollment and verification. The enrollment is a one-time process in which unique features of a person are stored in the database. During verification, the input features are compared with stored in the database. These features can be eye, iris, palm, gait, fingerprint or face.

   The RFID based attendance system requires users to carry RFID tags. This works by scanning the tag and the user needs to place cards on a reader to mark their attendance. This may raise the problem of fraud because an unauthorized person can come with authorized RFID and mark attendance. Also, this system is very time-consuming.

   The fingerprint scanner is another technique used for security and surveillance. This system stores fingerprints of a user as training data initially. In the field, it compares input scanning data with stored once in the database. But sometimes registered person may not get access due to injury or skin allergy. The sensors may be fooled by mold as well. Some optical sensors may not able to distinguish between pictures of the finger and the finger itself.

   Some of the application s needs to handle large users. In that case, users need to wait in a standing queue to get the access. The RFID and fingerprint scanning systems are stationary, can handle one user at a time.

   The RFID and fingerprint scanning security systems are not reliable due to its drawbacks. The reliable system which gives better accuracy and efficiency, A hybrid system which would be reliable for surveillance and security needs to be developed. Lots of biometric features like eye, iris, palm, gait, and face can be used to develop the surveillance system. In this way, the performance of surveillance systems can be improved.

   The face recognition is a most secure and efficient method of the all existing identification methods. The face recognition can be used in banks, hospitals, schools, college, research labs etc. The difficulty of taking attendance can be avoided by making use of the face recognition. This system provides an easy and secure way of taking attendance. This system continuously marks attendance by detecting face and simultaneously compares face with the database.

   The rest of the paper is organized as Section II describes existing theories and practices section III contains a proposed methodology for face recognition system and attendance report generation. Section IV describes hardware used for the system design. Section V shows the result. The conclusion is deliberated in section VI.

2. RELATED WORK

   Automated attendance marking is a challenging task in the field of computer vision and image processing. Many of the researchers have developed and implemented automated attendance marking systems [1], [2-6], [11-12]. The video- based attendance system is described in [13-14], [16]. The kinetics based system is described in [9]. The related work contains a study of existing systems and papers mentioned above.

   Author's study reveals the face recognition in three steps such as face detection, face recognition, and auto-tagging. The Haar cascade algorithm was used for face detection. The training of this system is done using the Ada Boost learning algorithm. This is a machine learning algorithm. An Android application had been developed using Android Studio. The camera is used to capture the image. The face detected by Haar cascade algorithm, the face recognition was done by comparing with data stored in the database. When the match found the result is tag name of a person. [7]

   In this paper, the recent advancement in the face recognition system was presented. The biometrics has been played an important role in the field of security, surveillance, crime prevention and many more. The various biometric parameters were such as iris, palm, eyes, nose etc were used for identification. The proposed

   System works in real time. According to an author, the face recognition was done in three stages such as face detection, feature extraction, and classification. The Viola-Jones algorithm was used for face detection. The PCA algorithm was used for feature extraction. The Ada boost algorithm has used for classification. The time required for recognition was 100ms. Raspberry Pi II minicomputer was used for the implementation of this system. [8]

   Author's study reveals that the face recognition system plays an important role in a presence system. In order to do this author had used a dual vision camera for implementation of an attendance system. This system will remove the drawbacks of falsification, proxy attendance and so on. This system produces dual images from both lenses and merged to produce a face database. The advantage of using dual vision is to avoid the use of face photo. This method is expensive as it requires two lenses [9].

   Ji-wan Park et al described the face recognition using sensor fusion. The face recognition is considered an emerging technology in today's world to deal with the security and

3. METHOD

   The proposed system is as shown in fig1. It uses OpenCV. OpenCV is freely available computer vision library for image processing. It focuses mainly on image processing, object detection, and video or image capture etc. It comes with basic programs and utilities. It has Python, C++, and Java etc interfaces and supports Windows, ios, Android and Mac.

   The face_recognizer class library of OpenCV recognizes faces from the python code or command line. The face_recognizer is built using dlibs state-of-art of face recognition.

   Fig. 1: Proposed System

   The proposed methodology of the system is as shown in fig 1. It illustrates the basic steps for database creation of the users. After that, the face recognition is done for further attendance marking. The results are obtained after comparing with the predefined database if the user is recognized it is considered as present otherwise considered as absent. If the user is not recognized then it is considered as a stranger entry. The .csv file will be created which contains whole day attendance record.

   privacy of individuals. In this paper, when a student is close to the seat guide robot, the robot will guide a student about his/her seat through the monitor. The seat guide robot guides student by searching student data such as name, id etc. This system is bulky as it requires robot. [10]

   Neha Patil et.al described the IOT based security system using the raspberry pi. The face recognition is used surveillance to fulfill the safety aspects. According to author surveillance is essential from small houses to big businesses. In this paper, the author proposed a system that uses a webcam to capture images based on motion or gestures. Whenever there is motion webcam capture the image and send the image to the cloud server. Also, that image is sent via mail as an alert message by the use of a Wi-Fi module. [11]

   According to related work and existing theories, computer vision plays an important role in the field of and surveillance security. The face recognition is the best technique for the authentication and verification of a person. There are some factors affecting the face recognition such as illuminations, pose variation, lightning and environmental changes, the distance between camera and person, camera quality and many more. In this paper, the effect of pose variation and a certain level of lighting condition changes are minimized by using face detection and recognition algorithms. This is the ear of IOT, the recognition results are updated onto the cloud.

   1. Training phase.

      The working of the enrollment phase is divided into three stages such as image acquisition, detection, and feature extraction and database creation.

      1. Image acquisition: The image is captured using the webcam.

      2. Face Detection: The Viola-Jones algorithm is used for face detection because of its high detection rate and ability to run in a real-time environment. Viola_Jones algorithm is effective for frontal faces and it can work up to 45 degrees around both vertical and horizontal axis. The face detection using Viola-Jones is done in four stages.

      3. Facial Feature extraction: The facial landmark detection is used for the facial feature extraction. It uses facial features like eyebrows, mouth, jawline, eyes, and nose. Total 128 face embeddings are estimated using these five facial features. These face encodings are stored in an array as known encodings.

      4. Database creation: The 128 face embeddings of captured image and name, class, and PRN are stored onto the database.

   2. Testing phase.

      The verification does in five stages such as follows:

      1. Video recording: The video is captured by using a webcam.

      2. Face detection: During this phase, every single frame is extracted from the video. The face detection algorithm is applied. Every frame is resized to ¼ size and converted from BRG to RGB form.

      3. Feature extraction: During this phase, the facial landmark detection algorithm is applied and facial landmarks are estimated and stored in an array as unknown encodings.

      4. Face recognition: During this phase, estimated face encodings and known face encodings are compared. If the match is found for the known encoding result is displayed. The result contains the name of the respective user.

      5. Data storage (Database on the Cloud): After the face recognition respective image and their associated logs are stored in the database. If both faces match then a respective image is stored in the database with logs. Otherwise, image and logs are stored in the database as stranger entry.

4. EXPERIMENTAL SETUP

   Fig 2 shows the hardware setup of proposed system. The Raspberry Pi is main component of the system. Power is given using battery charger. The processing is done by the credit card size board. The images and videos are captured using the webcam. The display unit (monitor) is connected through the HDMI connector.

   1. Raspberry Pi:

      The Raspberry Pi is a low cost, small size on-chip computer. Raspberry Pi has four onboard USB ports, onboard SD card, Wi-Fi, Ethernet, and HDMI support. It also has great community support. It is capable of doing anything that normal computer can do. Also, it is the ability to interact with the real world. Nowadays it is widely used for IOT projects.

   2. Webcam:

      Logitech webcam is used for this system. It has image capture and video capture capacity of 360P, 480P, 720P. When an image is captured, it can be saved anywhere through the USB connector or Wi- Fi. In this system captured image and videos are stored on Pi board through the USB port.

   3. Monitor:

   The display units like computer monitor and TV screen can be used to view the images and videos.

   Fig. 2: Experimental setup

5. RESULT AND DISCUSSION

   Results obtained by the proposed system are as shown below. Fig 3: shows the database creation of users. One image per user is captured by the webcam and stored for further processing.

   Fig 3: .CSV file

   Fig 4: shows the detection and recognition results for 10 samples with 1 meter distance between camera and user.

   Fig 5: shows the recognition results for 50 samples with 1.2 meter distance between camera and user.

   Fig 6: shows the recognition results for 150 sampleswith 1 meter distance between camera and user.

   15

   10

   5

   0

   face detection face recognition

   Fig 4: user name vs. face detection and recognition assuming 10 samples.

   Table 1: Data samples 1

   User Name	face detection	face recognition
   Chaitanya_Devale	10	9
   Dhanshri_Mali	10	8
   Pratiksha_Patil	10	9
   Manali_Mahajan	10	10
   Rushikesh_Shirkar	10	8

   Table 2: Face_detection

   		Frequency	Percent	Valid Percent	Cumulative Percent
   Valid	10	5	50.0	100.0	100.0
   Missi ng	System	5	50.0
   Total		10	100.0

   Table 3: Face_recognition

   		Frequency	Percent	Valid Percent	Cumulative Percent
   Valid	8	2	20.0	40.0	40.0
   	9	2	20.0	40.0	80.0
   	10	1	10.0	20.0	100.0
   	Total	5	50.0	100.0
   Missing	System	5	50.0
   Total		10	100.0

   Table 3 describes the result analysis of the system. According to table 1 the frequency and percentage values for five samples are as shown in above table.

   Rushikesh Manali_M Pratiksha_ Dhanshri_ Chaitanya

   151

   150

   149

   148

   147

   146

   145

   151

   150

   149

   148

   147

   146

   145

   Face

   detection

   Face recognition

   Face

   detection

   Face recognition

   Face recognition

   Face detection

   45 50

   Fig 5: user name vs. face detection and recognition assuming 50 samples.

   p>User Name	Face detection	Face recognition
   Chaitanya_Devale	48	48
   Dhanshri_Mali	50	49
   Pratiksha_Patil	47	47
   Manali_Mahajan	49	49
   Rushikesh_shirkar	49	48

   User Name	Face detection	Face recognition
   Chaitanya_Devale	48	48
   Dhanshri_Mali	50	49
   Pratiksha_Patil	47	47
   Manali_Mahajan	49	49
   Rushikesh_shirkar	49	48

   Table 4: Data sample 2

   Total 50 samples per user are used for the face detection and recognition assuming 1.2 meter distance between user and camera. Out of 50 samples Chaitanya_Devale got detected 48 time and recognized 48 times.

   The accuracy varies with variation in distance between the user and surveillance cameras.

   Table 5: Case Processing Summary

   		Face_detec tion	Face_recog nition
   Series or Sequence Length		10	10
   Number of Missing Values in the Plot	User-Missing	0	0
   	System-Missing	5	5

   Fig 6: user name vs. face detection and recognition assuming 50 samples.

   User Name	Face detection	Face recognition
   Chaitanya_Devale	150	148
   Dhanshri_Mali	150	149
   Pratiksha_Patil	150	147
   Manali_Mahajan	150	149
   Rushikesh_shirkar	150	149

   User Name	Face detection	Face recognition
   Chaitanya_Devale	150	148
   Dhanshri_Mali	150	149
   Pratiksha_Patil	150	147
   Manali_Mahajan	150	149
   Rushikesh_shirkar	150	149

   Table 3: Data sample 3

   Fig 7: user name vs. face detection and recognition assuming 50 samples.

   First, the system needs to register image for training i.e. need to create a training database. The images are captured using a webcam and stored into training dataset. The Logitech webcam is used to capture images.

   Now starts recording the video using a webcam. The face is detected as shown below.

   Fig 8: single face detection.

   A single face is detected from live video feed as shown in Fig 1. For face detection 68 point face landmark detection is used. It estimates 68 points on face for both training and testing images.

   Fig 9: multiple face detection.

   The multiple face detection can be done using this system as shown in Fig4. Three faces are detected at a time from input live video feed.

   Fig 10: face recognition result.

   The face recognition results are as shown in Fig 5. Two faces are recognized at a time. Initially recognition accuracy is less than 50%. By changing tolerance value and code optimization the accuracy is increased up to 90%.

   Fig 11: Face recognition.

   In Fig 5 Fig 6 spectacles are interchanged still system is able recognize faces. Unlike traditional face recognition system, compares pixels to pixel values and does recognize face even if frame is changed.

   Fig 12: face recognition.

   Fig 13: face recognition

   Fig 14: face recognition with illumination

   This system can recognize face in various light conditions but with less accuracy.

   Fig 15: face recognition with illumination

   After recognition, date and time corresponding to recognized faces are stored into database and attendance is marked and faces not recognized are stored into strangers table into database. The .csv file is created. After each transaction .CSV file is updated. The attendance record is maintained without students intervention. This as well system gives accurate result, marks effortless attendance also saves students time. This system avoids unauthorized entry into hostel.

   Fig 16: pose variation results.

   As shown in above fig, this system can recognize faces even if there is pose variation.

   Fig. 17 Training images

   Fig. 18: Attendance report on cloud

6. CONCLUSION

A face is an important identity of human beings. The proposed system uses a webcam to capture images for training and record video for testing. The system uses one image per user for training purpose. It reduces memory consumption as well as the processing power of Raspberry Pi. The system can work with little environmental change and pose variation as explained in the result section. The system gives 100% and about 95 % of face detection and faces recognition accuracy assuming 1-meter distance between camera and user. After recognition .csv file is and stored onto the cloud to maintain attendance records of the day. The system performance can be improved by integrating more facial features and by applying algorithms. Better camera resolution will further increase the performance of proposed system.

REFERENCES

1. K.Senthamil Selvi et al. (2014), FACE RECOGNITION BASED ATTENDANCE MARKING SYSTEM, International Journal of Computer Science and Mobile Computing, Vol.3 Issue.2, February-2014, pp. 337-342

2. Anusaya Tantak et al. (2017), Face Recognition for E- Attendance for Student and Staff, IOSR

   Journal of Computer Engineering (IOSR-JCE), Volume 19, Issue 2,

   Ver. IV (Mar.-Apr. 2017), pp. 89-94

3. Venkata Kalyan Polamarasetty et al.(2018), Attendance System based on Face Recognition,

   International Research Journal of Engineering and Technology (IRJET), Volume: 05 Issue: 04 | Apr-2018, pp. 4606-4610.

4. Mathana Gopala Krishnan et al. (2015), Implementation of Automated Attendance System using Face Recognition, International Journal of Scientific & Engineering Research,

   Volume 6, Issue 3, March-2015, pp. 30-33

5. Khem Puthea et al. (2017), A Review Paper on Attendance Marking System based on Face Recognition, 5th International Conference on Image Processing, Communication, Engineering and Technology (IPCET-17) Oct. 11-12, 2017 Bali (Indonesia) pp. 20-24

6. Ketan N. Mahajan and Nagaraj V. Dharwadkar (2016)classroom attendance system using surveillance camera, proceedings of international conference o onference on computing, omputing, communication and energy systems (iccces- ystems (iccces-16) In Association with IET, UK & Sponsored by TEQIP- In Association with IET, UK & Sponsored by TEQIP-II 29th -30th,

   Jan. 2016

7. Aayushi Bansal (2018) A factors affecting face recognition IJAMTES volume 8, issue II, pp. 669-672.

8. Viji, A. Pavithra (2017), A real time face recognition using Raspberry Pi II IJIRSET volume 3, issue 6,330-335.

9. Edy Winarno et al. (2017) ,Anti-Cheating Presence System Based on 3WPCADual Vision Face Recognition Proceedings of IEEE Conference of Electrical Engineering, Computer Science and Informatics IEEE 2017.

10. Ji-Gwan Park et al. (2013), Multiple Human Tracking using Multiple Kiects for an Attendance Check System of a Smart Class 10th IEEE conference of Ubiquitous Robots and Ambient Intelligence (URAI), October 31-November 2, 2013.

11. Neha patil et al. (2017), IOT Based Smart Surveillance Security System using Raspberry Pi, IEEE conference on signal processing April6-8 2017, pp. 0344-0348.

12. Nusrat Mubin Ara et al. (2017), Convolutional Neural Network Approach for Vision Based Student Recognition System 20th IEEE conference of Computer and Information Technology (ICCIT), 22-24 December, 2017.

13. Jalendu Dhamija et.al (2017)An advancement towards efficient face recognition using live video feed, International Conference on Computational Intelligence and Networks ,pp. 53-56.

14. Aditya Gupta et.al(2014) Video based real time face recognition system International Journal of Computer Science And Technology volume 5, pp. 132-136

15. Prof. Shivarudraiah B et.al (2018), IOT BASED SURVEILLANCE SYSTEM AND HOME AUTOMATION International Research Journal of Engineering and Technology (IRJET) Volume: 05, Issue: 05, pp 2031-2036.

16. Suchitra Khoje et al. (2017), Implementation of IOT based Smart Video Surveillance System, International conference on Advances in Intelligent Systems and Computing, pp 771-780.

17. Rumana Anjum, Vijaya Kamble, Student Tracking and Attendance Monitoring System Using RFID International Journal of Scientific Research in Computer Science, Engineering and Information Technology© 2017 IJSRCSEIT | Volume 2 | Issue 2| ISSN : 2456-3307

18. Devendra Gondole1, Prof. P. A. Salunkhe2, Face Recognition Based On Local Binary Pattern, ©2018 IJSRSET

    | Volume 4 | Issue 1 |Print ISSN: 2395-1990 | Online ISSN : 2394-4099 Themed Section : Engineering and Technology.

19. Luca Anzalone, Data Driven Investoor, Training alternative Dlib shape predicator models using python, 2018, [online]. Available: https://medium.com/datadriveninvestor/training- alternative-dlib-shape – predictor-models-using-python- d1d8f8bd9f5c

20. Betaface,[online].Available: https://www.betafaceapi.com/demo.html#page-header

21. Wikipedia, the free encylopedia, Viola-Jones object detection framework, 2015, [online]. Available:https://en.wikipedia.org/wiki/Viola%E2%80%93Jones

    _object_detection_framework. .

22. Mali, Dhanshri and RTP, Ramesh (March 21, 2019)., Real-Time Smart Surveillance System Using Raspberry Pi In Proceedings of International Conference on Sustainable Computing in Science, Technology and Management (SUSCOM-2019), February 26 – 28, 2019, Amity University Rajasthan, Jaipur, India.