Automated Face Detection System based on Deep Metric Learning

Automated Face Detection System based on Deep Metric Learning

Suvam Chowdhury

Department of Information Technology JIS College of Engineering

Kalyani, India

Dibendu Kundu

Department of Information Technology JIS College of Engineering

Kalyani, India

Saswati Pal

Department of Information Technology JIS College of Engineering

Kalyani, India

Rupashri Barik

Department of Information Technology JIS College of Engineering

Kalyani, India

Sagnik Bhadra

Department of Information Technology JIS College of Engineering

Kalyani, India

Biswajit Mondal

Debdeep Sarkar

Department of Information Technology JIS College of Engineering

Kalyani, India

Department of Information Technology JIS College of Engineering

Kalyani, India

AbstractIn this pandemic situation it is very much important to proceed for any kind of contactless authentication system. To serve this purpose, face detection can help to authorize a person to make his or her presence in an organization. In this paper, we have proposed an algorithm based on deep metric learning for detecting the face of a person. The system will capture the image of a person standing in front of camera and will extract the features from it and will match these features with existing database for validating the person. To get accuracy in face detection several numbers of images of a person has been trained. Hence an automated system will help to identify a person and will mark the attendance of the employee as well as will approve an authorize person to get entry in a restricted area.

KeywordsImage Processing, Face Detection, Feature Extraction, Haar Training, Deep Metric Learning

1. INTRODUCTION

   For any organization, it is always required to have a proper attendance tracking system to record the attendance of an employee. It should also record the in-time and out-time of the employee. In this digital era, it is very evident to use an automated system for recording attendance of the employee and it can be done easily using some electronic devices. Some of the organizations are using Bio-metric devices for this purpose, mostly finger prints are used as bio-metric authentication. But in this pandemic situation it is better to use a touchless attendance monitoring system. Face detection can be useful for attendance monitoring system. It may also help to approve entry of individual after the system identifies the employees face after matching with database.

   Manual system exerts unwanted pressure on the person to be correct and keep track of all the information. Unintentional mistakes are created again which are again needed to be corrected. Manual systems put pressure on people to be correct in all details of their work at all times, the problem being that people arent perfect, however much each of us wishes we were. With manual systems the level of service is dependent on individuals and this puts a requirement on management to run training continuously for staff to keep them motivated and to ensure they are following the correct procedures. It can be all to easy to accidentally switch details and end up with inconsistency in data entry or in hand written orders. This has the effect of not only causing problems with customer service but also making information unable be used for reporting or finding trends with data discovery. Reporting and checking that data is robust can be timely and expensive. This is often an area where significant money can be saved by automation. It takes more effort and physical space to keep track of paper documents, to find information and to keep details secure. When mistakes are made or changes or corrections are needed, often a manual transaction must be completely redone rather than just updated. With manual or partially automated systems information often has to be written down and copied or entered more than once. Systemisation can reduce the amount of duplication of data entry.

   Another impact of manual systems is on Customer service. Customer queries can be difficult to respond to as information is stored in different places and may even require that you find the right person before being able to respond.

   Manual systems of time and attendance management can lead to many problems such as:

   1. Inconsistent data entry and generate errors

   2. It is a time consuming and costly process to produce reports.

   3. Duplication of data entry is possible

   4. Lack of security

   5. Reduction in sharing information and customer services.

   6. System is dependent on good individuals.

   7. Large ongoing staff training cost.

   There is always the obvious balance of cost vs benefit, as an organization grows and adds people and functions the benefits of systems increases. Often when an organization is growing its the time that they need to put in a new system, however this is also the time that there is significant pressure on existing resources and its the hardest time to make a change.

2. BACKGROUND STUDY

   A face detection [1] system matches human faces with a digital image stored in system database and validate whether the detected face is authorized or not. Smart aattendance [2] marking system is an application based on these techniques. Here faces are recognized using deep learning. It makes use of face embedding during which each face is converted into a vector and this system is named deep metric learning. Maintaining the Integrity of the Specifications

   Image Processing is the use of a digital computer to process digital images using algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analogue image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions digital image processing may be modelled in the form of multidimensional systems.

   Face detection is a technology getting used during a sort of application that identifies human faces in digital images. Face detection also refers to the psychological process by which humans locate and attend to faces during a visual scene.

   In machine learning, pattern recognition, and image processing, feature extraction starts from an initial set of measured data and builds derived values (features) intended to be informative and non-redundant, facilitating subsequent learning and generalization steps, and in some cases leading to better human interpretations. Feature extraction is claimed to dimensionality reduction.

   Computer vision algorithms have contributed greatly to developing very efficient visual search workflows using Convolutional Neural Networks. Since data volumes are increased in recent times, visual perception models are ready to recognize the objects and generalize the image features at scale. However, during a challenging classification setting where the number of classes is large, there are several constraints one must address to style effective visual search workflows. The aim here is to calculate the sum of all the image pixels lying in the darker area of the haar feature and the sum of the pixels lying in the lighter area of the haar

   feature. And after that finding out the difference between them.

3. METHODOLOGY

   Here we discuss about the proposed algorithm for recognising faces using deep learning [10] [11]. Here we use face embedding [12] technique in which each face is convered into a vector and this technique is called deep metric learning. Deep Metric Learning uses Neural Networks to

   automatically learn discriminative features from the images and then compute the metric i.e. similarity and dissimilarity between images. For implementing this face detection system Python with Django framework has been used along with HTML. OpenCV [3] [4] [5] has been used to develop real- time computer applications. It is a cross-platform library. It mainly focuses in image processing, video capturing and object detection. The main objective of using NumPy is that it provides faster array up to 50x faster than Python list. Since, arrays are frequently used in image data, where resources and speed are required. The entire process has been further divided into three simple steps:

   1. Face Detection

      The very first task we perform is detecting faces [6] in the image or video stream. With face detection [7] we can get the information we need to perform. Now that we know the exact location/coordinates of the face, we extract this face for further processing ahead.

   2. Feature Extraction

      After completing the whole process of face detection [8], we come over Feature Extraction to reduce the dimensionality of the selected features. Now that we have already cropped the face out of the image, then we extract features from it. Face Embedding is being used here to extract the features out of the face, which will represent the original data for effective learning. Feature extraction [9] is useful here because we have dataset and need to reduce the number of resources without losing any important or relevant information. A neural network takes an image of the persons face as input and outputs a vector which represents the most important features of a face. In machine learning, this vector is called embedding and thus we call this vector as face embedding. Fig. 1 shows the vector for face embedding.

      Fig:1. Face Embedding

      While training the neural network, the network learns to output similar vectors for faces that look similar. For example, if I have multiple images of faces within different timespan, of course, some of the features of my face might change but not up to much extent. Therefore, in this case the vectors associated with the faces are similar or in short, they are very close in the vector space. Take a look at the above diagram for a rough idea.

   3. Comparing Faces:

   Now the face embeddings for each face in the data saved during a file, the subsequent step is to recognize a replacement image that's not in the considered data. Therefore, the initiative is to compute the face embedding for the image using an equivalent network we used above then compare this embedding with the remainder of the embeddings that have been obtained. It will recognize the face if the generated embedding is closer or almost like the other embedding. Hence, different images have been passed the embeddings for some images have not been saved but embeddings for some images have been saved. Thus, once comparing different embeddings with the prevailing ones is for recognizing the authenticated person.

   Fig:2. Face Classifications

   Here Fig. 2 represents classification based on extracted features from the face. Hence the faces of different persons have been identified and the features have been extracted from there and the identified faces have been classified in same group.

   Below Fig.3 represents the proposed algorithm for the automated face detection system. The application uses systems webcam to test, for sampling the application have created 36 samples of each person so that the application can acquire high accuracy. The aim of the test is to ascertain the accuracy in face detection employing a sort of samples. The application tends to urge higher accuracy for a more specific sort of image, where the location of the face is extremely detectable and ideal. Instead, the application uses various images where the face is not always located dead centre or

   ones where there is a group of people (more than 1 face in an image). The application also wants to find out if there is any bias to the algorithm when it comes to non-human faces.

   Fig:3. System Flow of the proposed algorithm

   The application is going to use the detect Multiscale module from OpenCV [13] [14]. What this does is create a rectangle with coordinates (x, y, w, h) round the face detected within the image. This contains code parameters that are the foremost important to think about. The worth indicates what proportion the image size is reduced at each image scale. A lower value uses a smaller step for downscaling. This allows the algorithm to detect the face. It has a worth of x.y, where x and y are arbitrary values to be set by the user. This parameter specifies what percentage neighbours each candidate rectangle should have. A higher value leads to less detections but it detects higher quality in a picture. User can use a worth of X that specifies a finite number.

   Here, the minimum object size, by default, is (30,30). The smaller the face within the image, it's best to regulate the minsize value lower. In order to run OpenCV [15] [16] [17], just to confirm the execution it from the prompt under the working directory where you dumped all of your files.

4. RESULTS & DISCUSSIONS

   Here dataset of human faces has been considered. Once the camera is open 36 no of shots have been captured and has been stored as the dataset of a person. Now for taking attendance the system camera will capture the image of the person and will match it with the existing dataset. If the

   captured image is matched then the attendance will be recorded.

   Fig:2. Sample Dataset of a Face

   Fig: 2 represents the dataset captured by systems camera to test, for sampling 36 samples of each person have been created so that we can acquire high accuracy [18].

   Fig: 3. Image of Face Detected

   Fig. 3 shows the result of the application and it represents that the face of the person has been detected so the attendance also can be recorded.

   Here, at first the face image has been captured for a person and then the features have been extracted from the face images. Next the face images have been trained and the dataset has been prepared. Hence, the face image has been captured for recording attendance and the captured image has been compared with stored dataset so the person will be recognized and the person will be authenticated and the required actions may be taken.

   The entire system is an approach for automated system for an attendance system. The quality of our sample images is quite low as are in grayscale, but the accuracy of the recognition model is best for extraction [19] and processing of said images. This technology can be adapted to various other fields which deal with facial identification [20].

5. CONCLUSION

Face recognition based automatic attendance management system has been developed for the purpose of reducing the errors that occur in the traditional (manual) attendance taking system. Our system will save the time, reduce the amount of work for the processors who has to manage the attendance and also it will replace the stationery material with electronic devices by which it will be in more secured manner, no one can access easily without the permission of attendance manager. It reduces the amount of human resource required for the purpose. Hence our system has been developed with the expected results. This study shows our facial detection and recognition model works in two separate modules, one being the main interface through which we register users and create sample datasets, and the other working in the background fetching and identifying using the webcam. The images are then stored in our database with other parameters as well. The model will then check with the images in our data set when a new session is created, marks the user present. In case of a situation where our modelcan detect a face but cannot identify the user, the output will be displayed as an error and ask the user to rebuild the dataset. The data is then mapped on to Calendar to visualize the data easily. We tested our model with 108 images retrieved from the webcam, the model reveals a recognition accuracy of 75%.

REFERENCES

1. Akash S Chachad*1, Soyeb M Khan*2, Rahul S Yadav*3, Dr. Pradip Mane* 4, 05/May-2021, Face Recognition (Hog Face Algorithm) For Automated Attendance System, International Research Journal of Modernization in Engineering Technology and Science, e-ISSN: 2582- 5208.

2. Petcharat Pattanasethanon, Charuay Savithi, 2012. Human Face Detection and Recognition using Web-Cam. Journal of Computer Science 8 (9): 1585-1593, 2012, ISSN 1549-3636.

3. Mamata S.Kalas, 2014. Real Time Face Detection and Tracking Using OpenCV. Proceedings of IRF International Conference, 5th & 6th February 2014, Pune India. ISBN: 978-93-82702-56-6.

4. Kruti Goyal, Kartikey Agarwal and Rishi Kumar, 2017. Face Detection and Tracking Using OpenCV. International Conference on Electronics, Communication and Aerospace Technology ICECA 2017.

5. Vincent Tabora, 2019. Face Detection Using OpenCV with Haar Cascade Classifiers. https://becominghuman.ai/face-detection-using- opencv-with-haar-cascade-classifiers-941dbb25177

6. Smart Attendance System using Computer Vision and Machine Learning Dipti Kumbhar#1 , Prof. Dr. Y. S. Angal*2 # Department of Electronics and Telecommunication, BSIOTR, Wagholi, Pune,

   India.

7. Face Recognition Based Student Attendance System with OpenCV CH. VINOD KUMAR1 , DR. K. RAJA KUMAR2 1 PG Scholar, Dept of CS& SE, Andhra University, Vishakhapatnam, AP, India. 2Assistant Professor, Dept of CS& SE, Andhra University, Vishakhapatnam, AP, India.

8. Automatic Attendance System Using Face Recognition. Ashish Choudhary1,Abhishek Tripathi2,Abhishek Bajaj3,Mudit Rathi4 and

   B.M Nandini5 1,2,3,4,5 Information Science and Engineering, The National Institue of Engineering,

9. Face Recognition based Attendance Management System using Machine Learning Anushka Waingankar1, Akash Upadhyay2, Ruchi Shap, Nevil Pooniwala4, Prashant Kasambe5.

10. https://www.superdatascience.com/blogs/opencv-face-recognition

11. https://towardsdatascience.com/face-recognition-how-lbph-works- 90ec258c3d6b

12. https://www.pyimagesearch.com/2018/09/24/opencv-face- recognition/

13. http://nxglabs.in/cloud/impact-biometric-attendance-system- educational-institutes.html

14. http://nxglabs.in/cloud/impact-biometric-attendance-system- educational-institutes.html

15. https://iopscience.iop.org/article/10.1088/1757- 899X/263/4/042095/pdf

16. http://www.ijsrp.org/research-paper-0218/ijsrp-p7433.pdf

17. https://www.theseus.fi/bitstream/handle/10024/132808/Delbiaggio_ Nicolas.pdf?sequence=1&isAllowed=y

18. Zhao, W., R. Chellappa, P.J. Phillips and A. Rosenfeld, 2002. Face recognition: a literature survey, Technical Report CAR-TR-948, Centre for Automation Research, University of Maryland.

19. Priyanka Dhoke and M.P. Parsai, A MATLAB Based Face Recognition Using PCA with Back Propagation Neural network

    International Journal of Innovative Research in computer and Communication Engineering, 8(2): 89-95.

20. Dewi Agushinta, R., Adang Suhendra, Sarifuddin Madenda and H.S. Suryadi, 2011. Face Component Extraction Using Segmentation Method on Face Recognition System, Journal of Emerging Trends in Computing and Information Sciences, pp: 67-72.