Social Distancing and Face Mask Detection From CCTV Camera

Social Distancing and Face Mask Detection From CCTV Camera

1Vishwesh M S, 2 Nikhil D K

1, 2 Student, Department of Computer Science,

JSS Academy of Technical Education, Bengaluru, Karnataka,

3 Savita S

3Assistant Professor,

Department of Computer Science Engineering, JSS Academy of Technical Education, Bengaluru,

Abstract: – The novel coronavirus also called Covid-19 had a huge effect on different sectors in many territories and imposed governments across the world to urge lockdowns to avert novel coronavirus transmission. This little step of wearing a face mask, following social distancing would save plenty of lives as the spread of the novel coronavirus could be mitigated. This theme consists of social distancing noticing and face mask detection for the events of disease like novel coronavirus can be solved by maintaining social distancing as well as wearing/putting on its face mask. This used to develop a Mask Detection using OpenCV, Keras/TensorFlow and also Deep Learning. This System can easily integrated/implemented to various embedded devices with limited computational capacity that uses MobileNetV2 architecture. System will detect face masks in photos/images and in real-time videos.

Keywords: – OpenCV, Keras/TensorFlow, deep learning, COVID-19, social distancing detection, face mask awareness, Convolutional Neural Network

1. INTRODUCTION

   The spread of novel coronavirus disease has created a most crucial health crisis across the world. In country like China the spread of severe acute respiratory syndrome coronavirus (COVID-19), a new severe infection respiratory disease was grown in Wuhan, China that infected 7,723 people and more than 500 deaths in China was reported. Till now there is no any update about any antiviral medicine or any vaccines against COVID-19. The virus has spread rapidly across the global world. At this moment, WHO (World Health Organization) recommended that people around the world should wear masks to prevent the risk of novel coronavirus transmission and also should maintained social distance of at least 3m between two people to prevent the spread of virus. Public service will be provided only if they are wearing mask and maintaining safe social distancing. This paper describes the approach to prevent the increase of coronavirus by monitoring in real-time if any person is maintaining social distance and wearing face masks in public places. During lockdown, the government of India has encouraged the people to come up with mask substitution due to lack of PPE available. Hence the masks are not standard class and it comes with non-identical colours, has various shapes and sizes.

2. RELATED WORK

   There is lots of open-discussion about deep learning approaches for person detection, but there is no any theory about distance measurement between two persons. This inspires us to come up with the new algorithm to break this

   problem. We reviewed below related information and existing methods.

   • Some app was created called Face Mask Alert App it was developed and process by Leeway Hertz software solution. This app sends an alert message to the users by enforcing them to wear masks.

   • In some company like Uber that they confirmed to CNN Business that its required to cover the face with mask for both drivers and passengers in any countries like India, USA the technology was developed to detect whether the drivers in Uber are followed by those rules.

   • Febri Eye, a camera that comes with additional analytics with both social distancing monitoring and face mask detection which generates an alarm or an alert in case of any violations. This technology was implemented by the Telangana state, India.

   • The widely used for face detection is the Viola Jones Face Detector this uses cascaded Haar features. Li et al another model for face detector it was a Multi-View Face Detector using features called surf.

   • The core model for person detection is MobileNetV2. This model is used because it required a framework to implement at a minimum speed of 3-4 frames to second. The advantage of this MobileNetV2 model is compute price as compared to a normal 2D convolution method.

   Limitations

   Many image processing techniques, mathematical formulae were proposed to compute a distance between two objects(persons) like Manhattan distance formula. Jong Bae proposed a medium to detect the distance of two persons applying smart phone based thermal rear camera that works in poor light environment. This method was as obtained 81% accuracy.

   In our proposed method uses a Euclidean Distance formulae for calculating distance between object here object means persons, this method was accuracy of 93%.

3. OBJECTIVES

   The main objective of our system is that will identify weather the person in frame is wearing the face mask and also maintaining distance between two people. The system provides two bounding boxes one for face and one person. If the person is wearing mask properly them our system will put the green bounding box across the face, else the bounding box will be in red colour. Similarly, the person in frame should maintain minimum of 6 feet distance then our system will put a green bounding box across the person, else bounding box

   will be in red colour. Status is shown in the screen, containing number of people in frame and number of people who are not maintaining distance of 6 feet, not in face mask will be displayed in status bar that bottom of the screen. Our proposed model can be integrated with the camera (CCTV camera) also in the web camera to impede the COVID-19 transmission.

4. METHODOLOGY

Figure: System Architecture

In the proposed system architecture, first our system takes input for evaluating the input can be in form of photo snap, live streaming videos and recorded videos. What our system does is it reads the input frame by frame if it reaches to EOF (End OF Frame) then the system will stop, else continues using algorithm called YOLOv3 (You Only Look Once) it gets the bounding boxes for face and person in the frame. Then using different algorithm for face mask detection and social distance detection can be computed, if the person in frame is wearing mask and maintaining distance then bounding box will be displayed in green colour if the person is not maintaining social distance and not wore face mask then bounding box will be in red colour.

FACE MASK DETECTION

Face mask detection is of two steps, one is Face Detection another is Face Mask Classifier. If any person hasnt wearing a mask then an alert get triggered in the monitoring screen. In this section it is described about the approachs taken to build the models.

1. Face Detection

   CNN based Deep Learning model is used for face recognition, we used this model because it has a great advantage compared to other models, some advantages are CNN model can detect the faces even in low resolution [420 X 420]. Also used MobileNetV2 model of accuracy of 94.2%, pixels of 90 X 90 is the base for face size that can be detected. The output gets displayed with the bounding box across the face, then this crop face is loaded into the face mask model.

2. Collecting Data

   To train face mask model, we used Face Detection model. Custom datasets are collection in our project consisting of real- time images of a person face with and without protection of face mask. The dataset that we have collected is 3835 photo snaps and then split into two classes, one is with_mask another is without_mask with_mask has 196 images and

   without_mask has 1916 images. Initially we have collected more than 4500 photo snaps in which handful are rejected/deleted for being blurred, not cleared. Here we can see data trimming was done. The dataset is separated into 80% of training data and 20% of testing data this is done using the help of sklearn lib. The images used for the training set is roughly around 3067 images and for testing data around 764 images.

   Sample data for with_mask

   Sample data for without_mask

3. Mask Detection

   OpenCVs face detector based on the Single Shot Multi-box Detector (SSD) framework which comes with MobileNetV2 architecture. To obtain a bounding box (x, y) coordinates for an object in this case object means mask, we need to apply the photo snap with object detection. Single Shot Multi-box Detector (SSDs) are originally developed by Google, they are between R-CNN and YOLOv3 methods of object detection. This are more straightforward algorithm and faster than RCNNs. The current model is combined with both MobileNetV2 architecture and Single Shot Detector (SSD) framework, so our model will have the quick, efficient deep learning-based method for mask detection.

4. Limitations

The proposed model does have some limitations. The application posses some difficulty in detecting the face from images when the face is half or partially visible to cameras, even in low lighting condition our model fails to detect the face.

SOCIAL DISTANCE DETECTION

1. Person Detection

   The Proposed model uses ResNet50 which is a subclass of convolution neural network and MobileNetV2 for the process of person detection using the framework of TensorFlow. The key feature of this model is that it is able to detect multiple classes of objects at the same time. On the downside, this model will require more computations for more accurate results. The GPU acceleration is enabled which helps in performing faster computation compared to previous models. Various set of features such as eyes, nose, mouth,

   arms, movement of body are extracted in order to get effective results. Histogram equalization is used to increase the contrast of input video.

2. Distance Computing

   The model used in this application has a very effectively trained MobileNetV2 for person detection in the video. The model will take the video frame as the input and output a list of coordinates in a bounding box in a rectangular shape across each and every person detected in the frame. The rectangular bounding box is represented as [ x-min, y-min, width, height]. Each and every person in the video frame will have a centroid for the resulting bounding box. By calculating the distance between two centroids, the model is going to calculate the distance between two people. The calculation of distance between two centroids is done using the Euclidean Distance formula. Euclidean distance is generally used to calculate the distance between any two real valued vectors. if the computed distance is less than 3 feet then the person is not maintaining social distance if the distance is 6 feet or greater than 6 feet then the person is maintaining a safe distance.

   Real-world coordinates i.e., P = (p1, p2, · · ·, pn), obtain the corresponding list of pedestrian distance D which is straightforward. The pedestrian i, the closest distance cam be calculated by dmin = min (di, j), j = i {1, 2, · · · n}. On the computed value of dmin, we further calculate the two metrics for distancing.

3. Limitations

   Illumination condition: Different quality of the camera and the lighting conditions directly affect the detection of the person. Uncontrolled Background: Difficult to detect the person and calculate the distance in an assorted environment.

   FUTURE SCOPE

   This method was developed with an efficient way for the people who are not wearing face mask and not maintaining social distance and notified to officials by email. As a future enhancement, we can predict/detect time at which it gets crowded and heat map can be plotted in an accurate way.

   CONCLUSION

   In this project we have used a recent techniques in the field of computer vision and also in the deep learning. A custom dataset can be created using Google/Bing Search API, Kaggle datasets and RMFD dataset. The proposed system will correctly detect the presence of face mask and person is in the safe the distance. The system is accurate, since we have used the MobileNetV2 architecture for detecting face mask and for distance computing we used Euclidean distance formula. Thus, it makes easier to deploy our model to embedded system like Raspberry Pi, Goole Coral etc. We believe that this approach will enlarge the safety of the individuals during the pandemic.

   BIBLIOGRAPHY

   1. https://keras.io/api/applications/mobilenet/#mobilenetv2-function

   2. https://arxiv.org/abs/1704.04861

   3. https://www.pyimagesearch.com/2019/06/03/fine-tuning-with- keras-and-deep-learning/

   4. Wang Chen, Horby Peter W, Hayden Frederick G, Gao George F. A novel coronavirus outbreak of global health concern. The Lancet. 2020;395(10223):470473. doi: 10.1016/S0140- 6736(20)30185-9.

   5. Matrajt L, Leung T. Evaluating the effectiveness of social distancing interventions to delay or flatten the epidemic curve of coronavirus disease. Emerg Infect Dis. 2020

   6. https://www.who.int/emergencies/diseases/novel-coronavirus- 2019/advice-for-public

   7. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting- sick/cloth-face-cover.html

   8. https://www.who.int/news-room/detail/03-03-2020-shortage-of- personal-protective-equipment-endangering-health-workers- worldwide