Social Distancing and Face Mask Detection with Reporting

Social Distancing and Face Mask Detection with Reporting

Hemanth Kumar H J1, Monish V2, Nithin B3, Siddalingappa B4, Subhashini B N5

1, 2, 3,4Department of Electronics and Communication, Vivekananda Institute of Technology, Bangalore, India

5Associate Professor, Department of Electronics and Communication, Vivekananda Institute of Technology, Bangalore, India

Abstract: The global pandemic of COVID-19 has severely impacted the world and has now infected more than eight million people worldwide. Wearing face masks and following safe social distancing are two of the enhanced safety protocols need to be followed in public places in order to prevent the spread of the virus. To create safe environment that contributes to public safety, we propose an efficient computer vision-based approach focused on the real-time automated monitoring of people to detect both safe social distancing and face masks in public places by implementing the model on computer to monitor activity and detect violations through camera(webcam/Phone) or a Network connected device. After detection of breach, computer sends an alert message including real-time Photograph of the individual violating Covid Protocols to a Telegram Private Group made for this very cause. In this proposed system modern deep learning algorithm have been mixed with geometric techniques for building a robust model which covers three aspects such as detection, and validation and report. Thus, the proposed system favors the society by saving time and curbing the spread of disease and reducing the overall case count. It can be implemented effectively in current situation where strict public safety measures is highly needed in situations such as public gatherings, shopping malls, cinemas, grocery store etc. Automated inspection reduces manpower to inspect the public and also can be used in any place.

Keyword: Deep Learning, Computer Vision, Convolutional Neural Networks (CNNs), Transfer Learning, public Safety, OpenCV, Tkinter-GUI, Telegram, COVID-19

1. INTRODUCTION

   Since COVID-19 has become a pandemic, the entire world is finding ideas and method to stop the spread of it. As the ground rule to stop the spread is to maintain social distance and wearing a mask while going out. The novel COVID or COVID-19 began spreading during 2019 December at first from China, the city guest Wuhan. In china, the infection is started from the creatures and spread generally as a pandemic circumstance everywhere on the world. Corona Virus infection communicates to other however coordinates actual contact with the influenced patients and through air. The infection straight forwardly hit the lung cells through respiratory arrangement of the patients and permits it to recreate the infection and makes an extreme irresistible problem in an exceptionally limited ability to focus. Our main motive, Face mask detection with Social Distancing is the task of identifying an already detected object as that person wear mask or not and

   they are walking with maintaining Social Distance to each other.

   1. Face Mask Detection

      Face Mask Detection Platform utilizes Artificial Network to perceive if a person does/doesnt wear a mask. The application can be associated with any current or new IP cameras to identify individuals with/without a mask. The face mask detection process starts from the image acquisition using a camera. The imaging device and the modules are developed using TensorFlow and Open-CV programming detect the face and evaluate all faces point that to detect is that face wear mask or not. If person wear mask they will be in safe zone it display as green rectangle-box with safe alert where if person dont wear mask then it will be shown in red rectangle-box and with the message of Alert as well.

   2. Social Distancing Detection

      Social Distancing detection will detect that two or more persons in a single frame are walking with maintain social distancing with at least 0.80 meter of range with each other. By using Euclidean Distance method, it will detect that persons maintaining or following social distancing under guidance of WHO. If they maintain then it will shown in Green Rectangular-box with safe alert message where if they dont following social distancing then system will display an alert message with red rectangular-box.

   3. GUI(Graphical User Interface)

      A friendly Graphical User Interface(GUI) is necessary for all individuals irrespective of their technical knowledge to operate, hence we tried to create a GUI based system using tkinter which consists of primary six blocks containing images for its respective use case. The blocks include options such as Facemask, Social Distancing, Facemask & Social Distancing, Manual inform, Automatic inform and Quit.

   4. Automated Reporting Option

      The automated or manually captured image and information needs to be reported at the receiving end, hence we used telegram as a medium for real time data receiving source. The script is including in the code which sends image with a message to notify the violation process. The user can either opt to perform a manual capture or automate the process using Automatic inform option.

2. FEASIBILITY STUDY

   It is not feasible to manually track the implementation of this policy. Technology holds the key here. We introduce a Deep Learning based system that can detect instances where face masks are not used properly. Our system consists of a dual stage Convolutional Neural Network (CNN) architecture capable of detecting masked and unmasked faces and can be integrated with pre-installed CCTV cameras.

   This will help track safety violations, promote the use of face masks, and ensure a safe working environment. The study is carried out by analyzing the necessary technologies involved in the previous research and formulating the efficient model that help the people in real-time.

   Application development using keras and Tensorflow is most widely used in current trends, henceforth we are suggesting the Python based image processing and machine learning technique to achieve the robust structure the feasibility of considering a diversity of cases using the proposed method with a high level of confidence and accuracy in social distancing monitoring and risk assessment with the help of Deep Learning and Computer Vision.

3. TECHNOLOGY USED

   • Tensorflow.

     Tensorflow is used to classify images. It creates an image classifier using a keras.Sequential model, and loads data using preprocessing.image_dataset_from_directory.

     1. Efficiently loading a dataset off disk.

     2. Identifying over fitting and applying techniques to mitigate it, including data augmentation and Dropout.

   • Scikit Learn.

     scikit-image is an image processing Python package that works with NumPy arrays which is a collection of algorithms for image processing.

   • Computer Vision (OPEN CV)

     OpenCV is a free open source library used in real-time image processing. Its used to process images, videos, and even live streams.

   • Tkinter GUI.

     The tkinter package (Tk interface) is the standard Python interface to the Tk GUI toolkit.

   • Telegram Bot.

     Bots are third-party applications that run inside Telegram. Users can interact with bots by sending them messages, commands and inline requests. You control your bots using HTTPS requests to our Bot API.

     3.1 Additional Methods And Techniques

   • Pre trained Yolov3 is used to detect the people./p>

   • Pre trained SSD is used to detect faces.

   • Trained MobileNetV2 is used as face mask classifier.

   • Euclidean Distance is used to calculate social distancing violations.

4. LIBRARIES USED.

   • face-recognition

   • imutils

   • Keras

   • matplotlib

   • opencv-pythonpandas

   • scikit-learn

   • tensorflow

   • tensorflow-gpu

   • scipy

   • pillow

   • requests

   • ppy

5. STRUCTURE

   1. Euclidean Distance

      In Both.py the modules tensorflow, Keras, numpy and SciPy is used to compute distance between the two person using Euclidean Distance.

      The Euclidean distance measure is generalized by the Minkowski distance metric shown in fig (5.a)

      Fig (5.a)

      Where r is parameter. The following are the three most examples of Minkowski distances.

      • r = 1. City block (L1 norm) distance. A common example is Hamming distance, which is the number of bits that are different between two objects that have only binary attribute, i.e., between two binary vector

      • r = 2. Euclidean distance (L2 norm)

        r =

        Compute the Euclidean distance for one dimension. The distance between two points in one dimension is simply the absolute value of the difference between their coordinates. Mathematically, this is shown as |p1 – q1| where p1 is the first coordinate of the first point and q1 is the first coordinate of the second point. We use the absolute value of this difference since distance is normally considered to have only a non-negative value.

        Take two points P and Q in two dimensional Euclidean space. We will describe P with the coordinates (p1,p2) and Q with the coordinates (q1,q2). Now construct a line segment with the endpoints of P and Q. This line segment will form the hypotenuse of a right triangle. Extending the results obtained in Step 1, we note that the lengths of the legs of this triangle are given by |p1 – q1| and |p2 – q2|. The distance between the two points will then be given as the length of the hypotenuse.

        Use the Pythagorean Theorem to determine the length of the hypotenuse in Step 2. This theorem states that c^2 = a^2 + b^2 where c is the length of a right triangle's hypotenuse and a,b are the lengths of the other two legs. This gives us c = (a^2 + b^2)^(1/2) = ((p1 – q1)^2 + (p2 – q2)^2)^(1/2). The distance between 2 points P = (p1,p2) and Q = (q1,q2) in two dimensional space is therefore ((p1

        – q1)^2 + (p2 – q2)^2)^(1/2).

        Extend the results of Step 3 to three dimensional space. The distance between points P = (p1, p2, p3) and Q = (q1,q2,q3) can then be given as ((p1-q1)^2 + (p2-q2)^2 + (p3-q3)^2)^(1/2).

        Generalize the solution in Step 4 for the distance between two points P = (p1, p2, …, pn) and Q = (q1,q2, …, qn) in n dimensions. This general solution can be given as ((p1- q1)^2 + (p2-q2)^2 + … + (pn-qn)^2)^(1/2).

   2. Face Mask Detection.

   In face mask.py we make use of YOLO-v3 model, one of the most frequently used versions of the YOLO family, which comprises the state-of-the-art object detection system for the real-time scenario and it is amazingly accurate and fast.

   Fig (5.b) YOLOv3 network architecture

   The YOLO-v3 algorithm uses a Darknet-53 as a backbone for feature detection. The full YOLO-v3 consists of 106 layers, including convolutional, residual, and up-sampling layers. The architecture of YOLO-v3 network is shown in fig (5.b). The YOLO algorithm uses only one forward propagation pass through to detect objects on a single input image. YOLO-v3 is better in detecting small objects thanks

   to the up-sampled layers which can preserve the fine- grained features of the small objects.

   We employ YOLOv3 to take its advantages for high accuracy and real-time processing.

   Fig (5.c) the flowchart of the proposed algorithm

6. DEMONSTRATION.

   In order to demonstrate the proposed system in a real application, we built a whole system for real-time face mask detection, Social distancing and Reporting (Automated and Manual) with tkinter GUI is installed at the entrance or gates of colleges, building or factories. The system consists of a computer and camera.

   Fig (6.a) the system of Detection

7. SCREEN OF THE RESULTS

1. GUI

   Fig (7.1) GUI of the Project

2. Face Mask

   Fig (7.2) Face Mask Detection

3. Social Distancing

   Fig (7.3) Social Distance Detection tested at Grocery store.

4. Face Mask and Social Distancing

   Fig (7.4) both face mask and social distancing tested at Grocery store

5. Telegram message.

   Fig (7.a) Message reported by Telegram Bot

1. CONCLUSION.

   Corporate giants from various verticals are turning to AI and ML, leveraging technology at the service of humanity amid the pandemic. Digital product development companies are launching mask detection API services that enable developers to build a face mask detection system quickly to serve the people amid the crisis.

   We proposed an approach that uses computer vision and MobileNet V2, SSD architecture to help maintain a secure environment and ensure individuals protection by automatically monitoring public places to avoid the spread of the COVID-19virus and assist the police by minimizing their physical surveillance work in containment zones and public areas where surveillance is required by means of camera feeds. The technology assures reliable and real-time face detection of users wearing masks. Besides, the system is easy to deploy into any existing system of a business while keeping the safety of the people involved. This proposed system will operate in an efficient manner in the current situation when the lockdown is eased and helps to track public places easily in an automated manner. We have addressed in depth the tracking of social distancing and the identification off face masks that help to ensure human health. So, the Social Distancing and face mask detection system is going to be the leading digital solution for most industries, especially retail, healthcare, temples, shopping complex, metro stations, airports and corporate sectors.

2. FUTURE SCOPE.

Objects are usually recognized by their unique features. There are many features in a human face, which can be recognized between a face and many other objects. It locates faces by extracting structural features like eyes, nose, mouth etc. And then uses them to detect a face. Typically, some sort of statistical classifier qualified

then helpful to separate between facial and non-facial regions

Human faces have particular textures which can be used to differentiate between a face and other objects. Moreover, the edge of feature scan helps to detect the objects from the face.

The above-mentioned use cases are only some of the many features that were incorporated as part of this solution. We assume there are several other cases of usage that can be included in this solution to offer a more detailed sense of safety. Several of the currently under development features are listed below in brief:

• Coughing and Sneezing Detection: Chronic coughing and sneezing is one of the key symptoms of COVID-19 infection as per WHO guidelines and also one of the major routes of disease spread to non-infected public. Deep learning-based approach can be proved handy here to detect & limit the disease spread by enhancing our proposed solution with body gesture analysis to understand if an individualist coughing and sneezing in public places while breaching facial mask and social distancing guidelines and based on outcome enforcement agencies can be alerted.

• Temperature Screening: Elevated body temperature is another key symptom of COVID-19 infection, at present scenario thermal screening is done using andheld contactless IR thermometers where health worker need to come in close proximity with the person need to be screened which makes the health workers vulnerable to get infected and also its practically impossible to capture temperature for each and every person in public places, the proposed use-case can be equipped with thermal cameras based screening to analyze body temperature of the peoples in public places that can add another helping hand to enforcement agencies to tackle the pandemic effectively.

REFERENCES

1. S. Wang Chen, Horby Peter W, Hayden Frederick G, Gao George F. A novel coronavirus epidemic of global concern for health. It's the Lancet.2020

2. Matrajt L,Leung T. Evaluating the efficacy of social distancing strategies to postpone or flatten the curve of coronavirus disease.

   Emerg Infect Dis, man. 2020

3. Chen, S., Zhang, C., Dong, M., Le, J., R., M., 2017b. Using rankingcnn for age estimates, in: IEEE Conference on Computer Vision and Pattern Recognition

4. Sandler, Mark, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen, "Mobilenetv2: Inverted residues and linear bottlenecks," IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510-4520. 2018.

5. Lin, Tsung-Yi, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie, "Type Pyramid Networks for Object Detection," IEEE Conference Proceedings on Computer Vision and Pattern Recognition,

6. Masita, K. L., Hasan, A. N., and Satyakama, P., 2018. Pedestrian identification by mean of R-CNN object detector. IEEE Latin American Conference on Computational Intelligence (Nov. 2018)

7. R. Girshick, Fast R-CNN, in Proc. IEEE International Confrence Computer Vision, Dec. 2015

8. P. Viola and M. Jones, "Fast object detection using an enhanced cascade of simple features," Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

9. X. Zhu and D. Ramanan. Face detection, pose estimation,and landmark position in the wild. In IEEE

10. S. S. Mohamed, N. M. Tahir and R. Adnan, "Background modelling and background subtraction efficiency for object detection," 2010 6th International Colloquium on Signal Processing & its Applications, Mallaca City, 2010

11. Joseph Redmon, Ali Farhadi, YOLOv3: An Incremental Improvement,2018, arXiv:1804.02767.

12. Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping- YangChen, Jun-Wei Hsieh, and I-Hau Yeh, CSPNet: A new backbone thatcan enhance learning capability of cnn, IEEE Conference on ComputerVision and Pattern Recognition Workshop (CVPR Workshop), 2020.

13. Hui, YOLOv3, Medium, 2020. [Online].

14. Zhanchao Huang, Jianlin Wang, DC-SPP-YOLO: Dense Connectionand Spatial Pyramid Pooling Based YOLO for Object Detection, 2019,arXiv:1903.08589.

15. Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, JunsukChoe and Youngjoon Yoo, CutMix: Regularization Strategy to TrainStrong Classifiers with Localizable Features, 2019, arXiv:1905.04899

16. Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. DropBlock: A reg- ularization method for convolutional networks, Advances in NeuralInformation Processing Systems (NIPS), 2018, pages 10727 10737.

17. Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, andDongwei Ren, Distance-IoU Loss: Faster and better learning forbounding box regression, Proceedings of the AAAI Conference onArtificial Intelligence (AAAI), 2020.

18. Zhuliang Yao, Yue Cao, Shuxin Zheng, Gao Huang, and Stephen Lin,Cross-iteration batch normalization, 2020, arXiv:2002.05712.

19. Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So KweonCBAM: Convolutional block attention module, European Conferenceon Computer Vision (ECCV),2018, pages 319.

20. Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia, Path aggrega-tion network for instance segmentation, IEEE Conference on ComputerVision and Pattern Recognition (CVPR), 2018, pages 87598768.

21. Coronavirus Disease 2019 (COVID-19), Centers for Disease Con- trol and Prevention, 2020, Available: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting- sick/social-distancing.htm