Face Mask Detection using Learning Based Approaches

Face Mask Detection using Learning Based Approaches

Shreya Dutta*, Rupashri Barik*, Nilay Mukherjee*, Swagata Mukherjee*, Darakshan Noor*, Smriti Dey*, Sutapa Das*

Department of Information Technology JIS College of Engineering

Kalyani, Nadia, India

Abstract COVID-19 is an unparalleled crisis leading to a huge number of casualties and security problems. To reduce coronavirus spread, people should wear masks and should take safety measures to protect themselves. It is very much needed to restrict people to join any gathering without face mask. Hence an automated face mask detection system is required while a person is availing any public transport or attending any public meeting or event. This paper aims to make an automated system that will detect whether a person is wearing a face mask or not. The proposed method captures the image of the person and the input image to be compared to a specific classifier. Next a feature extraction layer is used to extract the feature map and the model is trained in a way that it will produce the final code ward. In the next step quantization has been applied to get the histogram. Here, the layer is used to similarity measure. Hence, the output is loaded in the system with different classifiers and the output also can be stored for future reference. As a surveillance task performer, it can also detect a face along with a mask in motion. The proposed system has various applications in public transports, in any organization, institutions, etc. People need to be detected with a face mask to help the society as well.

KeywordsDeep Learning, Feature Extraction, Face Mask Detection, Haar-like Features, Image Complexity Prediction Function

However, wearing the mask face causes the problems like fraudsters and thieves to take advantage of the mask, steal, and commit crimes without being identified. Community access control and face authentication have become very difficult tasks when most of the face is hidden by a mask. Hence, detecting the face mask and recognizing the person behind the face mask is very important.

Face mask detection [1] is essential to avoid spread of coronavirus. Recognition of face is a popular and significant technology in recent years. Previously, detecting the Face of a person and facial expression was done using different machine learning techniques like support vector machines but the work towards identifying whether a person is wearing a face mask or not is a challenging task and there is no large dataset available for it. In Deep Learning, when the raw data is given as input to the layers it learns features on its own that is machine automatically identifies important features or patterns from the training data. With the advancement of technology and time more reliable methods for human recognition with a face mask can be implemented.

1. INTRODUCTION

   Corona Virus which is commonly known as COVID-19 is an infectious disease that causes illness in the respiratory system in humans. The term Covid 19 is sort of an acronym, derived from Novel Corona Virus Disease 2019. Corona Virus has affected our day-to-day life and has affected millions of people, who are either sick or are being killed due to the spread of this disease. The COVID-19 virus can be spread through contact and contaminated surfaces. There is so much essential equipment needed to fight against the Coronavirus. One of such essentials is Face Mask. Firstly, a face mask was not mandatory for everyone, but scientists and doctors have recommended everyone to wear a face mask as the day progresses. So, to detect whether a person is wearing Face Mask or not is an essential process to implement in the society currently that can be used for various applications like at the airport, hospitals, offices, schools, etc. This system can be of great importance at airports to detect travellers whether they are wearing a mask or not and at schools to ensure students are wearing a face mask for their safety.

   A primary focus of the researchers during the ongoing coronavirus pandemic is to come up with suggestions to handle this problem through rapid and efficient solutions.

   Fig. Process of Mask Detection

   Therefore, the application of image extraction, image analysis, and using the deep learning method can lead to more accurate detection of the mask and can help to control the problem. In this context, the author suggested the application of feature extraction so that it can learn from the examples and store it for future reference to give more intelligent output with an accuracy of 90%. This method introduced a new approach using Haar Like Feature with 95% accuracy so that when the person covers the face with a piece of cloths for substitution of mask it gives the output as with mask, and when it faces is covered with paper or hand it will give output as no mask.

   A deep CNN (convolutional neural network) model is proposed in this model, this model is open-source and accessible in public, the detection accuracy is high, it proves that the model makes good predictions in learning and improved screening through learning.

   Before coronavirus, some people put on masks to protect themselves from air pollution. Now protection against

   coronavirus is a mandatory countermeasure, according to the WHO. Indeed, wearing a mask is an effective method of blocking 80% of all respiratory infections. Also, the WHO recommends practicing physical distancing to mitigate the spread of the virus. All over the world, governments are struggling against this type of virus. Many organizations enforce face mask rules for personal protection. Checking manually if individuals entering an organization are wearing masks is cumbersome and possibly conflicting. In this context, the authors proposed a deep learning-based model to prevent human-to-human transmissions of the SARS-CoV-2 and detect faces with or without a mask. Two different datasets with over different images are used to train and test the model.

2. BACKGROUND STUDY

   1. Digital Image Processing

      Digital image processing methods perform some operations on an image, in order to get an enhanced image or to extract some useful information from it. It is a type of signal processing in which input is an image and output may be an image or characteristics/features associated with that image. It helps to process, analyse the image, compare it with the preserved data and produce the output. Computer vision [2] is a field that includes processing, analysing, and understanding images in general high dimensional data from the real world to produce numerical and symbolic information or it is a technology of science and machines that see it obtain information from images. the transformation of visual images into descriptions of the world that make sense to the processes and can take appropriate action. This image understanding can be seen as the information from image data using models constructed with the aid of learning theory.

   2. Machine Learning

      Machine learning [3] is a method of teaching prediction based on some data. It is a branch of artificial intelligence, which numerically improves data in the algorithm and the performance of the system is improved. It helps to tag people and objects, mage recognition, fraud system, detect skin cancer, and self-driving cars. Machine learning is using data to answer questions. Using data means training and answering questions is prediction and this prediction can be made using different approaches of machine learning. In supervised learning, there are several data points or samples described using predictive variables r features, and the target variable our data is represented in table structure. In this, the machine is taught using labelled data. Game supervised learning is building a model its the ability to predict the target variable. So, the input and the output are labelled. Unsupervised learning is a machine learning task of uncovering hidden patterns from unlabelled data. The machine is trained on unlabelled data without any guidance. the machine is forced to build a compact internal representation of its world and then generate imaginative content from it. An unsupervised Learning algorithm might decide that the data lives in two different clusters. In reinforcement learning, machine or software agents interact with an environment reinforcement learning agents that can automatically figure out how to optimize their behaviour

      given a system of reward and punishments reinforcement learning draws inspiration from behavioural psychology. Reinforcement learning differs from supervised learning by not needing labelled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected. Instead, the focus is on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge). The agent interacts with its environment by producing actions and discovering errors or rewards.

   3. Deep Learning

      Deep learning [3] is a powerful set of techniques for learning using neural networks. Neural networks are a beautiful biologically-inspired programming paradigm that enables a computer to learn from data. These are learning algorithms. Deep learning is a subset of machine learning. And a subset of artificial intelligence which is achieved through algorithm, deep learning is achieved through the structure of the human brain. The machine is taught some features and through the neural network, it is learned by the machine. Neural networks process images or data through a neural network with a hidden layer existing between them, they are transformed through channels from layer to layer. It is applied in customer support, medical care, self-driving cars, etc. Deep learning- based model is helpful for detecting masks over faces in public place. The high accuracy of model is also due to highly balanced face mask centric dataset achieved through Random over-sampling with data augmentation over original dataset. In Deep Learning, when the raw data is given as input to the layers it learns features on its own that is machine automatically identifies important features or patterns from the training data. Machine learning is good for classifying images. The machine is taught some features and through the neural network, it is learned by the machine. Neural networks process images or data through a neural network with a hidden layer existing between them; they are transformed through channels from layer to layer. It is applied in customer support, medical care, self-driving cars, etc.

   4. Feature Extraction

      Feature extraction [9, 10] is a part of the dimensionality reduction process, in which, an initial set of the raw data is divided and reduced to more manageable groups. The most important characteristic of these large data sets is that they have a large number of variables. It transforms the raw data into a processed feature and also it preserves the information of the original data, it gives us new features which are the combination of the original feature, in the next which would be compared with the raw input data. Picture Complexity is the details of an image or the variety classified in the image. The quality or condition of the image is being difficult to understand or lacking in simplicity, the complexity of an image is defined on the set of target classes and train the data models according to that.

   5. Haar Like Feature

   Haar Like feature [4] is used in the feature-based algorithm some sort of statistical classifier qualified then helpful to separate between facial and non-facial region. They owe their name to their intuitive similarity with Haar wavelets and were

   used in the first Real-time face detector. The key advantage of a Haar-like feature [13] over most other features is its calculation speed. A Haar-like feature [14] takes the neighboring regions to detect the image pixels and in the last sum up the pixel intensities in each region and calculate the difference between these sums. This difference is then used to categorize the subsection of the images; this property can be used to detect the face mask in human faces. An example of a Haar-like feature is the detection of a mask in two rectangular blocks, one is above the eyes, that block is without a mask, and another is below the eyes which block is with a mask.

   A. Das et al. [5] had illustrated the learning and performance task of the model. Using basic ML tools and simplified techniques the method has achieved reasonably high accuracy. It can be used for a variety of applications. Wearing a mask may be obligatory in the near future, considering the Covid-19 crisis. Many public service providers will ask the customers to wear masks correctly to avail of their services. The deployed model will contribute immensely to the public health care system. In future it can be extended to detect if a person is wearing the mask properly or not. The model can be further improved to detect if the mask is virus prone or not

   i.e. the type of the mask is surgical, N95 or not.

   S. Sethi et al. [6] described a deep learning-based approach for detecting masks over faces in public places to curtail the community spread of Coronavirus is presented. The proposed technique efficiently handles occlusions in dense situations by making use of an ensemble of single and two-stage detectors at the pre-processing level. The ensemble approach not only helps in achieving high accuracy but also improves detection speed considerably. Furthermore, the application of transfer learning on pre-trained models with extensive experimentation over an unbiased dataset resulted in a highly robust and low- cost system. The identity detection of faces, violating the mask norms further, increases the utility of the system for public benefits.

3. PROPOSED METHODOLOGY

   In the proposed work, face mask detection is achieved through deep neural networks because of their better performance than other classification algorithms. But training a deep neural network is expensive because it is a time- consuming task and requires high computational power. Deep- learning-based transfer learning is applied here to train the network faster and cost-effectively. Transfer learning allows transferring the trained knowledge of the neural network in terms of parametric weights to the new model. It boosts the new models performance even when it is trained on a small dataset.

   Here an image is taken as input, then two different methods were included D-fast and D-slow which are used as single and two stage detectors respectively. Then an input image is passed as input to compare the complexity, if it is already there then it has soft complexity, otherwise hard complexity, and then input the data as trained data.

   Algorithm of Image Complexity Predictor Function

   Step 1: Take an image as input.

   Step 2: Take a function D-fast which is used as a single-stage detector.

   Step 3: Take a function D-slow which is used as a two-stage detector.

   Step 4: Take an input variable C to check the image complexity.

   Step 5: Take an input variable R to store the output.

   Step 6: If image complexity is soft, store the two-stage detection of the image into variable R.

   Step 7: Else image complexity is not soft, store the single- stage detection of the image into variable R.

   Step 8: Store the set of region proposals into variable R. Next, an input is taken as input and then an example is taken to make as an example and then input images are taken to compare. Some feature vectors are collected then it is a dataset is taken as input and st the different classifiers. Then input image is taken and it has to compare to a specific classifier.

   Algorithm of Haar-like feature

   Step 1: Take an image as input.

   Step 2: Use the feature extraction layer to extract the feature map.

   Step 3: Take an input variable Vi to refer to the number of feature vectors extracted from the ith image.

   Step 4: Take an input variable F to refer to the set of all the feature vectors.

   Step 5: Take an input variable Vk to refer to the number of neurons.

   Step 6: The neuron centers are learned to get the final code word.

   Step 7: Quantization is applied to extract the histogram with a pre-defined number of bins. Each bin is referred to as a code word.

   Step 8: The layer is used as a similarity measure. Step 9: Take an input variable X as a feature vector.

   Step 10: Take an input variable Ci as a center of the neuron. Step 11: The output is collected which contains a histogram of the global quantized feature vector.

   Step 12: The data set is loaded into the system with different classifiers.

   Step 13: Load the face mask classifier model to detect faces in the image or video stream.

   Step 14: Apply the classifier in different inputs.

   Step 15: Classify the right input as With Mask and the wrong input as Without Mask.

   Following the above discussed approaches, the automated face mask detection has been implemented using some basic Machine Learning packages like TensorFlow, Keras, OpenCV and Scikit-Learn. The image dataset has been prepared as well for a person. Here, the deep learning model contains one input layer, a greater number of hidden layers, and one output layer.

4. RESULTS & DISCUSSION

   In this section image of a person has been captured and has been analyzed to detect whether the person has worn a mask or not to prevent the spread of novel coronavirus.

   Fig 2: No Mask Detected

   Fig 3: Not a proper Mask

   In fig. 2, the person has not worn the mask so it results that no mask is there. Fig. 3 illustrates that the person has covered his face with a paper and as a result it shows red mark after identifying the face region and it also indicates that no mask has been detected.

   Fig 4: No Mask Detected

   Fig 5: Mask Detected

   The person has covered his face by his hand in fig. 4. Hence it also produces no mask detected. Whereas in fig. 5, it can be seen that the person is appearing with a proper mask and as a result it has been marked green and the result shows that the mask has been detected.

   Fig 6: Mask Detected

   Here the face has been covered with some clothes which is a substitute of a face mask and it can be considered as a valid approach. Hence, it produces that the mask has been detected.

   TABLE I: COMPARISON AMONG DIFFERENT METHODS

   Algorithm	Detection Rate (%)	False Rate (%)	Missed Rate (%)	Detection Time(s)
   CNN	91.85	16.60	9.42	1.67
   Deep Learning	94.46	12.82	7.37	0.950
   Proposed Algorithm	98.88	8.89	2.20	0.260

5. CONCLUSION

One novel approach of face mask detection has been proposed in this paper. Here, Haar-like features are digital image features used in object recognition. Human faces might be occluded by other objects such as a facial mask in real- world scenarios. This makes the face recognition process a very challenging task. The deep learning-based method and quantization-based technique achieve a high recognition performance. The main challenge in face mask detection is that to recognize the person while wearing mask. This embedded vision-based application can be used in any working environment such as public place, station, corporate environment, streets, shopping malls, and examination centers, where accuracy and precision are highly desired to serve the purpose. It can be used in smart city innovation, and it would boost the development process in many developing countries. The method attains accuracy up to 95.77% to detect the face mask. This proposed method may help the society by preventing the spread of novel coronavirus.

REFERENCES

[1] Toshanlal Meenpal, Ashutosh Balakrishnan, Amit Verma Facial Mask Detection Using SemanticSegmentation.

[2] https://Computer-Vision-Talks.Com/2011-07-13- Comparison-Of-

The-Opencv-Feature-DetectionAlgorithms/

[3] Mohamed Loey, Gunasekaran Manogaran, Mohamed Hamed N. Taha ,Nour Eldeen M. Khalifa A Hybrid Deep Transfer Learning Model With Machine Learning Methods For Face Mask Detection In The Era Of The Covid-19 Pandemic.

[4] Chinmay Patil, Haar Cascades On Face Mask Detection, published in International Journal of Scientific Research & Engineering Trends, Volume 7, Issue 2, 2021.

[5] Arjya Das; Mohammad Wasif Ansari; Rohini Basak, Covid-19 Face Mask Detection Using TensorFlow, Keras, and OpenCV, Published in 2020 IEEE 17th India Council International Conference (INDICON)

[6] Shilpa Sethi, Mamta Kathuria , Trilok Kaushik, Face mask detection using deep learning: An approach to reduce risk of Coronavirus spread, Elsevier Public Health Emergency Collection.

[7] M. Rahman, S. Mahmud, J. Kim, Md. M. Manik, Md. M. Islam (2020) published a document aimed at developing a system for determining whether a person uses a mask or not and informing the relevant authority in the smart city network.

[8] H. Anandakumar and K. Umamaheswari, A bio-inspired swarm intelligence technique for social aware cognitive radio handovers, Computers & Electrical Engineering, vol. 71, pp. 925937, Oct. 2018. doi:10.1016/j.compeleceng.2017.09.016

[9] D. Bhamare & P. Suryawanshi (2019) summarize and analyze many well-known techniques in the multiple steps of the pattern recognition system and identify the analysis and application topics

[10] R. Arulmurugan and H. Anandakumar, Early Detection of Lung Cancer Using Wavelet Feature Descriptor and Feed Forward Back Propagation Neural Networks Classifier, Lecture Notes in Computational Vision and Biomechanics, pp. 103110, 2018. doi:10.1007/978-3-319-71767-8_9.

[11] Savvides, M., Heo, J., Abiantun, R., Xie, C., and Kumar, B. V. (2006, May). Class dependent kernel discrete cosine transform features for enhanced holistic face recognition in FRGC-II. IEEE (Vol. 2, pp. II-II).

[12] Vu, N. S., and Caplier, A. (2010, September). Face recognition with patterns of oriented edge magnitudes. (pp. 313-326). Springer, Berlin, Heidelberg.

[13] Abusham, E. E., Jin, A. T., Kiong, W. E., and Debashis, G. (2008).

Face recognition based on nonlinear feature approach.

[14] G. K. Jakir Hussain and S. Natarajan, Subpixel Based Image Scaling Using Continuous Domain Analysis, International Journal of Computational Research and Development, Volume 1, Issue 2, Page Number 92-96, 2016.

[15] G. K. Jakir Hussain, K. S. Tamilselvan, R. Tamilanban and M. Vinoth Saravanan, A Comprehensive Review of Multimodal Medical Image Fusion Techniques, Innovations in Information and

[16] https://github.com/chandrikadeb7/Face-Mask-Detection