Pneumonia Detection Using Transfer Learning

Pneumonia Detection Using Transfer Learning

Maahi Patel1, Ayush Sojitra1, Zeel Patel1, Mohammed Husain Bohara2

1U.G Student, Dept. of Computer Science and Engineering

Devang Patel Institute of Advance Technology and Research (DEPSTAR), Charotar University of Science and Technology (CHARUSAT), Changa 388421, India

2Assistant Professor, Dept. of Computer Engineering

Devang Patel Institute of Advance Technology and Research(DEPSTAR), Charotar University of Science and Technology (CHARUSAT),Changa 388421, India

Abstract: Pneumonia is a life-threatening illness that affects the lungs of humans. This disease is either caused by viral infection or bacterial infection. This is a kind of disease which makes breathing very difficult and aching for a person. Pneumonia is a menacing ailment in respiratory, therefore it is necessary to detect pneumonia as early as possible. We present you with systematic model for the diagnosis of pneumonia which is trained on digital images of chest X-ray to accurately detect pneumonic lungs. This will help the medical practitioners a lot.

Keywords: Pneumonia, Chest X-ray Images, Computer Aided Diagnosis(CAD), ConvNet(CNN), Residual Networks (ResNet), Transfer Learning, VGG16, VGG19.

1. INTRODUCTION

   Pneumonia is fatal illness which causes a severe respiratory contamination in the lungs. As the lungs gets inflamed with pus & fluids which leads to breathing issues amongst people having pneumonia. Generally, bacterial pneumonia is more threatening and causes more severe symptoms. People with weak immunity system, elderly patients & children below age of 5 are more likely to be affected by this disease. Pneumonia becomes the reason for the death of around three lakh children in India and to our astonishment half of the pneumonia deaths are of children in India. Most of the time the disease is being ignored. Many lives can be saved if it is detected on time.

   Chest X-rays, CT scan of the lungs, MRI of the chest, ultrasound of the chest, radiograph of the chest, and lung biopsy are all tests used to diagnose pneumonia.[1] Even sometimes the viral pneumonia or bacterial images are wrongly classified by the radiologists, leading to inaccurate remedy which leads to worsening condition of the patients. There is a need for a system that can help the radiologists diagnosing pneumonia from X-ray images of chest.

   Currently many AI based solutions [2] are used to solve biomedical complications (breast cancer detection, brain tumor detection, etc). The use of ML and DL methods on chest X-rays is common since they are inexpensive and there is no lack of data for feeding distinct ML models. These models give accuracy similar and even much more than a particular radiologist in the prediction of the disease. CNNs performs very well while working with images and hence, it is widely accepted among the researchers [3]. Transfer learning models like, VGG16, VGG19 and ResNet50 are few of the very successful Image Net dataset models having pre-trained weights. We have used these models and trained them to categories X-ray images of chest into normal or pneumonia detection taking very less duration, thus can be used for faster and earlier pneumonia detection.

   The paper is divided into 5 sections as follows: 1st gives brief introduction of the subject, clearly states our purpose for taking this project, addresses its importance and relevance, what motivated us to take up this research work and states our objective. While, Section 2 gives in-depth background knowledge about machine learning concepts we will use for our research. Section 3 explains about methods which we have use for our work, dataset used for training and testing different models, explaining model architectures and flowchart. Section 4 showcases output we have achieved using various CNN models [4] (VGG16, VGG19 and ResNet) and compares their performance by comparing accuracies and losses and also confusion matrices. Section 5 concludes paper by providing the best-suited model for our subject and also importance of this research work. All the references which we have referred have been listed.

2. BACKGROUND INFORMATION ON MACHINE LEARNING METHODS

1. ConvNet or CNN

   In Convolutional Neural Network, the model first learns the features from the images and based on that it helps in classification. The reason for popularity of CNN is its very good execution while classifying. CNN was founded on the idea of recognising handwritten zip codes. It's a feed-forward kind of NN. [4]. Composition of multiple layers of artificial neurons forms Convolutional neural networks [5]. There is first the Convolutional Layer: which is responsible for learning the characteristics of the images input, then there is Max-Pooling Layer: it performs max-pooling operation by selecting max value from kernel matrix, and then comes Fully Connected Layer: all the neurons of a particular layer are connected to the neurons of the next layer to prepare the network for classification capabilities. Figure 2.1 explain the CNN architecture very well.

   Figure 2.1. CNN Architecture

2. Transfer Learning

   It is a method of ML in which the already trained model which has previously acquired knowledge can be used for solving other different problems. In this process, pre-trained models are utilized as the inception for classification, and not training once again with randomly initialized weights. These models are trained on very large datasets but we can also use them on smaller datasets by performing data augmentation [6] [7]. Figure 2.2 explain the different components of transfer learning and its working model.

   Figure 2.2. Idea of Transfer Learning

3. Pre-trained Convolutional Neural Network

   We have used three very well-known and widely used pre-trained DL CNNs: VGG and ResNet, for pneumonia detection.

   1. VGG

      VGG is a form of a CNN model. When VGG was tested it had attained an accuracy of 92.7%. It was trained and tested on over fourteen million images and there were thousand different classes. The simplicity of this network is, it uses only 3×3 convolutional layers that are put on the top of each other. With the help of max-pooling, the size is reduced. There are 2 fully-connected layers, both of them with 4,094 nodes and after that there is a SoftMax [8]. VGG16 and VGG19 both were used to see the performance.

      1. VGG-16 is ConvNet having deep 16 layers. 2. VGG-19 is ConvNet having deep 19 layers.

   2. ResNet

ResNet which is short for Residual Networks, was priorly developed for two problems namely vanishing gradient and degradation problem. The three types are: ResNet18, ResNet50 and ResNet101, they are named after the number of layers they possess respectively. We have used ResNet50 for the Pneumonia detection [9].

1. METHODOLOGY

   1. Dataset

      The dataset was downloaded from Kaggle.com. Chest X-ray images were utilized for feeding our network. It comprises of total 5,860 chest X rays in of two categories, one is Pneumonia and the other is Normal.

      Out of total images, 5,206 images were deployed for training the model and 634 images were utilized for testing purpose and rest of them are used for the validation dataset.

      Table 3.1: Dataset

      TRAINING SET	NORMAL	1341 IMAGES
      	PNEUMONIA	3875 IMAGES
      TEST SET	NORMAL	234 IMAGES
      	PNEUMONIA	390 IMAGES
      VALIDATION SET	NORMAL	8 FILES
      	PNEUMONIA	8 FILES

      The Figure indicates the samples of two different categories.

      Example of Normal Case

      Example of Normal Case

      Example of Pneumonia Cases

      Example of Pneumonia Cases

      Figure 3.1 . Different samples of chest x-ray

      For the study, Google Colab was used for training the model, for the evaluation and testing these algorithms. First of all, all the images are pre-processed, after that we perform data augmentation on all the training images and at last, we train by using the algos: VGG16, VGG19 and ResNet50 and we test these three algorithms on our dataset.

      Table 3.2. Parameters used for training

      Software Pre-trained CNN Image Size Optimizer models

      VGG16 224×224

      Google Colab Adam

      VGG19 224×224

      ResNet50 224×224

   2. Pre-Processing

      The foremost steps of building a model is pre-processing of the data and one of the most vital steps here was to re-size all the input image for the three machine learning technique in 224×224. Each of the three algorithms i.e., VGG16, VGG19 and ResNet50, the image was re-sized to 224×224 for input. All the images to the respective architectures were also normalized accordingly.

   3. Data Augmentation

      We want to have the best solution for the diagnosis of pneumonia from X-ray and as we know, that CNNs work better with large datasets [9]. We know that the size of our database right now is not very large. Because we have less amount of data comparatively, parameters are compromised and so the generalization is poor. In training these algorithms, we often do not have very big datasets, so we enlarge the available dataset by using a technique called data augmentation. This technique is helpful as it solves the drawback of limited dataset and thereafter, we fine-tune these algorithms and make them ready for efficiently detecting pneumonia [10]. One of the advantages of using this technique is that this technique prevents overfitting of the data by enlarging the size of the train data. Hence, we need not worry about the issue of overfitting.

      In image augmentation settings used are:

      Table 3.3. Augmentation Settings

      Technique rescale shear_range zoom_range	Setting 1./255 0.2 0.2
      horizontal_flip	True

   4. Architectures Tuning:

      The very first step performed on the images of chest X-Ray was pre-processing and the next step was normalization so that data is ready for training the network. After that using data augmentation techniques, we processed the data more efficiently. The layers of the algorithms used were not trained because the weighs are fixed already [11]. We set the last layers of VGG16, VGG19 and ResNet50 to be False as we need to classify only 2 categories and these algorithms classify thousands of categories with great accuracy. So in order to decrease computational load, we did this and added our own layer (Flatten and Dense) with activation function as softmax [12]. The optimizer that we used is Adam optimizer.

      Here is the model summary.

      Figure 3.2. The Algorithm

      Here is workflow of the experiment.

      Figure 3 .3. Schema of the experiment conducted

   5. Evaluation Metrics

      To check how good our model is, we need to perform some evaluation techniques using our test dataset after the training process is over. The evaluation techniques used were accuracy, recall, precision, F1score.

      What do these terms mean and how are they calculated has been mentioned below. When we do the evaluation using test data, True Positive (TP) depicts number of pneumonia images that are recognized as pneumonia, True Negative (TN) for number of normal images that are recognized as normal, False Positive (FP) depicts number of normal images that are recognized as pneumonia, False Negative (FN) for number of pneumonia images that are recognized as normal.

      • Accuracy: Gives us the percentage of correct predictions made for the test data.

      • Precision: It is fraction of TP among all of the sample which were expected to belong in a particular class.

      • Recall: It calculates the percentage of total positive results classified by the algorithm.

      • F1-score: It conveys a balance between precision and recall.

2. RESULTS AND DISCUSSION

   The results we got on performing evaluation metrics on each of the models is shown in Table 4. We have evaluated the performance of following three ConvNet models: VGG16, VGG19, and ResNet50.

   For VGG16 we got an accuracy of 96.7%. When tested on validation data, we got a validation accuracy of 89.58%. For VGG19 we got an accuracy of 95.6%. When tested on validation data, we got a validation accuracy of 91.19%. For ResNet50, we got an accuracy of 88.14%. When tested on validation data, we got a validation accuracy of 79.6%.

   For the further evaluation of each classifier CNN model, precision, recall, and f1-score are considered and evaluation measures [13]. Graphs for accuracy and loss have also been plotted and is shown in Table 4.1 & 4.2. Results of confusion matrix for each of the algorithm is also shown.

   TABLE 4.1. Summary showing testing accuracies and testing losses for different models. VGG16 was able to attain the maximum accuracy and the minimum loss amongst all.

   MODEL	ACCURAC		LOSS	VAL_LOSS
   VGG16	Y 96.7%	89.58%	0.09	0.35
   VGG19	95.6%	91.19%	0.12	0.29
   ResNet50	88.14%	79.6%	0.29	0.51

   MODEL	ACCURAC		LOSS	VAL_LOSS
   VGG16	Y 96.7%	89.58%	0.09	0.35
   VGG19	95.6%	91.19%	0.12	0.29
   ResNet50	88.14%	79.6%	0.29	0.51

   VAL_ACCURACY

   TABLE 4.2. Graphs of 3 different models comparing Accuracies and Losses

   Model

   Training Acc and Val Acc

   Training Loss and Val Loss

   VGG16 VGG19

   ResNet50

   Below are the confusion matrix for VGG16, VGG19 and ResNet50 respectively.

   TP: Pneumonia detected as Pneumonia FP: Normal detected as Normal

   TN: Normal detected as Pneumonia FN: Pneumonia detected as Normal

   TABLE 4.3. Shows precision, recall and f1-score calculated for each model.

   	MODEL CRITERION	NORMAL	PNEUMONIA
   	Precision Recall	0.87 0.98	0.95 0.76
   	VGG16 F1-Score	0.92	0.85
   	Precision Recall	0.94 0.92	0.87 0.90
   	VGG19 F1-Score	0.93	0.88
   	Precision Recall	0.76	0.95
   	ResNet50 F1-Score	0.98	0.48
   		0.86	0.64

3. CONCLUSION

The work represents the three different methods of transfer learning for diagnosing pneumonia very efficiently. Three distinct algorithms were first trained and validated for classifying the two categories. Among the three algorithms it was observed that VGG16 (96.7%) and VGG19(95.6%) performs nearly same, VGG16 being slightly better whereas ResNet model has lowest accuracy of 88.14%. Pneumonia is an infectation in one or both lungs. Precision and recall for models VGG16, VGG19 and ResNet is (87%, 94%, 76%) and (98%, 92%, 98%) respectively. VGG16 and VGG19 shows promising result with lesser value loss promising result. ResNet model have maximum value loss and less accuracy amongst three model. Pneumonia is a fatal disease and is a cause of morbidity and mortality. Because of this menacing disease, there are a lot of children who lose their lives every year. There is always a solution for a problem, and the solution to this is timely intervention and treatment of this disease. There are not many radiologists who can study these X-Rays [14] [15]. With the help of this study, it can be helpful to our medical field. VGG16 shows great performance in diagnosis. Hence, it can be a great contribution.

REFERENCES

1. Ayan, Enes, and Halil Murat Ãœnver. "Diagnosis of pneumonia from chest X-ray images using deep learning." 2019 Scientific Meeting on Electrical- Electronics & Biomedical Engineering and Computer Science (EBBT). Ieee, 2019.

2. Stephen, Okeke, et al. "An efficient deep learning approach to pneumonia classification in healthcare." Journal of healthcare engineering 2019 (2019).

3. Bougias, H., et al. "Identifying pneumonia in chest X-rays: Comparison between different transfer learning methods." European Congress of Radiology- ECR 2020, 2019.

4. Hashmi, Mohammad Farukh, et al. "Efficient pneumonia detection in chest xray images using deep transfer learning." Diagnostics 10.6 (2020): 417.

5. Chouhan, Vikash, et al. "A novel transfer learning based approach for pneumonia detection in chest X-ray images." Applied Sciences 10.2 (2020): 559.

6. Liang, Gaobo, and Lixin Zheng. "A transfer learning method with deep residual network for pediatric pneumonia diagnosis." Computer methods and programs in biomedicine 187 (2020): 104964.

7. Jain, Rachna, et al. "Pneumonia detection in chest X-ray images using convolutional neural networks and transfer learning." Measurement 165 (2020): 108046.

8. Rahman, Tawsifur, et al. "Transfer learning with deep convolutional neural network (CNN) for pneumonia detection using chest X-ray." Applied Sciences 10.9 (2020): 3233.

9. Labhane, Gaurav, et al. "Detection of pediatric pneumonia from chest x-ray images using cnn and transfer learning." 2020 3rd International Conference on Emerging Technologies in Computer Engineering: Machine Learning and Internet of Things (ICETCE). IEEE, 2020.

10. Maghdid, Halgurd S., et al. "Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms." Multimodal Image Exploitation and Learning 2021. Vol. 11734. International Society for Optics and Photonics, 2021.

11. Puneet Gupta, Pneumonia Detection Using Convolutional Neural Networks, International Journal for Modern Trends in Science and Technology, 2021.

12. Modi, Shrey, and Mohammed Husain Bohara. "Facial Emotion Recognition using Convolution Neural Network." 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS). IEEE, 2021.

13. Perumal, Varalakshmi, Vasumathi Narayanan, and Sakthi Jaya Sundar Rajasekar. "Detection of COVID-19 using CXR and CT images using transfer learning and Haralick features." Applied Intelligence 51.1 (2021): 341-358.

14. Brima, Yusuf, et al. "Transfer Learning for the Detection and Diagnosis of Types of Pneumonia including Pneumonia Induced by COVID-19 from Chest X-ray Images." Diagnostics 11.8 (2021): 1480.

15. Cha, So-Mi, Seung-Seok Lee, and Bonggyun Ko. "Attention-Based transfer learning for efficient pneumonia detection in chest X-ray images." Applied Sciences 11.3 (2021): 1242.

16. Khalifa, Nour Eldeen M., et al. "A study of the neutrosophic set significance on deep transfer learning models: An experimental case on a limited covid- 19 chest x-ray dataset." Cognitive Computation (2021): 1-10.