Neonatal pain detection using triple CNN

Neonatal pain detection using triple CNN

ICCIDT – 2023 Conference Proceedings

Akshaya Anish Kumar1 B-tech Student

Computer Science &Engineering Mangalam Colleg Of Engineering

Bobin Varghese2 B-tech Student

Computer Science &Engineering Mangalam Colleg Of Engineering

Abstract – Convolutional Neural Networks are used nowadays because of its successful application in medical image analysis, image recognition etc. Triplet Convolutional Neural Network which is specifically designed for neonates for the effective detection of neonatal pain. The N-CNN used here is applied on a real world dataset which contains images of the neonates at the time of pain while being hospitalized in Intensive Care Unit. It is clear from the experiments that N-CNN used here is very efficient in classifying the pain when compared to the existing standard.

KeywordsConvolutional neural assessment, clinical network, pain facial expression

applications, analysis.

I. INTRODUCTION

For many years Pediatricians believe that neonates do not have the ability to sense the pain. But later there exists many methods for detecting the neonatal pain. The main drawback of these methods are they are discontinuous and not consistent in determining the pain of neonates. Since neonates cant able to communicate correctly it is very important to determine the cause of pain for the proper pain management. Convolutional Neural Network were used because of its wide successful application in medical image analysis, object recognition, image recognition etc. It provides pain relevant features as they have the ability to learn and extract features at multiple levels of abstraction.Triple CNN which is introduced and designed specifically for determining the pain of neonates using facial expressions. Convolutional neural networks (CNNs) have gained much popularity in the last decades due to the wide range of its successful applications in medical image analysis, object and emotion recognition. In this paper, we investigated the use of a novel lightweight neonatal convolutional neural network as well as other popular CNN architectures for assessing neonatal pain. We experimented with various image augmentation techniques and evaluated the CNN architectures using two real-world datasets [COPE and neonatal pain assessment dataset (NPAD)] collected from neonates while being hospitalized in the intensive care unit. The experimental results demonstrate the superiority and efficiency of the novel network in assessing neonatal pain. They also suggest that the automatic recognition of neonatal pain using CNN networks is a viable and more efficient alternative to the current assessment. Triplet Convolutional Neural Network which is specifically designed for neonates for the effective detection of neonatal pain. The N-CNN used here

Devu VS3 B-tech Student

Computer Science &Engineering Mangalam Colleg Of Engineering

Amritha Soloman 4

Assistant Professor Computer Science &Engineering Mangalam Colleg Of Engineering

is applied on a real world dataset which contains images of the neonates at the time of pain while being hospitalized in Intensive Care Unit. It is clear from the experiments that N- CNN used here is very efficient in classifying the pain when compared to the existing standard.

II RELATED WORK

Pain Assessment Using Handcrafted Features: It presented a handcrafted based method to detect facial expressions of pain using different variations of Local Binary Pattern (LBP) descriptor. Specifically, Local Ternary Pattern (LTB), Elongated Local Binary Pattern (ELBP), and Elongated Local Ternary Pattern (ELTP) texture descriptors were applied to COPE infant dataset [6] to extract pain-relevant features.

In the pre- processing stage, the images were re-sized, aligned, cropped to obtain the exact facial region, and divided into blocks or cells of 25 × 25. Then, the texture descriptors listed above were applied to these blocks to extract pain- relevant features. To classify the images (204 images) of 26 subjects as pain or no- pain, an ensemble of Radial Basis SVMs was built and evaluated on a testing set.

The results showed that ELTP texture descriptor achieved the highest (approx. 0.93) Area Under the Curve of Receiver Operating Characteristic curve (AUC of ROC) as compared to other texture descriptors. In addition to LBP descriptor, Celona and Manoni applied HOG (Histogram of Oriented Gradients) descriptor to 2 × 2 blocks of 8 × 8 pixel cells with an overlap of half the block and histograms of 9 bins evenly spread from 0 to 180 degrees. Applying this descriptor to 224 × 224 gray-scale image generates 26244-dimensions feature vector (729 regions ×

4 blocks × 9 bins). This feature vector was reduced to 175 dimensions using Principal Component Analysis (PCA) followed by L2 normalization. Using the features extracted by HOG descriptor with SVM achieved 81.75% average accuracy (i.e., accuracies averaged across 26 subjects). Other hand crafted based methods for neonatal pain assessment can be found in [10]

• [13]. The challenge of manually designing handcrafted descriptors and extracting the best set of features has motivated researchers to use Convolutional Neural Networks.

  CNNs learn and extract relevant features, at multiple levels of abstraction, directly from the source data or images. These networks achieved state-of-the-art results in many applications, including clinical and emotion recognition applications.

  2) Pain Assesment Using Deep Features: As is well known training solid CNNs requires large and well- annotated datasets (e.g., ImageNet – approx. 1.2 million images and 1000 classes). In practice, it is restively rare, especially in the medical domain, to find large and well- annotated datasets. Therefore, transfer learning and data augmentation concepts were introduced to handle the lack of data issue. Transfer learning is the process of applying the knowledge that was learned in one domain to another relevant domain. Recently, transfer learning has become the de-facto method for analyzing medical images because it allows researchers to extract, using models trained on large datasets, relevant features from small medical datasets. Celona and Manoni [9] applied transfer learning method to static images of COPE dataset (26 infants and 204 images) to classify these images as pain or no pain. In particular, the presented method used deep features extracted by a pre-trained CNN (VGG-Face) to train Support Vector Machine (SVM) model. Testing the trained model on unseen data (i.e., leave-one-subject-out cross validation) achieved 82.42% average accuracy. Combining the extracted deep features with the handcrafted features (e.g., LBP) improved the pain classification and yielded an average accuracy of 83.78%. Zamzmi et al. [14] used different pre- trained CNNs to recognize the pain of neonates. Particularly, VGG-Face, which was trained on a face dataset for face recognition, and VGG-F,M,S, which were trained on ImageNet for image classification, were applied to extract deep features from neonates faces. The features that were extracted by the pre-trained CNNs were used to train Naive Bayes, Nearest Neighbors (kNN), Support Vector Machines (SVMs), and Random Forests (RF). The proposed pipeline achieved up to 90.34% accuracy. Data augmentation is the process of creating synthetic data, using image processing techniques, that are large enough to train CNN from the scratch. Examples of common image processing techniques that are used for image augmentation include histogram modification, noise addition, zooming in/ut, cropping, geometric transformations via random translation, rotation, and flipping, and elastic deformations [15]. In addition to the traditional methods, deep neural networks such as Generative adversarial networks (GANs) are commonly used for image augmentation. Data augmentation is a mandatory pre- processing step for the vast majority of deep learning methods.

  Add layers to the architecture. The layers are Convolution, Maxpool, Dropout, Linear, Softmax.

  The architecture layers will be created with the help of Keras Library.

  We will use tripple CNN as modification

  1. Preprocessing:

     Histogram equalization

     • Normalization of images in the dataset

  2. Feature Extraction

     The preprocessed images are taken and features extracted using keras.

  3. Training

     The architecture is trained with the features obtained along with the labels

     The architecture after the training is saved as model

  4. Prediction

     • Neonatal Image is inputted.

     • Image preprocessed as in step 2

     • Image features are extracted as in step 3

       The model file is loaded and predicted with the features.

       The result is shown as the Neonatal is in pain or not.

  5. GUI development

1. System Architecture

2. DFD LEVEL 0

A. Proposed System

III METHODOLOGY

We use an advanced version of the convolutional neural network called Triplet CNN for assessing the type of neonatal pain by facial images. The machine learns from examples like the human brain and forms a logic in CNN to classify which category it belongs to. We obtained a neonatal pain classification image dataset from a public source and trained the preprocessed images with Triplet CNN. The architectural layers of CNN are created using the Keras library in Python.The trained model categorizes the images into four different stimuli: pain stimuli, rest/cry stimulus, air stimulus to the nose, and friction stimulus.The different stages of implementation are as follows;

1. CNN Architecture Creation:

D. DFD LEVEL 1

IV.RESULT

The current practice for assessing neonatal pain is inconsistent because it depends highly on the observers bias. Additionally, it is discontinuous and requires a large number of well-trained nurses to ensure the proper utilization of the pain scale. The discontinuous nature of the current practice as well as the intra- and inter-observer variations may result in delayed intervention and inconsistent treatment of pain. Since pain assessment is the cornerstone of pain management, developing automatic and continuous scales that generate immediate and more consistent pain assessment is crucial. This paper investigates the use of a novel Neonatal CNN (NCNN) along with other two well-known CNNs (ResNet and VGG-16) for pain assessment application. All the networks were evaluated using a real-world dataset collected from 31 neonates hospitalized in the NICU. To the best of our knowledge, this paper is the first to fully exploit the use of different CNN architectures for pain assessment application. The experimental results showed that the pain assessment performance achieved by N-CNN is comparable to, if not better than, ResNet50 and VGG-16. These results are encouraging and suggest that the automatic recognition of neonatal pain is a viable and more efficient alternative to the current standard of pain assessment. By continuing to explore the use of CNNs for developing a highly accurate pain assessment application, we hope to improve the effectiveness of pain intervention while mitigating the short- and long-term outcomes of pain exposure in early life.

1. FUTURE SCOPE

   For many years Pediatricians believe that neonates do not have the ability to sense the pain.Since neonates cant able to communicate correctly it is very important to determine the cause of pain for the proper pain management. So here we develop a tool to detect neonatal pain using advanced deep learning techniques.

   Detect neonatal pain from videos.

   Implementation of the system in the NICU for automated monitoring of neonatal.

   Improve the performance of the system by using transfer learning based feature extraction and classification.

   Ongoing work includes integrating other pain indicators such as body movement and crying sound to facial expression to obtain a multimodal network for neonatal

   pain assessment. The multimodal approach for pain assessment is necessary because it allows to assess pain during circumstances when not all pain responses are available.

2. CONCLUSION

   We proposed Triplet CNN using deep learning algorithms.The machine learns from examples like the human brain and forms a logic in CNN to classify which category it belongs to. We obtained a neonatal pain classification image dataset from a public source and trained the preprocessed images with Triplet CNN. The current practice for assessing neonatal pain is inconsistent

   because it depends highly on the observers bias. Additionally, it is discontinuous and requires a large number of well-trained nurses to ensure the proper utilization of the pain scale. The discontinuous nature of the current practice as well as the intra- and inter-observer variations may result in delayed intervention and inconsistent treat ment of pain. Since pain assessment is the cornerstone of pain management, developing automatic and continuous scales that generate immediate and more consistent pain assessment is crucial

3. ACKNOWLEDGEMENT

The authors would like to thank Principal Vinodh P Vijayan,Neethu Mariya John, H.O.D, Faculty of Computer Science, for their appropriate guidance, valuable assistance and helpful comments during the proofreading process.

REFERENCES

[1] Marchant, Neonates do not feel pain: A critical review of the evidence, Biosci. Horizons Int. J. Student Res., vol. 7, pp. 19, Sep. 2014.

[2] D. Cruz, A. M. Fernandes, and C. R. Oliveira, Epidemiology of painful procedures performed in neonates:

A systematic review of observational studies, Eur. J. Pain, vol. 20, no. 4, pp. 489498, 2016.

[3] Field, Preterm newborn pain research review, Infant Behave. Develop., vol. 49, pp. 141150, Nov. 2017.

[4] Zamzmi, R. Kasturi, D. Goldgof, R. Zhi, T. Ashmeade, and Y. Sun, A review of automated pain assessment in infants: Features, classification tasks, and databases, IEEE Rev. Biomed. Eng., vol. 11, pp. 7796, 2017.

[5] . Nanni, S. Brahnam, and A. Lumini, A local approach based on a local binary patterns variant texture descriptor for classifying pain states, Expert Syst. Appl., vol. 37, no. 12, pp. 78887894, 2010.

[6] . Brahnam, C.-F. Chuang, F. Y. Shih, and M. R. Slack, Machine recognition and representation of neonatal facial displays of acute pain, Artif. Intell. Med., vol. 36, no. 3, pp. 211222, 2006.

[7] M. N. Mansor and M. N. Rejab, A computational model of the infant pain impressions with Gaussian and nearest mean classifier, in Proc. IEEE Int. Conf. Control Syst. Comput. Eng., 2013, pp. 249253.

[8] U. Rahman, D. J. Jobson, and G. A. Woodell, Multi-scale retinex for color image enhancement, in Proc. 3rd

IEEE Int. Conf. Image Process., vol. 3, 1996, pp. 1003 1006.

[9] . Celona and L. Manoni, Neonatal facial pain assessment combining hand-crafted and deep features, in Proc. Int. Conf. Image Anal. Process., 2017, pp. 197204.

[10] G. Zamzami, G. Ruiz, D. Goldgof, R. Kasturi, Y. Sun, and T. Ashmeade, Pain assessment in infants: Towards spotting pain expression based on infants facial strain, in Proc. 11th IEEE Int. Conf. Workshops Autom. Face Gesture Recognit. (FG), vol. 5, 2015, pp. 15.