Detecting the Comatose Patient’s Recovery Scale in a Neuro ICU by Tracking Eye State, Using Tablets in Android Platform

Detecting the Comatose Patient’s Recovery Scale in a Neuro ICU by Tracking Eye State, Using Tablets in Android Platform

Asha Yeldose*, Anju S.S*

*Department Of Computer Science and Engineering Amal Jyothi College of Engineering, Kottayam-686518, India

Abstract

Automated emotion state tracking is a crucial element in the computational study of human communication behaviors. It is important to design robust and reliable emotion recognition systems that are suitable for real world applications to enhance analytical abilities to support human decision making. Human emotions can be recognized by several approaches such as speech, gesture, facial images, and physiological signals. As the physiological signals originate from the activity of the Autonomous Nervous System (ANS), they cannot be triggered by any conscious or intentional control. Sympathetic nerves of the ANS get activated when a person is positively or negatively excited, raising heart rate, blood pressure and respiration rate and decreasing heart rate variability (HRV). Of these, HRV seems to be the ideal candidate to provide indications on emotional changes that are less obvious to perceive visually, at the same time easy to capture using wearable sensors. Gathering a high quality database of physiological signals is vital for the development of emotion recognition system. It is easy to gather meaningful data in case of image and audio recognition as the integrity of the data can be seen or heard by nonspecialists. However, a good physiological data cannot be determined by nonspecialists. The physiological signals being the activity of ANS, the emotions must be naturally elicited on the subjects in order to obtain good data.

We are currently implementing a Tablet PC based system for automation of the Medical Neuro Intensive Care Unit (NICU) in Jubilee Mission Medical College (JMMC), Trichur. The emotional status is a parameter of great clinical significance for patients in Nero ICU. The system will provide the time sequence of images of patient from his time of admission till the current time with annotations of the external environments like medication history, vitals, episodes, external stimuli at time of taking the image, etc. These images may be examined by clinicians. We proposed to implement the above as part of the ICU automation

system in JMMC. The viability of the scheme critically depends on the ability to integrate the bed side monitors to the automation system, adoption of the proposed scheme by the clinicians and the interest and effort put by them. The corpus created can be used as base material for further study in this area.

1. Introduction

   The Intensive Care Unit (ICU) is the arena in which many succeed and many others perish in the intense battle between life and death. As the name suggests ICU can be expanded as I See You". That means someone is observing us continuously. As we are humans its not possible to observe the patients in every single second. There comes the need of machines. Computers and computer-based applications become more and more sophisticated and increasingly involved in our everyday life. Whether at a professional, a personal or a social level, it becomes ever more important that we are able to interact with them in a natural way, similar to the way we interact with other human agents.

   Our implementation area is the neuro ICU at Jubilee Mission Medical College Trichur. In neuro ICU the behavior of patient is important. A patient's response to the outside world is highly significant. To monitor the patient's all emotions in a short span is not effective. So the patient under comma stage is chosen for our study. With this research our prime importance is to identify whether a patient shows any sign of recovery from comma especially by identifying their eye openings from images captured in different times in a day. The image capturing is made without any human interaction there by taking care of not adding an extra load to nurse's daily routine. The whole said process should be performed in a tablet. Tablet is preferred because of its ease in handling, small size etc., the main important feature of this will be it requires only very few user interactions, say one.ie, to activate the application once the .apk file of app is loaded to the tablet successfully.

   JMMC-The Jubilee Mission Medical College is one of the biggest healthcare centers in Kerala. It is generally known by the name Poor Man's Hospital. Even then the hospital management was not ready to accept that name in the stream of service. They always wanted to give their patients a great care and good service. So they become ready to go hand in hand with technology. They promoted new researchers as they know they can grasp things fast and can imagine a new era for JMMC along with new technological trends. As our team approached them we too had no idea of what and how we should give colors to their dreams, and we had no time to automate their complete system as a whole. So it was a good opinion to start from ICU, and we chose Neuro ICU.

   In NICU, I got a chance to observe the daily working there, continuously for about 48 hrs. Learned the hardship of states, especially that of nurses and doctors. There I observed the battle between life and death there. In ICU each seconds matters. For humans its not adorable to be conscious on every second. So we have to be dependent on machines. As NICU is for patients who have some neurological disorders, their small movements matters a lot for doctors to decide on their medication or on their further diagnosis. From that thought this research topic come to my mind. A tablet with a camera monitoring the patients under comma to see any recovery, by analyzing their images shot on different intervals.

   ICU is characterized by periods of heavy load of urgent tasks to be finished within precious time and manual entry of information into the medical record will be either postponed or dropped. This creates a statistical skew in the information captured that will affect the analysis based on them. That means time management is critical inside ICU. Nurses are very busy with their schedule. They are busy with:

   1. Take Over

   2. Patient Care

   3. Requesting Laboratory services

   4. Reporting

      1. ICU in charge

      2. Doctors

      3. Bystanders

   5. Monitoring Equipment

   6. Taking directions from doctors

   In such a scenario it will be nice to think about a system that looks upon a patient continuously to know whether his eyes are moving or whether he produces any sound. It will be nice as it increases the percentage of patient's recovery if it could identify the eye openings by patients under comma.

2. The Current Neuro ICU scenario

   Our customers were the NICU staffs, where each input and output matters a lot. As the Neuro ICU deals with neurological disorders the emotions of the patient in different conditions are valuable or inevitable. Normally during the time of checkups doctors test their responses with some natural techniques. Asking them their names, about their where about, checking their eye and pupil, testing their level of response to pain etc., are some of them. Among those the eye is very important. A well experienced staff can look at the patient eye and can say whether he will be recoverable or not. And they can measure their depth of consciousness thus.

   So our applications main focus was to help the staffs in that. For that primary importance was given to eyes. Among tha the fast achievable point was identifying a comatose person. As a first step input it was necessary to monitor patients through our system. Considering a continuous monitoring was somewhat risky as the work space is inside an ICU, and video monitoring may create bandwidth problem. So took images as the measure for monitoring. Images will be captured and check for any sound. Both are stored for further processing. The whole operations take place periodically.

3. Conceptual Architecture

   Our proposed system is based on eye tracking so as to detect the comatose patients recovery. For this a sample database of patient images is of prime importance. But as our implementation area is the neuro ICU, (from here onwards it will be referred as NICU), getting sample images of patient is not applicable so the database consists of some models enacting the eye opening or closing.

   The conceptual architecture of the system is shown in Fig. 1.All the system works under an Android platform.

4. Proposed System

   As Fig 1 suggests our system has basically 7 sections. Using a tablet camera first the images are captured and then an image database is created. After that we perform the basic 7 operations.

   1. Image Compression

      The inputted images may be bitmap images. Chances are there for OOM error, i.e., out of memory error when a bitmap is loaded to memory. So there is the need of image compression. Image is compressed to a predefined width and height. Once these width or

      height is set all the images that are loaded to memory under this step will be scaled accordingly.

   2. Face & Eye Detection

      For performing face and Eye detection the android face detector API is used. Actually the Android Face Detector API works on identifying eyes in human face images. So it would be a better choice to go with this API for face detection. Once the face is detected for getting the eye part we will draw a rectangle covering both the eyes. See Fig. 2.which shows a sample image with eye region covered with a rectangle when a simple api is executed on an emulator.

      Model images as input

      Model images as input

      Performs Face Detection

      Performs Face Detection

      Detects eyes

      Detects eyes

      Convert image to Black & White

      Convert image to Black & White

      Count the no of black pixels

      Count the no of black pixels

      Checks the count with threshold

      Checks the count with threshold

      Fig 2: Sample image with eye region covered with a rectangle, in an emulator.

   3. Converting image to Black and White

      It will be easier if we could convert the image to black and white. As we have to detect the eyes are open or not we have to check whether there is pupil, which is black. If pupil can be detected we can conclude that eyes are open else eyes are not. For converting the image to black and white first scan through all the pixels in the image. Get the colour of each pixel. Then with a threshold set the black and white image.

   4. Checking the Eye Opening

      Once the image is converted to black and white we will loop through the rectangular region so as to get the count of black pixel. With lot of processing over images we could get a threshold value for pixel count to see whether eyes are open or closed. Compare the obtained count with the threshold and toast an alert accordingly. See Fig 3 for a closed eye image, and Fig 4 for open eye image, which are calculated through pixel count against a threshold.

      No

      Is Eye Open?

      Yes

      Toast Alert

      Toast Alert

      Fig 1: Flowchart of the conceptual Architecture Model

      Fig 3: Sample image with eyes closed.

      Fig 4: Sample image with eyes open.

5. Experiment Results

   The images used here are that of the Spiderman fame Mr. Tobey Maguire. As getting live images are difficult. These images are downloaded from internet. When the procedure is performed over a variety of images the result obtained was satisfactory.

6. Conclusion and Future Work

   In this research I have pointed out a simple mechanism to detect the coma state recovery of patients in NICU. Its a simple work it can be extended to detect the emotion of patients through facial expressions, through sound or through physiological signals.

7. Acknowledgement

   First and foremost I would like to thank the God Almighty for his tremendous support to come out with this research. Without his blessings I am nothing. Then I would like to thank Dr. George Varkey for his guidance and all other staffs in Amal Jyothi College. Special thanks to JMMC team. Last but not the least a hearty thanks to my family for their supports and encouragements. Indirect thanks to Mr. Tobey Maguire for those wonderful images.

8. References

1. Ramneet Singh Chadha, Umesh Sharma, Vijay Uniyal : Design &Implementation Aspects of Operation Theatre Management Module in HMIS,Proceedings of ASCNT 2010

2. William R. Hersh: Electronic Medical Record: Promises and Problems -Biomedical Information Communication Center. Journal Of The American Society For Information Science 46(10):772-776,1995

3. Mandalapu Sarada Devi, Dr Preeti R Bajaj, Driver Fatigue Detection Based on Eye Tracking , First

   International Conference on Emerging Trends in Engineering and Technology.

4. Amarjeet Singh Cheema, Prakhar Misra, Pragya Sharma : Clinical Data Abstraction,Proceedings of ASCNT-2011.

5. Saeid. Fazli, and Parisa. Esfehani: Tracking Eye State for Fatigue Detection International Conference on Advances in Computer and Electrical Engineering (ICACEE'2012) Manila (Philippines) Nov. 17-18, 2012