Image-Based Automatic Pulse Rate Monitoring System Using PC Webcam

Image-Based Automatic Pulse Rate Monitoring System Using PC Webcam

Isayiyas Nigatu Tiba and Li Li, AssociateProfessor & Supervisor of Master Degree Student

Department of Electronics Engineering, Tianjin University of Technology and Education.

China

Abstract

Pulse rate is one of the most major physiological parameter to be measured in-home health monitoring system. As it is well said in English proverb prevention is better than cure pulse rate monitoring in our routine daily activity plays the main role in preventing an illness than to treat it. In this paper, a new, low price easily accessible method to monitor human pulse rate by using PC webcam is presented. The method implements the Haar cascade classifier algorithm presented in OpenCV to detect human face. By limiting a region of interest (ROI) in the face and extracting the raw RGB trace to a video frames in the ROI, independent component analysis (ICA) is applied to determine the strongest source signal used to estimate the pulse rate in beat per minute (PBM). A graphical user interface is also designed to make this application more convenient for a user.

1. Introduction

   In recent years, the usage of home health monitoring system is getting increased as diseases like cardiovascular disease is the most dangerous one. According to [14], heart disease is the number-one killer compared to all cancers combined. In this paper automatic cardiovascular pulse rate monitoring system using a webcam integrated with personal computers is described. Besides introducing a new application in image processing, this method believed to contribute a lot in monitoring the human heartbeat by measuring cardiovascular pulse rate at home.

   Even if there are different methods (possibilities) to measure cardiac pulse, the golden and mostly used in hospitals and health care center is electrocardiogram (ECG) [15]. However, recording electric potential generated by the heart requires appropriate application of the electrodes, and it is too complicated and annoying in home conditions [9, 10].

   Non-contact, long-term monitoring human heart rate is of great importance to home health care. Recent studies show that Photo-plethysmography (PPG) can provide a means of heart rate measurement by detecting blood volume pulse (BVP) in human face [10].

   This system is based on a novel theory introduced by (Ming-Zher Poh, Daniel J. McDuff, and Rosalind W. Picard) [7, 11]. It is a novel methodology for recovering the cardiac pulse rate from video recordings of the human face, which can be considered as a new application in digital image processing as shown in figure 1.

   Figure 1. Principle of pulse rate monitoring system

   The Heart Disease Health Center describes the normal range of a resting heart rate (after resting 10 minutes) in beats per minute, as shown in Table 1 below, according to age1.

   Even though there is a wide range pulse rate of normal beat, an unusually high or low heart rate may indicate underlying problem. It is highly recommended to call/see a doctor if any of the following symptoms getting to happen. An irregular or rapid heartbeat, (palpitations),Palpitations can be persistent or may come and go (episodic), Chest pain, Dizziness, Fainting, Lightheadedness, and Shortness of breath.

   Table 1: Normal resting heartbeat rate for different fitness level.

   Age or fitness level	Beats per minute (bpm)
   Babies to age 1:	100-160
   Children ages 1 to 10:	70-120
   Children ages 11 to 17:	60-100
   Adults:	60-100
   Well-conditioned athletes:	40-60

2. Methods

   1. Experimental setup

      This experiment is done in a small studying room illuminated with sufficient amount of light (artificial/natural) about 300 to 400 lux. The setup consisted of the webcam integrated with laptop (TOSHIBA Web camera -HD), and heath care device called OMRON HEM-

      720C. The webcam used to record video frames in color (24-bit RGB with 3 channels × 8 bits/channel) at 10 frames per second (fps) with pixel resolution of 640 × 480 and saved in AVI format on the laptop.

      OMRON HEM is an upper arm automatic digital blood pressure monitor.

      Figure 2. OMRON HEN-720C

      It isvalidated according to the British Hypertension Society (BHS) protocol for use for general populations. It reads systolic and diastolic blood pressure values, and pulse rate in a minute. In this experiment, the pulse rate value (bpm) is used as a reference to validate the experimental results for all participants.

      There are 10 participants (6 males and 4 females) between the ages of 26-37 years were enrolled for this study. The sample featured participants of both genders, different ages and with varying skin colors (Asians, Africans and Caucasians). Informed consent was obtained from all the participants prior to the start of each study session.

   2. ROI selection

      The region of interest (ROI) is selected as a rectangle containing about 60% the face region at the first frame of video recording. As shown in figure 3.

      Figure 3. The detected face captured by webcam.

      The ROI contains three primary color red, green and blue (RGB) channels.

   3. Methods of analysis

      The blind source separation problem is an approach or method to extract the underlying source signals from a set of mixtures, where the mixing matrix is unknown. This situation is common in acoustics, radio, medical signal, image processing hyper spectral imaging, and other areas. A source means here an original signal,

      1. independent component, Blind means that we know very little, if anything, on the mixing matrix, and make little assumptions on the source signals.

         ICA is one method, perhaps the most widely used, for performing blind source separation [10]. In this study the ICA model made the following assumptions about the observed signal [11].

         1. Statistical independence between each of the sources Si from the sources vector S is assumed.

         2. Non-gaussianity is maximized.

         3. The mixing matrix must be square and full rank, the number of mixtures must be the number of sources. And the mixtures must be linearly independent from each other.

      The fundamental source signal of interest in this study is the blood volume pulse (BVP) that propagates throughout the body. During the cardiac cycle, volumetric changes in the facial blood vessels modify the path length of the incident ambient light such that the subsequent changes in amount of reflected light indicate the timing of cardiovascular events[7][11].

      Data will be made by recording a video of the facial region with a webcam primary or (RGB) color sensors. Each color sensor of the webcam records a mixture of original source signal with some difference in weight.

      Let us now denote the observed signal from the red, green and blue color sensors as:

      () , () and () respectively.

      () : – Is the amplitude of recorded signals.it can be considered also as a function of an average of all pixels in the facial region.

      According to the above assumptions, the number of recoverable sources cannot exceed the number of observations.

      Thus, we assume three underlying source signals

      represented by:

      () , () and ()

      The observed signals are observed mixtures o source, it can be written as:

      =

      =

      () = ()

      (1)

      for i = 1, 2, 3.

      For ease of the analysis in this study, let us write the matrix representation of the system:

      = + +

      = + +

      = + +

      =

      To apply the ICA algorithm firstly, the ROI must be

      Observation Vector X

      Observation Vector X

      Mixing matrix A

      Mixing matrix A

      Recoverable source vector S

      Recoverable source vector S

      decomposed to the RGB channels. Figure 4 shows the ROI is decomposed to the three channels (RGB) for every number of frames captured per-second.

      The mixing matrix A contains the mixture coefficients aij.

      source frame

      red channel

      green channel blue channel

      = . (2)

      Let us denote the unmixing matrix as W, and the output or the recoverable source vector Y

      1 = (3)

      By substituting equation (3) in to (2) we will have:

      = .

      = 1.

      = .

      = .

      Figure 4. Single frame representation of ROI decomposition in to RGB

      After the ROI is decomposed to the RGB channels the raw trace data needs to normalize. The normalized raw trace is determined as:

      = ( ) , i=1, 2, 3

      Where and are the mean and standard deviation of the data [7, 8, 11].

      RED

      0.1

      0

      -0.1

      9 10

      9 10

      0 1 2 3 4 5 6 7 8

      = . (4)

      0.1

      9 10

      9 10

      0

      GREEN

      Finally, this equation (4), has a lot to tell about the independent components or the underlying sources we

      wanted to extract.

      -0.1

      0.1

      0

      -0.1

      0 1 2 3 4 5 6 7 8

      BLUE

      0 1 2 3 4 5 6 7 8 9 10

      Figure 5. The raw RGB traces.

      =

      The normalized raw traced signals are then decomposed in to independent source signals by using

      the ICA algorithm,based on the (joint) diagonalization of cumulant matrices (JADE) [7,15].

      In the final step the source signals are transformed to frequency domain using the most favourite method FFT, for BVP estimation [8] as shown in Figure 6. The heart rate (HR) is computed as 60* (estimated BVP in Hz).

      				0 0.5			1.5				2.5
      	0 0.5				1.5			2.5

      				0 0.5			1.5				2.5
      	0 0.5				1.5			2.5

      60

      40

      The application is designed to return two different heartbeat rates, the Training Heart Rate (THR), and the Actual Heart Rate (AHR).

      The training heart rate is a target heart rate used to determine the safe training zone. This benefits more athletes, or people who are looking for weight loss and fitness improvement. According to Karvonen [16], THR is calculated as:

      20

      0

      200

      100

      0

      150

      100

      50

      1 2 3

      1 2 3

      = ()

      = . (6)

      =

      0

      0 0.5 1 1.5 2 2.5 3

      Figure 6. Spectra of separated source signals.

3. Application Design

   Since this experiment is undergone in a MATLAB environment a user interactive application is designed as shown below, using MATLAB GUI.

   Figure 7. GUI of the application

   % +

   (7)

   The resting heart rate can be determined by using a medical device or manual method3 (Counting from zero, take your 1minute heart rate in your neck) [17], before getting out of bed. Doing this for three days in a raw will maximize the accuracy. If it is unknown, an estimated 70 bpm will be taken as a resting HB. User will be asked to enter his/her age and resting heart rate, and the application returns his/her maximum training heart rate on the left side of the interface. This can be done before starting the webcam.

   It doesnt require a user to be an advanced computer literate to use this application. The start button will start the webcam and run the algorithm to start processing. It takes 15 20 seconds to display the calculated current heart rate of the user. The image shown will be the first video frame captured from webcam. The remaining video frames will be shown in independent HIGHGUI window called by a wrapper function from c++ environment. The current heartbeat is computed for each 10 consecutive frames, and plotted on the bottom

   side. Finally, the mean of 10 beats is computed and displayed numerically on upper right corner. Compared to Table 1, the application displays a feedback on the right side. If the measured heartbeat rate is out of the normal range3, the user is advised to see a doctor or do more medical checkup for further investigation.

   The user can stop execution of the program or exit the application.

4. Results

   Particip ant Id.	Participant Condition	Webcam Pulse rate (bpm)	OMRON HEM- 720C Pulse rate (bpm)
   1.	Normal	65.0533	65
   2.	After Exercising	117.7439	100
   3.	Normal	90.52	76
   4.	Resting	64.6549	60
   5.	Normal	72.970999	72
   6.	After Exercising	103.3630	100
   7.	Normal	61.955534	67
   8.	Resting	63.400246	60
   9.	After exercising	120.4671	110
   10.	Normal	70.07894	68

   Particip ant Id.	Participant Condition	Webcam Pulse rate (bpm)	OMRON HEM- 720C Pulse rate (bpm)
   1.	Normal	65.0533	65
   2.	After Exercising	117.7439	100
   3.	Normal	90.52	76
   4.	Resting	64.6549	60
   5.	Normal	72.970999	72
   6.	After Exercising	103.3630	100
   7.	Normal	61.955534	67
   8.	Resting	63.400246	60
   9.	After exercising	120.4671	110
   10.	Normal	70.07894	68

   Table 2. List of sample pulse rate measured by the application compared with the medical device OMRON HEM-720C.

5. Conclusion

   The application designed in this paper seems effective and easy to use in daily monitoring of home care. It is a low-cost method, and leads a way to show an approach which is easily scalable for simultaneous assessment of multiple people in front of a camera.

   It also extends attention to digital image processing is increased to a new dimension or principle that, we are able to extract physiological parameters by processing video frames recorded using a basic webcam.

   However, further study has to be made performed to design a real time dynamic application to able to install independently to MATLAB. Therefore, designing a real time application in other platform like c++ and java (in visual studio) will be the one future works.

6. References

[1]. The McGraw-Hill Companies, Inc, Fitness Zone Online the Heart.

[2]. The McGraw-Hill Companies, Inc, Measuring

Your Exercise Heart Rate.

[3]. Nursecom Educational Technology, An ECG primer ,2003

[4]. Lorenzo Scalise, Non-Contact Heart Monitoring, Dipartimento di Ingegneria Industriale e Scienze Matematiche, Università Politecnica delle Marche, Italy

[5]. Mohammad Ali, Akbari,Kamran, Hassani,John D Doyle,Mahdi Navidbakhsh,,Maryam

Sangargir,Kourosh Bajelani,and Zahra Sadat Ahmadi, Digital Subtraction Phonocardiography (DSP) applied to the detection and characterization of heart murmurs,2011.

[6]. Santiago Lopez, Pulse Oximeter Fundamentals and Design,2012.

[7]. M.Z. Poh, D.J. McDuff, and R.W. Picard,

Noncontact, automated cardiac pulse measurements using video imaging and blind source separation, 2010.

[8]. Haiying Xia, Zhouxiao Bao, Haomiao Jiang, MOBILE CARDIAC PULSE

MEASUREMENTS, Electrical Engineering, Stanford, 2011.

[9]. T. Pursche (Faculty of Electrical, Information, Media, Engineering), J. Krajewski (Faculty of Experimental Business psychology), and Reinhard Moeller (Faculty of Electrical, Information, Media, Engineering), Video-based Heart Rate Measurement From Human Faces, University of Wuppertal, Germany,2012.

[10]. John Allen, Photoplethysmography and its application in clinical physiological measurement, Physiol. Meas.28 (2007) R1 R39, 2007.

[11]. Ming-Zher Poh*, Daniel J. McDuff, and Rosalind W. Picard,Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam ,IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 58, NO. 1, JANUARY

2011.

[12]. A. Noulas and B. Kr¨ ose, EM detection of common origin of multi-modal cues, inProc.

ACM Conf. Multimodal Interfaces, 2006, pp.201208.

[13]. P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, inProc. IEEE Conf Comput. Vis. Pattern Recognit., 2001, pp. I-11I-518

[14]. Rosalind W. Picard and Charles Q. DV,

Monitoring Stress and Heart Health with Phone and Wearable Computer, Massachusetts Institutes of Technology, Offispring, vol 1, NO 1, 2002

[15]. Frank G. Yanowitz, MD Professor of Medicine, INTRODUCTION TO ECG

INTERPRETATION, University of Utah School of Medicine,V8.0 (July 2012). J.F. Cardoso et al., On the performance of orthogonal source separation algorithms, in Proc. EUSIPCO. Edinburgh, UK, 1994, vol. 94, pp. 776779.

[16]. academics/physical education/Fitness Center

Heart Rate Reserve… How to Determine Your Heart Rate Training Zone,

[17]. The McGraw-Hill Companies, Heart Rate Measuring TechniquesTaking Your Pulse Manually Fitness Zone Online

1. http://www.webmd.com/heart-disease/pulse- measurement?page=2

2. http://www.briancalkins.com/HeartRate.htm

3. http://www.medicalnewstoday.com/articles/23 5710.php