Validation of Heart Rate Extraction using Video Imaging on A Built-in Camera System of A PC

Validation of Heart Rate Extraction using Video Imaging on A Built-in Camera System of A PC

Doreen Devasia, Dhanu Thomas, Lois Roy, Jilu James, Mrs. Tessy Annie Varghese

Dept of ECEAsst. Professor

Amal Jyothi College of EngineeringAmal Jyothi College of Engineering Kanjirappally, IndiaKanjirappally, India

Abstract- Nowadays, techniques for physiological monitoring typically require users to strap on bulky sensors, chest straps or sticky electrodes. This discourages regular use because the sensors can be uncomfortable or restraining. In this work, we propose a new method for real-time, contact-free measurements of heart rate without the need for external sensors. Users can have the experience of remote health monitoring. Since cardiac pulse needs to subtle color change of a skin, a pulsatile signal which can be described as photoplethysmographic signal can be measured through recording facial video using a webcamera. We explore the chances that the reliable heart rate can be measured remotely by the facial video recorded using web camera. During the initial steps, facial video was taken using the webcam. Then we detected the region of interest from each of the frames, and yielded the raw trace signal from each of the three RGB channels. To extract more accurate cardiac pulse signal, we applied independent component analysis (ICA) to the raw trace signal. The heart rate was extracted using frequency analysis of the raw trace signal.

Keywordsphotoplethysmographic signal(PPG), region of interest(ROI) independent component analysis(ICA), fast fourier transform(FFT)

1. NTRODUCTION

   As the burden on limited medical resource increases, there is a need for the low cost physiological measurement solution which is accurate and can be used beyond the clinical environment. Cardiac pulses leads to subtle color change of human skin.Because the subtle color change can be detected through monitoring the skin image using a web camera, a pulsatile signal which can be described as photoplethysmographic (PPG) signal can be measured [1][2]. Poh et al. methodology has been used to measure cardiac pulse remotely using video imaging and blind source separation [3].We recorded the facial video using the built-in webcam of a laptop with sunlight as illumination. We apply the independent component analysis (ICA) for extracting the cardiac pulse signal and frequency analysis was done to obtain the heart rate. In this paper, we explore the potential that reliable heart rate can be measured remotely by recording the facial video using the built-in camera of a PC [8].

   We followed Phos methodology for extracting cardiac pulse signal and heart rate. First, facial videos were recorded using the webcam. Face region of each frame was then detected as region of interest (ROI) using face detection algorithm [3]. We selected the green channel raw traces. We obtained even

   more accurate cardiac pulse signal when we applied the independent component analysis (ICA) to the raw trace signal [5]. The heart rate was extracted using frequency analysis of raw trace signal.

   1. METHODS

      A. EXPERIMENTAL SETUP

      We used a basic webcam embedded in a laptop to provide an interactive display. The same can be done using the front facing camera of smartphone. All videos were taken in color with atleast 25 frames per second with pixel resolution 640×480. The user is visible to the webcam and the LCD monitor can be used to project information onto the reflective surface of the mirror. Also an external webcam can be setup connected to a lap running the analysis software in real time.

      Participants were seated at a table in front of a laptop at a distance of approximately 0.5 m from the built-in webcam. Each video recording was conducted one minute. We asked them to sit without movement and stare at the webcam while taking the video and normal sunlight was used for illumination.

      B.BLOCK DIAGRAM

      Image Frame

      Image Frame

      Face Detection

      Face Detection

      Channel Separation

      Channel Separation

      Mean of separated channels

      Mean of separated channels

      Independent Component Analysis

      Independent Component Analysis

      Fast Fourier Transform

      Fast Fourier Transform

      We used a basic webcam embedded in a laptop to provide an interactive display. We obtained RGB channels from each frame and the mean of the pixel values are calculated.We apply the independent component analysis (ICA) on the

      3 1 () for each i=1,2,3. This can be represented compactly by the mixing equation

      x(t)=As(t) (1)

      where the column vectors x(t)= [x1(t),x2(t),x3(t)]T and the square 3 × 3 matrix A contains the mixture coefficients aij. The aim of ICA is to find a separating or demixing matrix W that is an approximation of the inverse of the original mixing matrix A whose output

      s(t)=Wx(t) (2)

      is an estimate of the vector s(t) containing the underlying source signals. According to the central limit theorem,a sum of independent random variables is more Gaussian than the original variables. Thus, to uncover the independent sources, we must maximize the nonGaussianity of each source. In practice, iterative methods are used to maximize or minimize a given cost function that measures non-Gaussianity such as kurtosis, negentropy or mutual information.

      D.HEART RATE EXTRACTION

      During the cardiac cycle, the change of volume in the facial blood vessels causes the subsequent changes in amount of reflected light. The RGB color sensors pick up the tiny changes and the changes indicate reflected plethysmographic signal. First, we separated each frame and the region of interest (ROI) was detected.The ROI was then separated into the three RGB channels and spatially averaged over all pixels in the ROI to yield a red, blue and green measurement point for each frame and form the raw traces 1(), 2()and

      3()respectively. We normalized the raw RGB traces as

      follows:

      detected face for extracting the cardiac pulse signal. The heart () = ()

      (3)

      rate was extracted using frequency analysis of raw trace

      signal.

      C.INDEPENDENT COMPONENT ANALYSIS

      In this study, the underlying source signal of interest is the cardiovascular pulse wave that propagates throughout the body. Volumetric changes in the facial blood vessels during the cardiac cycle modify the path length of the incident ambient light such that the subsequent changes in amount of reflected light indicate the timing of cardiovascular events. By recording a video of the facial region with a webcam, the RGB color signal along with other sources of fluctuations in light due to artifacts such as motion and changes in ambient lighting conditions. Given that hemoglobin absorptivity differs across the visible and near-infrared spectral range, each color sensor records a mixture of the original source signals with slightly different weights. These observed signals from the red, green and blue color sensors are denoted by x1(t),x2(t) and x3(t) respectively, which are amplitudes of the recorded signals (averages of all pixels in the facial region) at time point t. In conventional ICA the number of recoverable sources cannot exceed the number of observations, thus we assumed three underlying source signals, represented bys1 (t),s2(t) and s3(t). The ICA model assumes that the observed signals are linear mixtures of the sources, i.e. () =

      for each i=1,2,3 where i µand are the mean and standard deviation of () respectively. The normalization transforms

      (t) and ï() which is zero-men and has unit variance. The

      normalized raw traces are then decomposed into three independent source signals using ICA. In this report, we used

      the joint approximate diagonalization of eigenmatrices (JADE) algorithm developed by Cardoso. Typically the second component contained a strong plethysmographic signal. For the sake of simplicity and automation, we always selected the second component as the desired source signal.

      Finally, we applied the fast Fourier transform (FFT) on the selected source signal to obtain the power spectrum. The pulse frequency was designated as the frequency that corresponded to the highest power of the spectrum within an operational frequency band. Despite the application of ICA in our proposed methodology, the pulse frequency computation may occasionally be affected by noise. To address this issue, we utilize the historical estimations of the pulse frequency to reject artifacts by fixing a threshold for maximum change in pulse rate between successive measurements. If the difference between the current pulse rate estimation and the last computed value exceeded the threshold (we used a threshold of 12 bpm in our experiments), the algorithm rejected it and searched the operational frequency range for the frequency

      corresponding to the next highest power that met this constraint. If no frequency peaks that met the criteria were located, then the algorithm retained the current pulse frequency estimation

   2. EXPERIMENTAL RESULTS

      We did not observe plethysmographic information in either the red, green or blue raw traces, but from the three independent sources recovered by ICA. The cardiovascular pulse wave was clearly visible in the second component.

      Fig2 . Obtained result

      To illustrate the effect of ICA, we first evaluated the accuracy of heart rate measurements obtained directly from the raw traces by designating the pulse frequency as the frequency that corresponded to the highest power (within the operational frequency band) of the raw green trace spectrum (Fig. 2). The green channel trace was chosen because it reportedly contains the strongest plethysmographic signal among all three channels.

   3. CONCLUSION

We have described, implemented and evaluated a novel methodology for recovering the cardiac pulse rate from video recordings of the human face and demonstrated an implementation using a simple webcam with ambient daylight providing illumination. To our knowledge, this is the first demonstration of a low-cost method for non-contact heart rate measurements that is automated and motion- tolerant. Moreover, we have shown how this approach is easily scalable for simultaneous assessment of multiple people in front of a camera. Given the low cost and widespread availability of webcams, this technology is promising for extending and improving access to medical care. Although this paper only addressed the recovery of the

cardiac pulse rate, many other important physiologicalparameters such as respiratory rate, heart rate variability and arterial blood oxygen saturation can potentially be estimated using the proposed technique. Creating a real-time, multiparameter physiological measurement platform based on this technology will be the subject of future work.

REFERENCES

1. C.G. Scully, J. S. Lee, J. Meyer, A.M. Gorbach, D. Granquist-Fraser,

   Y. Mendelson, and K.H. Chon, Physiological Parameter Monitoring from Optical Recordings With a Mobile Phone, IEEE Transactions.,

   Biomedical Engineering, vol. 59, no. 2, p.303-306, 2012

2. E. Jonathan and L. Martin, Investigating a smartphone imaging unit for photoplethysmography, Physiological Meas., vol. 31, no. 11, p. N79, 2010.

3. M. Z. Poh, D. J. McDuff, and R. W. Picard, Non-contact, automated cardiac pulse measurements using video imaging and blind source separation, Opt. Expr., vol. 18, pp. 1076210774, May 2010

4. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, "Remote plethysmographic imaging using ambient light," Opt. Express, vol. 16, pp. 21434-21445, Dec. 2008

5. A. Hyvärinen, Independent component analysis: algorithms and applications, Neural Networks, vol. 13, pp. 411-430, June 2000

6. A. Hyvarinen, Survey on independent component analysis, Neural Comput. Surveys, vol. 2, pp. 94128, 1999

7. J. F. Cardoso, Blind signal separation: Statistical principles," Proc. of the IEEE, vol. 86, no. 10, pp. 2009-2025, Oct. 1998

8. V. M. Asunción, P. O. Hoyer and A. Hyvärinen, Equivalence of some common linear feature extraction techniques for appearance-based object recognition tasks, IEEE Trans Pattern Anal Mach Intell., vol. 29(5), pp. 896-900, May 2007