Real-Time Embedded Classifiers for Human Activity Recognition

Real-Time Embedded Classifiers for Human Activity Recognition

Real-Time Embedded Classifiers for Human Activity Recognition

Prajakta Kishore Kalangutkar

Department of Electronics

K.C.College of Engineering & Management Studies & Research

Kopri Thane (E) kalangutkar.prajakta@gmail.com

Hansraj Guhilot

Principal

1. ollege of Engineering & Management Studies & Research

   Kopri Thane (E) hansraj.g@gmail.com

   Abstract With the advent of computer systems, it is now possible to develop a real-time computing system for the recognition of human activities which will aid the doctors and care-givers to keep a track of the patients day to day activities and responses who are required to follow a well defined exercise routine, in diseases such as diabetes, obesity or heart diseases. This paper discusses a real-time implementation of an embedded classifier that aims at recognizing the various human activities by recording the heart rate variability of the subject. Certain novel modifications to the previously published model are implemented in Arduino and MATLAB and better accuracy is achieved.

   Keywordsreal-time; activity recognition; heart rate variability

   1. INTRODUCTION

      The first work on recognition of human activities was carried out in late 90s, however the technology for recognition of human activities needs improvement, as some of the aspects are yet to be resolved completely, that include; (i) collection of the data set; (ii) implementation of feature extraction and real time classification algorithms; (iii) construction of a small, quick and portable system; (iv) collection of real-time data with minimum delay.

      As per the studies related to the functioning of the heart, it has been observed that, the period of heartbeat is not constant and changes over time with various human activities. Variations in heart rate and their respective periods are called heart rate variability (HRV). The HRV obtained from long- term and short-term interval has been widely used to analyze the autonomic functions in patients with neurological dysfunction, cardiovascular diseases, diabetes, etc. Thus incorporating more detailed information of the heart rate variability would be beneficial for interpreting the role of physical activities.

      Along with the advances in computing systems, the advances in serial communication also helps in easy and quick transfer of the data from the sensor in real-time, using Arduino and MATLAB to process the data and implement the classifier algorithms.

   2. LITERATURE SURVEY

      transform to retrieve time-varying characteristics of heart rate variability under different physical activities. Their work showed that the heartbeat fluctuations in higher frequencies were greatest during lying and smallest during standing.

      Mohammed Fikri Azli Bin Abdullah, et al., cited in their paper that selection of classifier algorithm mainly depends on the capability of the processing platform to execute the algorithm [2].

      Dominic Magrurie and Richard Frisby compared two classification algorithm used in activity recognition [3]. The various features were extracted using a time window of 3.4 seconds to analyze the response of each classifier algorithm.

      Al-Busaidi, A.M. described in his paper [4] the development of innovative low-cost educational platform to study and control a biped robot in real-time using MATLAB and Arduino board. MATLAB was used as a control and visualization environment, while the Arduino board was utilized as a final controller for the servo motors as well as a Data Acquisition Card (DAC). The paper claims that it was important to emphasize that it merely focused on the hardware and software development which can be utilized to accomplish many tasks like controlling the walking motion.

      O´scar D. Lara and Miguel A. Labrador claimed that despite human activity recognition (HAR) being an active field for more than a decade, there are still key aspects that, if addressed, would constitute a significant turn in the way people interact with mobile devices. The paper surveys the state of the art in HAR based on wearable sensors [5]. A general architecture is first presented along with a description of the main components of any HAR system. The paper also proposed a two level taxonomy in accordance to the learning approach (either supervised or semi-supervised) and the response time (either offline or online). In the survey, the HAR systems have been characterized that rely on wearable sensors in two levels.

   3. PROPOSED IMPLEMENTATION

      The proposed implementation model mainly consists of three blocks as shown in Fig. 1.

      Hsiao-Lung Chan, et al. [1] observed that the period of heartbeat is not constant and changes over time. Variations in heart rates and their respective periods are called heart rate

      Data Acquisition

      Feature Extractio

      Classifi- er Alg.

      variability (HRV). The paper incorporates discrete wavelet Fig.1. Proposed Implementation Model

      Data Acquisition: It consists of acquiring the heart rate data in real time, using a pulse sensor from a particular set of people of varying ages who are asked to perform some activities.

      Feature Extraction: It consists of extracting the Heart Rate Variability (HRV) and Beats per Minute (BPM). Further various features such as mean, standard deviation, etc are extracted t feed the classifier algorithms.

      Classifier Algorithms: The algorithms processes the various features extracted in order to classify the activities.

      Figure 2 shows the detailed implementation model.

      1. Data Acquisition

         The data set is collected in real-time using a Pulse Sensor Amped by Joel Murphy and Yury Gitman for Arduino [14] which is essentially a photoplethysmograph, used for non- invasive heart rate monitoring. The data from this pulse sensor is extracted using Arduino [17], which is a single-board microcontroller and can be graphically visualized using Processing [18], an open source programming language and integrated development environment (IDE) built for the electronic arts, new media art, and visual design communities with the purpose of teaching the fundamentals of computer programming in a visual context, and to serve as the foundation for electronic sketchbooks [19]. The data from the Arduino Microcontroller can be further transferred to MATLAB via serial communication [20], as it has an interactive development and debugging capabilities.

      2. Feature Extraction

         The main features to be extracted are the Heart Rate Variability (HRV) and the Beats per minute (BPM). This is done using the pulse sensor and the principle given in [21]. The heart pulse signal that comes out of a photoplethysmograph is an analog fluctuation in voltage, and it has a predictable wave shape [21]. The depiction of the pulse wave is called a photoplethysmogram, or PPG. Arduino amplifies the raw signal from Pulse Sensor, and normalizes the pulse wave around V/2 (midpoint in voltage). Pulse Sensor and Arduino responds to relative changes in light intensity. If the amount of light incident on the sensor remains constant, the signal value will remain at 512 (midpoint of ADC range). Signal is directly proportional to the light level. Light from the LED, that is reflected back to the sensor changes during each pulse. The goal is to find successive moments of instantaneous heart beat and measure the time between, called the Inter Beat Interval (IBI). Once the pulse is detected and the heart rate variability (HRV) and the beats per minute (BPM) are caculated, the data is transferred serially to MTALAB [16].

         Once the data is obtained in MATLAB, the following features are extracted to train the classifier algorithm.

         • Mean represents DC component of the signal over a window time frame;

         • Standard Deviation discriminates the similar data value for differing activities;

           DATA ACQUISITION

           FEATURE EXTRACTION

           ARDUINO HRV

           BPM

           SERIAL COMM

           MATLAB

           MEAN,SD

           ENERGY, CORR.

           CLASSIFIER ALGORITHM

           RECOGNIZED ACTIVITY

           Fig.2. Block Diagram of Implementation Model

         • Energy gives the measure of intensity of movement of squared discrete FFT magnitude;

         • Correlation differentiated the activities that involve transition.

      3. Classifier Algorithm

      A number of classifier algorithms have been already proposed to classify human activities based on features extracted, although in this paper an attempt has been made to classify these activities empirically by recording the HRV of five subjects of varying ages, performing two different activities in real-time. The real time HRV data which is transferred serially to MATLAB is passed through a classifier function to recognize the activity. It has been noted that with varying ages, there is a variation in the HRV. Thus age factor is taken as one of the inputs to the classifier algorithm in order to achieve better accuracy than the previously implemented algorithm. Figure 3 shows the flow of implemented classifier algorithm.

Initialize Serial Communication

Is port

available

?

NO YES

Read HRV & BPM values and display in MATLAB

User Input: Enter age of the subject

Call the Classifier function

Compute the feature extracted parameters

Fig.4. Processing Output for HRV & BPM Values

algorithm. Figure 5 shows a snapshot of gathering data into PLX-DAQ from Arduino. Table 1 shows the average HRV of 5 subjects, with the age factor varying between 20 and 30, for two different activities. Table 2 shows the average HRV of 5 subjects, with the age factor varying between 50 and 60, for two different activities

Display Recognized Activity

Is value Unstable

?

YES

NO

EXIT

Fig.3. Flow of Classifier Algorithm

1. RESULTS

   The HRV values obtained on the serial monitor of the Arduino can be graphically visualized using Processing software, as shown in figure 4. Since the data is real-time, the data is stored into an excel sheet, using the PLX-DAQ in order to train the

   Fig.5. Obtaining data from Arduino & Storing in Excel using PLX-DAQ

   TABLE I. AVERAGE HRV RECORDED FOR 5 SUBJECTS WITH AGE GROUP BETWEEN 20 & 30

   	Sitting	Standing
   Subject 1	708	635
   Subject 2	706	620
   Subject 3	755	625
   Subject 4	746	607
   Subject 5	756	605

   TABLE II. AVERAGE HRV RECORDED FOR 5 SUBJECTS WITH AGE GROUP BETWEEN 50 & 60

   	Sitting	Standing
   Subject 1	885	782
   Subject 2	892	780
   Subject 3	905	725
   Subject 4	891	740
   Subject 5	902	722

   With the help of this data, the classifier algorithm was trained giving a result with an accuracy of around 88% as the algorithm is based on empirical logic. Moreover, the consideration of age factor helps in obtaining a better accuracy.

   Figures 6 and 7 show respectively, the snapshots of output result for recognized activity and the vales of the features extracted.

   Table 3 shows the various feature extraction parameters and the accuracy obtained from the real time data.

   Fig.7. Feature Extraction parameters of the real-time data

   TABLE III. VARIOUS PARAMETERS & %AGE ACCURACY OBTAINED IN REAL-TIME

   Parameters	Output values
   Mean	892.3
   Std. Deviation	93
   Correlation Coefficient	0.053
   Energy	3.2
   Accuracy	88%

2. FUTURE SCOPE

As the incoming signal is superimposed with various kinds of disturbances, we would suggest implementation of various transformation techniques, such as the wavelet transform, in order to process the incoming signal in real-time and achieve a higher percentage of accuracy.

Further, the idea can also be implemented on Field Programmable Gate Array (FPGA), to make the entire system stand-alone.

ACKNOWLEGMENT

We are grateful to all those who have contributed in making this idea a success with their continuous guidance and patience.

ACKNOWLEDGMENT (Heading 5)

REFERENCES

Fig.6. Recognition of Activities from the Real Time data obtained, when the subject is Standing

1. Hsiao-Lung Chan, Shih-Chin Fang, Yu-Lin Ko, Hui-Hsun Huang, and Chun-Hsien Lin, Heart rate variability characterization in daily physical activities using wavelet analysis and multilayer fuzzy activity clustering IEEE Transactions on Biomedical Engineering

   ,Jan.2006,vol.53,no.1,pp.133-139

2. Mohd Fikri Azli bin Abdullah, Ali Fahmi Perwira Negara, Md. Shohel Sayeed, Deok-Jai Choi, Kalaiarasi Sonai Muthu, Classification algorithms in human activity recognition using smartphones World Academy of Science, Engineering and Technology 68, 2012, pp.422-429

3. Emmanuel Munguia Tapia, Stephen S. Intille,William Haskell, Kent Larson, Julie Wright, Abby King, and Robert Friedman Real-Time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor.

4. O´ scar D. Lara and Miguel A. Labrador Department of Computer Science and Engineering University of South Florida, Tampa, FL 33620 A survey on human activity recognition using wearable sensors.

5. David Naranjo-Hern´andez, Laura M. Roa, Javier Reina- Tosinaand Miguel A´ ngel Estudillo-Valderrama, SoM: A smart sensor for human activity monitoring and assisted healthy ageing IEEE transactions on biomedical engineering, Nov.2012, vol. 59, no. 11, pp.3177-3183.

6. Maguire, D., Frisby, R. Comparison of feature classification algorithm for activity recognition based on accelerometer and heart rate data, 9th. IT & T Conference, Dublin Institute of Technology, Dublin, Ireland, 22nd.-23rd. October, 2009.

7. Wansuree Massagram, Noah Hafner, Mingqi Cehn, Luca Macchiarulo, Victor M.Lubecke, and Olga Boric-Lubecke, Digital heart-rate variability parameter monitoring and assessment ASIC, IEEE Transactions on Biomedical Circuits and Systems, Feb.2010, vol.4, no.1,pp. 19-26.

8. Liming Chen, Chris D. Nugent, Hui Wang, "A knowledge- driven approach to activity recognition in smart homes," IEEE Transactions on Knowledge and Data Engineering, June 2012, vol. 24, no. 6, pp. 961-974.

9. Dipali Bansal, Munna Khan, A.K. Salhan, "A review of measurement and analysis of heart rate variability," iccae, pp.243-246, International Conference on Computer and Automation Engineering (iccae 2009), 2009.

10. Al-Busaidi, A.M. ; Dept. of Mech. & Ind. Eng., Sultan Qaboos Univ., Muscat, Oman, Development of an educationalenvironment for online control of a biped robot using MATLAB and Arduino, Published in Mechatronics (MECATRONICS) , 2012 9th France-Japan & 7th Europe-Asia Congress on and Research and Education in Mechatronics (REM), 2012.

11. Allen Y. Yang, Data Collection Protocol for Wearable Action Recognition Database (WARD).

12. Heart Rate Variability Analysis Scientific Background- http://www.biocomtech.com/hrvscientific, visited on 19/5/2013.

13. Heart Rate Variability Analyzer – http://zone.ni.com/reference/en-XX/help/373698A- 01/bioapps/hrv_analyzer/, visited on 19/5/2013.

14. Pulse Sensor – http://www.pulsesensor.com/, visited on 29/4/2013.

15. Introduction to Data Acquisition and Processing – http://www.seas.upenn.edu/~ese111/Fall2012labs/Lab5.pdf, visited on 1/5/2013.

16. Arduino to MATLAB read in sensor data http://robotgrrl.com/blog/2010/01/15/arduino-to-matlab-read-in- sensor-data/, visited on 9/5/2013.

17. Download software / Purchase Arduino – www.arduino.cc/, visited on 11/4/2013.

18. Download Processing with examples – http://www.processing.org/, visited on 29/4/2013.

19. Processing (Programming language) from Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki/Processing_%28programming_lang uage%29, visited on 30/4/2013.

20. Serial Communication between Arduino and MATLAB – http://home.iitb.ac.in/~rahul./ITSP/serial_comm_matlab.pdf, visited on 1/5/2013.

21. Pulse Sensor Amped Code & Guide – http://pulsesensor.myshopify.com/pages/pulse-sensor-amped- arduino-v1dot1, visited on 6/5/2013.

22. PLX-DAQ Data Acquisition for Excel – http://www.parallax.com/ProductInfo/Microcontrollers/PLXDA QDataAcquisitiontool/tabid/393/Default.aspx, visited on 20/6/2013.

23. Prajakta Kishore Kalangutkar, Santosh Saraf, Sheetalkumar Mallikarjun Onkari, Human Activities – Their Classification, Recognition & Ensemble of Classifiers, International Journal of Computer Applications,August 2013, vol.76-No.14,pp.0975- 8887.