Performance Analysis of Various Kernelized SVM Based on HFD and Correlation Dimension During Partial Limb Movement Imagery

Performance Analysis of Various Kernelized SVM Based on HFD and Correlation Dimension During Partial Limb Movement Imagery

Somsirsa Chatterjee Puneet Mishra Chirag Pathania

Department of BME, Guru Jamb- heshwar University of Science and Technology, Hisar, Haryana

Department of BME, Guru Jamb- heshwar University of Science and Technology, Hisar, Haryana

Department of Distance Education, Guru Jambheshwar University of Science and Technology, Hisar, Haryana

Abstract

Neuroprosthetic devices controlled by EEG are being developed to cater the needs of those who are differently abled. These devices are also finding applications in improving their quality of life. As the EEG signal is frac- tal in nature, in this paper, we investigate the temporal changes in the fractal dimension and correlation dimen- sion corresponding to partial limb movement imagery, namely: finger movement imagery, distal limb movement Imagery, proximal limb movement imagery. The classifi- cation was done by various kernelized support vector machines. A novel Sugeno type fuzzy inference system was developed to evaluate the performance of the clas- sifiers. The complexity measures did vary with time, and had unique trends. This type of analysis would be helpful in calibration of the neuroprosthetic devices on the basis of imagery complexity which depend on the character of the sensorimotor rhythms of an individual for function- ing.

1. Introduction

   Applications of Chaos Theory and Nonlinear analyti- cal methods helped us to get a closer look into the brain dynamics with the help of EEG signals. This approach relies on a transition from the phase space and trajectory generation. Fractal dimension of the trajectories leading to strange attractors is the measure of the effective number of degrees of freedom in the chaotic dynamical system, thus, quantifying its complexity [8, 9, 10]. Feature extrac- tion is important as it affects classification and computa- tion speed considerably. Complexity measures have been successfully integrated with various domains to exemplify the innate complexity of the data with nonlinear proper- ties, though in this paper it is being used for the first time in partial limb movement imagery based BCI applica- tions. [4]

   Brain Computer Interfaces translates the brain activity using efficient pre-processing, feature extraction and clas- sification. Computer interfacing with brain can be broadly categorized into four classes:

   1. Acquisition of the neural activity;

   2. Imagery Extraction of the action pertaining to the particular activity involved;

   3. Implementation of the desired action with the help of the prosthetic effectors; and

   4. Feedback, either through intact sensation like vi- sion, or generated and applied by the prosthetic device [1, 2, 3, 4, 25].

      Fractal analysis allows investigation of relevant events which are relatively shorter than those which can be detected by means of other linear and non-linear tech- niques. Higuchis Fractal Dimension is an estimation of attractor dimension which is calculated in the time do- main as it becomes essential to retrieve maximum infor- mation from short time domain data.

2. Details Of The Experiment And Data Description

   Mostly investigators use C3, C4 and Cz of the 10-20 electrode system to analyze the complexity of brain sig- nals, since they are placed on the scalp over the motor cortex which is associated with voluntary control of movements. The mining of relevant features which de- scribes the discriminative signal properties still remains to be the primary step after the signal acquisition. Feature mining [11, 12, 23, 25] is the fundamental step for devel- oping an efficient interface. To improve the performance of the classifiers it is crucial to select relevant features associated with the imagery.

   This study is based on the recordings of the EEG sig- nal done using NeuroWin EEG Acquisition System de- signed by NASAN India Pvt. Ltd. Ag/AgCl electrodes were used as the brain interface, and the data was record- ed at a sampling frequency of 250 Hz, which was further

   filtered using an IIR bandpass filter between 0.01 Hz and 35 Hz. 3 channel electrodes: C3, Cz and C4 were selected and were placed according to the International10-20 Elec- trode placement system. The sensitivity of 100 µV is to be taken and an additional 50 Hz notch filter has been im- plemented to get rid of the line noise. The sampling fre- quency of the amplifier was set to 250 Hz, i.e., 250 bits of data were saved for each second. The data was also saved in ASCII format and for further processing of the data MATLAB was used.

   This experiment was designed to quantitatively ana- lyze the temporal evolution of the fractal nature of motor imagery through repetitive training of the same subject. The experiment was performed for 9 days on a male sub- ject of 21 years of age, who had no prior exposure to BCI related environment. The feedback session was designed where the subject relaxed on a chair with armrests. The duration of the experiment was 9 days, divided into three phases (each phase corresponding to a week). The inter- vals between the days were kept constant for all the phas- es. Each day the participant was asked to perform three sets of tasks. The instructions of the task were given to the participant through an audiovisual stimulus.

   Figure 2: Timing schematic of the stimulus used in the experiment.

   Therefore, each trial lasted for 7 seconds. Hence, the total data set for each session comprised of 1750 × 3 × 30 data. The training and testing data were chosen ran- domly to avert any systematic feedback. Sensorimotor rhythms occurring in the frequency band: 8-30 Hz was filtered using a digital IIR bandpass filter of order 14.

3. Feature Extraction

   1. Correlation Dimension

      Fractal nature may arise from the criticality of a self- organized system. Self-regulating complex systems of the human body are the primary generators of biopotentials. Linear spectral methods do not have the flexibility to detect cumulative phase properties of nonlinear signals which is its characteristic feature. These phase properties are generated from the coupling processes of different modes, which in turn lead to nonrandom phase structure.

      Correlation Dimension is one of those measures which help us determine the attractor dimension in the state space. The evaluation of Correlation Dimension incorpo- rates a long time trajectory in the state space, where points are laid on a spherical orbit of radius r, and then the Euc- lidean distance is calculated between each pair of points. Grassberger and Proccacia defined the correlation func- tion through the following equation [7]:

      = lim

      1

      Figure 1: Schematic of the Experimental Methodology.

      1

      ( )

      (1)

      Each session consisted of 30 trials. In the first ses- sion, the participant was asked to imagine self-paced fin- ger movements. The participant performed this task in one session. In the 2nd session the subject imagined self paced distal limb movements, and in the 3rd session motor im- agery of self paced proximal limb movement was per- formed. C3, C4, Cz electrodes were considered relevant for the study. The audiovisual stimuli started with a blank screen for 2 second, followed by an audiovisual cue (+ with a beep for 2 seconds) which signaled the subject to get ready for the cue about the Limb Type (Left/Right) which lasted for 3 seconds before reverting bck to the

      blank screen.

      where, is the Euclidean distance between the points of the spherical orbit and s is the unit step function. For a lot of attractors, this C(r) exhibits a power law dependence on r, as 0; that is

      lim =

      0

      (2)

      Based on this relationship, the following expression defines the Correlation Dimension:

      = lim ln ()

      0

      ln

      (3)

      The dimension d is always a fraction in the case of chaotic attractors.

   2. Higuchi Fractal Dimension

      Any curve which is fractal in nature can be subdivided into k similar curves (k = k1, k2 kmax). The length of this curve is proportional to k-D , where fractal dimension (D) determine the complexity of the curve. A simple curve will have D equal 1, while for a curve which nearly fills out the plane, D is close to 2. [4]

      Construction of phase space and data embedding are not necessary for calculating the fractal dimension by Higuchis method. [5, 6, 24] The algorithm computes a new time series based on a given finite time series:

      y = {y (1), y (2), y (N)}, by the following equation:

      = , + , , + .

      parable Support vector machines belong to the category of kernel based classification methods which possess some innate advantages, like:

      • Its ability to generate non-linear decision bounda- ries using linear classification methods.

      • The application of kernel functions which enables users to use it on the data which has no fixed di- mensional vector space representation.

        SVM training always tries to find the global minimum and its performance depends on the type of kernel se- lected, where the error penalty parameter is user-defined. [13, 14, 15] SVM gained popularity primarily because of its promising features like better empirical performance. The formulation based on Structural Risk Minimization (SRM) principle, was shown to be superior to Empirical Risk Minimization (ERM) principle used in conventional neural networks. SRM minimizes the upper bound on the

        for = 1, 2, . . . ,

        = 8

        (4)

        expected risk, while ERM minimized the error on the training data.

        For a binary classification problem, if the training data is labelled as xi, yi , i = 1, , l, yi +1 , 1 , xi Rd . Suppose there is a hyperplane which separates the two

        Both m and k are integers which indicate the initial time and the time interval respectively. The length, () of each curve is calculated as

        classes (the separating hyperplane). The point x which lie on the hyperplane satisfies the equation . + = 0, where is normal to the hyperplane. is the per- pendicular distance between the hyperplane to the origin,

        1

        and is the Euclidean norm of . Let, +& be the

        = +

        =1

        1

        shortest distance from the hyperplane to the nearest posi- tive or negative examples respectively. The margin of the

        + 1

        1. The Fractal Dimension thus is computed by:

           generated hyperplane would be defined as + + . The primary aim of any type of SVM is to find the hyperplane with the largest margin. If we assume that all training data satisfy the following constraints:

           . + +1, = +1 (7)

           =

           log2

           ()

           . + 1, = 1 (8)

           log2

4. Feature Classification Using Kernelized Support Vector Machines

   These equations can be combined to form the resulting equation as follows:

   . + 1 0, (9) If the vectors are distributed non-linearly, then it be-

   comes essential to use a kernel function to map the data into a higher dimensional hyperspace wherein a multi-

   The primary idea revolves around the formation of an

   optimal hyperplane which perfectly separates multi- dimensional data into binary or multiple classes. In some cases the data is not easily separable using linear methods, in this situation SVM plays a significant role by introduc- ing the concept of kernel induced feature space. In this concept, the data to be analyzed or classified is extended to a higher dimensional space which makes it easily se-

   dimensional hyperplane can be used to segregate the data. Kernel functions correspond to an inner product in some expanded hyperspace. Mercers Theorem states that every semi-positive definite symmetric function is a kernel. The dot product , , represents Gram Matrix (a matrix of dot products in the Euclidean space). Prior to this process, each data point is mapped into the higher

   dimensional hyperspace via some transformation

   : . For kernalized SVM, the differentiating function would be of the form

   = (. () + ) (10)

   Kernel for the dot product in the higher dimensional feature space would be

   , = . () (11) The polynomial kernel function is given by:

   , = . + (12)

   Four membership functions were defined correspond- ing to a level of accuracy achieved (individual or aver- aged):

   1. Poor: Sigmoid function with parameters [-240 0.80]

   2. Average: Gaussian function with parameters [0.05 0.85]

   3. Good: Sigmoid function with parameters [240 0.90]

   4. Excellent: Sigmoid function with parameters [240 0.98]

   The rule base of the FIS consisted of six rules:

   i.

   where, is a tunable parameter. Firstly, a linear kernel

   , = . + (13) Secondly, a quadratic kernel

   , = . + 2 (14)

   Finally, a polynomial kernel of order 3 (Cubic kernel)

   ii.

   iii.

   iv.

   , = .

   + 3 (15)

   In this paper we compare the performance of the po- lynomial kernel in classification of Left/Right limb using partial limb movement imagery (i.e. finger, distal portion of the limb, proximal portion of the limb). In order to find the tendency of the complexity pattern we tested the data recorded in the final day (9th day of the experiment) with all the previous recordings we had obtained in the earlier 8 days of the experiment. The SVMs were trained dis-

   v.

   =

   vi.

   =

   The output level for each rule is weighted by the fir- ing strength of the rule, which is given by:

   tinctly with the data obtained previous 8 days pertaining

   to each session, with the aim of testing the data obtained

   =

   1

   ,

   2

   (16)

   in the 9th day. The resulting performance of the classifier would give us the probability of correct classification for those particular sessions while the classifier is trained using the data obtained at a prior date. [24]

   where, represents probabilistic OR and 1 and 2 are the membership functions

   of 1and 2 respectively. Final output is the weighted

   =1

   average of all the outputs, computed as

   = 1

   (17)

   4.1. Performance of the classifier

   The performance of a classifier is generally measured

   =1

   =1

   by its mean probability of corect classification over all trials. However, it may not be the actual measure of its accuracy as the accuracy varies with the training set. In this paper we present a novel Sugeno-type fuzzy inference system (SFIS) which measures the performance not only by averaging its performance over all trials but also takes into consideration, the performance of the classifier for each individual trial.

   For any two inputs, 1 2 the Sugeno-type FIS returns an output: = 1 + 2 + .

   , =

5. Experimental Results And Analysis

   The following table shows the changes of correlation dimension and fractal dimension at the beginning and end of each phase of the experiment:

   TABLE 1: Categorization of Correlation Dimension measures at the beginning and end of each phase per session.

   TABLE 2: Categorization of Fractal Dimension measures at the beginning and end of each phase of every session.

   Classification Performance Analysis

   Fig. 3: Fluctuations of Classification accuracy of polynomial SVM during temporal evolution based on Correlation

   Dimension Measures

   Figure 4: Fluctuations of Classification performance of polynomial SVM during temporal evolution based on Fractal

   Dimension measures.

   All the classifiers were consistently accurate with 97% accuracy except for one case where the classifier accuracy was 96.81% (training by 1st day 1st session using correla-

   tion dimension feature). The classification of each session took less than 1 second to compute. The linear kernel had the fastest computation as compared to the quadratic and

   the cubic kernel with a mean of 1.5121 s, and a standard deviation of 0.1091.

6. Discussion

   We know that as we do mental practice of new movement forms, new sensorimotor programs are gener- ated, or the older movement related sensorimotor pro- grams get modified. From this experiment it is visible that as the mental practice for the imagery is repeated the new imagery forms a linear or polynomial relationship with the older imagery [24]. The classification performance of the Finger movement imagery using fractal dimension shows that the imagery of the final day of phase 3 (Index:

   9) forms a linear relationship with the imagery taken the previous day (Index 8), while it forms a polynomial rela- tionship with the imagery taken in the first day (Index 1). During the experiment, it was seen that the fractal dimen- sion of the C3 electrode was always distinctly higher than that of C4 in each day of the experiment, thereby confirm- ing that the contra-lateral portion of the brain is activated during the imagination of the bodily actions. It can also be seen that the dimension of the chaotic attractor of the EEG sequence observed on the final day is related to its preceding day imagery more accurately by a polynomial function while it relates to the attractor of an imagery tak- en two weeks prior by a quadratic function.

   Distal Limb Movement Imagery classification analy- sis portrays the fact that during the intermediate stages of the experiment, both the fractal dimension and the corre- lation dimension have a cubic relationship with the data obtained in the final day of the experiment. It may be a result of the modifications of the central sensorimotor programs corresponding to the distal limb movement im- agery.

   The proximal limb movement imagery classification analysis also showed distinct patterns which manifest due to the temporal evolution of the Fractal dimension and Correlation dimension parameters. The data obtained from these sessions showed a unique trend of having a cubic relationship with the data obtained in the recent past, while it has a linear relationship with the data ob- tained in the distant past.

7. Conclusion

   This type of analysis using the temporal evolution of the fractional dimension parameters would help us in ca- librating the control signals of the neuroprosthetic arm. Moreover, the performance analysis using the Sugeno- type FIS would not only provide us with the classification accuracy averaged over time but would also help us to evaluate the performance on the basis of the fluctuations that are a result of the individual trial characteristic. The computation time of the linear kernel SVM was compara- tively lower than that of the quadratic and cubic kernel SVM.

8. References

1. A.B. Schwartz, X.T. Cui, D.J. Weber, D.W. Moran, Brain Controlled Interfaces: Movement Restoration using Neural Pros- thetics., Neuron, Vol.52, October 2006, pp. 205-220

2. M.A. Lebedev, Nicoleis, Brain-machine interface: Past, present and future, Trends Neuroscience, Vol. 29(9), Septem- ber 2006, pp.536-546

3. R.A. Anderson, S. Musallam, B. Pesaran, Selecting the signals for a brain-machine interface, Curr Opin Neurobiology,

   Vol.14 (6), December 2004, pp.720-726

4. U Guclu, Y Gucluturk, C.K. Loo, Evaluation of fractal dimension estimation methods for feature extraction in motor imagery based brain computer interface, Procedia Computer Science (3), 2011, pp 589-594

5. C.K. Loo, A. Samraj, G.C. Lee, Evaluation of Methods for estimating fractal dimension in motor imagery based Brain Computer Interface, Discrete dynamics in Nature and Society, 2011, Article ID: 724697.

6. J. Theiler, Estimating Fractal Dimension, 7 (6), J. Opt.

   Soc. Am. A, June 1990, pp 1055-1073

7. P. Grassberger, I. Procassia, Measuring the strangeness of strange attractors, Physica D 9, 1983, pp 404-424.

8. W. Klonowski, J. Ciszewski, W. Jernajczyk and K. Nied- zielska, Application of Chaos Theory and Fractal Analysis for EEG-signal processing in patients with Seasonal Affective Dis- order, Proceedings of 1999 International Symposium on Nonli- near Theory and its Applications (NOLTA'99), Waikoloa, HA, U.S.A., pp 339-342.

9. M. Roopei, R. Boostani, R.S. Rohani, M.A. Taghavi, A. Azimifar, Chaotic based reconstructed phase space features for detecting ventricular fibrillation, Biomedical Signal Processing and Control (5), 2010, pp 318-327

10. M. Phothisonothai, M. Nakagawa, Fractal based EEG data analysis of Body parts movement imagery tasks, J. Physiol.

    Sci. Vol. 57, No. 4; Aug. 2007; pp. 217226

11. R. Boostani, B. Graimann, M.H. Moradi, G. Pfurtscheller, A comparison approach toward finding the best feature and classifier in cue-based BCI, Med. Biol. Eng. Comput. 45 (2007) 403412.

12. W.S. Noble, What is a support vector machine, Nature Biotechnology, 24(12), 2006, pp 1564-1567

13. B. Scholkopf, C.J. Burges, A.J. Smola, Advances in Ker- nel Methods: Support Vector Learning, MIT Press, 1999

14. C.W. Hsu, C.C. Chang, C.J. Lin, A practical Guide to Support Vector Machine, Technical Report, Department of Computer Science, National Taiwan University, 2010

15. A. Bordes, S. Ertekin, J. Weston, L. Bottou, Fast kernel classifiers with online and active learning, Journal of Machine Learning Research, September 2005, pp 1579- 1619.

16. C. Vidaurre, N. Kramer, B. Blankertz, A. Schlogl, Time Domain Parameters as a feature for EEG-based Brain Computer Interface, Neural Networks 22, 2009, 1313-1319

17. B. Mahmoudi, A. Erfanian, Single channel EEG based prosthetic hand grasp control for amputee subjects. Proceedings of the Second Joint EMBS/BMES Conference, Houston, Texas,

    USA, October 2002

18. A. Schloegl, Dynamic spectral analysis based on an auto- regressive model with time-varying coefficients, IEEE-EMBC and CMBEC, 1995

19. S. Darvishi, A. Al-Ani, Brain-computer interface analysis using continuous wavelet transform and adaptive neuro-fuzzy

    classifier, Proc. 29th Int. Annu. Conf. IEEE Eng. Med. Biol.

    Soc., August 2007, pp. 3220-3223

20. Q. Xu, H. Zhou, Y. Wang, J. Huang, Fuzzy support vector machine for classificationof EEG signals using wavelet based features. Medical Engineering & Physics 31, 2009, pp. 858-865

21. P. Herman, G. Prasad, T.M. McGinnity, D. Coyle, Com- parative analysis of spectral approaches to feature extraction for EEG-based motor imagery classification. IEEE Trans. Neural sys. Rehab eng. 16(4), August 2008, pp. 317-326

22. S. Bhattacharyya, A. Khasnobish, S. Chatterjee, A. Konar,

    D.N. Tibarewala, Performance analysis of LDA, QDA and KNN algorithms in left-right limb movement classification from EEG data, International conference on systems in medicine and biology, IIT Kharagpur, 2010, pp 126-131.

23. A. Khasnobish, S. Bhattacharyya, A. Konar, D.N. Tibare- wala, K Nearest neighbor classification of left-right limb movement using EEG data., Proceedings of International Con- ference on Biomedical Engineering and Assistive Technologies, NIT Jalandhar, 2010.

24. S. Chatterjee, S. Bhattacharyya, A. Khasnobish, A. Konar,

    D.N. Tibarewala & R. Janarthanan, Study of Inter-Session Variability of Long Term Memory and Complexity of EEG Signal, Proceedings of Third International Conference on Emerging Trends of Information Technology, Indian Statistical Institute, Kolkata, November 30 December 01, 2012, pp 106- 109.

25. S. Bhattacharyya, A. Khasnobish, A. Konar, D.N. Tibare- wala & A.K. Nagar, Performance analysis of Left/Right Hand movement classification from EEG signal by intelligent algo- rithms, IEEE Symposium on Computational Intelligence, & CAMB, 2011.