Classification of Emotions From EEG Signals using Machine Learning Algorithms

Classification of Emotions From EEG Signals using Machine Learning Algorithms

Mrs. Nilkamal More

Department of Information Technology K.J.Somaiya College of Engineering Mumbai

Ameya Tathavadkar

Department of Information Technology K.J.Somaiya College of Engineering Mumbai

Meet Shah

Department of Information Technology K.J.Somaiya College of Engineering Mumbai

Varun Parmar

Tanish Shah

Department of Information Technology K.J.Somaiya College of Engineering Mumbai

Department of Information Technology K.J.Somaiya College of Engineering Mumbai

Abstract Emotion recognition with the help of computers has increased its popularity in recent times.Our project mainly focuses on recognizing a particular emotion from human brain activity which is measured by electroencephalogram (EEG) signals.In this project the type of classifier we are using is Support Vector Machine(SVM) and Random Forest(RF). We have used DEAP dataset which is preprocessed to eliminate noise from EEG signal.The following EEG signal is decomposed into 5 bandwidths for feature extraction.Following to this Principal Component analysis is done to find the standard deviation,mean,variance to find the proximity of the EEG signal to one of its classifier.Depending upon its classification emotion is detected.

Keywords:-Emotion Recognition, electroencephalogram (EEG), Wavelet Decomposition, SVM,ANN.

I. INTRODUCTION

Emotions act as a basic medium to express feelings and have a major effect on the social skills of a person and how they perceive the world. Human emotion can be detected by a plethora of factors like facial-expressions, body-language and physiological signals.But these methods are superfluous when our main aim is to detect the inner emotion of that person.For example Lets take an example of the person A, he was happy while viewing his favourite watch but he was not happy with its price. In this case, the person A will not show his sorrow because of the price on his face but this problem can be overcome by Emotion detection using EEG device with the help of Brain computer Interface (BCI). Thus proving that emotion detection using EEG signals is advantageous than other methods.The Brain computer Interface is responsible for reading the brain waves of a person and also acts as a sole connection between computer and human brain so there is no chance of recording a false emotion.

There are so many problems based on emotion:-

There are so many disabled people who find it difficult to express their emotion.It can act as a feedback for any service or product which can later be analyzed for the development of the product.People who are mentally unstable and they try to hide their actual emotion.These problems that were mentioned can be solved with the help of Brain-Computer Interface.EEG has the rapid response time and are inexpensive than other methods.In this study, the DEAP Dataset of 32 participants is used to evaluate the classifier to obtain their respective result in finding the emotion of the people.Electroencephalogram(EEG) is the recording of the electrical activity of the human brain in terms of current. It is a noninvasive technique which is used on the skull of a human to receive electrical impulse produced from neuron activation in the brain. For the following type of study to be conducted, EEG electrodes are placed on the specific region of the scalp. EEG electrodes are capable of measuring electrical signal from the human brain in the range of 1 to 100 microvolt ( V).

The EEG signal has a range of 4.0 – 45.0 Hz. Further EEG signals can be categorized to bands of different frequency ranges named as alpha, beta, theta, delta,and gamma as shown in the table Fig 1. In the following paper we have used 32 channels s01, s02, s03, s04, s05, s06, s07, s08, s09, s10, s11, s12, s13, s14, s15, s16, s17, s18, s19, s20,s21, s22, s23, s24, s25, s26, s27, s28, s29, s30, s31,s32.

The method For Principal Component Analysis (PCA) used in this is Standard Deviation, Mean, Variance. The classifier used to detect the emotion with great accuracy is Support vector machine(SVM) and Random Forest.This paper is organized in such a manner that Section 2 gives the literature survey of other studies which is followed by Section 3 which tells the system design & component of design. Section 4 gives the evaluation & error analysis and Section 5 gives the conclusion & future study.

1. SYSTEM DESIGN

   II. LITERATURE SURVEY

   As discussed in [3], "Electroencephalogram (EEG) is one of the most reliable physiological signals used for detecting the emotional states of human brain. In this work the decomposition and automatic classification is achieved for two human emotions obtained from Electroencephalograph signals. Emotion is a natural communication in human life, the emotional state of a person defines their interaction with other people or objects. The EEG signals are most widely used in the medical field to analyse the patient condition, because it contains more information about human brain task. The subjects were described for different emotions such as smile and anger. The EEG signals are recorded by using ENOBIO 8 device, and pre-processing of the signal is done by designing the FIR filter. For the feature extraction the multi-wavelet transform is observed and for classification purpose KNN classifier, SVM classifiers are observed. In multi-wavelet Transform method we extract 8 statistical parameters for each sample."[3] As mentioned in [5],"Emotion recognition from EEG signals is a subject of interest for both engineers as well as psychologists .In Brain Computer Interface systems, EEG based classification and analysis of human emotions is a latest and difficult field that has gained momentum in the past few years. In the recent work, human emotion recognition was carried out using Multi- Wavelet Transform."[5]

   As stated in [8],"Psychological changes in humans are the result of emotions which occur due to activities in daily life. To understand these changes in behavioral pattern, research on a active computing has emerged. Emotions are an integral part of our daily lives, based on which in this paper an investigation have been made to analyze the impact of positive and negative emotions using Electroencephalogram (EEG). Three classes of emotions namely calm, anger and happiness have been studied. The EEG signals are recorded in real time from 10 subjects while watching different emotions video clips of 2 minutes each. Next, the fractal dimension feature has been extracted from raw EEG. To further detect emotional states, the extracted features have been classified using Support Vector Machine (SVM) with radial basis function (RBF) kernel with an average accuracy of 60%. The proposed methodology shows that emotions recognition is possible from EEG signals."[8]

   Fig. 1. System Design of EEG Signal Analysis

   DEAP is EEG emotion signal dataset that is available online.The dataset used in this project was unprocessed as it may contain some disturbances in form of noise.The dataset has to be cleaned from all the artifacts using a bandpass filter of 4-45 Hz. The obtained signals are then downsampled to 128 Hz. It is necessary to clean the signal to increase the Signal to Noise ratio. If Signal to noise ratio is good the the signal is pellucid and can be easily analyzed to detect the emotion.

   This clean signal will have many frequency component. So, it is necessary to divide the signal into particular frequency band to increase the understanding of the EEG signal. The downsampling of Signalis performed using the wavelet decomposition or by connecting a filter bank. Wavelet decomposition is used to divide the EEG signal into 5 frequency band respectively as shown in Fig 1. After we obtain the frequency band for each channel then it is necessary to analyse each frequency band of respective channel. The output acquired from the decomposer is passed to PCA block which in turn analyse the Band frequency and produce the feature vector for each frequency band. This process is performed for all the channel and a feature vector is obtained for each analysis. Now, its time to classify the feature vector. So, machine learning algorithm are used to classify the feature vector. The feature vector along with label is passed to the classifier during training and then the efficiency is checked by varying the dataset size or PCA technique.

   A. Dataset

   "The Deap dataset comprises of two parts:

   An online self-assesment test was conducted where 120 one-minute music video clips were shown to

   14-16 volunteers and each clip was rated based on 4 factors namely, valence,arousal,dominance and liking.

   The member evaluations, physiological accounts and face video of a trial where 32 volunteers viewed a subset of 40 of the above music videos.EEG and physiological signs were recorded and every member likewise appraised the recordings as above.

   The preferred spelling of the word acknowledgment in America is without an e after the g. Avoid the stilted expression one of us (R. B. G.) thanks …. Instead, try R. B.

   G. thanks…. Put sponsor acknowledgments in the unnumbered footnote on the first page.Online Ratings is the arrangement of all the individual video evaluations that were gathered during the online self-appraisal. The evaluations were gathered utilizing a tool. Members evaluated valence, arousal,liking and dominance utilizing Self-appraisal mannequins (SAM) on a discrete 9-point scale.

   Member Ratings is the arrangement of all the member video appraisals that were gathered during the experiment. . Arousal, valence, liking, and dominance were appraised straightforwardly after every preliminary on a constant 9-point scale utilizing a standard scale. Self-appraisal mannequins were utilized to envision the evaluations for arousal, valence, and dominance. For liking (i.e.Did you truly like the video?),emoticons, for example, approval or disapproval were used.Proximity was evaluated after the finish of the analysis which was mapped on a 5-point number scale (from "tuning in just because" to "hear it out on regular schedule"). The Proposed dataset is down tested to 128 Hz. Electrooculography (EOG) antiquities are disposed of with the assistance of Band pass channel. The EEG channels were reordered with the goal that they all follow the Geneva order as above. The information was fragmented into 60-second preliminaries and a 3-second pre-trial baseline removed."[7]

   In this assignment, labels are extracted into separate file and data of each channel is extracted into separate file. data from each channel is stored in row wise versus time in column for each trail,per person.

   1. Pre-processing

      EEG is a very sensitive signal, It is highly susceptible to any kind of noise.Because of this dirty EEG signal which is blended with noise it is very difficult to analyze the EEG signal.So,there are many steps to reduce or remove the noise from the EEG signal. We can increase the accuracy of the EEG signal to noise ratio (SNR) with the help of a good EEG acquisition system. The EEG signal is recorded with an impeccable instrument so the chance of noise being added at the time of recording was minimized.So,this way the noise from the EEG signal is also eliminated during the pre-processing stage.As, we know that there are plethora artifacts produced by a different kind of human activity such as noise added when u blink your eye, muscles movement etc. have the frequency which is less than 3 Hz. There are other noises like power line interface and many others that have frequency ranges greater than 45 Hz. So basically, the range of the signal that we are interested in is 4 –

      45 Hz. We can obtain this band of frequency by using a bandpass filter. The bandpass filter is designed to pass 4-45 Hz of frequency and block the rest of the frequency.

   2. Basic Concepts of components of system

      DEAP dataset which consist of EEG singals recorded for 32 participants.The dataset was preprocessed using the bandpass filter which blocks all the frequency which are not in the range of 4-45 Hz.Thus this will remove all the artifacts or noise that is blended with the EEG signal.The obtained signal is now clean and the features can easily be obtained from the clean data using Wavelet decomposition.The decomposition is done on level 4 using the order 2 Daubechies wavelet. After that, different PCA method is used to analyze the component.

      1. Wavelet Decomposition: Wavelet decomposition is a method in which the EEG signal is decomposed into different bandwidth.Hence the EEG is divided into 4 levels as delta,theta,alpha and gamma for feature extraction.

         where,

         x[k] is the original signal.

         g[n k] represents the Impulse Response of the signal. y[n] represent the final signal after performing the convolution with Impulse Response

         The above equation (1) is used to decompose the respective EEG signal into two frequency bands that are high pass and low pass.Low frequency band is further decomposed as shown in Fig 3.

         Fig 3. Decomposition of EEG signal to 4 level

         As we can infer from the above figure, the EEG signal is decomposed into 5 bandwidth CD1,CD2,CD3,CD4,CA4 having different levels which corresponds to gamma,beta,alpha,theta and delta respectively where

      2. Principle Component Analysis: We use Principle Component Analysis to statistically analyze a data which in this case is EEG signal.There are several methods to analyze the data such as mean,variance and standard deviation.

      • Empirical

        where,

        xi represent the feature vector.

        n it is length of the vector.The above calculated mean is placed in equation(2) into mean vector of the dataset of mX1 dimension.

      • Standard deviation

        where

        is the standard deviation of the vector. si is the vector of N length.

        mean is the mean calculated above in equation (2).

        In the equation(3) the standard deviation is applied for vector and the data is analysed using the standard deviation.

      • VARIANCE

      where

      xi is the feature vector of N length.

      mean is the mean calculated in equation (2).

      The equation (4) is used for the vector to calculate the variance of each value in the vector.

   3. Classifier

   There are various machine learning algorithm that can be used to learn the data along with the label.In this study, 2 different type of classifier is evaluated by using the different Principle Component Analysis method. The classier evaluated in this project are Support Vector Machine (SVM), Random Forest.

   1. Support Vector Machine: We can solve both classification and regression problems using SVM algorithm.In this study, we will classify the data into labels( happy, excited, sad, bored, sleepy, calm, peaceful, nervous, angry,annoying, relaxed, pleased).Now lets discuss the working of SVM for binary classification(depending on valence and arousal) which is equivalent for multi-label classification. In this, the data is plotted and a line in case of 2-D or hyper-plane for multi dimension which act as a decision boundary. The concept of the dot product of two parameters valence and arousal is used to take the decision.

      So basically, we find the point projected on the plane and classify into its equivalent emotion lying within that decision boundary.

      where,

      xi represent feature vector. yi represent another vector.

      this represent the angle between those plane. N it is length o the feature vector.

      Fig. 4. Fig 6: Multi-Dimensional Emotion Space Model

   2. Random Forest: Random Forest is also known as Random Decision Forest because it is based on ensemble learning.In this, a plethora of decision trees is constructed during the training of the dataset. Later on, it uses the decision tree to predict the mode of classes of each individual decision tree. It corrects the over fitting of the decision trees.The algorithm works in 2 phases,where phase I is done by using bootstrap. In this step, lets create random samples with n-sized tuples from the training data group with replacement, then arrange trees. Repeat this step for k times so that you get k random trees.Now we will move on to phase II where for each unclassified tuple will be run on the classifier for each random tree to classify the signal to one of its emotion.So,by finding the combination estimates from the k trees its classified into one of its emotion depending upon the majority vote.This process of combining the estimated values of many trees is called bagging.Because of this process the random tree obtained from a single tree consist of different size and shape.

2. EVALUATION AND ERROR ANALYSIS

   So in order to determine emotions we have used two classifiers Random Forest and Support Vector Machine,Lets first see how we have classified using Random Forest Classifier.Every input entered is considered in the window size of 2 seconds and step size is considered of 0.125 seconds which further sampled at a frequency of 128 Hz.Here before using the classifier we have used fast fourier transform which helps to split the signal into various frequency bands such as theta,alpha,low beta ,high beta

   ,gamma bands which are effectively used for recognizing the emotion.For evaluating purpose of random forest we have used 512 decision trees in order to get accurate results which is observed these as the optimal number of trees by trial and error method,which is then executed by splitting into 6 parallel jobs

   .In case of SVM,initial only the SVC classifier was implemented for the purpose of testing and training. But because of unsatisfactory accuracy values, we made a few changes.

   Firstly we used feature scaling for preprocessing of data accompanied with the implementation of PCA. We then implemented pipe-lining along with cross validation Grid Search in order to find the best fit parameters for the SVC kernal. Based on these parameters we obtained the highest possible accuracy for different emotion parameters.We also tried different test and training set sizes and came to the conclusion that 85 percent for training and 15 percent for testing worked the best.Having implemented these additional methods along with the conventional SVM we were able to increase the accuracy by 5 percent.

   Fig. 5. Fig 7::Final accuracy of our classifiers

3. CONCLUSION

   The aim of this application was to help mentally or physically challenged person to convey its feelings and let people know about its sentiments.This would help the disabled person to communicate with the world.The use of EEG signals as a mode of communication between a human and a machine represents one of the biggest challenges in signal.We have used DEAP dataset which consist of EEG singals recorded for 32 participants.The dataset was preprocessed using the bandpass filter which blocks all the frequency which are not in the range of 4-45 Hz.Thus this will remove all the artifacts or noise that is blended with the EEG signal.The obtained signal is now clean and the features can easily be obtained from the clean data using Wavelet decomposition.It endeavors to introduce different procedures that can be utilized to perceive feelings utilizing discourse and EEG mind signals.The wavelet transform decomposed the EEG signals for feature extraction and based on the calculated parameters values the classification takes place.We have used two classifiers in this project Support vector machine and random forest and finally Binary

   classification by using two parameters valence and arousal.With the help of these values multi label classification is done to detect its emotion .In this project we were successful in classifying the emotion based on EEG signal. As we can see from the above experiment,we found that the Random Forest classifier was giving better efficiency than SVM. Wavelet Decomposition used in Random Forest classifier shows better accuracy than Standard deviation PCA technique used in SVM classifier.

4. FUTURE WORK

The scope of this project can be further increased in the future. So to extend this project we would like to implement this in real time collecting data from the NeuroSky device and recognize the emotion and displaying the output on the screen.The future work includes a further research on EEG for different and more number of emotions as right now our project can classify into 12 emotions depending upon its valence and arousal values, so there are many hybrid emotions which cannot be detected from this project.Furthermore, Cognitive reaction can likewise bethought of as future work utilizing EEG signals and understanding the various phases of how the mind forms on each circumstance as per condition. Whats more, utilize enormous number of tests to get high precision to enhance the system proficiency.

REFERENCES

1. Robert Horlings, Dragos Datcu, Leon J.M. Rothkrantz. Emotion Recognition Using Brain Activity.

2. Thejaswini S, K M Ravikumar, Jhenkar L, Aditya Natraj, Abhay K

   K. Analysis of EEG Based Emotion Detection using SVM.

3. Rashmi G1,Mr. Chethan Balaji, Rashmi C. Human Emotion Classification From EEG using SVM and K-NN Classifier For BCI

4. Ratna Astuti Nugrahaeni,Kusprasapta Mutijarsa. Comparative analysis of machine learning KNN, SVM, and random forests algorithm for facial expression classification.

5. Swati Vaid , Preeti Singh and Chamandeep Kaur(2015). Classification of Human Emotions using Multiwavelet Transform based Features and Random Forest Technique.

6. Barjinder Kaura, , Dinesh Singha, Partha Pratim Roy. EEG Based Emotion Classification Mechanism in BCI.

7. http://www.eecs.qmul.ac.uk/ , for DEAP dataset.

8. Nilkamal More,V.B Nikam,Biplab Banerjee. Machine learning on high performance computing for urban greenspace change detection: satellite image data fusion approach.