Brain Activity Recognition using EEG Signal: A Review

Brain Activity Recognition using EEG Signal: A Review

Prof. Madhav H Thigale

Assistant Professor, Department of E&TC,

Dr. D.Y. Patil Institute of Engineering Management & Research Akurdi, Pune (India)

Abstract — Analysis on the human bodies have never diminished and explorations on it has never came to an end. An analysis of EEG for study of cognitive process for bio-medical appliances is a current research area. The EEG aids to understand the brain and its activities by offering thorough information of the present status of the brain. This is a necessary data since it could aid in identifying or preventing varied abnormalities or diseases or damages in brain. For that, the precise study of EEG signals is important. This survey begins with the assessment of 25 papers on Brain Activity Recognition (BAR). The diverse employed schemes in reviewed articles are analyzed that include classification and optimization techniques like Support Vector Machine (SVM) and so on. In addition, the performance of the reviewed articles is examined and the better performances are analyzed. At last, chronological review is done and the challenges on BAR is described.

Keywords Brain Activity Recognition; EEG; Support Vector Machine; Accuracy; Error.

Nomenclature

1. INTRODUCTION

   BAR is one of the major areas of study in the area of brain science in current years. It has the ability to significantly alter traditional applications of brain research, including the diagnosis of nervous system illnesses, auxiliary services for the old and disabled, device control, disease monitoring, etc. BAR can increase independence and assist persons with limited behavioural abilities and execute routine daily tasks [2] [3]. In addition, as DL technology continues to advance and mature, EEG based BAR is becoming increasingly accurate, which significantly raises the utility of this technology in the context of real-world diagnosis, treatment, and supported living [7] [8]. EEG readings primarily indicate brain activity. Electrodes are applied to the scalp in order to non-invasively record the voltage changes of brain neurons. The scalp surface or cortex is the reflection of electro-physiological actions of brain nerve cells, and the EEG measures changes in radio waves that occur during brain activity. EEG can detect the present level of brain activity since it is dispersed in several frequency bands in accordance with various brain activities [11] [12]. Although extensive research on BAR has been done recently, it still confronts several difficulties, like multi-person and multitask categorization. A number of EEG feature identification techniques are first suggested, with an emphasis on EEG feature representation.

   The evaluation of these studies reveals that the recognition and classification algorithm's accuracy is insufficient to satisfy the EEG application standard under current demand. This is mostly due to the complexities and high dimension of the EEG signal, which have a significant impact on the classification accuracy rate of the feature space of the actual EEG data [5] [6]. Additionally, the literature study mostly focuses on the identification of single-person binary activity in the brain, which is confined in the larger and more complicated scenarios.

   IJERTV13IS120091

   (This work is licensed under a Creative Commons Attribution 4.0 International License.)

   Therefore, the detection of brain activity in situations involving several people and objects has drawn more and more interest. Another drawback is the decline in identification accuracy brought on by the interaction of outside noise and EEG's poor signal-to-noise ratio (SNR). Conventional EEG feature analysis takes a lot of time and requires a lot of attention [8] [9].

   The contributions are as follows.

   1. Reviews existing works on BAR and offers description on the similar topic.

   2. Analyses the performances in adopted articles and examines their better performances.

   3. Performs chronological review that shows the year of publication of adopted articles.

      Here, section II briefs extant works on BAR. The review on employed schemes and analysis on performance are given in section III. The research gaps are in Section IV and conclusion is given in section V.

2. LITERATURE REVIEW

   1. NN based works

      In 2022, Asadzadeh et al. [3] identified the pattern of each emotion by employing the sLORETA technique to map from scalp sensors to brain sources. Then, sLORETA sources are employed as the nodes of basic graph in a GNN for EEG based emotion identification. Finally, a GNN classifier is used to identify the label of hidden emotions.

      In 2021, Mehmet et al. [7] discussed the topic of accurate emotion identification when listening to music. In response to this issue, an EEG-based emotion detection model is created, and a new DL based emotion recognition approach is suggested. EEG signals are generally collected from various channels and converted to spectrograms for the purpose of classifying emotions.

      Gholami et al. [14] goal in 2018 was to model and visualise the brainwave activity patterns cause by marketing aspects of products based upon the spatio-temporal interactions among the continuous EEG data streams. In this study, attention bias- induced spatiotemporal brain patterns were analysed using brain like SNN models.

      Shepelev et al. in 2018 [19] developed a unique NN method for the real-time identification and categorization of temporospatial EEG signals. Experiments showed that user learning increased control speed, classification accuracy for EEG patterns, and recognition accuracy.

      In 2021, Liu et al. [23] used EEG signals with DL in order to categorise criminal emotions. The study also developed a system for identifying EEG data based on NN model. The outcomes demonstrate that the approach suggested in this research has some real-world applications.

      Li et al. [24] in 2018 employed HCNN to categorise the positive, neutral, and negative emotion states. To train the

      HCNNs, the differential entropy features were taken from various channels as 2D maps. HCNN were effective in recognising emotions, particularly on beta and gamma waves.

   2. SVM based works

      In 2021, Chitti et al. [4] created a technique that may automatically ascertain a person's level of attentiveness, or mental condition of vigilance. This technique was applicable to many different fields. Here, SVM was deployed as the fatigue driving identification system.

      The categorization of EEG signals generated by the piano's first octave musical notes as stimuli was first published by Konstantina et al. [18] in 2020. A SVM classifier was provided with the extracted ERSP. Finally, a better accuracy of 70% was obtained using SVM.

      A voting method based on ensemble SVM was suggested by Taheri et al. [25] in 2020. The EEG wave was transformed to varied representation using the DCT, FT, CSP, and EMD. In addition, SVM was deployed in this work to categorise the generated feature vectors.

   3. Correlation based works

      Chen et al. [11] in 2015 used a methodical 3 level validation methodology to determine if fNIRS could accurately capture the core components of bottom-up acoustic processing. The association among changes in cerebral activity and haemoglobin concentrations were investigated using fNIRS- EEG study with auditory and visual stimulation in 24 subjects. Yuvaraj et al. in 2015 [15] examined the impact of emotion regulation on inter-hemispheric EEG coherence in PD. The use of EEG emotion in medical practice to identify possible neurophysiologic anomalies may become more widespread as a result of this study.

   4. Optimization based works

      Pane et al. in 2019 [22] acquired EEG signals from an EEG public dataset with 4 categories. A hybrid feature extraction scheme was deployed to derive EEG features from 3 domains of time, frequency, and wavelet. Further, RF was employed for categorization of emotions. The parameters of RF model were tuned using GSO.

   5. Other works

      Uzefovsky et al. in 2019 [1] merged genetic as well as brain image data for examining the impact of genotype in respond to cognitive empathy. 2 among 5 OXTR SNPs analysed were related with brain actions and 3 of SNPs correlated with diagnose status for predicting brain actions. It was the first study to examine OXTR and brain activities in autism.

      Yu et al. [2] in 2021 identified general pattern of rearrangement of NN in entire human brain on attaining the threshold of recognition of imageries at uncertain conditions. The data were analysed in the context of multilevel and multichannel systems for making decisions on visual signal.

      In 2019, Juneja et al. [5] evaluated numerous aspects (time domain, frequency domain, and LSE features) under an ELM classifier in order to detect human activities. The findings of several tests have verified that the suggested scheme has increased accuracy in comparison to other classification approaches.

      In 2020, Farashi et al. [6] exploited the alterations in the brain functions using MST structure for emotion categorization based on EEG data. The data were used to assess the frequency components of emotional states that were most instructive. The MST graph was subsequently taken out of the FCM, and its properties were applied to the identification of emotions.

      Ganin et al. [8] in 2018 explored the BCI-potential P300's to identify hazy attention foci. Presumably, this technique may be used to gauge a subject's sensitivity to emotigenic stimuli that could be helpful for the instrumented assessment of heightened moods or emotional perception deficiencies.

      Pushkin et al. [9] in 2017 created the control signals for the display of sensory inputs at certain stages of the main biorhythm. This was accomplished by implementing real time digital EEG filters in certain leads and evaluating the background rhythm's immediate amplitude and phase.

      Hiyoshi et al. in 2015 [10] correlated the human emotion judgement of brain. This study used multimodal stimuli to intentionally disturb the emotion system of the brain to determine if such emotional stimuli may cause repeatable and reliable alterations in EEG data.

      In 2020, McAssey et al. [12] analysed whether healthy left and right handed people differ in terms of their behavioural traits and neurological correlates of vection. Also behavioural traits such vection perceived strength, onset delay, presence, and duration were noticed.

      In 2016, Nowicka et al. [13] recorded the EEG of 15 younger men with ASD and 15 one to one control persons. The ERP, coherence, ERD/S and DTF approaches were used to evaluate the EEG data. The names were graphically categorised into four categories: own, close friends, famous, and unknown.

      In 2015, Han et al. [16] examined the brain waves using image stimuli that elicited N1 reactions. Analysis was done on the relationship between the transient stages of pre-stimulus EEG waves and N1 amplitudes.

      In 2018, Zafar et al. [17] investigated whether EEG might be used to decipher fMRI-like patterns of brain activity. Due to the high quality of the data, fMRI only needs data from a small portion of the brain to decipher the patterns of brain activity. In this work, information is derived from dispersed activation patterns of the brain using MVPA, which was utilised for both the analysis of EEG and fMRI data.

      Patel et al. [20] in 2021 provided insights on EEG-based emotion recognition, sought to provide a concise synopsis of the different entropy based approaches employed for emotion

      classification. This paper also explored present and future trends and analysed on emotion identification by utilising entropy to extract features that achieved increased recognition rates.

      In 2022, Xiao et al. [21] introduced a brand-new technique for EEG emotion identification. It is known as the 4D-aNN. First, 4D representations of raw EEG data were created. A CNN was then used to handle with the spectrum and spatial data of 4D representation, and the suggested 4D-aNN employed spectral and spatial attention methods.

3. REVIEW ON EMPLOYED SCHEMES AND ANALYSIS ON PERFORMANCE

   1. Deployed Schemes

      The deployed approaches in the considered articles are exposed in Fig. 1. The analysis is done under, (a) SVM schemes; (b) Correlation based schemes (c) NN based schemes (d) optimization schemes and (e) other schemes. Among optimization schemes, GSO is adopted in [22], CNN + LSTM is adopted in [21], SVM is adopted in [4] [18] [25]. SVM is deployed as classifier in maximum of the works as it is effectual in scenarios, where the sizes are higher than the sample count. SVM is much effectual in high dimension space and poses efficient memory. In addition, Pearson correlation based schemes is adopted in [11] [15]. Gollin method is used in [2] and wavelet analysis is adopted in [16]. NN based methods like DGCNN is adopted in [3], DNN is adopted in [7], SNN is adopted in [14], NN is adopted in [19] [23] and HCNN is adopted in [24]. Transform based scheme is adopted in [9] [12]. The other models include MarsBaR [1] ELM classifier [5] Graph theory [6] Fisher linear discrimination [8] Welch periodogram method [10] DFT [13] MVPA [17] Fuzzy entropy

      Varied schemes on brain activity recognition systems

      CNN + LSTM

      SVM

      Correlation based schemes

      Wavelet analysis

      GSO

      [21]

      [4] [18] [25]

      [11] [15]

      [16]

      [22]

      NN based schemes

      Transform based scheme

      DGCNN [3] DNN [7] SNN

      [14] NN [19] [23] HCNN
      [24]

      [9] [12]

      Marsbar [1] Gollin [2] ELM classifier [5] Graph theory [6] Fisher linear discrimination [8] Welch periodogram method

      [10] DFT [13] MVPA [17] Fuzzy entropy [20] RF [22]

      Others

      [20] RF [22].

   2. Performance Analysis

      Table I reveals the wide-ranging performances examined in the reviewed works on BAR in EEG signals. In Table I, it is observed that 18 works have examined accuracy, which is the most examined measure in almost all works. The accuracy measure is adopted in majority of the works and has offered 72% of the total contribution. The mean metric has contributed about 32% and SD was examined in 7 papers that contribute about 28% of the entire works. The error metric was examined in 5 of the reviewed works and it offers about 20% of the contribution. Time cost, Jittered period, Median, Correlation, False negative and Fatigue error rate are examined in each works, i.e. [24] [12] [12] [6] [5] and [4], which has contributed around 4%. Sampling rate was examined in [10] and [13]. FDR as well as activation level are examined in [16] and [14] that provided about 4% of the whole contribution.

      TABLE I. Varied performances in existing BAR techniques

      Measures	Citations
      Accuracy	[3] [5] [6] [7] [8] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25]
      Mean	[1] [9] [11] [15] [18] [19] [23] [25]
      Standard deviation	[1] [11] [18] [19] [21] [23] [25]
      Error	[9] [10] [15] [19] [22]
      Time cost	[24]
      Jittered period	[12]
      Median	[12]
      Sampling rate	[10] [13]
      Correlation	[6]
      False negative	[5]
      Fatigue error rate	[4]
      FDR	[16]
      Activation level	[14]

   3. Analysis on Maximum Performance

      The highest performances in each reviewed paper for BAR is shown in Table II. Fig. 2 demonstrates how well each extant works exhibits its betterment on accuracy, error and standard deviation. In the reviewed works, accuracy computed in [25] has achieved a high value of 99.7% and mean used in [25] has obtained a better value of 96.34. The accuracy using SVM has accomplished high value due to its advantages in high dimensional space and high recognition rates. Standard deviation deployed in [25] has gained less deviated value of

      1.87. Likewise, a minimal error of 0.059 is accomplished in [22]. A less FDR of 1% is obtained in [16] and a less false negative value of 41.18% is accomplished in [5].

      TABLE II. Highest performances gained in reviewed works

      Sl. no	Citation	Metric	Higher performance
      1	[25]	Accuracy	99.7%
      2	[25]	Mean	96.34
      3	[25]	Standard deviation	1.87
      4	[22]	Error	0.059
      5	[16]	FDR	1%
      6	[5]	False negative	41.18%

      (a)	(b)
      (c)

      Fig. 1. Analysis on (a) Accuracy (b) Error and (c) Standard deviation

   4. Chronological Analysis

      The chronological study of extant reviewed works is exposed in Fig. 3. In this survey, 7 works are made available between 2022 and 2021. 7 works are published between 2020-2019. 7 works were published among 2018-2016. A minimal of 4 works was published between 2015 and 2014. Thus, maximum articles are adopted between the year 2022 and 2016.

      Fig. 2. Chronological Analysis

4. RESEARCH GAPS

   BAR is a most hopeful research area for past few years [26]. Activities of brain are generally characterized by EEG signals that record the voltaic fluctuation of brain neuron with the electrodes located on scalp in a non-invasive manner. Even though BAR is broadly examined over the previous decade, it yet meets with numerous challenges like multi-class and multi- person classification. Regardless of numerous analyses on multi-person EEG classification, there needs enhancement on conventional techniques regarding classification accuracy. Subsequently, the majority of extant appliances that deploy EEG classification for diagnosing diseases require only binary classification (abnormal or normal). The brain controlled robot is a novel technique depending upon robot control and BCI mechanisms. This mechanism permits the brain to regulate a robot to carry out a sequence of actions. The EEG signal holds an imperative role in the BCI technology. Even though BCR have broad study prospects, there exist numerous difficulties and challenges that require being solved [27].

   • EEG wave has activeness, non-linearity, and time varying causalities. It is a nonlinear, dynamic time-varying, very compound system [26].

   • Numerous EEG patterns still include issues like fatigue and discomfort, and lower data transfer rate. Thereby, better distinctive EEG signals must be chosen via testing [27].

   • If online signal processing could not be processed in actual time, the robot could not ensure its real time controlling accuracy.

   • The BCI system would include diverse working effects owing to individual variations. It is moreover very demanding to learn how to adjust the constraints so that individual variations would not have an effect on the system [27].

5. CONCLUSION

This research made the review on extant BAR approaches.

• This survey begins with the assessment of 25 papers on BAR.

• The diverse employed schemes in reviewed articles were analysed that include classification and optimization techniques like SVM and so on.

• In addition, the performance of the reviewed articles was examined and the better performances were analyzed.

• At last, chronological review was done and the challenges on BAR was described.

REFERENCES

1. Florina Uzefovsky , Richard A. I. Bethlehem, Simone Shamay-Tsoory, Amber Ruigrok, Rosemary Holt, Michael Spencer, Lindsay Chura, Varun Warrier, Bhismadev Chakrabarti, Ed Bullmore, John Suckling, Dorothea Floris, and Simon Baron-Cohen, The oxytocin receptor gene predicts brain activity during an emotion recognition task in autism, Molecular

   Autism, vol, 10, o.12, 2019, https://doi.org/10.1186/s13229-019-0258-

2. Shelepin, K.Y., Shelepin, Y.E. Rearrangement of the Activity of Neural Networks in the Human Brain on Reaching the Recognition Threshold for Fragmented Images. Neuroscience and Behavioral Physiology, vol. 51,

   pp. 229237, (2021). https://doi.org/10.1007/s11055-021-01061-x.

3. Shiva Asadzadeh, Tohid Yousefi Rezaii, Soosan Beheshti & Saeed Meshgini, Accurate emotion recognition using Bayesian model based EEG sources as dynamic graph convolutional neural network nodes, Department of Biomedical Engineering, (2022) 12:10282.

4. Chitti, S., Kumar, J.T. & Kumar, V.S. EEG Signal Feature Selection Algorithm and Support Vector Machine Model in Patient's Fatigue Recognition. Arabian Journal for Science and Engineering, (2021). https://doi.org/10.1007/s13369-021-06206-1

5. Kapil Juneja, Chhavi Rana, Individual and Mutual Feature Processed ELM Model for EEG Signal Based Brain Activity Classification, Wireless Personal Communications, vol. 108, pp. 659681, https://doi.org/10.1007/s11277-019-06423-w.

6. Farashi, S., Khosrowabadi, R. EEG based emotion recognition using minimum spanning tree. Phys Eng Sci Med 43, 985996 (2020). https://doi.org/10.1007/s13246-020-00895-y

7. Mehmet Bilal Er, Harun Çi, brahim Berkan Aydilek, A new approach to recognition of human emotions using brain signals and music stimuli, Applied Acoustics, Volume 175, 2021, 107840, ISSN 0003-682X, https://doi.org/10.1016/j.apacoust.2020.107840.

8. Ganin, I.P., Kosichenko, E.A. & Kaplan, A.Y. Properties of EEG Responses to Emotionally Significant Stimuli Using a P300 Wave-Based BrainComputer Interface. Neurosci Behav Physi 48, 10931099 (2018). https://doi.org/10.1007/s11055-018-0672-7

9. A. A. Pushkin, L. V. Lysenko, A. G. Sukhov, A. V. Vdovjuk, and I. V. Scherban, Regulation of the Functional State of the Human Brain by Real-Time FrequencyPhase Synchronization of Sensory Stimuli with the EEG Rhythm, Biomedical Engineering, Vol. 51, No. 1, pp. 2832,

   May, 2017, https://doi.org/10.1007/s10527-017-9678-7

10. K. Hiyoshi-Taniguchi M. Kawasaki T. Yokota H. Bakardjian H. Fukuyama, A. Cichocki, F. B. Vialatte, EEG Correlates of Voice and Face Emotional Judgments in the Human Brain. Cogn Comput 7, 1119 (2015). https://doi.org/10.1007/s12559-013-9225-0

11. Ling-Chia Chen, Pascale Sandmann, Jeremy D. Thorne Christoph S. Herrmann, Stefan Debener, Association of Concurrent fNIRS and EEG Signatures in Response to Auditory and Visual Stimuli, Brain Topogr 28, 710725 (2015). https://doi.org/10.1007/s10548-015-0424-8

12. Michaela McAssey · James Dowsett· Valerie Kirsch,· Thomas Brandt· Marianne Dieterich, Different EEG brain activity in right and left handers during visually induced self-motion perception Journal of Neurology, 267, pp. 7990, (2020). https://doi.org/10.1007/s00415-020- 09915-z

13. Anna Nowicka, Hanna B. Cygan, Pawe Tacikowski, Pawe Ostaszewski and Rafa Ku, Name recognition in autism: EEG evidence of altered patterns of brain activity and connectivity, Molecular Autism 7, 38 (2016). https://doi.org/10.1186/s13229-016-0102-z

14. Gholami Doborjeh, Z., Doborjeh, M.G. & Kasabov, N. Attentional Bias Pattern Recognition in Spiking Neural Networks from Spatio-Temporal EEG Data. Cogn Comput 10, 3548 (2018). https://doi.org/10.1007/s12559-017-9517-x

15. R. Yuvaraj M. Murugappan Norlinah Mohamed Ibrahim, Kenneth Sundaraj Mohd Iqbal Omar Khairiyah Mohamad, R. Palaniappan, M. Satiyan, Inter-hemispheric EEG coherence analysis in Parkinsons disease: Assessing brain activity during emotion processing. J Neural Transm 122, 237252 (2015). https://doi.org/10.1007/s00702-014-1249-

    4

16. Li Han Zhang Liang, Zhang Jiacai, Wang Changming Yao Li Wu Xia, Guo Xiaojuan, Improving N1 classification by grouping EEG trials with phases of pre-stimulus EEG oscillations. Cogn Neurodyn 9, 103112 (2015). https://doi.org/10.1007/s11571-014-9317-x

17. Raheel Zafar Nidal Kamel · Mohamad Naufal· Aamir Saeed Malik · Sarat

    C. Dass · Rana Fayyaz Ahmad, Jafri M. Abdullah· Faruque Reza,, A study of decoding human brain activities from simultaneous data of EEG

    and fMRI using MVPA, Australas Phys Eng Sci Med, vol. 41, pp. 633 645, (2018). https://doi.org/10.1007/s13246-018-0656-5.

18. Konstantina Tsekoura Amalia Foka, Classification of EEG signals produced by musical notes as stimuli, Expert Systems with Applications, Volume 159 (Cover date: 30 November 2020), Article 113507, 30 May

    2020

19. I. E. Shepelev, D. M. Lazurenko, V. N. Kiroy, E. V. Aslanyan, O. M. Bakhtin, and N. R. Minyaeva, A Novel Neural Network Approach to Creating a BrainComputer Interface Based on the EEG Patterns of Voluntary Muscle Movements, Neurosci Behav Physi, vol. 48, pp. 11451157 (2018). https://doi.org/10.1007/s11055-018-0679-0.

20. Pragati Patel, Raghunandan R and Ramesh Naidu Annavarapu, EEG- based human emotion recognition using entropy as a feature extraction measure. Brain Inf. Vol. 8, no. 20 (2021). https://doi.org/10.1186/s40708-021-00141-5.

21. Guowen Xiao, Mengwen Ye, Bowen Xu, Zhendi Chen, Quansheng Ren, 4D attention-based neural network for EEG emotion recognition. Cogn Neurodyn, vol. 16, pp. 805818 (2022). https://doi.org/10.1007/s11571-

    021-09751-5

22. Evi Septiana Pane,· Adhi Dharma Wibawa,· Mauridhi Hery Purnomo,. Improving the accuracy of EEG emotion recognition by combining valence lateralization and ensemble learning with tuning parameters, Cogn Process, vol. 20, pp. 405417 (2019). https://doi.org/10.1007/s10339-019-00924-z

23. Liu, Q., Liu, H. Criminal psychological emotion recognition based on deep learning and EEG signals. Neural Comput & Applic, pp. 33, vol. 433447 (2021). https://doi.org/10.1007/s00521-020-05024-0.

24. Li, J., Zhang, Z. & He, H. Hierarchical Convolutional Neural Networks for EEG-Based Emotion Recognition. Cogn Comput 10, 368380 (2018). https://doi.org/10.1007/s12559-017-9533-x.

25. Taheri, S., Ezoji, M. & Sakhaei, S.M. Convolutional neural network based features for motor imagery EEG signals classification in braincomputer interface system. SN Appl. Sci. 2, 555 (2020). https://doi.org/10.1007/s42452-020-2378-z

26. Xiang Zhang, Lina Yao, Dalin Zhang, Xianzhi Wang, Z. Sheng, and Tao Gu, "Multi-Person Brain Activity Recognition via Comprehensive EEG Signal Analysis", In Proceedings of Mobiquitous 2017, MELBOURNE, AU. Nov. 2017, 10 pages. DOI: 10.475/123 4.

27. Ziyang Huang, Mei Wang, A review of electroencephalogram signal processing methods for brain-controlled robots, Cognitive Robotics, vol. 1, pp. 111-124, 2021.