Eminence – A Real-time Mental Health Monitoring System

Eminence – A Real-time Mental Health Monitoring System

Harsh Kumar, Mayank Sethia, *Priyal Agrawal, Aditya K, Shreyas Gonjari, Davuluru Phani, Abhay Singh Chauhan, Chirag Phor, Santhi V

School of Computer Science and Engineering Vellore Institute of Technology Vellore, India

Abstract Suicide has been a big problem for many years. The idea behind this paper is to reduce the increasing numbers of suicide. For the same purpose a mental health diagnostics system and a monitoring application is designed based on EEG Signals using Neurosky Mindwave Headset as well as the vocal output of the patient.The project measures the attention level and meditation level using speech recognition and neurons potential difference. The application classifies the emotion and based on which some necessary actions are carried out. that includes messaging the emergency contacts, notifying the user of his condition, etc. The application will provide suggestions and answer all frequently asked questions (FAQs) with the help of an AI chatbot.

Keywords Mental Health, Neurosky Mindwave Headsets, EEG, Emotion Recognition, Suicide Prevention

1. INTRODUCTION

   Suicide is one among the major reasons of death across the globe.As stated by health World Organization (OMS), approximately 800,000 humans commit suicide per year across the globe and the fact that worries the most is that, for every suicide that happens, many other suicide attempts had occurred. Suicide is a very complex problem, with many causes that are related to one another. The most significant cause for this is the mental illness that happens due to depression[1]. More than 350 million people worldwide of all the ages and from different-different communities suffer from depression[2] .

   Mental health is a very important part of human health & is much more than the absence of mental illnesses. It refers to the wide range of tasks either directly or indirectly associated with the mental well-being, averting of mental chaos, and treatment & rehabilitation of the people influenced by mental disorders. WHO estimated that a load of mental health problems is of the level of 2,443 DALYs per 100,000 populations, and the age-tuned suicide rate per 100,000 populations is 21.1 in India. It is evaluated that, the economic loss in India because of mental health conditions in India, is

   1.03 trillion of 2010 dollars, between 2012-2030 [3]. In accordance of the seriousness of this problem & the outcome of psychological results, it might be mandatory to keep the patient on notice, medicate it or internalize it so that he or she does not try to attack himself or herself [4]. Due to all such reasons, the variant sections of health and science have been working to identify the traits that are associated with the suicidal tendency in the case to prevent suicides from happening.

   Due to the expanding population, the number of professional psychiatrists is inadequate to serve the needs of patients. The main problem is the unavailability of a real-time mental health monitoring system for the people as well as the unavailability of a quantifiable approach for the psychiatrists to evaluate the mental health of the patients.

   EEG (Electroencephalography) is an electrophysiological method to analyze the brain activity and detect abnormalities in it.Neurosky Mindwave is a headset device which captures EEG values form the user who is wearing it. The headset contains electrodes which capture the very minute electrical impulses created by the synapses between the neurons.It basically records the electrical activity of the brain. It is a non-invasive method. The electrodes are placed on the scalp and behind the ears [19].

   The idea for this problem is using the Neurosky Mindware Device to record the EEG signals of the patient. We developed an algorithm to classify the signals or waves. We have used Convolutional Neural Networks (CNN) which is a Deep Learning System that can infer a hierarchical representation of input data that facilitates categorization. These classifications would then be associated with various emotional tags such as stress, happy etc. These data would then be sent to the cloud. We will also develop a mobile app that will monitor all these. If the value crosses a particular threshold it would generate an alert in the app and also send an alert message to the emergency contacts of the person. The app would also contain some emergency tool-kits which can be used by a patient in times of emergency and need. Apart from these, we have other functionalities like finding nearby hospitals on the map. Also, a chatbot is developed which will help the user to suggest the actions to be taken.

2. LITERATURE SURVEY

   Chiang, W., Cheng, P., Su, M., Chen, H., Wu, S., & Lin, J. came across various methodologies that were used to deal with the suicide prevention, the Taiwan suicide prevention center( TSPC) has given mind survey Feeling Thermometer which is able to determine the suicidal tendency with the precision rate of 85%. It has also been done to provide suicidal tendency observational system on social media sites like facebook, Instagram, twitter . The main problem with this methodology was that only people who volunteered for the procedure could be diagnosed and also most of the people may lie on the test there is no proper validation of the test [5].

   In 2011 Liu, Y., Sourina, O., & Nguyen, M. K. focus on recognizing the inner emotions of a person using theElectroencephalogram (EEG) signals using Arousal-valence emotional model to quantify and classify

   basic emotions to measure 6 main types of emotions[8]. In 2014 Z. Zhou, H. Jiang and X. Song, have proposed a system for emotion recognition based on the EEG values. The EEG values were captured using Neuroscan 64. A comparison between different kernels for the SVM is also carried out by the authors which evidently showed that the linear kernel is the best suitable kernel for SVM with an overall accuracy of 89%[12].

   G. Lv, S. Hu, and X. Lu have stated and have briefed about the partially continuous Hidden Markov Model (HMM) and then went on to propose a Dynamic Bayesian Network (DBN) for speech based emotion recognition. The proposed DBN had three layers. As emotion changes dynamically the traditional DBN cannot handle the nuances of the emotional state changes due to the limited number of states in it[11]. Later Zhengwei Huang and team also proposed a model for Speech Emotion Recognition (SER) in 2014 to study affect-main characteristics automatically and effectively via semi-supervised CNN (semi-CNN). The training of semi-CNN consists of two steps: first is unsupervised and another is semi-supervised. The model consists of an input layer, a convolutional layer and an SVM classifier. This approach leads to robust and stable recognition performance by disentangling composite environments which comprise of other factors like speaker and noise[7].

   In 2015 Y. Gao, H. J. Lee, R. M. Mehmood work dealt withHuman Emotion recognition using deep learning. The proposed algorithm learns and classifies the EEG signals simultaneously. The 3 layer RBMs are used and backpropagation is used for tuning all the subjects. More accuracy is achieved by deep learning compared to conventional methods[17].In 2016 Ackermann, P. and team author stressed on the importance of automatic emotion recognition. A novel approach, which involves attribute extraction, selection &further classification of EEG data or brain waves based which is recorded using pulse meters or galvanic skin sensors.[9].Than in 2016 Ghare, P. S and team also focused for Emotion recgnization via training the classifier using LMS algorithm, Daubechies function (db4), decomposition based on delta, alpha and beta, and SVM classifier. In this process a GUI wasdeveloped for Human emotions recognition algorithm. There are many applications of this work like the diagnosis of diseases, interfacing, etc[16]. Berrouiguet, S and group proposed an E-Health mobile application for suicide prevention. The application proposed is based on a method practiced by doctors and medical practitioners to read as well as evaluate patients physical and psychological conditions. The main reason to come out with such an application was to relieve the patients from their hesitancy to tell their problems to a doctor. One of the important features of the application was the Ecological Momentary Assessment (EMA). The use of a highly secure data warehouse approved by public authorities which almost

   eliminated the privacy issues[10].

   In 2017 Calderon-Vilca proposed a method of simulating analytically generated data set to tells us about the number of

   youth with the suicidal tendency, the assessment of 3 algorithms of the machine learning as an outcome is created on the basis of trees. The main disadvantage in the machine learning process is that there is not much data available on the suicides of the youth and since the universities do not reveal details of the suicide leads to incomplete data on the suicides[6].

   T. Lv, J. Yan and H. Xu proposed an Emotion Recognition Method using the AdaBoost classifier. AdaBoost classifier is a combination of a number of weak classifiers. The weak classifiers are initially trained on the same data and then combined based on an algorithm to give better results and thus making it a stronger classifier. The AdaBoost classifier performed much better than the widely used classifier with an accuracy over 90%[13].In the same year Cheng, C., Wei, X., & Jian, Z used a preprocessing technique for preserving the association of EEG data based on CNN, then the parameters of ERACNN are adjusted.They obtained an accuracy of 83.45% for the 2-category emotion recognition algorithm and were also able to achieve an accuracy of 68.8% for the 3 category recognition algorithm[15]. Lahane, P., & Thirugnanam, M. stated in a paper how K-Nearest Neighbor (KNN) classifier combined with frequency cepstral coefficient(FCC) techniques gives better performance than others. The results show how FCC (accuracy of 90%) outperforms KDE (accuracy of 80%) by quite a margin[18].

   In 2018 the primary aim of the Teo, J., & Chia, J. T. is to permit constantly dependable emotion recognition forvirtual reality(VR) stimuli via means of Wearable EEG. The classification accuracy was between 65-89% only using KNN classifiers and SVMs. The work done evidently depicts that the usage of Wearable EEG may be consistently and reliably expandedto identify emotions produced throughvisual immersive stimuli[14].

   In all the above models deep learning came out to give very high accuracy. There are a lot of models present to detect emotion using EEG signals and speech recognition but no such quantitative model proposed till now to measure the mental health using EEG signal.

3. ARCHITECTURE

   Fig1. Architectural Model

   • Neurosky Mindwave – The hardware device which capture the EEG values form the mind. It outputs the EEG power spectrum (alpha, beta, gamma waves etc). These values will be further processed for emotion recognition. An algorithm is designed for processing this data. The device consists of a headset, a sensor arm and an ear clip. The ground electrodes of the headset are on the ear clip and EEG electrodes are on the sensor arm, on the forehead above the eye.

   • Voice Based Emotion Recognition – First start with voice capturing, followed by feature extraction and then classification of the person emotions.

   • Real Time Location – The offline real time location of the person can be sent to the emergency contacts of the person without internet.

   • One Click Panic Button

   • All in one health kit – Features such as Call emergency services, first aid Guide, torch, compass will be included in app.

   • AI Chat-bot- Assistant to help you out during emergencies using AI.

4. METHODOLOGY

   Electric waves are brain impulse which can be estimated by computing the electrical movement close to the scalp of a human. The frequency of the above mentioned mind waves varies from 1-100Hz The signal is divided into 5 wave types based on frequencies. The following table describes the 5 different types of waves along with their frequencies and the condition of mind at that time.

   Fig2: Hardware Methodology

   The bands of frequencies are sampled at the standard of 512hz and afterward conveyed to ADC in 12-bit resolution. Each Brainwave category is given distinct discrete esteem. Afterward, the processor computes the potential Fast Fourier Transformation (FFT) for all distinct Brainwave categories and once the values of FFT are received the processor searches for the configuration pins. At that point, the processor yields the computerized FFT values for every Brainwave type sequentially by means of the Tx pin at the bandwidth through configuration pin.

   The mid flex board gets these sequential points and after that does its calculation internally for broadcasting these waves via a wireless medium. Attention and Meditation of an individual are shown by the reading of these values. Attention demonstrates the focus level of a human essentially put as steady mental action. Meditation demonstrates relaxation or calmness. Along with the attention value of the patient, it is important to estimate the confusion score or how confused a patient is during the psychiatry session. We have used Random Forest Algorithm for the same. The readings obtained are then pushed to the firebase. And the real-time graphs are monitored for attention level and meditation level through the Android app. The confusion score is used by the psychiatrist.

   S No.	Wave Type		Frequency (in Hz)	Mental conditions
   1.	Delta Wave		0.1-3.5	Deep sleep, dreamless sleep, loss of consciousness
   2.	Theta Wave		4-7	Short term memory, relaxation, creative, intuition, imaginary
   3.	Alpha Wave		7.5-12.5	Relaxation, calmness, conscious, tranquil
   4.	Beta Wave	Low	13-15	relaxed stillconcentrated, integrated
   		Mediu m	16-20	Thinking, awareness of surrounding
   		High	21-30	Alertness, agitation
   5.	Gamma Wave		31-100	Motor functions, fully conscious

   S No.	Wave Type		Frequency (in Hz)	Mental conditions
   1.	Delta Wave		0.1-3.5	Deep sleep, dreamless sleep, loss of consciousness
   2.	Theta Wave		4-7	Short term memory, relaxation, creative, intuition, imaginary
   3.	Alpha Wave		7.5-12.5	Relaxation, calmness, conscious, tranquil
   4.	Beta Wave	Low	13-15	relaxed stillconcentrated, integrated
   		Mediu m	16-20	Thinking, awareness of surroundng
   		High	21-30	Alertness, agitation
   5.	Gamma Wave		31-100	Motor functions, fully conscious

5. EXPERIMENTAL RESULTS

   Table1. Categorization of Brain Waves along with frequency and mental conditions

   No pre-existing data set was available so a data set was created based on the readings of Neurosky Mindwave Mobile

   1. The data set consists of attributes: delta, theta, alpha1, alpha2, beta1, beta2, gamma1, gamma2, tstamp. Than further implementation is done on this data-set. The hardware process is done in 5 steps.

      Fig3: Resulting meditation, attention level

      We have a dashboard that have 4 dials indicating attention, meditation, confusion and overall score. The plot at the bottom indicates the confusion scores throughout the conversion while the patient is wearing the Neurosky device.

      Classifier	Accuracy Score
      KNN	0.6214
      Naive Bayes	0.6573
      XGBoost	0.6259
      Random Forest	0.9592

      Fig4: Dashboard

      A heat map has been plotted for correlation, that also have the attributes of meditation, attention, and different waves.

      Fig5: Heat-map Correlation

      The scores obtained were pushed to the firebase and the attention level and meditation level is plotted, which can be monitored in real-time by the patient through the app.

      Fig6: Real Time Monitoring on app

      Several classifiers were tested to identify the confused state. Random Forest Classifier ended up giving the highest accuracy score of 0.9592.

      Table2: Comparison of accuracy of different algorithms

6. CONCLUSION

The paper stated a very unique solution to detect emotions using EEG signals and the Neurosky Mindware device which can be used by the psychiatrist to detect how prone the patient is towards suicide. Our USP is our unique algorithm which we have used to classify the EEG signals. With this system, we have successfully received the meditation level, attention level, and confusion score properly that further helps us to predict the mental state of the person. Random Forest Algorithm turns out to be a very good algorithm that depicts with such a high accuracy that is 95%. Also integrating the entire system with the help of a mobile application can increase its reach to a large number of people further.

REFERENCES

1. Heath world organization – OMS. Suicide, August 2017.Retrived from http://www.who.int/mediacentre/factsheets/fs339/es/consulted.

2. Media centre. (2016, December 19). Retrieved from https://www.who.int/mental_health/advocacy/en/#Factsheets

3. World Health Organization, Mental health in India. (2017, July 13). Retrieved from http://www.searo.who.int/india/topics/mental_health/about_mental health/en/

4. Suicidio y comportamiento suicida : MedlinePlus enciclopedia médica. (n.d.). Retrieved from https://medlineplus.gov/spanish/ency/article/001554.htm

5. Chiang, W., Cheng, P., Su, M., Chen, H., Wu, S., & Lin, J. (2011). Socio-health with personal mental health records: Suicidal-tendency observation system on Facebook for Taiwanese adolescents and young adults. 2011 IEEE 13th International Conference on E-Health Networking, Applications and Services. doi:10.1109/health.2011.6026784

6. Calderon-Vilca, H. D., Wun-Rafael, W. I., & Miranda-Loarte, R. (2017). Simulation of suicide tendency by using machine learning. 2017 36th International Conference of the Chilean Computer Science Society (SCCC). doi:10.1109/sccc.2017.8405128

7. Huang, Z., Dong, M., Mao, Q., & Zhan, Y. (2014). Speech Emotion Recognition using CNN. 2014 In Proceedings of the 22nd ACM international conference on Multimedia. Florida, US: ACM, 801 804

8. Liu, Y., Sourina, O., & Nguyen, M. K. (2011). Real-Time EEG-Based Emotion Recognition and Its Applications. Lecture Notes in Computer Science Transactions on Computational Science XII,256-277, vol 6670. Springer, Berlin, Heidelberg. doi:10.1007/978-3-642-22336-5_13

9. Ackermann, P., Kohlschein, C., Bitsch, J. A., Wehrle, K., & Jeschke, S. (2016). EEG-based automatic emotion recognition: Feature extraction, selection and classification methods. 2016 IEEE 18th International Conference on E-Health Networking, Applications and Services (Healthcom). doi:10.1109/healthcom.2016.7749447

10. Berrouiguet, S., Billot, R., Lenca, P., Tanguy, P., Baca-Garcia, E., Simonnet, M., & Gourvennec, B. (2016). Toward E-Health Applications for Suicide Prevention. 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE). doi:10.1109/chase.2016.37

11. G. Lv, S. Hu and X. Lu, "Speech emotion recognition based on dynamic models," 2014 International Conference on Audio, Language and Image Processing, Shanghai, 2014, pp. 480-484. doi: 10.1109/ICALIP.2014.7009840

12. Z. Zhou, H. Jiang and X. Song, "EEG-based emotion recognition using wavelet features," 2014 IEEE 5th International Conference

    on Software Engineering and Service Science, Beijing, 2014, pp. 585-588. doi:10.1109/ICSESS.2014.6933636

13. T. Lv, J. Yan and H. Xu, "An EEG emotion recognition method based on AdaBoost classifier," 2017 Chinese Automation Congress (CAC), Jinan, 2017, pp. 6050-6054. doi:10.1109/CAC.2017.8243867

14. Teo, J., & Chia, J. T. (2018). Deep Neural Classifiers For Eeg-Based Emotion Recognition In Immersive Environments. 2018 International Conference on Smart Computing and Electronic Enterprise (ICSCEE). doi:10.1109/icscee.2018.8538382

15. Cheng, C., Wei, X., & Jian, Z. (2017). Emotion recognition algorithm based on convolution neural network. 2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE). doi:10.1109/iske.2017.8258786

16. Ghare, P. S., Paithane, A. N., Human Emotion Recognition using Non Linear and Non Stationary EEG Signal, International Conference on Automatic Control and Dynamic Optimization Techniques, 2016, pp. 1013-1016.

17. Y. Gao, H. J. Lee, R. M. Mehmood, "Deep learninig of EEG signals for emotion recognition", Multimedia & Expo Workshops (ICMEW) 2015 IEEE International Conference on, pp. 1-5, 2015.

18. Lahane, P., & Thirugnanam, M. (2017). A novel approach for analyzing human emotions based on electroencephalography (EEG). 2017 Innovations in Power and Advanced Computing Technologies (i-PACT). doi:10.1109/ipact.2017.8245056

19. MindWave. (n.d.). Retrieved April 04, 2019, from https://store.neurosky.com/pages/mindwave

20. Sezer, A.; nel, Y.; Seçkin, A. Ç. & Uluçnar, U. (2015). An investigation of university students' attention levels in real classroom settings with neurosky's mindwave mobile (EEG) device. Intenational Educational Technology Conference, stanbul, Turkey, 27 29 May, pp.88 101.