Breath Assisted Communication Device for Speech Impaired / Elderly People with Patient Request Tracking and Fall Detection System

Breath Assisted Communication Device for Speech Impaired / Elderly People with Patient Request Tracking and Fall Detection System

Vignesh Pranav B1, Avinash Bhat B2, Ananth Bogar B N3, Bharath B S4, Sujan Kumar A5

1, 2, 3, 4 Under Graduate Student,

5Assistant Professor,

Dept. of ECE, APS College of Engineering, Bangalore, India

Abstract People who are incapable of speaking need to utilize an Augmentative and Alternative Communication (AAC) device to express themselves. Recent estimates show that most people suffering from such developmental disabilities cannot afford current AAC devices. Thus the existing AAC devices are almost inaccessible to those in need and are also ineffective in terms of their operation. In this paper, a cost effective AAC device is designed that allows the user to interact with his environment using only his breath which is both reliable and accurate. The basic needs of the patient are effectively managed using patient request tracking system which monitors the services provided by a nurse to the patient. Also, a fall detection system based on tri-axial accelerometer is added to detect accidental falls of patients or elderly people which works with high detection accuracy.

Keywords Augmentative and Alternative Communication (AAC), basic needs, breath, fall detection, patient request tracking.

1. INTRODUCTION

   Speech is one of the most efficient and widespread forms of communication among humans. For most people the process of speech occurs spontaneously and without any effort. But, there are many individuals who are unable to use speech as a primary method of communication. Estimates show that approximately one hundred and five million people worldwide suffer from some kind of speech disorder. These individuals need to use alternative methods of communication to express their thoughts and needs. The devices which provide these alternative methods of communication are referred to as augmentative and alternative communication (AAC) devices.

   While there are many AAC devices available, most of them are bulky, expensive, inefficient and are difficult to handle. This paper focuses on developing an affordable and easy to use AAC device which fulfills the primary function of interaction and expression for the user. The AAC device being designed uses breath as a mode of communication as it is easy to control breath than other senses.

   A patient request tracking system is also designed that can effectively manage the needs of the patient using the AAC device due to the non-availability of such a robust system. In the current system, we believe on trust of employee or CCTV cameras which are not feasible. When the patient raises a

   request using the device, it is important to ensure that the need is catered to on time and the feedback of the patient regarding the service provided is also significant. These two factors are taken into consideration in the design of the request tracking system.

   Elder people are prone to falling from beds or wheelchairs because of decreasing stability and body weakness. Estimates show that approximately half a million fall related deaths are reported each year globally. A part of this paper focuses on creating a real-time, high accuracy, and low-cost fall detection system that employs a detection algorithm based on tri-axial accelerometer data. This data is monitored continuously so that the patient can get immediate attention if fall occurs.

   The proposed AAC device and the patient request tracking system combined with the fall detection system ensure that all the basic needs of the patients are taken care of satisfactorily and that the patient gets timely help in case of a fall. This provides a sense of comfort and security to both the patient and the concerned people.

2. EXISTING AAC DEVICES

   1. Switch access scanning

      This technology makes use of a matrix that consists of either letters or symbols over which an indicator moves periodically in a cyclic manner, highlighting each element [1]. The user has to wait until the indicator moves over the desired element and then press a button to select it. For processing and display, tablet computers are used which consume a lot of power. This technology demands continuous attention of the user during the selection process, which may cause stress and fatigue. These devices are bulky and are generally expensive.

   2. Eye Trackers

      This technology consists of a video camera, which tracks the movement of the eyes, on the basis of which an indicator is moved on a screen displaying a matrix consisting the users requirements [2]. The tracking operation requires detailed images of the eyes. Hence, expensive, high resolution video cameras are required. Further, the user is required to continuously stare at the screen and to maintain a steady head

      position, which may strain the user [3]. This technology cannot be utilized by subjects with prior vision problems.

   3. Brain Computer Interface (BCI)

   This technology employs sensors that measure the neural activity of the user to synthesize what the user is trying to express [4]. The processing of neural signals is complex and requires expensive, high-end processors. This technology is still in its infancy stages [5]. Ethical issues may hinder the development of this technology.

3. PROPOSED SYSTEM

   In this paper, a simple and economical system to communicate using breath patterns for people who have difficulty in speaking is designed. Patients and elderly people are prone to falling from beds or wheelchairs due to their weak constitution. So a fall detection system is also proposed. In addition, the feature of tracking the requests by patients and satisfactory response by nurse or caretaker is incorporated in the proposed system. The block diagram of proposed system is shown in Fig. 1

   Fig. 1. Block diagram of proposed system

   The patient is required to give two distinguishable exhales (hard and soft) which are converted into electrical signals using a microphone (ADMP401) placed under the nose or mouth of the patient depending on the convenience. These signals which are analog in nature are converted into digital signals using an analog to digital converter (MCP3008). The digital signals are received by the Raspberry Pi 2. Algorithms are performed on the received signals which look for patterns to distinguish between hard and soft breaths. Soft breaths are represented as S and hard breaths as H. Based on the combination of these representations, a predefined breath pattern is selected and the voice file attached to the corresponding message (words/sentences) is played out as speech using a speaker. In this paper, 8 words are implemented in a 3-bit table which covers all the basic needs of the patient. The breath patterns and their corresponding messages are shown in Table I.

   The patient request tracking system records the requests by patients and the subsequent responses by the nurses. If the response is acceptable and provided within a stipulated time frame, then patient ends his request by pressing a switch. If the service is not provided within the time period or not up to the patients standards, then the end time is automatically recorded with failed service description.

   Table I Breath Patterns and the Corresponding Messages

   The accelerometer (MPU6050) is used to detect accidental falls. When a patient/elderly person accidentally falls from his bed, there is an increase in acceleration which is measured using the tri-axial accelerometer attached to the patient. Acceleration data frm a single axis is inadequate to detect a fall occurrence since the center of gravity changes when the user assumes different positions resulting in false indications. So to make the device sensitive to falls in all directions and for accurate detection, the resultant magnitude of acceleration along all three axes(x, y and z) is taken [6]. If the resultant value is higher than the threshold value, it indicates that the patient has fallen down and the same is announced on the speaker so that appropriate action can be taken.

4. SYSTEM IMPLEMENTATION

   1. Determination Of Thresholds Of Hard And Soft Breaths

      About 50 series of breath samples from 5 people were obtained to establish the range for hard and soft breaths. It was determined that the peak of soft breath lies approximately in the range of 2 – 3.1V and that of hard breath in the range of 3.1 – 3.3V. Fig. 2 to Fig. 5 show four such series along with the peak voltages of the breaths. These waveforms were obtained using the line chart of Microsoft Excel.

      Fig. 2. Breath sample-1 (Hard breath:3.3V)

      Fig. 3. Breath sample-2 (Soft breath:2.76V)

      Fig. 4. Breath sample-3 (Hard breath:3.3V, Soft breath:2.49V)

      Fig. 5. Breath sample-4 (Hard breath:3.3V, Soft breath:2.70V)

   2. Breath to speech

      The flowchart for breath to speech conversion is illustrated in Fig. 6. When input switch is pressed, the Raspberry Pi 2 reads the voltage values of the breaths sensed by the microphone through the ADC (analog to digital converter) which converts the analog data into digital. The Raspberry Pi

      2 and ADC communicate through the SPI interface and exchange 3 bytes of data with each other containing the 10 bit digital data (ADC to Pi) and information on channel selection and input configuration (Pi to ADC).

      Fig. 6. Breath to speech flowchart

      After obtaining the samples of a breath, the peak value (initially set to 0) is compared with the current value of a sample. If current value is greater than the peak value then the current value becomes the new peak value otherwise the old peak value is retained. After going through all the samples the peak value of the breath is determined. Then the value is analyzed to determine if the breath is soft or hard and the corresponding light emitting diode (LED) glows. After 3 such breaths are sensed by the Raspberry Pi 2, the breath pattern is compared with the pre-defined patterns to identify the type of request made by the patient and corresponding message is played out as speech and the nature of request along with date and time is stored in the database.

   3. Patient request tracking

      The Service provided to the patient by nurse or caretaker is monitored using patient request tracking system. The flowchart for patient request tracking is illustrated in Fig. 7. When the patient raises a request (using breath) like wants water, juice, wants to go to toilet etc, a timer will start countdown of one minute using the system time as reference. Before one minute, if the service is fulfilled by nurse then patient will end the request by pressing the service switch. Then the request is closed with satisfactory remark. In case service is not fulfilled within one minute or not fulfilled satisfactorily, the patient can indicate the same by not pressing the switch. All the data regarding the services provided are logged on to a database with time. In this way, the patients concerned people or doctors can track if that nurse/caretaker is providing proper service.

      Fig. 7. Patient request tracking block diagram

   4. Fall detection

   Accidental falls are detected by comparing the acceleration readings with a preset threshold. The flowchart for fall detection system is illustrated in Fig. 8.

   Fig. 8. Fall detection system flowchart

   In the initialization part, the required header files are included and the variables are declared. In the next step, the circuit connections are checked and if there are any misconnections or faulty components, a connection failed message is displayed. If all the connections are proper and the components in working condition, a ready message is displayed and the control moves to the next step. Once the network setup is successful, the Raspberry Pi 2 starts reading data from the accelerometer using the I2C protocol. In the next step, the SMV (signal magnitude vector) [7] is calculated using (1):

   (1)

   where ax, ay and az are acceleration along x, y and z axes respectively.

   The SMV value is compared with a preset value of threshold. In this case the threshold value is set to 2g (g = 9.80665 ). If the SMV value is greater than or equal to the threshold value, it is established that the patient has fallen down and an alert message is played out on the speaker. If the SMV value is less than the threshold value, the Raspberry Pi 2 continues to read data from the accelerometer.

   Fig. 9 shows the hardware implementation of the proposed device.

   Fig. 9. Working model of the designed device

5. RESULTS

   The breath to speech module was tested by 20 healthy people in controlled environment (with almost no noise) and then in natural environment in order to keep the results fair. The results showed an accuracy of 99% in controlled environment and an accuracy of 95% in natural environment when the microphone was kept really close to the users nose or mouth. It was later deduced that any error which occurred was due to the false input by the users or due to the surrounding noise and not because of the device.

   For each breath exhaled by the patient, number of samples and the peak value of the breath gets displayed on the display screen. When the input switch is released, the voice file corresponding to the breath pattern given out by the user is played using the speaker. Then timer of 1 minute is started. If service is provided within the time limit, then Request Serviced in time message is displayed as shown in Fig. 10.

   Fig. 10. Request serviced in time execution

   If service is not fulfilled within 60 seconds of making the request, then Request timed out message is displayed as shown in Fig. 11.

   Fig. 11. Request not serviced in time execution

   After a particular action is taken on the necessity requested, the name of the request, nature of the service, date and time at which the action was taken towards the request are stored in a

   text file as shown in Fig. 12. This can be used by the doctors or patients concerned people to check if the nurse or caretaker is properly taking care of the patient.

   Fig. 12. Service history stored in database

   When the SMV value is greater than the threshold, a fall occurs. The accelerometer detects this occurrence and the voice file indicating that the patient has fallen down is played using the speaker. The first 3 values displayed in Fig. 13 represent the raw acceleration values measured along the 3 axes and the fourth value always represents a zero value when SMV value exceeds the threshold.

   Fig. 13. Fall detection execution

6. ADVANTAGES

   • Affordable: The designed device can be made available at a fraction of the cost of existing AAC devices, making it easily accessible to the ones in need.

   • Compact: The device uses minimal components making it compact, lightweight and also comfortable for the user handling the device.

   • Portable: The device has a size similar to that of a smartphone and can also run on a rechargeable battery thus making it portable.

   • Ease of use: The device is fairly easy to use once the user is trained with the pre-defined breath patterns. Because of its simplicity in terms of handling and operation, the user can quickly get accustomed to the device.

   • Faster: The use of Raspberry Pi which has a better computing and faster processing capacity makes the device run smoother and faster.

   • Comfortable: The non-usage of bulky units such as wheelchair, head gear etc., maes the device comfortable to operate and the employment of breath as the medium of interaction reduces greatly the strain on the user.

   • Flexible operation: The device can also be easily reconfigured to suit the requirements of each individual.

7. CONCLUSION AND FUTURE SCOPE

In this paper, a cost effective device was designed with high focus on accuracy, convenience and reliability which can re-invent the AAC systems. Current AAC systems have deviated from the main goal of allowing the speech impaired to freely interact with the world by including monitors, headgears or eye gears. This makes the devices bulky, slow, high-priced and difficult to operate. But the device implemented here uses breath as a mode of communication enabling the patient to ask for his basic needs with little effort. It also has other added advantages such as being affordable, compact and comfortable to use. The fall detection system decreases the frequency of fall happenings of elderly people/patients by judging the occurrence of falls timely and accurately. This mainly helps the elderly people or patients with weak constitution in getting immediate medical attention to prevent magnifying of injury if incurred. The inclusion of patient request tracking system compels the nurse to deliver acceptable services to the patients demands. It also provides a sense of comfort and security to both the guardians and the patient. This device would be useful in medical institutions or hospitals. It would be useful in the home environment. This device can be used by people with developmental disabilities like Locked-In Syndrome, Amyotrophic Lateral Sclerosis, Tetraplegia, people who have gone through tracheotomy surgery, people with vocal cord paralysis and brainstem stroke syndrome, aged people who are either semi-paralyzed or not able to communicate properly and many more.

A limited number of words are implemented in this paper, which suffice the basic needs of the patient. However the device can be further enhanced by implementing the Morse code to frame words, process them and output complete sentences allowing the user to interact better with his/her environment. The device can also be made to learn the cognitive and language demands of each individual user and adapt or adjust over time as the user learns his/her device. The speaking rate may also be increased by automatically completing sentences using auto-prediction. More advanced algorithms can be used for adaptively estimating and removing background noise from breath signals allowing the

device to be entirely operated using only the breath of the user. GSM facility can be added to notify nurses about the requests if they are away from patient and also to send messages to their guardians about the quality of services. Fall detection system can be enhanced to distinguish the falls from other body movements to reduce false fall detections. The patient request tracking system can be improved by maintaining transparency of medicine dosage administered to the patient and keeping a record of the dose history. Also, preserving documentation of the frequency of patients demands helps the nurse or caretaker to better understand the patients necessities.

REFERENCES

1. https://www.googlesciencefair.com/projects/en/2014/dde56de014edae3 e

   9516ac84d426641f372ba076d6f6b60a4809306f43b58bc6

2. http://professionals306.blogspot.in/2007/05/eye-tracking- assistive-technology.html

3. http://www.diku.dk/~panic/eyegaze/node27.html

4. Rutger J. Vlek, David Steines, Dyana Szibbo, et al., Ethical issues in braincomputer interface research, development, and dissemination, Journal of neurologic physical therapy, vol. 36, pp. 94-99, June 2012.

5. http://jhorst1.blogspot.in/2009/10/brain-computer- interfaces-advantages.html

6. Jing Luo, Bocheng Zhong, Dinghao Lv, Fall monitoring device for old people based on tri-axial accelerometer, International Journal of Advanced Computer Science and Applications, vol. 6, no. 5, pp. 34-39, 2015.

7. Jin Wang, Zhongqi Zhang, Bin Li, Sungyoung Lee, and R. Simon

Sherratt, An enhanced fall detection system for elderly person monitoring using consumer home networks, IEEE Transactions on Consumer Electronics, vol. 60, pp. 23-29, March 2014.