EOG Based HMI For Paralysed People To Control Electrical Devices

EOG Based HMI For Paralysed People To Control Electrical Devices

K. M. Mamatha1, Sumalatha.S2, Nalini. S3

1Department of Electrical and Electronics, Vishveshvaraya Technological University, II SEM, M.Tech, Dr.Ambedkar Institute of Technology, Banagalore-560056.

2Assistant Professor Grade-III, Department of Electronics and Communication,

Acharya Institute of Technology, Bangalore-560090.

3Associate Professor, Department of Electrical and Electronics, Dr.Ambedkar Institute of Technology, Bangalore-560056.

Abstract

Bio-based human machine interface (HMI) has attracted more and more attention of researchers all over the world in the recent years. In this paper, a HMI system based on electro-oculogram (EOG) is proposed. It transforms electric potentials recorded by horizontal and vertical EOG into computer/pattern re-organization module wirelessly, in order to control external equipment according to eye movements. The objective is to design a HMI to control electrical devices using EOG signals. The HMI output is used to move the cursor on the graphic display which has several buttons and each button on clicking by blinking of eyes activated corresponding electrical devices. The RF interface between acquisition and processing part and application so that is easy to handle and easy to install in homes and hospitals.

1. Introduction

   Bio-based HCI (Human Computer Interface) [3] has the potential to enable severely disabled people to drive computers directly by bio-electricity rather than by physical means. A study on the group of persons with severe disabilities shows that many of them have the ability to control their eye movements, which could be used to develop new human computer interface systems to help them to communicate with other persons or control some electrical instruments.

   1. Electro-oculogram Principle

      Electro-oculography (EOG) is a new technology of placing electrodes on users forehead around the eyes to record eye moments. EOG is a very small electrical potential that can be detected using electrodes. EOG signals haves the characteristics as follows: the amplitude is relatively high (15-

      200µV), the relationship between EOG and eye movements is linear.

      The electro-oculogram (EOG) is an electrical signal produced by the potential difference between retina and cornea of the eye [1]. This difference is

      due to the large presence of electrically active nerves in the retina compared to the front of the eye i.e., cornea. Many experiments show that cornea part is a positive pole and the retina part is the negative pole in the eye ball [1]. Eye movements will respectively generate voltage up to 16µV and 14µV per 1º horizontal and vertical way. The typical waveforms generated by eye movements are shown in figure 1.1 [1].

      In figure 1.1, positive or negative pulses will be generated when the eyes rolling upward or downward the amplitude of pulse will be increased with the increment of rolling angle, and the width of the positive (negative) pulse is proportional to the duration of eyeball rolling process.

      Figure 1.1: Typical EOG waveforms

2. Methodology

   In our HCI system, four to five electrodes are employed to attain EOG signals [1]. Figure 2.1 shows the electrode placement.

   Figure 2.1: Electrode placements

   1. and 4 for detecting vertical movement

   2. and 3 for detecting horizontal movement 5 is for reference

      Blink detection is a separate algorithm based on EOG signals.

      1. Block Diagram of the Proposed System

         alarm generates a 30sec ring when clicked working slightly different from other appliances which needs human intervention to be reset.

         Processing for eye movement

         Processing for eye movement

         Acquisition and Processing Part

         Electrodes used to sense eye signals

         Instrumentati on Amplifier

         ,active filter

         ADC

         Electrodes used to sense eye signals

         Instrumentati on Amplifier

         ,active filter

         ADC

         Application Part

         Graphics LCD

         Graphics LCD

         Embedded Controller For data acquisition using I2C

         RF TX Module

         RF TX Module

         RF RX

         Module

         RF RX

         Module

         Figure 2.3: Internal circuit diagram for acquisition part

         Devices to be controlled by eye movement s

         Display Screen and Device controlle r

         Eye Movem ents output

         In the above figure 2.3, op-amps U1A and U1B are used to adjust the gain of the signals acquired from left and right movement of the eye. Similarly we use op-amps U2A and U2B to adjust the gain of signals acquired from up and down movements of the eye. The op-amps U1C and U2C are the first order low pass filters for right-left channels. We use a two channel amplifier [2] one for left-right and other for up-down and blink movement.

         Figure 2.2 Block diagram

         1. Brief Description

   In the above figure 2.2, outputs of the electrodes are amplified and filtered using instrumentation amplifier and active filter. Using an ADC, the signal is digitized and this completes the acquisition of the signal. Acquired signal from all the electrodes are sent via RF interface for further processing. Finally the eye movement and eye blink are extracted and sent as commands to drive on screen cursor, using eye movements cursor is controlled using eye blinks click operation is done, the controller is connected to a relay board. The clicks on the button turn the switch on or off. Hence the appliances get turned on or off. The

3. User Interface and End Devices

   The two application programs to test the system are the GLCD and the patient assistant software. The GLCD users make cursor move up, down, left and right to select a letter in the table. The devices are selected by voluntary eye blinking.

   A 128*64 graphic LCD is used. This GLCD comes with a EL backlight and a four wire analogue touch screen. The GLCD is controlled with the ATMega16, the GLCD used is HG1286418C-VA with a KS0108 controller is used. The display has 8 bits and 5control bits.

   Figure 3.1: circuit diagram of application part

   In the above figure 3.1, the interfacing of GLCD with the micro-controller, connection of GLCD and the controller is shown.

4. Implementation

   Software: KEIL, Flash magic and Hyper-terminal PC requirements: Pentium 4 PC or higher,

   OS: Win XP or higher, minimum 1GB RAM.

   The following flowcharts in the figure 4.1, shows the process of decoding the eye movement data. Digital data from ADC is the input for each channel, check for straight sight and avoid noise. This is done by checking that signal is not varying much in some band near centre as shown in figure

   1. When signal goes UP for sufficient time > 200ms [3], then its right eye movement in case of L-R and up movement in case of U-D, but if signal goes down depending upon the channel. Any of the case if it comes back before sufficient time then movement is ignored but in case of up-down movement, if signal is up for >50ms to <100ms [3] then it is considered as blink movement which is shown in figure 4.2 and 4.3.

Initially we have to look in the straight direction [2] and signals are stabilized; now it is turn to give commands using eye movements.

Figure 4.1: Flow chart for processing straight signal

The left, right, up, down and blink commands are sent via RF transmitter module and received through the RF receiver module at the application part end. These ommands are decoded and send to the application controller which then drives the cursor and operates buttons on GLCD.

Figure 4.2: flowchart for left-right detection

Figure 4.3: flowchart for up-down and blink detection

Figure 4.4 gives the flowchart to perform he initialization of GLCD.

Figure 4.4: flowchart to initialize GLCD

The figure 4.5 shows the flowchart to perform menu execution button on GLCD.

Figure 4.5: flowchart for programming of user interface buttons on GLCD

Conclusion

The EOG based HMI can be designed and implemented successfully. Intermediate output of the system is eye movements and eye blinking signals. These signals are then processed into commands at UART interface. These commands are sent to the application part which interprets and moves cursor accordingly and button is clicked using the same output and corresponding relay is toggled hence appliance is controlled.

Scope of Improvement

The major factors affecting the system include drift in the eye movements and by implementing even more efficient algorithm, high end circuitry and embedded system software.

Even it can improved by employing a larger GLCD which can accommodate more icons. A zig-bee module can be used instead of RF, where in a star topology can be used to link a network devices can be controlled through centralized server.

References

1. Andreas bulling, Jamie A. ward, Hans gellersen, Gerthar troster, Eye movement analysis for activity recognition using electrooculgraphy IEEE transactions on pattern analysis and machine intelligence, vol 33, NO. 4, April 2011.

2. David Raamirez Perez, Human Machine Interface Communication using EOG for handicapped people, Congress on Instruments and Applied Sciences.

3. Siriwadee Aungsakun, Angkoon Phinyomark, Pornchai Phukpattaranont and Chusak Limsakul, Development of robust electrooculgraphy (EOG)- based human-computer interface controlled by eight-directional eye movements, International Journal of Physical Sciences, vol 7(14), pp. 2196- 2208.

4. I2C bus specification (version 2.1), from NXP Semiconductors (Philips).

5. Datasheets of all the ICs used in the system.

6. Manuals of Keil software.

7. C and the 8051, Thomas w.Schultz.

8. The 8051 Microcontroller, Kenneth J Ayala.

9. http://en.wikipedia.org/wiki/Eye_tracking.