Voice And Vision Controlled Wheelchair For Disabled

Voice And Vision Controlled Wheelchair For Disabled

G Uday Kiran (A), N Nithesh Chakravarthi (B), K R Radhakrishnan (C)

(A)Pursuing B.E degree program in Electronics and Communication engineering in PSG College of Technology, Coimbatore.

(B)Pursuing B.E degree program in Electronics and Communication engineering in PSG College of Technology, Coimbatore.

(C)Asst. professor in PSG College of Technology, Coimbatore.

Abstract

In this paper, a new method is proposed beside the conventional method, to control the motorized wheelchair using voice signals and Infrared Oculography (IROG) signals. A user dependent voice recognition system, Eye ball movement tracking system (IROG), GSM module and an Emergency LED and Obstacle Detection (ELOD) system has been integrated in this wheelchair. The IROG method allows the user to look around freely while the wheelchair navigates automatically to the desired goal point. Voice recognition system performs pre-defined activities based on the voice commands (forward, backward, left, right, stop, send) given by the user. GSM module sends the predefined messages to family members and family doctor with a single voice command (send). The ELOD system handles the obstacle detection by restricting the motion of the wheelchair if an obstacle is detected and turns on ultra-bright LED if it is dark outside.

1. Introduction

   Conventional Joystick controlled wheelchairs [3] are not suitable for hand crippled and paralysis patients. There are many people with neural disability caused by accidents which leads to lose their ability to move the voluntary muscles. Speech is an ideal method for robotic control and communication. The speech recognition circuit functions independently from the wheelchairs main intelligence(CPU).This is advantageous because it does not take any of the wheelchairs main CPU processing power for word recognition. The CPU merely polls the speech circuits recognition lines occasionally to check if a command has been issued by the user to the wheelchair.

   This can even be improved by connecting the recognition line to one of the wheelchairs CPU

   interrupt lines. Thus, a recognized word would cause an interrupt, alerting the CPU that a recognized word had been spoken by the user. The advantage of using an interrupt is that we can avoid polling the circuits recognition line occasionally, further reducing any CPU overhead. Another advantage of this stand-alone speech-recognition circuit (SRC) is its programmability. SRC can be programmed and trained in order to recognize the unique words. The SRC can be easily interfaced to the wheelchairs CPU.

   The wheelchair control is further enhanced by adding vision control [1]. The robot will automatically move to a particular direction as the patient moves his/her eye in that particular direction. The eyeball sensor basically works with the infrared sensors to detect the movement of the eye ball direction. Lack of obstacle detection in conventional systems is rectified in this project and an emergency LED system is added to improve the visibility in darkness [4]. A GSM module is also integrated to the system allowing patient to send sms to family members and doctor in case of emergency. The main objectives of this project are given below

   1. To design an electronic system which can be installed in any commercial joystick controlled wheelchairs.

   2. To control the wheelchair by means of vocal commands, as well as by eye movement.

   3. To incorporate a safety sensor system in the wheelchair which would permit obstacles to be detected.

   4. The control system had to be open and modular, in the sense that future enhancements could be made.

      Another aspect of importance with regard to the design was that the final prototype should be

      economically priced for its later manufacture and commercialization

2. Proposed Architecture

   1. VOICE Mode

      In this driving mode, the wheelchair is controlled by means of various voice commands such as forward, backward, left, right, stop, and send [2]. The speech recognition process can generally be divided in many different components illustrated in Fig.2. The first block, which consists of the acoustic environment plus the transduction equipment (microphone, preamplifier and A-D converter) can have a strong effect on the generated speech representations. For instance additional impact generated from additive noise or room reverberation can be taken. The second block is intended to deal with these problems, and also deriving acoustic representations that are both good at separating classes of speech sounds and effective at suppressing irrelevant sources of variation. The third block must be capable of extracting speech specific features of the pre-processed signal. This can be done with various techniques like cepstrum analysis and the spectrogram. The fourth block tries to classify the extracted features and relates the input sound to the best fitting sound in a known 'vocabulary set' and represents this as output. The SRC can be prepared by either HM2007 or EASYvr speech IC, testing is done on both circuits and were able get satisfactory results from both circuits.

      Fig. 1. Proposed architecture

      Fig. 2. Speech Recognition Process

   2. VISION Mode

      In this driving mode, the wheelchair is controlled by means of various eye ball movements and eye blinking. The eye ball movement tracking is done by implementing Infrared Oculography. The technology works by the user wearing a tracking device on their face like a pair of glasses which has IR sensors on it. The IR sensor sends a signal to a central unit via infra- red LEDs, tracking eye movement precisely right down to the exact position of the iris, which then relays the message to the core system of the wheelchair. User must look in the direction they wish to move and the wheelchair responds. The eye blinking stops the wheelchair.

      Sensor assembly: Infrared transmitters and receivers are housed into a single unit and fixed in front of the goggles. Necessary wirings go through the goggles to the sensor processing. The user must wear the goggles for the sensor to work. The sensor unit consists of the following sections Oscillator: A 2 kHz oscillator is powered by an external 12v which provides the transmission signal to the infrared transmitter. IR transmitter: It transmits the IR waves when signaled by the oscillator. IR receiver: It receives the reflected IR waves from the eye. The buffered output is the analogue output from the corresponding receivers. The analog outputs from the sensor are then converted to digital using an analog to digital converter which is present in the microcontroller. The user should look straight initially when he/she wears the goggles as those values will be stored as base values for the straight view. Then comparing the present values with

      the base values we can detect the corresponding eye movement.

      Fig. 3. IR Sensor working

      Fig. 4. Difference in IR reception after the movement in eyes

      Fig. 5. Model of Eye sensor

   3. GSM and ELOD System

   The GSM module can be operated differently in voice and vision modes. In voice mode, emergency sms can be sent by speaking the respective command (send). In vision mode, to send a message panic button must be pressed as the number of eye movements are limited to identify. Panic button can be used in voice mode also.SIM900 GSM module can be used for this application. The results we obtained are good enoughfor the application.

   ELOD system stands for Emergency LED and Obstacle Detection system. Lack of Obstacle detection is one of the major drawbacks of the conventional systems, which makes it difficult for the patient who is navigating the wheelchair backwards [5]. In this project an ultrasonic sensor is used to detect the obstacles in the way. Ultrasonic ranging module HC – SR04 provides 2cm – 400cm non-contact Measurement function, the ranging accuracy can reach to 3mm.By using few of these sensors we can build an efficient obstacle detection system [4]. We are programming such that whenever the wheelchair/robot encounters an object it stops automatically, thus reducing the burden and danger involved. By installing multiple ultrasonic sensors on wheelchair in different places, the system efficiency can be improved a lot.

   A Cadmium Sulfide (CdS) Light-Dependent Resistor is used in the prototype to measure the light intensity. Even if we have a best navigation prototype, if it is in dark or dim environments we cant do anything if we dont have sufficient visibility. So, by installing photocells we can measure the light intensity and if it is dark, the ultra-bright led will automatically turned ON [5]. Ultra-bright LED are best choice for this application as they occupy less space and consumes less power when compared to other torches(or other sources) available.

   Two sensors in the same direction are not activated simultaneously in order to eliminate interference between them [5]. The decision as to which sensors are activated at any time should be made. Each photocell sensor will act a little differently than the other, even if they are from the same group. The variations can be really large, 50% or higher! For this reason, they shouldn't be used to t determine precise light levels in lux or milli candela. Instead, you can expect to only be able to determine basic light changes. When adding more CdS sensors, you find that the temperature is inconsistent, this indicates that the sensors are interfering with each other when switching the analog reading circuit from one pin to the other. It can be fixed by delaying the two reading one from other and tossing out the first one.

   Fig. 6. Location of Ultrasonic sensors

   Fig. 7. Flowchart for Emergency LED

   Fig. 8. Flowchart of Obstacle detection

   Fig. 9. GSM module parts

3. Implementation

   The whole design is implemented on two wheeled robot prototype. SRC and Eye ball sensor are interfaced with the robot. Sensor system is also integrated to the robot. ATMEGA328 is used as microcontroller which is the heart of the system. As for SRC, either HM2007 or Easyvr IC can be used. Satisfactory results can be expected from either of the ICs in the noise limited environment.SIM900 is used as the GSM module (have to be programmed separately). As for eye ball sensor, it prepared by arranging IR sensors to the normal goggles (with or without glasses) and readings are compared

   Fig. 10. L293D configuration

   using comparator connected to goggles. Care must be taken to choose the mic for voice recognition as its sensitivity plays the main role. Due to small size of robot chasis, multiple ultrasonic (HC-SR04) and photocells (CdS) are not used. GSM is integrated to voice module and to a push button which acts as a panic button. As we mentioned before this module can be used in vision or voce mode but not both at the same time.L293D is used as a motor driver.

   Fig. 11. A model of Eye ball sensor

   Fig. 12. Robot chasis with basic sensor system in voice mode (Eye ball sensor not connected in this picture)

   Fig. 13. A model of final prototype

4. Result Analysis

   The speech recognition was able to recognize the commands which were earlier proposed (forward, backward, left, right, stop, send). However, recognition failed for some commands which are phonetically similar. Eye Ball sensor is based on infrared oculography, hence its operation is highly dependent on stability of infrared sensors. The SR module was able to comprehend the commands in a noise-free environment, but it was seen that the recognition failed in a noisy-environment. This may be due to the sensitivity of the microphone in the SR module.

5. Conclusion

   A robot which operates based on the human voice and vision is of vital importance in various industrial applications where the work to be accomplished is physically impossible by human beings, but can be successfully accomplished by a trained robot. These type of robots are indispensable in situations which are drastic and dangerous to life forms. Thus, using it in medical field for the sake of physically challenged is possible.

   By using proper IR sensors in eye ball sensor, favourable results are obtained. But we observed that vision control is not that efficient when used as an only control option, as it strains eye a lot. But by combining it with voice control the system can be made efficient. The SR module was able to comprehend the commands in a noise-free environment, but it was seen that the recognition failed in a noisy-environment. This may be due to the sensitivity of the microphone in the SR

   module .By improving the microphone sensitivity and adding the wireless control functionality the system can improve even further. The wheelchair has been made speaker dependent, but its design can be extended to make it speaker independent thereby improving its performance. The size and speed of the wheelchair can be made variable to cater the needs of various applications. The ELOD system and IR sensors are economical and the SRC also comes with affordable price. Thus making the prototype economical and affordable to all kinds of people. Another version of prototype can be implemented in future in such a way that it can be integrated to existing joystick controlled wheelchair by making it more economical.

6. References

1. Arai K, Purwanto D (2007) Electric wheelchair control with the human eye. 7th International Conference on Optimization: Techniquesand Applications, December 2007.

2. Wouter Gevaert, Georgi Tsenov, Valeri Mladenov, Senior Member, IEEE, NEURAL NETWORKS USED FOR SPEECH RECOGNITION in Journal Of Automatic Control, University Of Belgrade, Vol. 20:1-7, 2010.

3. Borenstein, J. and Koren, Y. The vector field histogram fast obstacle avoidance for mobile robots, IEEE Transactions on Robotics and Automation, 7(3):278-288,1991.

4. Manuel mazo, francisco j. Rodriguez, josi~ l. L,~zaro, jesi~is urei~a, juan c. Garcia, enrique santiso, pedro revenga and j. Jesi~is garcia(1995), Wheelchair for Physically Disabled People with Voice, Ultrasonic and Infrared Sensor Control, Autonomous Robots, 2, 203-224 ,1995.

5. Lou, R.C. and Lin, M.H. Robot Multi-Sensor Fusion and Integration:Optimum Estimation O f Fused Sensor Data. IEEE Trans. on Robotics and Automation,RA-2(3):1076-1081,1988