Smart White Cane – A Sophisticated and Compact Walking Aid for Blind-Dumb Deaf Communication with GEO Location Indication

Smart White Cane – A Sophisticated and Compact Walking Aid for Blind-Dumb Deaf Communication with GEO Location Indication

Chandana L, Gajula Lavanya, Lavanya R, Likhitha B, Mrs. Veena V Pattankar

IV Year B.E Computer Science and Engineering, Global Academy of Technology, Karnataka, India.

Assistant Professor, Department of Computer Science and Engineering, Global Academy of Technology, Karnataka, India.

Abstract:- Physically disabled people like blind are sensitive to getting in contact with whatever obstruction which might pass by them while performing daily activities like walking, subjecting them to the threat of injury due to falling and it could also cause great damage to them. The target of our project is to develop a smart walking aid with measurement of distance from obstacles in front of them using IoT and it is compatible with Blind-dumb- deaf. In our project we are enriching the stick with a variety of sensors. The stick contains ultrasonic sensors for distance measurement. In the project, pulse sensor and temperature sensor continuously checks pulse and body temperature and Vibration motor is used as a secondary output. OLED will display the data and it helps the deaf to seek help. Raspberry Pi microcontroller processes the information which later produces a voice for alerting the blind person as the output. GPS live tracking systems will be used to track the blind people. Overall, the system notifies the hurdles to the vision-impaired through the auditory output through which they can walk safely without any problem. The details will be stored in Google Firebase and the data will be monitored using an Android app.

Keywords:- Raspberry Pi microcontroller, MCP 3008, Ultrasonic sensor, Temperature sensor, Pulse sensor, OLED, GPS, Google Firebase, Android App.

1. INTRODUCTION

   Worldwide, the number of vision-impaired, partially sighted, near-blind individuals are around 1.0-1.5 billion. In India alone itself, there are 40-50 million people which is almost 20% of the overall population of the World. Vision-impaired, partially sighted people are the ones who have complications and difficulties even to recognize basic things around them which are small. Individuals with visual acuteness of 6/60 and the transverse distance of eyes with both open is equal to or less than 20 degrees. These individuals are considered to be blind.

   The major dilemma of blind is how to traverse their way in any direction to reach a particular destination. The above numbers specify the requirement of blind sticks across the world. Smart White Cane is an inventive walking aid reasoned for visually impaired individuals to enhance steer Man ship. The Smart White Stick is designed with the help of IoT technology.

   Internet Of Things – IoT is nothing but an inter- network of physical objects called things consisting of various softwares, sensors, electronics which are inter- connected to a network connectivity. This enables the physical objects to send and retrieve the data.

   A Smart White Cane is an Electronic Technology using IoT Technology to help visually impaired individuals and also assist deaf and dumb. The proposed stick or device is a Raspberry pi based automated hardware that can help a vision impaired to detect barriers or objects in front of him/her correctly and assist dumb with OLED screen with pre-installed instructions with the help of keys which help them to seek help in proper conditions. The hardware used for the device consists of a Microcontroller pin and ultrasonic sensor for obstacle detection. The proposed sticks design is easy to use and also the cost of the stick is quite less compared to others available in the market. The power consumption of the proposed stick is low and can be operated easily. The system is made up of an ultrasonic sensor and camera as input and the output is given through voice commands.

   To measure the distance from the object, Ultrasonic Sensors are used since they use sound waves to calculate distance by reflection of them. The Raspberry Pi camera detects the human. Pulse sensor and temperature sensor continually checking and health Status and Vibration motor detects accidents like if a person falls down then it will give information to an authorized person. Data is then sent to Raspberry Pi microcontroller for data processing and running various modules which further produces output through voice. Also the system includes the GPS live tracking system and guides the path for blind people. The device notifies blind individuals of the various barriers through the audio output through which they can walk safely without any problem. We are storing each to the cloud called Google Firebase and anyone can monitor data using web and Android apps. It's designed with the MIT app inventor tool using this app we can monitor anywhere in the world.

2. REVIEW OF LITERATURE

   Over the years, from 1921 when the first walking stick was designed, a bunch of testing and analysis have taken place in developing a compact and sophisticated walking aid for the blind people. Each of the research included some common points and drawbacks.

   Some of the reviewed papers are:

   1. Multi-Functional Blind Stick for Visually Impaired[1]:

      The paper used ultrasonic sensors for distance measurement from the objects and GSM modules are used

      for tracking and provides information to the respective individuals using apps. However, the usage of the GSM modules for tracking will cause bandwidth lag and low data rate.

   2. Smart Stick for Blind People[2] :

      The paper used ultrasonic sensors for distance measurement from the objects in the range of 4m, uses radio frequency to detect the blind stick and the controller used is Arduino UNO. However, the proposed solution has no feature of detecting the blind person.

   3. Smart Cane For Visually Impaired [3]:

      This system has also used ultrasonic sensors for distance measurement from the objects, vehemence noting module, wet spotting module is used to detect water under the stick, brightness noting, hole and steps noting module using IR sensor and sends notification using GSM modules. However, Infrared radiation has lower frequency and has drawbacks during dark times. GSM requires higher bandwidth hence, it causes bandwidth lag.

   4. Smart Assistance Navigational System for Visually Impaired Individuals [4]:

      This system is designed with an Arduino board and the objects are detected using ultrasonic and infrared light rays. However, Usage of RF transmitters and receivers can have interference from other radio waves.

   5. Smart Electronic Stick for Visually Impaired using Android Application and Googles Cloud Vision [5]:

      This system is designed with RIR sensors, ultrasonic sensors and GPS. However, RIR sensors do not detect long wavelength infrared radiations.

   6. IoT based smart walking cane for typhlitis with voice assistance [6]:

      The paper used ultrasonic sensors for distance measurement from the objects. The latitude and longitude of the physically disabled are located using GPS. However, because of using infrared sensors, infrared frequencies are affected by hard objects.

   7. Advanced Voice Based Blind Stick with Voice Announcement of Obstacle Distance [7]:

      The paper used ultrasonic sensors for distance measurement from the objects, Panic switch, and accelerometer are provided but due to usage of GSM modules it will cause bandwidth lag.

   8. An Intelligent sensor based stick for Blind and Deaf [8]:

      Obstacles are detected using proximity sensors, a short circit sensor is used to detect the water under the stick and a vibration motor is used as a secondary output for deaf. Proximity sensors detect only metallic objects and they are affected by the temperatures which are not suitable for obstacle or object detection.

3. EXISTING WORK

   Problem Statement

   Now-a-days, numerous exploration or inquest efforts are emphasized on the blueprint of Smart Walking Aid which helps the blind in travelling. Recently, some of the new advanced solutions in the blind stick helps the blind to traverse themselves. An additional objective why ultrasonic sensors are better to use, is because of its low cost. Furthermore, ultrasound emitters and detectors are easily movable elements which can be easily carted in absence of complicated circuits. The stick also helps the person to search the blind stick, whenever it is misplaced. When a fellow tries to search the stick, by using an Android app he connects to the module, by clicking any button in the app then the buzzer gives a signal to the blind person.

   Existing System

   In Order to resolve the complications confronted by the blind people, we have some development in the field of IOT to generate smart blind sticks, smart belts etc., Almost all of these gadgets were developed using ultrasonic sensors/IR (Infrared) sensors to find the obstacles near to the blind person. Since the IR sensors can detect only shorter range objects which sometimes put the blind people in trouble.

   This reason makes the ineffectiveness of using that type of stick in case of dangerous conditions. Further, some of the sticks in the market use RFID(Radio Frequency Identification)sensors to detect the obstacles. Still, using the RFID avoids people using the stick only to domestic climates; those are the areas which are labelled only using RFID sensors.Alternately,using RFID sensors increases the cost to develop the stick which can be used in the outdoor climates, since many obstacles need to be labelled. Some of the blind sticks use GPS (Global Positioning System) and GSM (Global System for Mobile) modules for navigation purposes to blind , for sending SMS in emergency situations or to make a call if the person is in danger.

   The foremost limitation of the current system is that it depends excessively on the GSM module. This is because the GSM module requires more power consumption. Also, GSM modules provide low internet speed in contrast to other mobile applications which sometimes cause ineffective outcomes for GPS tracking. In Addition, the total amount required to build the stick also increased and it reduces the quality of the stick. Since the accessible sticks have insufficient competency and productiveness, we recommend a fitter solution that helps the physically disabled people like blind.

4. PROPOSED METHOD

   In order to vanquish the drawbacks of currently available sticks, we have proposed a system which utilizes a mobile application alternatively in place of global systems for mobile applications (GSM), the mobile application is used to track the stick holders location which helps in tracking the person, alert messages will be popped up during difficulty or panic situations and for additional operations. Accordingly, the major supremacy of the recommended

   technique is that maximum work of the information deciphering will be carried out by the mobile application. In consequence, the task which requires the microcontroller's memory and CPU time will be diminished and is narrowly limited just to detect the obstacles, temperature and pulse detection and biometric identification like face features determining tasks. Therefore, the charge required by the stick is drastically reduced. Additionally, the usage of IoT connectedness in place of cellular networks like global systems for mobile application modules eliminates the time lag that occurs due to slow network connectivity and also provides better outcomes for GPS navigation. Also this approach reduces the total price and briefness of the blind cane. In addition, the recommended technique also delivers numerous other attributes which are not present in the available procedures, like detecting and recognising human faces and providing alert messages using Googles cloud vision Application Program Interface.

   Therefore, the recommended technique in addition to the extra facilities being provided, helps to improve the total execution more efficiently than the previous devices.

   Fig. 1 Block Diagram of the Proposed Solution

5. IMPLEMENTATION The proposed model is divided into 4 Modules:

1. Obstacle Detection Module

2. Smart Health Check-Up Module

3. Data Processing and Android Application Module

4. Face Recognition Module

1. Obstacle Detection Module:

   Object detection is one of the important tasks that must be performed to help the blind to navigate easily from one place to another, especially in an unknown region. To detect the objects and holes we are using ultrasonic sensors. When the obstacle is detected within a given range then the gap between the individual and the object is gauged and the distance alertness will be provided to the blind to navigate easily.

   The Ultrasonic Sensor works on the principle that is similar to RADAR which transmits a signal and analyses the distance between the target object by apprehending the thrown back signals. It has three working parts: a transmitter, a circuit to control and an ultrasonic receiver. The Ultrasonic

   Transmitter will generate 40 KHz Ultrasound to detect the objects. This wave then propagates through air and if there is any object detected in its propagation, the wave will hit the object and bounce back. That bounced wave will be received by the receiver. The propagation time that is hitting the object and travelling back time will be calculated and based on the duration it took to travel the distance will be calculated and given as the output.

   Fig. 2 Obstacle Detection Modules Output

   Fig 2. Depicts the Output Of the Obstacle Detection Module. The input to the ultrasonic sensor is the sound wave travelling back from the object.

2. Smart Health Check-Up Module:

   The Elderly and disabled people increase the responsibility of caring for them when they are staying alone at home. Now- a-days, Health has become one of the most important concerns all over the world. Due to covid-19 pandemic everywhere pulse and temperature are checking, so people are significantly more worried about their health since cases are rising rapidly. Hence it is crucial to monitor the physical health of the patient. Due to various problems, patients health monitoring is very essential during all times.

   This module mainly consists of 2 sensors:

   • Pulse Sensor

   • Temperature Sensor Pulse Sensor:

     The pulse sensor measures the pulse rate by penetrating the infrared light into the fingertip and detects the heart rate by measuring the difference in the amount of blood circulation through which the number of times the heart beats will be calculated. Based on the change in the light transmitted heart beat is measured. Pulse sensors are used in restricted areas where radiance can effortlessly move-in like fingertip and auris externa.

     The vitals of the person holding the stick is input to the module and the output will display on the OLED provided to the stick and data will be sent to the cloud. We are using the Adafruit library for measuring the pulse rate. The Adafruit library is installed using the pip command.

     Fig. 3 Pulse Rate BPM Output

     Figure 3, depicts the heart beats found while testing the sensor. When the heartbeat is not found it produces the output as no beats found.

     Temperature Sensor:

     The sensor determines the degree or internal heat of a person's body. The sensor used in this project to measure the temperature is DHT11. It is a digital sensor for sensing temperature and humidiy with low-cost. DHT11 has the capacity to measure the temperature in the range of 0 to 50 degree Celsius. It has a sampling rate of 1Hz,that is it produces one reading for every second.

     DHT11 has a capacitor which consists of 2 electrodes and contains a dielectric between them. The capacitor's values change according to the change in the temperature and humidity. Based on the changes, temperature and humidity will be measured. The human vitals are given as input to the sensor and an adafruit library is used to measure the temperature values.

     Fig. 4 Output of the Temperature Sensor DHT11

     Figure 4, depicts the output from the DHT11 sensor while testing and this output will be later processed and is used to monitor the health condition.

3. Data Processing and Android Application Module:

   This module helps to transfer the data from the sensors to the cloud and mobile applications and vice versa. Cloud, Google firebase and MIT App Inventor are the technologies used in this module.

   Working Procedure:

   Raspberry pi is connected to the google firebase using firebase URL. The data from the GPS, temperature and pulse module is stored in the firebase.

   Fig.5 Data Storage in firebase

   The fig.3, depicts the storage of sensor data in the real-time database of google firebase. The values will be changed according to the change in sensors output. The MIT app is connected to the cloud using firebase URL and unique key. In the mobile application data will be displayed in the understandable format. When pulse rate and temperature varies abnormally or above the normal range, notification will be provided to the concerned people. From the latitude and longitude, location will be determined and based on it, a marker or pointer will be placed on the map and helps to track the person. Additional Find me option is provided in the app and by clicking the button buzzer will be activated which helps in finding the lost smart cane.

4. Face Recognition module:

Visibility is actually among the most crucial provided presents for any man. An individual created with a visual disability or maybe has lost the perception of his for some other factors such as for instance mishaps or even years element suffers a great deal for his whole lifetime. This will make the individual be determined by many other individuals to do his/her daily actions.

KNN algorithms are actually utilized for detection and face recognition that may be quickly employed for understanding the actual individual.

The face recognition system functionality recognizes the face of the person in front of the camera by matching facial features from the database of known faces. The input is a video of a person taken from a raspberry pi3 camera. If the face is identified properly, then the name of the recognized face is given as output through speakers/earphones.

The known peoples images will be collected for recognition to the blind using different modes. More than four face images of the person will be collected for the greater recognition of the face.

Machine Learning using KNN K-Nearest Neighbours

(KNN) is actually among the easiest algorithms employed in Machine Learning for classification as well as regression problems. KNN algorithms utilizes the information & classifies brand new data points depending on similarity methods (e.g. distance function). The distinction is actually accomplished by a vast majority of the vote to the friends of its. The information is assigned to the category that has the maximum nearest friends. Importing the required libraries needed to apply the KNN Algorithm in Python. The NumPy libraries for scientific computation are installed. Then, matplotlib.pyplot library is also imported for plotting the graph.

Detecting and recognition Face

The KNN model that is trained in the proposed work is actually utilized to identify the facial skin at the raspberry pi camera video feed. The video clip frame with experience detected will be given to the KNN design for face recognition. The KNN classifies the detected face with the repository. The face with the greatest matching score is actually selected. The selected ID of the largest matching facial skin is then converted to the title.

The title received is actually given to the content to the speech module to talk the title to the end-user.

Fig. 6 Face Detection and Recognition Output

V. RESULTS AND DISCUSSION

The solution proposed in the paper is a Smart Walking White Cane for individuals who are vision-impaired or blind. It is much more convenient, easy to use and the cost is quite less compared to the traditional sticks used by blind nowadays.

The main ambition of this Smart White Cane is to provide a sense of vision to the blind. This could be considered as a small step towards it.

Utilizing this Smart White Cane, it would be effortless and easy for the vision-impaired to execute daily chores like

traversing or navigating and routing through different routes, GPS Location Tracking, Obstacle Detection, also Face Recognition helps communication with people with much more essential and legitimate.

Consequently, by preserving the straightforward, easy blueprint of the cane currently used by them, a more compelling remedy can be furnished to solve the problems of the vision impaired individuals.

Therefore, in this project we are integrating the current trending and technologies for developing a project which would be a boon to the blind and hence benefits the society.

OUTPUT

Fig 7. Final Output of the Smart Cane

REFERENCES

1. Vanitha Kunta; Charitha Tuniki; U. Sairam Multi-Functional Blind Stick for Visually Impaired People, IEEE 2020.

2. N. Loganathan ; K. Lakshmi ; N.Chandrasekaran ; S.R. Cibisakaravarthi ; R.Hari Priyanga ; K.Harsha VarthiniSmart Stick for Blind People, IEEE 2020.

3. Sankari Subbiah; S Ramya; G Parvathy Krishna; Senthil Nayagam Smart Cane for Visually Impaired Based On IOT, IEEE 2020.

4. S. Divya; Shubham Raj; M. Praveen Shai; A. Jawahar Akash; V. Nisha Smart Assistance Navigational System for Visually Impaired Individuals, IEEE 2019.

5. Devshree Bharatia ; Priyanka Ambawane ; Piyush Rane Smart Electronic Stick for Visually Impaired using Android Application and Googles Cloud Vision ,IEEE 2019.

6. SathyaNarayanan E ; Gokul Deepan D ; Nithin B P ; Vidhyasagar P IoT based smart walking cane for typhlotic with voice assistance , IEEE 2018.

7. Shashank Chaurasia, K.V.N. Kavitha, An Electronic Walking stick for Blinds, ICICES – S.A.Engineering College, Chennai, Tamil Nadu, India, 2014.

1. Arvind R. Tamboli, Avinash N. Suryawanshi, Aman D. Bohra, Dipak D. Pal, Jagdish Y. Kapadnis, IoT based Smart Stick for Blind People, IJARIIE-ISSN (O)-2395-4396, Vol 3 Issue-2 2017.

2. Rupali A Tanpure, Pragati D Borkar, Madhuri R Falke, Monali Dhale, Advanced Voice Based Blind Stick with Voice Announcement of Obstacle Distance, IJIRST, Volume 4, Issue 8, January 2018.

3. Gaurav Sareen, Parminder Kaur, Komal Marwah, Hitesh Vohra, An Intelligent sensor based stick for Blind and Deaf, Australian Journal of Basic and Applied Sciences, 11(8) Special 2017.

4. Sharada Murali et. al. Smart Walking Cane for the Visually Challenged. Department of ECE, SSN College of Engineering, Chennai, India, August 2016.