Smart Assisting Visually Impaired Stick (SAVIS)

Smart Assisting Visually Impaired Stick (SAVIS)

Lim Tim Choong

Final Year Student, Bachelor of Electronics Engineering (Communication) with Honours

QUEST International University, Ipoh, Perak.

Mr. Karna Vishnu Vardhana Reddy

Senior Lecturer, Faculty of Science and Technology QUEST International University,

Ipoh, Perak.

Abstract The blind stick is one of the tools or equipment that been daily bring along to navigate and detect obstacle for blind or visually impaired people. This blind stick has its limitation that is can detect the ground obstacles by hitting and let the people to move either left or right direction. Besides that, blind people have also been faced some awkward situations such as hitting walker's leg by blind stick, injury by hitting the obstacles. Thus, this research project is to help them to overcome such awkward situations of using the blind stick.

The blind stick designed in this project use Raspberry Pi 3 B+ module as a main processor to compute the task activity. This blind stick will be implement with a night vision camera, which captures the image of the obstacle. After that image will be processed using digital image processing software to detect the edges. Then, system sends the PWM signal to motor wheel, which control the direction of the stick to avoid hitting obstacle. A GPS module is used to help blind people navigate in proper direction. Telegram app has been used for automatic sending the real-time GPS location and picture to a caretaker by pressing a button on the stick. Caretaker check the location of the bind people whether they moved away from the current location by sending a text message to the system in the stick. Apart from that, object identifying function also been implement into this project.

Keywords Blind stick, Obstacle detection, Route searching, Emergency message.

1. INTRODUCTION

   Due to nervous system scientist of classifying of human 5 sense list, eyes are one of the major and important senses in the human body. Human eyes are the one receives sensory to provide vision to the human. In 2018 world health organization (WHO) was announced the statistics in this globally of 7 billion people, there are 36 Million people are blind [1]. In this 36 Million of blind people, 82 percent of the blind people are over 50 years old. Therefore, due to this announcement, the number of vision impaired and blind people are starting to increase by population growth and aging [2].

   To overcome the vision inconvenient of blind people, Guilly d'Herbemont from France in 1931 [3] is invention and launches a white cane or blind stick to provide guiding movement for blind people to detect and avoiding hitting obstacle, balances, supports when they walking. Moreover, in 1942 Guide Dogs for the Blind (GDB) provide guide dog services to help them. Therefore, until today, these two methods were popular used to help blind people in the days of walking and crossroad to anywhere.

   Due to globalization and modernization era, they have a lot of electronic hardware and software were invented to solve the traditional limitation and ease their life. The machine and

   system have been used to help blind people are using electronic sensor help blind people to avoid blind people hitting an obstacle. The sensor has been used are such as ultrasonic sensor, RFID punching system, laser line and so on.

2. LITERATURE REVIEW

   In Dang, Q., Chee, Y., Pham, D., & Suh, Y.[4], line laser- based Vision system and Inertial Measurement Unit (IMU) sensor have been used. This project the line laser-based vision system is used to detect the obstacle distance and the type of obstacle. The IMU and camera used in this project is to improving the accuracy of an obstacle position and orientation pose.

   K.S.Manikanta et al.[5] integrated an intelligent blind stick with ultrasonic sensor to detect an obstacles by the ultrasonic waves after that if an obstacle is closer enough, the buzzer sound will give the signal feedback to the blind people. Moreover, this stick also uses to detect the water and alerts to the blind people by different buzzer sound. Another features in this stick is to helping blind people find out where their stick keeps by pressing a wireless RF remote button and the stick will buzzer out the sound.

   B. Li et al.[6] proposed a navigation system method to help blind people navigate in the indoor environment. This system used a software to develop a map to parse geometric information from architectural models and generate a semantic map. The Google Tango device are designing to leveraging the visual positioning service (VPS). Apart from that, the on- board RGB-D camera used to detect obstacle and avoidance approach based on a time-stamped map Kalman filter (TSM- KF) algorithm. To give the feedback signal to the blind people, speech-audio interaction and robust haptic interaction are been used.

   S. Dhananjeyan et al.[7] implement a system to provide path planning for blind people. This system is when the blind people pressing a button and speak the desired destination; the blind customer care will guide the path instruction to them. However, to ensure the blind people start to the destination or reach the destination, the blind people have to punching the radio frequency identification (RFID) system to provide location checkpoint and prevent wrong path. The ultrasonic sensor also used to prevent blind people collision to the obstacle.

   Alarden et al.[8] using the RGB-D camera and digital image processing of the Canny edge detector and probabilistic Hough line transform to identify the obstacle and giving the

   avoid direction feedback response to the blind people by haptic to avoid hitting the obstacle.

   Kanwal et al.[9] present a research of using Kinect camera with the Harris and Stephens corner detection method of digital image processing to detect the obstacle. After that, the feedback respond signal of the obstacle direction will giving by the haptic interaction to the blind people.

   Ruxandra et al.[10] present a research of using the sensor in the Smartphone and using the algorithm of Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Histogram of Oriented Gradients (HOG) features extracted and (Support Vector Machine) SVM to identify the direction of the obstacle. After that, send the feedback respond signal to the blind people by the haptic interaction.

3. PROBLEM STATEMENT

   • Inaccuracy of sensor detection

   • Needed sighted friend or family accompany to navigate unfamiliar places of outdoor or indoor.

   • Insufficient Guided dog supply to the visually impaired people and sensitive issue in some countries.

4. RESEARCH OBJECTIVE

   • To develop a Smart Assisting stick with autonomous wheel and camera to avoid blind people hit an obstacle and identifying any object.

   • To automatically send information message about their location to caretaker or guardian via the Telegram.

   • To implement GPS module with Google map to get destination route.

5. PROJECT DESIGN

   In this project, the Table 1 and Table 2 show the hardware and software that has been used in project. The function of each hardware and software used also been mention in Table 1 and Table 2.

   Hardware	Function
   Raspberry pi 3 B+ module	Control task activity of whole system.
   Night Vision Raspberry pi	Capture image to perform he digital image processing of edge detection.
   Global Positioning System (GPS) receiving module	Receiving latitude and longitude of coordinate.
   3V to 6V Direct current (DC) motor	Control the movement direction.
   L298N IC motor drive module	Control the Pulse Wave Modulation of voltage supply to the DC motor.
   Press Button and On-Off Switch	Control the function selection.
   Earphone	Voice out the information message.
   5V/2.1A Powerbank	Power supply to the Raspberry pi 3 Model B+ module.

   Hardware	Function
   Raspberry pi 3 B+ module	Control task activity of whole system.
   Night Vision Raspberry pi	Capture image to perform the digital image processing of edge detection.
   Global Positioning System (GPS) receiving module	Receiving latitude and longitude of coordinate.
   3V to 6V Direct current (DC) motor	Control the movement direction.
   L298N IC motor drive module	Control the Pulse Wave Modulation of voltage supply to the DC motor.
   Press Button and On-Off Switch	Control the function selection.
   Earphone	Voice out the information message.
   5V/2.1A Powerbank	Power supply to the Raspberry pi 3 Model B+ module.

   TABLE 1: LIST OF HARDWARE

   TABLE 2: LIST OF SOFTWARE

   Software	Function
   Python IDLE	Source code writing and debugging to support Raspberry pi 3 B+ module to perform activity.
   OpenCV 4	Perform the image digital processing of edges detection when capture the image by the night vision camera.
   Google Maps API	Support the search route function.
   Telegram API	Support the send emergency function and monitoring function.
   Google Cloud Vision API	Support the object identify function when capture the image by the night vision camera.
   eSpeak text to speech synthesizer	Support the function of text to voice conversion.

6. METHODOLOGY

   1. Block Diagram

      In this project, the input hardware is used 4 press button, GPS device and caretaker user equipment to send the data or signal to the main processor system. The output hardware is the L298N motor driver, caretaker user equipment and earphone to receive the data or signal send out by the main processor. The main processor is used to operate all the software activity function in this project. In Figure 1 shown that the overall hardware block diagram.

      Figure 1: Block Diagram of the system.

   2. Hardware Wire Connection

      In Figure 2 is the schematic diagram of the system hardware wiring connection. In this system L298N motor driver, 2 DC motor wheel, raspberry pi 3 model B+ board, GPS module, battery, and 4 pressing button switches are used in the project. In Figure 2 left side is the autonomous wheel with construct by 2 DC motor wheel and L298N motor driver. However, the raspberry pi 3 models B+ is the core of the project that operates the performance of input and output. The GPS module is used to get the real-time location of the blind people. The 4 pressing button switch is performed as the input to send a signal to the raspberry pi board to execute the operation. Moreover, the 2 separate battery is used as the power supply to the raspberry pi and motor driver.

      Figure 2: Hardware system wire connection.

   3. Flow Chart

      In this project, the flow chart is used to show the system flow of the whole project that used and also the source code of the software. In this project, the system flow of each function shown in the below Figure 3.

      Figure 3: Flow Chart of the whole project design function.

7. PROTOTYPR HARDWARE DESIGN

   In this project, the final output design of the total length of long, width and height of the blind stick is 16cm, 7cm and 107cm and the overall design is shown in Figure 4.

   Figure 4: Final output hardware project design.

   However, the total weight of this project is around 1kg after install all the hardware component and equipment onto the blind stick and Figure 5 shown the weight that been measured by the measurement scale.

   Figure 5: Total weight of the project design.

   In Figure 6 is show the front, back, left and right view of the bottom part hardware system design of the blind stick. The night vision camera, DC motor, L298N motor driver, 2 separate battery GPS module and raspberry pi 3 B+ module are located in the bottom part of the blind stick.

   Figure 6: Bottom Part of hardware system design.

   The top part hardware system design of the blind stick is located the holder of the function button and the earphone. The hardware system of the top part is show in Figure 7.

   Figure 7: Top part of hardware system design.

8. RESULT AND DISCUSSION

   In this project system, the function has been implemented onto the blind stick is obstacle avoiding function, sending emergency function, caretaker monitoring function, object identify function, object identify function and search route function.

   In the Obstacle avoiding function, the canny edges detection of digital image processing method with support by the software of Open CV 4 are used in the system. The Figure 8 show the output result mask of free obstacle path space and Figure 9 show the output result of the obstacle avoiding function and send the signal to the L298N motor drive and control the direction movement of the motor.

   Figure 8: Output mask of the canny edge detection of free path space.

   Figure 9: Output result of the obstacle avoiding function.

   In these 4 functions the time taken of each function process when button is press and recorded in the Table 1. The Table 2 show the accuracy percentage result of tester object been used to test the object avoiding function.

   TABLE 3: TIME RESPONSE OF EACH FUNCTION.

   Function	Time taken
   Start obstacle avoiding functions	2 – 3 second
   Sending emergency message function	10-12 second
   Caretaker monitoring function	10-13 second
   Object identify function	5-9 second
   Search route function	6-7 second

   Object	Indoor	Outdoor
   Human	80% detected	70% detected
   Trolley	60% detected	20% detected
   Boxes	90% percent	60% percent
   Table and chair	60% percent	–
   Car	–	50% percent
   Lamp pole	–	90% percent

   Object	Indoor	Outdoor
   Human	80% detected	70% detected
   Trolley	60% detected	20% detected
   Boxes	90% percent	60% percent
   Table and chair	60% percent	–
   Car	–	50% percent
   Lamp pole	–	90% percent

   TABLE 4: ACCURACY PERCENTAGE RESULT OF OBSTACLE AVOIDING FUNCTION TEST.

   For the object identify function, the output result is shown in Figure 11.

   Figure 11: Output result of Object identify function.

   For the search route function, the output result will voice out the text of the sentences with Figure 12 show.

   Figure 12: Output sentences voice out to through the earphone of search route function.

9. CONCLUSION

   In this paper, the method to make a blind stick to be more powerful function is presented. The main target of this project was also implemented successfully to provide an efficient way and multifunction blind stick to ease the life of visually impaired people. Apart from that, the problem of insufficient provide guide dog also been solve by this solution.

   In the caretaker monitoring and send emergency message function, the output result is shown in the Figure 10. The caretaker monitoring function, caretaker can send any text message to the system and the current image and location will send back to the user device of the caretaker via internet.

   Figure 10: Output result of the caretaker monitoring and send emergency message function.

10. FUTURE WORK

    The work could be improving in this project are adding the machine learning algorithm in order to perform the object identify function in offline condition. Moreover, the Grab service API can be added in this system to help visually impaired people booking a grab car to the far destination. Apart from that, the enhancement can be improving in this system is the search route function. This function can be improving to be an auto navigate until the destination reach.

    In hardware improving, GSM module can be adding to provide the data internet connection directly to the system and replacing the WIFI connecting mode. Increasing the power of DC motor wheel. Ultrasonic module can be adding to achieve more higher accuracy of object avoiding function.

11. REFERENCES

1. Vision impairment and blindness: Fact sheet, World Health Organization, 2018. [Online]. Available: https://www.who.int/news- room/fact-sheets/detail/blindness-and-visual-impairment

2. Desk, I. T, World Sight Day 2017: Statistics and facts about visual impairment and tips to protect your eyes, in India Today, 2017,

   October 12. [Online News]. Available: https://www.indiatoday.in/education-today/gk-current- affairs/story/world-sight-day-2017-facts-and-figures-1063009-2017-10- 12

3. Guilly d'Herbemont – Beginnings of The White Cane, [Online].

   Available: http://dherbemont.free.fr/en/textes/cannes.html

4. Dang, Q., Chee, Y., Pham, D., & Suh, Y., A Virtual Blind Cane Using a Line Laser-Based Vision System and an Inertial Measurement Unit,

   Sensors 2016, 95. doi:10.3390/s16010095

5. K.S.Manikanta, T. S. S. Phani, A .Pravin, Implementation and Design of Smart Blind Stick for Obstacle Detection and Navigation System, in International Journal of Engineering Science and Computing, Volume8, Issue No.8, pp 18785-18790, August 2018.

6. B. Li et al., "Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People", in IEEE Transactions on Mobile Computing, vol. 18, no. 3, pp. 702-714, 1 March 2019. doi: 10.1109/TMC.2018.2842751

7. Dhananjeyan, S., Sundaram, K. M., Kalaiyarasi, A., and Kuppusamy,

   P. G., Design and Development of Blind Navigation System using GSM and RFID Technology, in Indian Journal of Science and Technology, doi:10.17485/ijst/2016/v9i2/85809.

8. Aladrén, G. López-Nicolás, L. Puig and J. J. Guerrero, "Navigation Assistance for the Visually Impaired Using RGB-D Sensor with Range Expansion," in IEEE Systems Journal, vol. 10, no. 3, pp. 922-932, Sept. 2016. doi: 10.1109/JSYST.2014.2320639

9. Nadia Kanwal, Erkan Bostanci, Keith Currie, Adrian F. Clark, A Navigation System for Visually Impaired: A fusion of Vision and Depth Sensor, in Applied Bionics and Biomechanics. 2015. doi: 10.1155/2015/479857.

10. R. Tapu, B. Mocanu, A. Bursuc and T. Zaharia, "A Smartphone-Based Obstacle Detection and Classification System for Assisting Visually Impaired People," 2013 IEEE International Conference on Computer Vision Workshops, Sydney, NSW, 2013, pp. 444-451. doi: 10.1109/ICCVW.2013.65

11. Mohapatra, S., Rout, S. H., Tripathi, V., Saxena, T., & Karuna, Y., Smart Walking Stick for Blind Integrated with SOS Navigation System, in 2018 2nd International Conference on Trends in Electronics and Informatics (ICOEI). doi:10.1109/icoei.2018.8553935, 2018.

12. Sharma, P., A Step-by-Step Introduction to the Basic Object Detection Algorithms (Part 1), 2018, October 11. [Online]. Available: https://www.analyticsvidhya.com/blog/2018/10/a-step-by-step- introduction-to-the-basic-object-detection-algorithms-part-1/

13. Yiting Yi and Lunfu Dong, "A design of blind-guide crutch based on multi-sensors," 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Zhangjiajie, 2015, pp. 2288- 2292.doi: 10.1109/FSKD.2015.7382309.

14. Mugdha Surve, Priyanka Tambe, Sayali Walke, Pranoti Mane, Blind Navigation System Using Image Processing and Embedded System, in International Journal of Electrical, Electronics and Computer Systems (IJEECS). 4. 4-7, February 2016.

15. Singhpannu, G., Ansari, M. D., & Gupta, P, Design and Implementation of Autonomous Car using Raspberry Pi in International Journal of Computer Applications, 113 (9), 2015, pp. 22- 29. doi:10.5120/19854-1789

16. Obstacle Avoideance: Tutorial, Roborealm Vison for Machines, 2012. [Online]. Available: http://www.roborealm.com/tutorial/Obstacle_Avoidance/slide010.php? utm_source=rb-community&utm_medium=blog&utm_campaign=a- tutorial-on-using-machine-vision-for-obstacle-avoidance

17. Ayush Goyal, Using a GPS module (neo-7m) with Raspberry Pi 3, in Medium Corporation, October, 2018. [Online]. Available: https://medium.com/@DefCon_007/using-a-gps-module-neo-7m-with- raspberry-pi-3-45100bc0bb41

18. Heywood, M., GPIO PWM For Raspberry Pi H-Bridge DC Motor Control, in Bluetin Corporation, February, 2018. [Online]. Available: https://www.bluetin.io/python/gpio-pwm-raspberry-pi-h-bridge-dc- motor-control/