Smart Assisting Visually Impaired Stick (SAVIS)

Download Full-Text PDF Cite this Publication

Text Only Version

Smart Assisting Visually Impaired Stick (SAVIS)

Lim Tim Choong

Final Year Student, Bachelor of Electronics Engineering (Communication) with Honours

QUEST International University, Ipoh, Perak.

Mr. Karna Vishnu Vardhana Reddy

Senior Lecturer, Faculty of Science and Technology QUEST International University,

Ipoh, Perak.

Abstract The blind stick is one of the tools or equipment that been daily bring along to navigate and detect obstacle for blind or visually impaired people. This blind stick has its limitation that is can detect the ground obstacles by hitting and let the people to move either left or right direction. Besides that, blind people have also been faced some awkward situations such as hitting walker's leg by blind stick, injury by hitting the obstacles. Thus, this research project is to help them to overcome such awkward situations of using the blind stick.

The blind stick designed in this project use Raspberry Pi 3 B+ module as a main processor to compute the task activity. This blind stick will be implement with a night vision camera, which captures the image of the obstacle. After that image will be processed using digital image processing software to detect the edges. Then, system sends the PWM signal to motor wheel, which control the direction of the stick to avoid hitting obstacle. A GPS module is used to help blind people navigate in proper direction. Telegram app has been used for automatic sending the real-time GPS location and picture to a caretaker by pressing a button on the stick. Caretaker check the location of the bind people whether they moved away from the current location by sending a text message to the system in the stick. Apart from that, object identifying function also been implement into this project.

Keywords Blind stick, Obstacle detection, Route searching, Emergency message.

  1. INTRODUCTION

    Due to nervous system scientist of classifying of human 5 sense list, eyes are one of the major and important senses in the human body. Human eyes are the one receives sensory to provide vision to the human. In 2018 world health organization (WHO) was announced the statistics in this globally of 7 billion people, there are 36 Million people are blind [1]. In this 36 Million of blind people, 82 percent of the blind people are over 50 years old. Therefore, due to this announcement, the number of vision impaired and blind people are starting to increase by population growth and aging [2].

    To overcome the vision inconvenient of blind people, Guilly d'Herbemont from France in 1931 [3] is invention and launches a white cane or blind stick to provide guiding movement for blind people to detect and avoiding hitting obstacle, balances, supports when they walking. Moreover, in 1942 Guide Dogs for the Blind (GDB) provide guide dog services to help them. Therefore, until today, these two methods were popular used to help blind people in the days of walking and crossroad to anywhere.

    Due to globalization and modernization era, they have a lot of electronic hardware and software were invented to solve the traditional limitation and ease their life. The machine and

    system have been used to help blind people are using electronic sensor help blind people to avoid blind people hitting an obstacle. The sensor has been used are such as ultrasonic sensor, RFID punching system, laser line and so on.

  2. LITERATURE REVIEW

    In Dang, Q., Chee, Y., Pham, D., & Suh, Y.[4], line laser- based Vision system and Inertial Measurement Unit (IMU) sensor have been used. This project the line laser-based vision system is used to detect the obstacle distance and the type of obstacle. The IMU and camera used in this project is to improving the accuracy of an obstacle position and orientation pose.

    K.S.Manikanta et al.[5] integrated an intelligent blind stick with ultrasonic sensor to detect an obstacles by the ultrasonic waves after that if an obstacle is closer enough, the buzzer sound will give the signal feedback to the blind people. Moreover, this stick also uses to detect the water and alerts to the blind people by different buzzer sound. Another features in this stick is to helping blind people find out where their stick keeps by pressing a wireless RF remote button and the stick will buzzer out the sound.

    B. Li et al.[6] proposed a navigation system method to help blind people navigate in the indoor environment. This system used a software to develop a map to parse geometric information from architectural models and generate a semantic map. The Google Tango device are designing to leveraging the visual positioning service (VPS). Apart from that, the on- board RGB-D camera used to detect obstacle and avoidance approach based on a time-stamped map Kalman filter (TSM- KF) algorithm. To give the feedback signal to the blind people, speech-audio interaction and robust haptic interaction are been used.

    S. Dhananjeyan et al.[7] implement a system to provide path planning for blind people. This system is when the blind people pressing a button and speak the desired destination; the blind customer care will guide the path instruction to them. However, to ensure the blind people start to the destination or reach the destination, the blind people have to punching the radio frequency identification (RFID) system to provide location checkpoint and prevent wrong path. The ultrasonic sensor also used to prevent blind people collision to the obstacle.

    Alarden et al.[8] using the RGB-D camera and digital image processing of the Canny edge detector and probabilistic Hough line transform to identify the obstacle and giving the

    avoid direction feedback response to the blind people by haptic to avoid hitting the obstacle.

    Kanwal et al.[9] present a research of using Kinect camera with the Harris and Stephens corner detection method of digital image processing to detect the obstacle. After that, the feedback respond signal of the obstacle direction will giving by the haptic interaction to the blind people.

    Ruxandra et al.[10] present a research of using the sensor in the Smartphone and using the algorithm of Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Histogram of Oriented Gradients (HOG) features extracted and (Support Vector Machine) SVM to identify the direction of the obstacle. After that, send the feedback respond signal to the blind people by the haptic interaction.

  3. PROBLEM STATEMENT

    • Inaccuracy of sensor detection

    • Needed sighted friend or family accompany to navigate unfamiliar places of outdoor or indoor.

    • Insufficient Guided dog supply to the visually impaired people and sensitive issue in some countries.

  4. RESEARCH OBJECTIVE

    • To develop a Smart Assisting stick with autonomous wheel and camera to avoid blind people hit an obstacle and identifying any object.

    • To automatically send information message about their location to caretaker or guardian via the Telegram.

    • To implement GPS module with Google map to get destination route.

  5. PROJECT DESIGN

    In this project, the Table 1 and Table 2 show the hardware and software that has been used in project. The function of each hardware and software used also been mention in Table 1 and Table 2.

    Hardware

    Function

    Raspberry pi 3 B+ module

    Control task activity of whole system.

    Night Vision Raspberry pi

    Capture image to perform he digital image processing of edge detection.

    Global Positioning System (GPS) receiving module

    Receiving latitude and longitude of coordinate.

    3V to 6V Direct current (DC) motor

    Control the movement direction.

    L298N IC motor drive module

    Control the Pulse Wave Modulation of voltage supply to the DC motor.

    Press Button and On-Off Switch

    Control the function selection.

    Earphone

    Voice out the information message.

    5V/2.1A

    Powerbank

    Power supply to the Raspberry pi 3 Model B+ module.

    Hardware

    Function

    Raspberry pi 3 B+ module

    Control task activity of whole system.

    Night Vision Raspberry pi

    Capture image to perform the digital image processing of edge detection.

    Global Positioning System (GPS) receiving module

    Receiving latitude and longitude of coordinate.

    3V to 6V Direct current (DC) motor

    Control the movement direction.

    L298N IC motor drive module

    Control the Pulse Wave Modulation of voltage supply to the DC motor.

    Press Button and On-Off Switch

    Control the function selection.

    Earphone

    Voice out the information message.

    5V/2.1A

    Powerbank

    Power supply to the Raspberry pi 3 Model B+ module.

    TABLE 1: LIST OF HARDWARE

    TABLE 2: LIST OF SOFTWARE

    Software

    Function

    Python IDLE

    Source code writing and debugging to support Raspberry pi 3 B+ module to perform activity.

    OpenCV 4

    Perform the image digital processing of edges detection when capture the image by the night vision camera.

    Google Maps API

    Support the search route function.

    Telegram API

    Support the send emergency function and monitoring function.

    Google Cloud Vision API

    Support the object identify function when capture the image by the night vision camera.

    eSpeak text to speech synthesizer

    Support the function of text to voice conversion.

  6. METHODOLOGY

    1. Block Diagram

      In this project, the input hardware is used 4 press button, GPS device and caretaker user equipment to send the data or signal to the main processor system. The output hardware is the L298N motor driver, caretaker user equipment and earphone to receive the data or signal send out by the main processor. The main processor is used to operate all the software activity function in this project. In Figure 1 shown that the overall hardware block diagram.

      Figure 1: Block Diagram of the system.

    2. Hardware Wire Connection

      In Figure 2 is the schematic diagram of the system hardware wiring connection. In this system L298N motor driver, 2 DC motor wheel, raspberry pi 3 model B+ board, GPS module, battery, and 4 pressing button switches are used in the project. In Figure 2 left side is the autonomous wheel with construct by 2 DC motor wheel and L298N motor driver. However, the raspberry pi 3 models B+ is the core of the project that operates the performance of input and output. The GPS module is used to get the real-time location of the blind people. The 4 pressing button switch is performed as the input to send a signal to the raspberry pi board to execute the operation. Moreover, the 2 separate battery is used as the power supply to the raspberry pi and motor driver.

      Figure 2: Hardware system wire connection.

    3. Flow Chart

      In this project, the flow chart is used to show the system flow of the whole project that used and also the source code of the software. In this project, the system flow of each function shown in the below Figure 3.

      Figure 3: Flow Chart of the whole project design function.

  7. PROTOTYPR HARDWARE DESIGN

    In this project, the final output design of the total length of long, width and height of the blind stick is 16cm, 7cm and 107cm and the overall design is shown in Figure 4.

    Figure 4: Final output hardware project design.

    However, the total weight of this project is around 1kg after install all the hardware component and equipment onto the blind stick and Figure 5 shown the weight that been measured by the measurement scale.

    Figure 5: Total weight of the project design.

    In Figure 6 is show the front, back, left and right view of the bottom part hardware system design of the blind stick. The night vision camera, DC motor, L298N motor driver, 2 separate battery GPS module and raspberry pi 3 B+ module are located in the bottom part of the blind stick.

    Figure 6: Bottom Part of hardware system design.

    The top part hardware system design of the blind stick is located the holder of the function button and the earphone. The hardware system of the top part is show in Figure 7.

    Figure 7: Top part of hardware system design.

  8. RESULT AND DISCUSSION

    In this project system, the function has been implemented onto the blind stick is obstacle avoiding function, sending emergency function, caretaker monitoring function, object identify function, object identify function and search route function.

    In the Obstacle avoiding function, the canny edges detection of digital image processing method with support by the software of Open CV 4 are used in the system. The Figure 8 show the output result mask of free obstacle path space and Figure 9 show the output result of the obstacle avoiding function and send the signal to the L298N motor drive and control the direction movement of the motor.

    Figure 8: Output mask of the canny edge detection of free path space.

    Figure 9: Output result of the obstacle avoiding function.

    In these 4 functions the time taken of each function process when button is press and recorded in the Table 1. The Table 2 show the accuracy percentage result of tester object been used to test the object avoiding function.

    TABLE 3: TIME RESPONSE OF EACH FUNCTION.

    Function

    Time taken

    Start obstacle avoiding functions

    2 – 3 second

    Sending emergency message function

    10-12 second

    Caretaker monitoring function

    10-13 second

    Object identify function

    5-9 second

    Search route function

    6-7 second

    Object

    Indoor

    Outdoor

    Human

    80% detected

    70% detected

    Trolley

    60% detected

    20% detected

    Boxes

    90% percent

    60% percent

    Table and chair

    60% percent

    Car

    50% percent

    Lamp pole

    90% percent

    Object

    Indoor

    Outdoor

    Human

    80% detected

    70% detected

    Trolley

    60% detected

    20% detected

    Boxes

    90% percent

    60% percent

    Table and chair

    60% percent

    Car

    50% percent

    Lamp pole

    90% percent

    TABLE 4: ACCURACY PERCENTAGE RESULT OF OBSTACLE AVOIDING FUNCTION TEST.

    For the object identify function, the output result is shown in Figure 11.

    Figure 11: Output result of Object identify function.

    For the search route function, the output result will voice out the text of the sentences with Figure 12 show.

    Figure 12: Output sentences voice out to through the earphone of search route function.

  9. CONCLUSION

    In this paper, the method to make a blind stick to be more powerful function is presented. The main target of this project was also implemented successfully to provide an efficient way and multifunction blind stick to ease the life of visually impaired people. Apart from that, the problem of insufficient provide guide dog also been solve by this solution.

    In the caretaker monitoring and send emergency message function, the output result is shown in the Figure 10. The caretaker monitoring function, caretaker can send any text message to the system and the current image and location will send back to the user device of the caretaker via internet.

    Figure 10: Output result of the caretaker monitoring and send emergency message function.

  10. FUTURE WORK

    The work could be improving in this project are adding the machine learning algorithm in order to perform the object identify function in offline condition. Moreover, the Grab service API can be added in this system to help visually impaired people booking a grab car to the far destination. Apart from that, the enhancement can be improving in this system is the search route function. This function can be improving to be an auto navigate until the destination reach.

    In hardware improving, GSM module can be adding to provide the data internet connection directly to the system and replacing the WIFI connecting mode. Increasing the power of DC motor wheel. Ultrasonic module can be adding to achieve more higher accuracy of object avoiding function.

  11. REFERENCES

    1. Vision impairment and blindness: Fact sheet, World Health Organization, 2018. [Online]. Available: https://www.who.int/news- room/fact-sheets/detail/blindness-and-visual-impairment

    2. Desk, I. T, World Sight Day 2017: Statistics and facts about visual impairment and tips to protect your eyes, in India Today, 2017,

      October 12. [Online News]. Available: https://www.indiatoday.in/education-today/gk-current- affairs/story/world-sight-day-2017-facts-and-figures-1063009-2017-10- 12

    3. Guilly d'Herbemont – Beginnings of The White Cane, [Online].

      Available: http://dherbemont.free.fr/en/textes/cannes.html

    4. Dang, Q., Chee, Y., Pham, D., & Suh, Y., A Virtual Blind Cane Using a Line Laser-Based Vision System and an Inertial Measurement Unit,

      Sensors 2016, 95. doi:10.3390/s16010095

    5. K.S.Manikanta, T. S. S. Phani, A .Pravin, Implementation and Design of Smart Blind Stick for Obstacle Detection and Navigation System, in International Journal of Engineering Science and Computing, Volume8, Issue No.8, pp 18785-18790, August 2018.

    6. B. Li et al., "Vision-Based Mobile Indoor Assistive Navigation Aid for Blind People", in IEEE Transactions on Mobile Computing, vol. 18, no. 3, pp. 702-714, 1 March 2019. doi: 10.1109/TMC.2018.2842751

    7. Dhananjeyan, S., Sundaram, K. M., Kalaiyarasi, A., and Kuppusamy,

      P. G., Design and Development of Blind Navigation System using GSM and RFID Technology, in Indian Journal of Science and Technology, doi:10.17485/ijst/2016/v9i2/85809.

    8. Aladrén, G. López-Nicolás, L. Puig and J. J. Guerrero, "Navigation Assistance for the Visually Impaired Using RGB-D Sensor with Range Expansion," in IEEE Systems Journal, vol. 10, no. 3, pp. 922-932, Sept. 2016. doi: 10.1109/JSYST.2014.2320639

    9. Nadia Kanwal, Erkan Bostanci, Keith Currie, Adrian F. Clark, A Navigation System for Visually Impaired: A fusion of Vision and Depth Sensor, in Applied Bionics and Biomechanics. 2015. doi: 10.1155/2015/479857.

    10. R. Tapu, B. Mocanu, A. Bursuc and T. Zaharia, "A Smartphone-Based Obstacle Detection and Classification System for Assisting Visually Impaired People," 2013 IEEE International Conference on Computer Vision Workshops, Sydney, NSW, 2013, pp. 444-451. doi: 10.1109/ICCVW.2013.65

    11. Mohapatra, S., Rout, S. H., Tripathi, V., Saxena, T., & Karuna, Y., Smart Walking Stick for Blind Integrated with SOS Navigation System, in 2018 2nd International Conference on Trends in Electronics and Informatics (ICOEI). doi:10.1109/icoei.2018.8553935, 2018.

    12. Sharma, P., A Step-by-Step Introduction to the Basic Object Detection Algorithms (Part 1), 2018, October 11. [Online]. Available: https://www.analyticsvidhya.com/blog/2018/10/a-step-by-step- introduction-to-the-basic-object-detection-algorithms-part-1/

    13. Yiting Yi and Lunfu Dong, "A design of blind-guide crutch based on multi-sensors," 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Zhangjiajie, 2015, pp. 2288- 2292.doi: 10.1109/FSKD.2015.7382309.

    14. Mugdha Surve, Priyanka Tambe, Sayali Walke, Pranoti Mane, Blind Navigation System Using Image Processing and Embedded System, in International Journal of Electrical, Electronics and Computer Systems (IJEECS). 4. 4-7, February 2016.

    15. Singhpannu, G., Ansari, M. D., & Gupta, P, Design and Implementation of Autonomous Car using Raspberry Pi in International Journal of Computer Applications, 113 (9), 2015, pp. 22- 29. doi:10.5120/19854-1789

    16. Obstacle Avoideance: Tutorial, Roborealm Vison for Machines, 2012. [Online]. Available: http://www.roborealm.com/tutorial/Obstacle_Avoidance/slide010.php? utm_source=rb-community&utm_medium=blog&utm_campaign=a- tutorial-on-using-machine-vision-for-obstacle-avoidance

    17. Ayush Goyal, Using a GPS module (neo-7m) with Raspberry Pi 3, in Medium Corporation, October, 2018. [Online]. Available: https://medium.com/@DefCon_007/using-a-gps-module-neo-7m-with- raspberry-pi-3-45100bc0bb41

    18. Heywood, M., GPIO PWM For Raspberry Pi H-Bridge DC Motor Control, in Bluetin Corporation, February, 2018. [Online]. Available: https://www.bluetin.io/python/gpio-pwm-raspberry-pi-h-bridge-dc- motor-control/

Leave a Reply

Your email address will not be published. Required fields are marked *