Voice Controlled Smart Page Turner for Differently Abled

Voice Controlled Smart Page Turner for Differently Abled

Balachandra H N Dept. of ECE SJCE, Mysuru India

K Sowmya

Jnaneshkumar S Dept. of ECE SJCE, Mysuru India

K Sanjay Nayak Dept. of ECE SJCE, Mysuru India

Shreekanth T

Dept. of ECE Dept. of ECE

SJCE, Mysuru SJCE, Mysuru

India India

Abstract In this modern world, as the age increases, people suffer from diseases like arthritis and stroke. This limits the functionality of the body. Like motor activities which worsen with age, affects the aged people from being involved in reading the printed materials like books and magazines. The principle objective of this work is to provide a helping hand to elderly, disabled and blind people. E-books and digital contents are the only source of information for such people. In the long run it may affects their eyes. Though many books are not digitized yet. To meet their requirements this automatic page turning mechanism through voice commands would be very advantageous. After turning the page, the content of the page should be read, for that, the whole page is scanned and read out to the user. Hence it useful for aged, disabled and blind people. The pages would be turned according to the voice commands like next or previous pages. When the required command is given the suction cup which is fixed to a robotic arm is made to attach to the page and turns to the required side. The suction pump creates low pressure area inside the suction cup which helps in turning the pages.

Keywords Tesseract, espeak, node mcu, vacuum piston pump

1. INTRODUCTION

   The disabled people are restricted with the e-books and digital contents for gaining the world of knowledge or else they must depend on someone for the process of turning the pages (next pages and previous pages). The voice-controlled page turner will resolve the latter problems. It provides the user to give voice commands through Google-assistant from their mobile phones to control the page turning mechanisms. Once the desired page is turned the cameras will take photographs of the page and the pictures will be converted to text using tesseract libraries and text will be read out based on the user voice commands.

   The vacuum piston pump which is employed in this work, pulls the air from one nozzle and leaves it in other. The suction pressure created by the pump is controlled by an external pressure regulator. The turning mechanism is faster in this mechanism. The mobile cameras take photographs of

   the pages with high quality, which is essential for the py- tesseract engine to recognize every character in the page. The espeak libraries provides the voice output for the scanned contents.

   Section II presents the previous works that have been done on this work.

2. LITERATURE SURVEY

   • Ernest F Hangman had a patent on automatically adjustable page turner. This invention relates to a means for automatically and mechanically turning pages of an open book, magazine or other bound, printed material.[1]

   • Avi shachar had a patent on mechanical page turner. It is the object of the present invention to provide a leaf turner for books, bound music and the like which is adapted to be operated by simple mechanical means.[2]

   • Tim Bowmen and Ben shepard had a patent on page turner. In this page disks are rotated and pages turned by rotating a cable which has a roller that rotates a page disk. The cable is powered by a foot operated gear drive or an electric motor.[3]

   • Koichiro Kakinuma, Minoru Watanabe made a patent on automatic page turning device4. It comprises a carrier drum having a perforated peripheral wall and being bodily movable from a first position opposing an outer edge portion of the first page of the opened book to a second position in which the carrier drum opposes an outer edge portion of the second page, a suction device by which a vacuum is applied to the interior of the carrier in its first position. [4]

   • Lima Roberto Ponticelli had a patent on automatic page turning device using mechatronic articulated arm. The invention relates to a page-turning device including a mechatronic articulated arm by way of

     a mobile element, which arm can turn pages in both directions automatically, as well as enabling pages to be turned manually.[5]

   • Character Recognition using OCR Techniques Gustav Tauschek obtained a patent on OCR in Germany, followed by Hander who obtained a patent on OCR. Gustav Tauschek was also granted a US patent on his method. His machine was a mechanical device that use templates and a photo detector.[6]

   • Speech synthesis based on hidden Markov models Statistical parametric speech synthesis emerged in the late 1990s and has grown in popularity.[7]

   Section III presents details on system engineering issue.

3. SYSTEM DESIGN

   The smart page turner consists 6 major modules: Node mcu, speaker suction pump, relay, robotic arm and system. The Figure.1 shows the block diagram as shown below.

   Fig. 1. System block diagram of voice-controlled page turner.

   The node mcu provides wireless connectivity to the users mobile phone. Using the Google assistant installed on the phone the user gives the voice commands to turn the page left or right. When the robotic arm is about to turn the relay switch on the suction pump and it switches off when the page is turned. Then the photographs taken using the mobile cameras will be passed to the system and the system process it and converts it into text and using e-Speak library the text will be converted into audio output and will be read out according to users convenience.

4. HARDWARE IMPLEMENTATION

   Node MCU: Node MCU is used as a basic micro controller for providing the LAN to control the robotic arm and the users mobile. These two are connected through the Authorization-token generated by the Blynk project. Node MCU is based on the ESP8266 Arduino core. It controls on/off the relay board which controls the suction pump. The node MCU is powered up through the USB cable from the laptop.

   Robotic arm: The robotic arm contains two servo motors. One with high torque and the other with the low torque. The high torque is for turning action. It is the metal gear high torque double ball bearing design. It is the 30cm 3pin connector wire. The low torque servo is for holding the suction cup which page holding mechanism.

   List of the servo motors and their details are listed in Table I and Table II.

   Table1. SPECIFICATION OF HIGH TORQUE SERVO MOTORS.

   Parameters	Specification
   Size	41mm x 20 x 43mm
   Weight	55g
   Stall Torque	4.8V -13kg/cm, 6.0V-15kg/cm
   Operating speed without load	4.8Y-0.17 sec/60Ao, 6.0V- 0.14sec/60Ao
   Dead bandwidth	5 microseconds
   Operating voltage	4.8V-6V

   Table 2. SPECIFICATION OF LOW TORQUE SERVO MOTORS.

   Operating voltage

   Parameters	Specifications
   Size	22 x 11.5 x 27mm
   Weight	9 g
   Stall Torque	1.2kg / 42.3oz(4.8V); 1.6 kg / 56.4oz (6.0V)
   Operating speed without load	0.10sec/60 degrees
   Dead bandwidth	7usec
   3.0-7.2 Volts

   Suction pump: It is the mini air dc vacuum piston pump. This piston pump is used to create low pressure and to provide the suction mechanism. The specifications of the mini air dc vacuum piston pump is as shown in the below Table 3.

   Table 3. SPECIFICATIONS OF THE MINI AIR DC VACUUM PISTON PUMP.

   Parameters	Rating
   Input voltage supply	12V DC
   Current rating	Less than 1.5A
   Airflow without load	5 LPM
   Maximum pressure	Greater than 2000mmHg
   Noise level	55 dB

   Camera and speakers: The CMOS technology that is used in the smart phone camera is used to capture the input image of the textbook. The speaker is for the speech output of the content read.

5. DESIGN AND IMPLEMENTATION

   This section contains information on the design and implementation of the complete proposed model.

   1. Hardware Design.

      The components used for page turning mechanism are Node mcu, Servo motor (2), Vacuum pump, Suction cup, Flexible tube, Relay module and battery to power up the vacuum pump. The wooden piece is cut down to proper dimensions and one end of it is attached to high power servo motor which is held on the wooden base. The wooden piece is drilled accordingly to fix it into the base servo. Another lower power servo is attached into other end of the wooden piece which is required to rotate the suction cup to the required position so that turning the page would be easier. The two servo motors are controlled by the Node mcu. To control the vacuum pump, we have used a 4 channel relay module where the pump is conneted to a single channel.

      Fig. 1. Hardware implementation.

      The Figure 2. Shows the brief working of the hardware model where the Blynk android app is used to control the node mcu over the wifi and IFTTT is used to give voice commands through google assistant which in turn makes web request through Webhooks.

      The virtual pin is used in Blynk app to control the whole arm movements, that is, when we press on button in the Blynk app the arm moves form 0 degree to 180 degree and vice versa. The small servo motor is also programmed such that it always faces towards downwards such sucking is possible.

      The above process is mimicked using IFTTT, where we have created separate applets for turning the page towards right and turning towards left. These applets work as it receives voice commands through google assistant.

   2. Software Design

      The python v3. is used for implementing the software block and the following libraries are used pip3, pytesseract, eSpeak, ALSA, url.retrieve, PyAudio 2.11, PocketSphinx, Google API Client Library for Python.

      The page is turned on the command of the user and when the python code is made to run. The Figure 3. Explains the detailed working of the software part.

      Fig. 2. Software implementation block.

      It requires users voice command to proceed to the subsequence steps and if the command is start, the photographs of the pages turned up would be clicked by the mobile phones which have been placed on the top of the book. The image is then saved for further processes. The images orientation will be changed accordingly and pytesseract comes into play, it extracts the text content from the image and convert it into digital content. The entire content that have been scanned would be separated by each of the . It encounters through the process. Then the espeak will start reading out the content and when the sentence is completed it again requires users voice command; if the command is repeat, it repeats the last sentence or else the control is transferred is back to the initial step and the process repeats.

   3. Implementation

      The high torque servo which is fixed in the basement provides the pivot for the wooden arm. It has degree of freedom from 0 degree to 180 degree. It makes a semi- circle trajectory. The low power, low torque servo motor is fixed at the end of the wooden arm which holds the suction cup in position to provide proper orientation while releasing the page

      to the other side of the book. These two servo motors are controlled through the node MCU which in turn controlled by the virtual button of the Blynk app, which in turn is controlled by making the web requests through the Web hooks which is triggered by the Google assistant commands.

6. OUTPUT

   The initial setup is made as shown in the Figure 4, the input voice command is given to the google assistant in the phone.

   Fig. 4. The initial setup.

   When the command turn the page is given through google assistant, it makes web request using IFTTT and to blynk app and the node MCU gets the command. Now the Node MCU checks for the input and sends the command to the servo motors and the dc air vacuum piston pump. The robotic arm now lifts the page from right to left. The Initial setup shown in Figure 4 and a set of photos merged into one photo which has the turning of one page is shown in Figure 5. After turning the page, the cameras placed on the top will take the pictures of the pages and send it tesseract engine through ip web cam interface.

   Fig. 5. is the photo which is given as input to tesseract.

   The tesseract converts the image to text and the output is in Figure 7.

   Fig. 6. Image given to Tesseract as input.

   The accuracy of tesseract engine is almost 70 percent. It depends upon various factors like inclination of book, quality of camera, orientation etc. Now this text input is given to the espeak which reads this text. After each sentence there was a break delay of 30 seconds. If the user wants to stop reading or repeat the sentence or pause reading then they can give commands like repeat , stop, pause. These commands will do the respective actions. After reading of whole page, the next page or the previous page command can be given. The robotic arm gets the command and turns the page accordingly.

   Fig. 7. Tesseract Output.

7. CONCLUSION

   The voice controlled smart page turner is designed mainly for differently abled people. The target of this work is to create an innovative, user friendly and compact mechanism to read books. The implemented methodology to turn the page and read the book satisfies most of the desirable qualities. The work combines the advantages of image processing into the mechanical movement to make of the page to increase the interest of even avid readers. Along with the old, decreased arm and blind, it can also be used by common man who are book worms and find it difficult to make time to read books.

8. FUTURE SCOPE

The developed model can only turn one page at a time, it may be previous page or next page. In future some more features can be added like,

• User can give page number as input and the correct page number should be opened by recognizing the page numbers which will be usually printed in top or bottom portions of the books.

• Inspired by the JAWS software, paragraph can be repeated if required while reading large paragraphs.

• By sensing the index of books, the appropriate page should be opened without taking much time.

• The whole device now requires two parts one is for software and other one is for hardware. For software

  we are using PC and for hardware we are using Node MCU. In future both the parts should be integrated to one system.

• It should be user friendly as well, like it should bookmark the pages which the user may wish to continue from where they have left.

REFERENCES

1. Frederick K. Storm, Jr., Los Angeles, and Eldridge H. Smiley, Rosemead, Calif., assignors to Ernest F. Hag man, Glendale, Calif. Automatically Adjustabe Pageturner Application, May 13, 1957, Serial No. 658,780 15 Claims. (C. 40-104).

2. Avi Shachar, Doar Na Merom Hagali, Kibbutz Sasa, Israel , Filed MECHANCAL PAGE-TURNER Apr. 20, 1964, Ser. No. 360,933 6 Claims. (Cl. 4.0-104)

3. PAGE TURNER Inventors: Tim Bowman, 3566 Nes Circle, Salt Lake City; Ben Shepard, Box 501, Ivins, both of Utah Appl. No.: 09/004,682 Patent Number: 5,962,801 Date of Patent: Oct. S. 9 1999

4. Koichiro Kakinuma, Tokyo; Minoru Watanabe, Kanagawa, Automatic Page-Turning Device Inventors: both of Japan 73 Assignee: Sony Corporation, Tokyo, Japan (21) Appl. No.: 891,465 Patent Number: (45) Date of Patent: 4.942,482 Jul. 17, 1990

5. Automatic page-turning device using a mechatronic articulated arm Inventor Lima Roberto Ponticelli Priority date 2008-01-10 Family: ES (1)WO (1)DateApp/Pub NumberStatus 2009-01-

   07PCT/ES2009/070002

6. Gustav Tauschek in Germany, Paul Hande The Reading Machine (rst OCR device) of l (so called Statistical Machine) USA 1933 (U.S. Patent 1915993). 1935 Tauschek US patent (U.S. Patent 2026329).

7. Smith, R., 2007, September. An overview of the Tesseract OCR engine. In Document Analysis and Recognition, 2007. ICDAR 2007. Ninth International Conference on (Vol. 2, pp. 629-633). IEEE.