Integrating Wearable and Assistive Technology powered by Artificial Intelligence Empowering the Lives of Disabled

Integrating Wearable and Assistive Technology powered by Artificial Intelligence Empowering the Lives of Disabled

Integrating Wearable and Assistive Technology powered by Artificial Intelligence

Empowering the Lives of Disabled

Praveen Kumar

Student, Technical Head (AnalytiX) Don Bosco School, Alaknanda

New Delhi – 110070

Abstract The traditional notion of Disability as a weakness, being disabled is passé. Our approach to the differently abled needs a fundamental change, while living in a country having the largest population of blind people. Of the 37 million people across the globe who are blind, over 15 million are from India. The touchscreen technology and smartphones have largely been exclusionary towards the visually challenged in ensuring ease of use. There has further been an absence of newer technological innovations that caters specifically to their needs. My design Infigo provides assistive, rehabilitative, and adaptive features to the visually impaired user. The design has components of E- textiles, capacitive touch technology, wearable technology, embedded systems and artificial intelligence, ensuring that the user can very easily communicate with each other and, access other mobile features literally through their fingers while wearing the lightweight glove. It is conceived as a lightweight Glove which has a conductive fabric in the finger regions that facilitates capacitive touch, just like touch screens, so that the users can operate their phones and other devices through simple intuitive hand gestures. It acts a standalone cell phone embedded on a wearable glove with the keypad buttons embedded on the tips of ones finger. The other existing technologies are generally complicated and bulky to use. The existing voice assistant technological solutions are often cumbersome to access in public places which are usually noisy. The goal of my device is to provide user friendly experience and hassle free communication and mobility to the user. The device aims to use the most cutting edge technology to ensure ease of living and communicating for the differently abled.

Keywords Wearable technology, Artificial Intelligence, Soft Robotics, embedded systems, assistive technologies.

1. INTRODUCTION

   In this booming technological era, In order to survive the advent of technologies such as Artificial Intelligence, Internet of Things, Blockchain, Machine Learning and their components, mankind needs to upgrade. The device also also contributes towards integrating Artificial Intelligence with assistive technologies. It also contributes towards uprising of Artificial Intelligence in Health sector. The usage of A.I will not replace human intellect but rather improvise and assist it by taking the capabilities to newer heights. My device is a step

   in the same direction and with this integration the user can actually dial a number and talk to the person on the other side through the wave of his/her hand and through some intuitive touches, without any other connection or syncing it with other phone. It acts a standalone cell phone embedded on your wearable glove. In the common way that Indians count numbers, dates, months on the fingertips, they can dial numbers with this device and place calls. What makes it special is that it user friendly, it is not only lightweight but also comfortable as conductive thread has been used, instead of thick bulky wires and adapted the techniques of soft robotics in my device

2. FACTOR ANALYSIS

   Reducing a smartphone to a layer of fabric on our hands The system uses a GSM 900A module integrated with ATmega328 Microcontroller through Arduino Development Platform.

   1. Global System for Mobile Communications (GSM) is widely used standard drawn by European Telecommunications Standards Institute (ETSI) for Cellular Networks used in Mobile Phones. A GSM module is a specialized type of modem which accepts a Subscriber Identity Module (SIM) card and operates over a subscription to a mobile operator just like a mobile phone. From a mobile network operator perspective a GSM modem looks just like a mobile phone. In this project SIM900A GSM module is used. The SIM900A is a complete Quadband GSM solution in a SMT module which can be embedded in the customer applications. Featuring an industry-standard interface, the SIM900 delivers GSM 850/900/1800/1900 MHz performance for voice, SMS, Data, and Fax in a small form factor and with low power consumption. Arduino is an open source computer hardware and software company, project and user community that designs and manufactures microcontroller based kits for building digital devices and interactive objects that can sense and control the physical world.

   Fig 1.1 – GSM Module

   The GSM module is integrated with an an Arduino Board, which is a AVR microcontroller based board which follows the standard Arduino schematic and is flashed with the Arduino bootloader. All Arduino boards should be compatible with the Arduino IDE which can be used to program the Arduino boards. The Arduino IDE is also open source and anybody can contribute their libraries to the Arduino. The Arduino board has all the required circuitry to get the built-in AVR microcontroller running. Both digital and analog inputs and outputs are available in all Arduino boards. The Arduino boards can also communicate with other devices using standard communication ports like USART, IIC, and USB etc. It uses a ATmega328 microcontroller in Arduino Uno. The high-performance Microchip 8-bit AVR RISC-based microcontroller combines 32KB ISP flash memory with read- while-write capabilities, 1KB EEPROM, 2KB SRAM, 23 general purpose I/O lines, 32 general purpose working registers, three flexible timer/counters with compare modes, internal and external interrupts,serial programmable USART, a byte-oriented 2-wire serial interface, SPI serial port, 6- channel 10-bit A/D converter (8-channels in TQFP and QFN/MLF packages), programmable watchdog timer with internal oscillator, and five software selectable power saving modes. The device operates between 1.8-5.5 volts. The entire coding takes place on Arduino IDE in C++.

   Fig. 1.2 – ATmega328 AVR microcontroller

   3. Raspberry Pi® is an ARM based credit card sized SBC(Single Board Computer) created by Raspberry Pi Foundation. Raspberry Pi runs Debian based GNU/Linux operating system Raspbian and ports of many other OSes exist for this SBC. It has an Rasbian OS with 1GB of RAM and a 16GB Class 10 MicroSD Card having all the Files of Rasbian OS.

   • ANALYSING THE DESIGN

     The GSM900A Module has been integrated with the Arduino, through RX and TX pins, associated with the GND (ground) pins. The Arduino has been programmed in order to receive inputs from the keypad matrix (in this case the fingertips) and then further process the input and send the commands to the GSM module to perform the required function. On the basis of event driven programming, the flow of the events is based on the user input (action). The system waits for the user to give a input. The Arduino sends AT commands to the module in order to perform respective functions.

     A 4×3 Numeric Keypad is designed for the project, by using this type of keypad you can easily enter any number (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, * and #). In this mechanism, you can delete last entered digit, which means this device can also handle options for delete or for corrections.

     The keys * is assigned for receive a call and # is assigned to make a call on the entered number and terminate a call.

     Raspberry Pi 3 Model B is connected to its peripherals in order to enable usage of voice assstants (Alexa). It connects to the internet through WiFi chip already integrated in it and access all the features. The user can establish voice commands and interact with Amazon Alexa – A Smart voice assistant.

     Syncing with the smartphone which has multi-processing capability :

     Utilising the present computational power, sensors and capacity of a smartphone by syncing it with the glove. A specific apps is designed for communication between the HC- 05 Bluetooth module in the glove and the smartphone in order to enable the user to access all the features of a smartphone through voice commands and intuitive hand gestures. As per a Survey conducted by Google, we spend about 5 days in a year only lifting and picking up our smartphones from places and putting it in our pockets. So in order to eliminate excessive dependence on smartphone, a user can opt for a connected version of the glove and access all the features without the hassle of lifting a smartphone so many times. This would also result in betterment of health, as a smartphones comes with health issues such as screen fatigue, visual impairment etc. So now imagine, a user can access the complex features of a smartphones through simple intuitive hand gestures or just the flick of fingers and voice commands.

     Fig 1.3 – Syncing through WiFi and Bluetooth protocols.

     Fig 1.4 – 4*3 Keypad Matrix

     By using the principles of E-textiles, the usage of conductive fabric and thread took place while designing the device. This eventually made the device lightweight and increased the comfort. So that the user can simply communicate while bringing the thumb close to the ear and the last finger to the mouth, establishing a usual telephonic communication through the glove, removing the hassle of taking out the phone and then holding for a long time. While implanting the embedding portion, a lot of difficulty came into my way, as a very sensitive conductive threads is used in the project, they have tendency to get in touch with nearest thread, and the unintentional connection cannot be seen easily as small yarns gets connected. Maintaining the Integrity of the Specifications

     Device – INFIGO

     Integrating Artificial Intelligence as Smart Assistant

     The most important element of Infigo is that it has Artificial Intelligence in the form of smart assistants. Infigo has Alexa, the smart assistant by Amazon. It has been integrated on the device, namely Raspberry Pi, through Alexa Voice Service (AVS) Integration. The AVS Device SDK provides C++- based (11 or later) libraries that leverage the AVS API to create device software for Alexa-enabled products. It is modular and abstracted, providing components for handling discrete functions such as speech capture, audio processing, and communications, with each component exposing the APIs that you can use and customize for your integration. It also includes a sample app, which demonstrates the interactions with AVS.The smart assistant runs on the Raspberry pi on the Device with an USB Microphone (INPUT) and a mini speaker (OUTPUT) for Speech-to-text (STT) and Text to Speech(TTS) processes during conversation.

     The Raspberry pi has the Alexa Avs Sample app installed and the Sensory wake word is enabled. When the pi boots up, I have automated the process of opening three different terminals and running some commands to start Alexa. The first bat file executes the starting of Companion Service, the second runs the Java Client i.e the Alexa app and the third starts the wake word engine.

     The functioning of Alexa consists of:

     Audio Signal Processor (ASP) – Third-party software that applies signal processing algorithms to both input and output audio channels. The applied algorithms are designed to produce clean audio data and include, but are not limited to acoustic echo cancellation (AEC), beam forming (fixed or adaptive), voice activity detection (VAD), and dynamic range compression (DRC). If a multi-microphone array is present, the ASP constructs and outputs a single audio stream for the array.

     Shared Data Stream (SDS) – A single producer, multi- consumer buffer that allows for the transport of any type of data between a single writer and one or more readers. SDS performs two key tasks:

     • It passes audio data between the audio front end (or Audio Signal Processor), the wake word engine, and the Alexa Communications Library (ACL) before sending to AVS

       • It passes data attachments sent by AVS to specific capability agents via the ACL

   Wake Word Engine (WWE) – Software that spots wake words in an input stream. It is comprised of two binary interfaces. The first handles wake word spotting (or detection), and the second handles specific wake word models (in this case "Alexa"). Depending on your implementation, the WWE may run on the system on a chip (SOC) or dedicated chip, like a digital signal processor (DSP).

   Audio Input Processor (AIP) – Handles audio input that is sent to AVS via the ACL. These include on-device microphones, remote microphones, an other audio input sources.

   The AIP also includes the logic to switch between different audio input sources. Only one audio input source can be sent to AVS at a given time.

   Alexa Communications Library (ACL) – Serves as the main communications channel between a client and AVS.

   (ADSL): Manages the order and sequence of directives from AVS, as detailed in the AVS Interaction Model. This component manages the lifecycle of each directive, and informs the Directive Handler (which may or may not be a Capability Agent) to handle the message.

   Activity Focus Manager Library (AFML): Provides centralized management of audiovisual focus for the device. Focus is based on channels, as detailed in the AVS Interaction Model, which are used to govern the prioritization of audiovisual inputs and outputs.

   Capability Agents: Handle Alexa-driven interactions; specifically directives and events. Each capability agent corresponds to a specific interface exposed by the AVS API.

   With the help of these features, the user get access to the world in her/his hand. For instance user can order food, read audio books, book cabs (Ola, Uber), check weather, control Smart Home and appliances (through the concept of IoT), set alarm, listen to music and do innumerable task just by voice commands, without any physical interaction. Nowadays every smartphone has an assistant, but what makes it different from those, is that it works headlessly, and just by using the wake word ("alexa") and the assistant starts to listen to you, removing the hassle of pushing a button and it offers portability to the user.

3. THE TECHNOLOGICAL FACTORS USED IN THE DEVELOPMENT

   The usage of various cutting edge technologies in the device took its usability and productivity to newer heights.

   1. Wearable Technology

      Since mankind has started to move on the path of civilization technology has developed gradually. However, recently, some revolutionary changes such as the invention of electronic chips, GPS systems, and Wi-Fi systems, the internet, computers, sensors, and advancements in nanotechnology have transformed the entire world at an unprecedented rate. Wearable technologies are one of the most important fields

      which have evolved from these continuous technological advancements.

      – Embedded Circuitry on wearable cloth Although there is no clear and agreed definition in the extant literature, in the simplest form wearable technologies can be defined as the technological devices that are worn on a users body. Since the wearable technologies have been popular newly, the designs and functionalities of the wearable technologies are still relatively unexplored. The scope of wearable technologies is very broad and amorphous, and determining the characteristics and specifications of wearable technologies is very thorny. However, in the near future the evolution of wearable technologies, especially smart glasses and smart gloves andwatches, will almost be completed their evolutions and these

      Technological devices will be adopted by the societies and companies.

      2. Embedded Systems

      An embedded system is a programmed controlling and operating system with a dedicated function within a larger mechanical or electrical system, often with real-time computing constraints. It is embedded as part of a complete device often including hardware and mechanical parts. Embedded systems control many devices in common use today. Ninety-eight percent of all microprocessors are manufactured components of embedded systems or in simple words an embedded system is a microcontroller or microprocessor based system which is designed to perform a specific task.

      An embedded system has three components:

      • It has hardware.

      • It has application software.

      • It has Real Time Operating system (RTOS) that supervises the application software and provide mechanism to let the processor run a process as per

   scheduling by following a plan to control the latencies. RTOS defines the way the system works.

   Since the embedded system is dedicated to specific tasks, design engineers can optimize it to reduce the size and cost of the product and increase the reliability and performance.

   3. Smart Assistants (Artificial Intelligence)

   An intelligent or smart assistant is a software agent that can perform tasks or services for an individual. Rapid improvements in key underlying technologies – voice recognition and natural language processing (NLP) are making these smart assistants more capable of letting us use our various devices just by talking to them. The promise of these assistants, ranging from Apples Siri and Googles Assistant to the newcomer, Samsungs Bixby, is that someday we will each have our own personal, always-listening. AI which can respond to any wish and command, like Tony Starks Jarvis in the movie Iron Man. Its a future vision of computing pulled directly from the pages of science fiction. These are making inroads in every walk of our lives today by increasing our productivity.

   devices, by providing new technological solutions for primitive factors. It has three major components:

   Therefore, Infigo completely supports the idea of IoT by having Alexa on board with IFTTT configuration, so that the user can control smart home, appliances and other devices over a couple of voice commands.

4. OBJECTIVES OF THE TECHNOLOGY

The objective of the device is to facilitate a user friendly and hassle free experience to the visually impaired as well as the normal users to communicate and access features of a smartphone. It aims to empower the the disabled people, so that they be more efficient and productive. A number of people have blazed the trail in the past without technology at their side, how amazing it would have been, if they had technology at their side. But now when technology is booming day by day, it is our moral duty to develop assistive platforms for them.

ACKNOWLEDGMENT

This is to acknowledge contributions of my mentor Mrs Geetu Munjal, Mr. Saiby Mathew and Mr. Kumar Lal Babu, Dr. Mrs. Neelam Kumari, Ms. Preeti Kumari, teachers, friends and family for being a perennial source of guidance and support in this project.

4. Internet of Things (IoT)

The term Internet of Things generally refers to scenarios where network connectivity and computing capability extends to objects, sensors and everyday items not normally considered computers, allowing these devices to generate, exchange and consume data with minimal human intervention. There is, however, no single, universal definition. The Internet of Things (IoT) is the network of physical objectsdevices, vehicles, buildings, living beings and other thingsembedded with electronics, software, sensors, and network connectivity. It enables these objects to collect and exchange data without any human-to-human or human-to-computer interaction. Its interaction is considered as any-to-any. Internet of things is rapidly changing the perspective of connecting object and

REFERENCES

[1] MESUT ÇÇEK, WEARABLE TECHNOLOGIES AND ITS

FUTURE APPLICATIONS International Journal of Electrical,

Electronics and Data Communication, ISSN: 2320-2084

[2] Jaewoon Lee, Dongho Kim, Han-Young Ryoo and Byeong-Seok Shin , Sustainable Wearables: Wearable Technology for Enhancing the Quality of Human Life, MDPI.

[3] Zeinab Kamal Aldein Mohammeda, Elmustafa Sayed Ali Ahmedb, Internet of Things Applications, Challenges and Related Future Technologies, WSN 67(2) (2017) 126-148 [4] Tutorials Point, Internet of things.

[4] Reference Material, Internet of Things, RoboTryst, 2018 [6] Tutorials Point, Embedded Systems.

[5] The future of Wearable Technology.by Human Centric Publications., 2018.

[6] Capacitive Sensing Technology, E. Terzic et al., A Neural Network approach to Fluid Quantity Measurement in Dynamic Environments