Gesture Controlled Robot with Wireless Camera Monitoring

Gesture Controlled Robot with Wireless Camera Monitoring

B. Chaitanya Varma P. Manikanta

Electronics & Communication Engg. Electronics & Communication Engg.

St. Anns college of engineering & technology St. Anns college of engineering & technology Chirala, India Chirala, India

K.Venu Gopal

P.Venkateswaralu Reddy Asst. Prof

Electronics & Communication Engg. Electronics & Communication Engg.

St. Anns college of engineering & technology St. Anns college of engineering & technology Chirala, India Chirala, India

Abstract- The interaction between humans and machines increasing day by day. With the help of new embedded technologies the gap between machines and humans will be reduced and provides human task easily. For this type of applications gestures play crucial role. Gesture controlled robot which can be controlled by human gestures. The transmitter circuit in which sensor included i.e. accelerometer, the user needs to wear this device. The sensor will accept the movement of hand in a specific direction which will result in the motion of the robot in the respective direction. The system design is divided into two parts namely, Accelerometer part and robot part. The robot part consists microcontroller unit (MCU), which receives signal from transmitter circuit and sends the motor controlling signal to the motor driver IC. Then the robot can move forward, backward, leftward & rightward directions.

Keywords- Accelerometer, Gesture, Microcontroller unit (MCU), Motor driver, Robot.

1. INTRODUCTION

   The word Robot was first used in a 1921 play titled R.U.R. Rossums Universal Robots, by Czechoslovakian writer Karel Capek. Robot is a Czech word meaning worker [1]

   The main objectives of using robot are:

   1. Where man dares not venture.

      Robots have traditionally been put to use in environments that are too hazardous for man.

   2. To rescue pronto.

      Robots also work under precarious conditions, for search and rescue after disasters.

   3. We even make them go to war.

   Battle robots of various shapes and sizes were deployed to defuse landmines, search for criminals hiding in caves, search for bombs under cars.

   These days a number of wireless robots are being developed and put to various applications and uses. In order to enhance the contribution of robot in our daily lives

   we need to find an effective way of communicating with robots. For this purpose, there have been certain developments in area of human-machine interaction. One common form of communication is Gestures that are not only limited to face, body and fingers but also hand gestures. In order to increase the use of robot in places where conditions are not certain like rescue operations, robots can be made to follow the instructions of human operator and perform the task accordingly.

   A hand Gesture Control Robot is a kind of robot which is controlled by the hand gestures and not by using buttons. The robot is equipped with two sections- Transmitting section and Receiving section. In the Transmitting section, the Accelerometer is mounted on hand of the user capturing its gesture and moving the robot accordingly. For assigning proper levels to the input voltages from the accelerometer comparator IC is used. Encoder IC is then used to encode the four-bit data which will later be transmitted by an RF Transmitter module. In the receiving section, the received encoded data by RF receiver module is then decoded using a decoder IC which is then processed by a microcontroller and passed onto a motor driver to rotate the motors in a special configuration to move the robot in the same direction as that of the hand. So, the primary basic aim of design is to make the robot move as soon as the operator makes any gesture.

2. LITERATURE SURVEY

   1. Existing system

      At starting stage of robotics era the robots are controlled through a wire that means a physical connection between robot and user operating device [2]. Due to these reason the operating range is based on the length of wire. To overcome this drawback, wireless connection is introduced. In wireless connections robots are controlled with remotes, in this technology IR transmission used hence there must be line of sight between robot and operating device.

      After the remote controlling, gesture recognition with image capturing method is introduced. In this method Firstly, the human or user gives a command using hand,

      these gestures are captured by the camera, and the image from the camera of the robot is processed and shown to the user [3]. If the image is already registered in the library the command is given to the robot and if a new command is generated then by the updated library the command is recognized and given to the robot. The main problem in this idea, happens when the there is a very huge library of hand gestures. Due to many commands present in the library, when a hand gesture is captured and is sent to cross reference with the images in the library, it takes a load of time.

   2. Proposed system

      In the proposed system robot can be controlled by using humans hand gestures. The transmitter circuit consists of the accelerometer sensor, which is used to detect the tilting position of user hand and produce different analogue values and transmitted to receiver through RF transmitter. At receiver side these values will give command to the robots motors through MCU AT89C51 so that robot can moves in forward, backward, rightward and leftward directions.

      Advantages of proposed system

      1. Length of connecting wire is not considered because wireless connection is established.

      2. Due to RF transmission, no necessity for line of sight between robot and operating device.

      3. Instead of camera capturing accelerometer used as sensing device hence speed of operation will increase.

3. HARDWARE IMPLEMENTATION

   The hardware of the system can be divided into 3 sections, namely transmitter section, receiver section and camera section.

   1. Transmitting section

      Fig. 1: Transmitting section.

      The transmitter section consists 2 power supplies one for accelerometer i.e. 3.3V another 5V supply for remaining components.

      Accelerometer(ADXL335): – An Accelerometer is a kind of sensor which gives an analog data while moving in X, Y, Z direction or may be X, Y direction only depends on the type of the sensor. In accelerometer there is some arrow showing if we tilt these sensors in that direction then the data at that corresponding pin will change in the analog form [4].

      Comparator (LM324): – For the purpose to change the analog voltage into digital we use comparator which compare that analog voltage to a reference voltage and give a high or low voltage [5].

      Encoder (HT12E): – Encoder converts that parallel data into serial which is received by the comparator. HT12E has

      a transmission enable pin which is active low. When a trigger signal is received on TE pin, the 4 bits data transmitted through RF transmitter module [6].

      RF Transmitter Module (TX): – The transmitter module is working on the frequency of 433MHz and is easily available in the market at nominal cost. In the circuit, vcc pin is connected to the + terminal. The data pin is connected to the HT12E that is transmitted or we can say that encoded data. The next pin is GND that is connected to the ground terminal. Now the last pin ANT this is connected to a small wire as an antenna [7].

   2. Receiving secion

      Fig. 2: Receiving section.

      RF Receiver Module (RX): – The RF receiver module will receive the data which is transferred by the gesture device. It is also working as like the transmitter module- Connect the +vcc pin to the 5volt terminal. Connect the ground pin to the ground terminal. The data pin is then bit data [7].

      Decoder (HT12D): -Decoder converts that serial data into parallel which is received by the RF receiver module. The input data is decoded when there is no error or unmatched codes are found. A valid transmission in indicated by a high signal at VT pin that is pin no17 [8].

      Microcontroller unit (AT89C51): -The processing is the most important part of the robot. Till now we get the data from the decoder. Based on that data decisions must be made. Microcontroller used for our robot to give it a decision capability. Our microcontroller is made up by Atmel and the product name is AT89C51.

      Port 1 works as an input port while Port 2 is working as output port for our program. A crystal oscillator is attached to the pins 18 and 19 of the microcontroller. The oscillator creates an electrical signal of a very precise frequency which is used to keep track of time. Two capacitors are connected in parallel with the oscillator to remove unwanted frequencies [9].

      Motor driver (L293D): – It is also known as H-Bridge or Actuator IC. Actuators are those devices which gives the movement to do a task like that of a motor. In the real world there are different types of motors available which work on different voltages. So, we need a motor driver for running them through the controller. The output from the microcontroller is a low current signal. The motor driver amplifies that current which can control and drive a motor [10]. In most cases, a transistor can act as a switch and perform this task which drives the motor in a single direction.

      DC motors: – A machine that converts DC power into mechanical power is known as a DC motor. Its operation is based on the principle that when a current carrying

      conductor is placed in a magnetic field, the conductor experiences a mechanical force. DC motors have a revolving armature winding but non-revolving armature magnetic field and a stationary field winding or permanent magnet. The speed of a DC motor can be controlled by changing the voltage applied to the armature or by changing the field current.

   3. Camera section

   The system uses a smart phone with camera for continuous real time video streaming of the system and its surroundings. An IP-based Android application, running on the smartphone enables the system to transmit the real time video wirelessly. An IP based camera has been integrated with this system for real time video streaming. The video captured by the camera is transmitted over the internet and it can be viewed on any internet enabled device (Mobile Phone or Laptop) by entering the IP address, in the URL bar, provided by the IP based Camera running on the smart phone enables the system to transmit the real time video wirelessly [11]. For observing robot surroundings android mobile placed in front of robot.

4. SOFTWARE IMPLEMENTATION

   The present system can be implemented by using 3 software packages, namely

   1. Keil µvision 3 for compilation

   2. Proteus 8.0 for simulation

   3. Flash magic for code dumping.

   1. Keil Vision

      The debugger accurately simulates on-chip peripherals of 89C51 device. Simulation helps to understand hardware configurations and avoids time wasted on setup problems. With simulation, we can write and test applications before target hardware is available. The system program written in embedded C using Keil IDE software will be stored in Microcontroller. The industry-standard Keil C Compilers, Macro Assemblers, Debuggers, Real-time Kernels, Single- board Computers, and Emulators support all 89C51 derivatives. The Keil Development Tools are designed to solve the complex problems facing embedded software developers.

   2. PROTEUS

      It is a fully functional and procedural programming language created in 1998 by Simone Canella which incorporates many functions derived from other languages: C, BASIC, Assembly, and Clipper/dBase. The main usage of this language is transforming data from one form to another [12]. It was designed to be practical, readable and consistent. Its major points are powerful manipulation; comprehensibility of Proteus scripts; availability of advanced data structures: arrays, queues (single or double), stacks, bit maps, sets, AVL trees. It has a fully functional, procedural approach and variables are untyped, do not need to be declared, can be local or public and can be passed by value or by reference. New functions can be defined and used as native functions. There are three Data types that are supported by Proteus: integer numbers, floating point

      numbers and strings. It includes hundreds of functions for accessing file system; sorting data; manipulating data and strings; interacting with the user (console functions) calculating logical and mathematical expressions.

   3. Flash Magic

      It is used to dump the code to microcontroller from PC. Flash Magic is a free, powerful, feature-rich Windows application that allows easy programming of Philips FLASH microcontrollers [13]. Custom applications built for Philips microcontrollers on the Flash Magic platform Can be used to create custom end-user firmware Programming applications, or generate an in-house Production line programming tool. The Flash Memory In- System Programmer is a tool that allows in-circuit programming of FLASH memories via a serial RS232 link. Computer side software called Flash Magic is executed that accepts the Intel HEX format file generated from compiler Keil to be sent to target microcontroller. It detects the hardware connected to the serial port.

5. RESULTS AND ANALYSIS Accelerometer act as sensor in transmitting device. The accelerometer is used to measure tilting positions of hand in X&Y axis. Consider for forward(X+), Reverse(X-), right(Y+) and left(Y-). Accelerometer produce 4 different values for 4 directions. These analog values compare with reference voltage of comparator and produce digital values. These digital data are given to encoder which converts parallel data into serial form. These serial data transferred to receiving section through RF transmitter.

   At receiving side, the RF receiver receives the 4-bit serial data and send to decoder. Decoder converts serial data into parallel form. These parallel data given to microcontroller(AT89C51) and it act as a decision making device. Based on hand tilting position microcontroller sends relevant data to motor driver. Motor driver internally consists of two H-bridge circuits. H-bridge controls the DC motors with specific logic values for robot moves in forward, backward, left and right directions.

   Fig. 3: Transmitting device.

   Fig. 4: Receiving device with camera

   The direction of robot logic values which send from microcontroller to motor driver listed in below table

   Direction	Input-1 (P1.0)	Input-2 (P1.1)	Input-3 (P1.2)	Input-4 (P1.3)
   Forward	0	1	0	1
   Reverse	1	0	1	0
   Right	1	0	0	1
   Left	0	1	1	0

   Table.1: – Direction logic values of robot [14]

   Simulation results

   In the simulation we sent the relevant data to the Microcontroller (AT89C51) through switches. The Microcontroller processed the data and sent the information to the Actuator IC (L293D). The Actuator IC upon receiing information showed response by driving the DC motors.

   If the motor rotates in clockwise indicates with + sign and motor rotates in anti-clockwise indicates with sign. Initially all switches in off condition hence both motors in stop condition then robot will stop [15].

   If SW1=1 the data 1000 send to microcontroller port1 then both motors rotate in clockwise direction hence robot moves in forward direction.

   Fig.5: – Robot moves in forward direction

   If SW2=1 the data 0100 send to microcontroller port1 then both motors rotate in anti-clockwise direction hence robot moves in backward direction.

   Fig.6: – Robot moves in backward direction.

   If SW3=1 the data 0010 send to microcontroller port1 then left motor rotates in clockwise direction and right motor rotates in anti-clockwise direction hence robot moves in left direction.

   Fig.7: – Robot moves in left direction

   If SW4=1 the data 0001 send to microcontroller port1 then right motor rotates in clockwise direction and left motor rotates anti-clockwise direction hence robot moves in right direction.

   Fig.8: – Robot moves in right direction

6. CONCLUSION

   In this paper, an automated robot has been developed which works according to your hand gesture. The robot moves wirelessly according to palm gesture. The RF module is working on the frequency of 433 MHz and has a range of 50-80 meters. This robot can be upgraded to detect human life in earthquake and landslide by implementing the sensor accordingly. It can also be upgraded to bomb detecting robot as it has robotic arm it can also lift the bomb. GPS system can be added to the robot by the help of which its location can be tracked.

7. REFERENCES

1. Swetha N, Design and Implementation of Accelerometer based Robot motion and Speed control with Obstacle detection. IJSERT, volume 3, issue 2, pp 749-755, March- 2013.

2. Saikat patra et al., Controlling a robotic car. Electronics for you., volume 5, issue 2 , pp 95-96, October-2016.

3. Rohan Sheoran et al., A Study of Hand Gesture Technique to Communicate with a Robot. IJERT, Vol. 4 Issue 01, pp. 12- 13, January-2015.

4. Bonny Varghese; B. Thilagavathi, Design and Wireless Control of Anthropomorphic Robotic Arm. 978-1-4799- 6818-3/15/$31.00©2015 IEEE.

5. Roy Chowdhury,D (2003). Linear Integrated Circuits, 2nd.Ed.New Age International(p) Ltd, 207-212.

6. https://electronissome.com/ht12e-encoder-ic-remote-control- systems/

7. Diksha Goyal; Dr. S.P.S. Saini, Accelerometer Based Hand Gesture Controlled Wheelchair. IJET, volume 4, issue 2, pp 15-20, june-2013.

8. https://electronissome.com/ht12d-decoder-ic-remote-control- systems/

9. Ajay V Deshmukh, (2012). Microcontrollers, TATA McGraw Hill publications,

10. Muhammad Ali Mazidi, Janice Gillsple Mazidi, and Rollin

    1. McKiniay; The 8051 microcontroller and embedded systems, 2nd Ed. Pearson Eductation india,507-512.

11. http://en.wikipedia.org/wiki/IP_camera

12. https://www.elprocus.com/simulation-process/

13. https://www.engineersgarage.com/foruns/8051/how-to- dump-code-at89c51-microcontroller

14. Rahul Ranjan Singh et al., Wireless Controlled Robot Movement System Designed using Microcontroller. IJETT, volume 32, issue 5, pp 204-207, February 2016.

15. http://www.electronicshub.org/interfacing-dc-motor-8951- microcontroller.