Gesture Controlled Robotic Arm

Gesture Controlled Robotic Arm

Tandrima Goswami, Divyanshi Sharma, Rahul Pratyush Dept. of CSE

MIET

Greater Noida, India

Ankit Kumar Dept. of ME MIET

Greater Noida, India

Abstract: Robotics is an integral to present day engineering revolution. In this new industrial revolution, robotics has become a major influence in our industrial as well as day to day life. What has made it more important is the fact that integrating it with control electronics could be result in various interesting results. Some of these are: IOT based, Wi-Fi based, Bluetooth based, Gesture based, etc. In this paper an attempt has been made to make a study in gesture control. In the presented paper, a robot is designed whose motion would be controlled by hand control.

Keywords: Gesture Control, Kinematics, robotics

INTRODUCTION

21st century, the revolution period for the world where everyone saw or took a huge leap in almost every industry sector. During the time we saw the rise of AI, the revelation in civil engineering, the new age technologies, the more dependency of people on robots and what not. In this era of machines taking control of every complex works, interaction with machines has become more important then ever. Robots are classified into two types: Autonomous Robots like line sensing or edge sensing robots and Remote-Controlled Robots like Gesture Controlled Robots [1].

This research paper tells us about the robot that the authors came up with which is a Cartesian coordinate robot. A Cartesian system is that which works in three axes: X, Y and Z and same as a Cartesian robot works or revolves in three axes: X, Y & Z. Now it should be noted that not every linear system that works in the X-Y-Z axes is a cartesian robot, one notable exception is a type of robot that employs two base (X) axes in parallel.

HISTORY

The word ROBOT is originally originated from a Czech word robota which means work and the word robotics was inadvertently coined by science fiction author Issac Asimov in his 1941 story Liar! Initially robots were designed and introduced to help human workers in reducing their work load. While programmed robots offered a novel and powerful manufacturing tool, it became evident in the 1960s that the flexibility of these machines could be enhanced significantly by the use of sensory feedback. During the 1970s a great deal

of research work focused on the use of external sources to facilitate manipulative operations [2].

Today, robotics is viewed as an extensive field of work what it was few years back. Now too many aspects are related to robotics like dynamics, kinematics, AI, Machine Learning and more. Since the authors came up with gesture-controlled robots, the focus of the paper will be on remotely controlled robots only.

There are several ways to capture a human gesture that a machine would be able to understand. The gesture can be captured using a camera or a glove or Bluetooth or infrared waves, Acoustic tactile, optical or motion technological means [3].

ROBOT KINEMATICS

Kinematics is a branch of mechanics which describes the motion of the objects without regard to the forces which cause it. It includes concepts such as distance or displacement, speed or velocity, acceleration and it looks at how these values vary with time.

In robotics, we use kinematics to facilitate the discussion of the pose and also velocity of the robot, parts of robots (Sensors, Arms, Joints and End-Effectors), and object of interest.

Using kinematics to describe the spatial configuration of the robot gives us two problems which are referred as the forward (or direct) kinematics and the second is the inverse (or arm solution) kinematics.

FORWARD KINEMATICS

Forward kinematics refers to the use of kinematic equations of a robot to determine the Cartesian position and orientation of the end effectors from specified input values of the joint parameters.

It is a very basic problem in the study of mechanical manipulation. This is the static geometrical problem of computing the position and orientation of the end effector of the manipulator.

Fig 1.1 Forward Kinematics

INVERSE KINEMATICS

It refers to as reverse process. This problem is presented as the position and orientation of the end effectors of the manipulator; calculate all the possible sets of joint angles that could be used to attain this position and orientation. This is the fundamental problem in practical use of manipulators. [4] Since the independent variable is in a robot arm are joint variables, and a task is usually stated I terms of reference coordinate frame, the inverse kinematics problem is more frequently used.

Fig 1.2 Inverzse kinematics

KINEMATICS EQUATION

The existence or non-existence of a kinematics solution defines the workspace of a given manipulator. The lack of solution means that the manipulator cannot attain a desired position or orientation.

The kinematics equations of a serial chain of n links, with joint parameters are given by:

The kinematics equations of a serial chain of n links, with joint parameters i are given by

DENAVIT-HARTENBERG PARAMETER

The DH parameters were first appeared in 1955 (Denavit and Hartenberg, 1955) to represent a directed line which is nothing but the axis of a lower pair joint.

The DenavitHartenberg parameters are the four parameters associated with a particular convention for attaching reference

frames to the links of a spatial kinematic chain, or robot manipulator

.

Fig 2 Four Parameters

Forward kinematic study of manipulators involving spatial links with two consecutive joint axes at right angles to each other leads to recognizable deficiency of the D-H algorithm

The matrices associated with these operations are [10]

Similarly,

The use of the Denavit-Hartenberg convention yields the link transformation matrix, [i-1Ti] as

known as the Denavit-Hartenberg matrix.

CONTROL

Near minimum time control:when the dynamic equations and parameters of the manipulators are precisely known, finding minimum time strategies for the robot manipulator operations are algorithmically difficult. The available methods are practically limited. To minimize the task cycle time, it is desirable to move a manipulator at its highest speed to minimize the task cycle time.To investigate the time optimal control problem for mechanical [2] manipulators. The objective of minimum time control is to transfer the end effectors of a manipulator from an initial position to a specified desired position in minimum time.

The time optimal control problem is to find an admissible control which transfers the system from initial to final state while minimizing performance and subject to constraints [2]

This control method is usually too complex to use for robots with 4 or more degree of freedoms.

VARIABLE STRUCTURE CONTROL:

Class of systems with discontinuous feedback control is called variable structure systems. It is used for control of various processes in the steel, chemical and aerospace industries. Main feature of Vss is that is has the sliding mode on switching surface in which the system remains insensitive to parameter variations and disturbances.[2] The insensitivity property of Vss enables to eliminate the interactions among the joints of a manipulator. the sliding parameter has a stable property.

Vss can be used to design a variable structure and such structure does not require dynamic modeling of the manipulator.

NONLINEAR DE COUPLED FEEDBACK CONTROL:

Designing anear optimal control strategy for mechanical manipulators is done by a substantial body of nonlinear control theory. Most of the existing algorithms emphasize nonlinear compensations. Hemami and camana applied nonlinear control to a system which has a particular class of non-linearity.[2] Saridis and lee proposed an iterative algorithm for sequential improvement of nonlinear suboptimal control law which provides approximate optimal control and requires a considerable amount of computational time.

RESOLVED MOTION CONTROL:

It means that the motions of the various joint motors are combined and resolved into separately controllable hand motions which means several joint motors must run simultaneously at different time varying rates.

[2]This allows the user to specify the direction and speed of the robot. The desired motion of the manipulator is specified in terms of a time-base trajectory in Cartesian coordinates. The mathematical relationship between these two coordinates is important in designing efficient control in Cartesian space

GESTURE CONTROL

A computer or a machine can interact by recognizing and interpreting the human movements by mathematical algorithms. Humans can interact with the computers without physically touching them. These gestures can originate from any bodily movements but moments of face and hands are commonly used.

[7] Gesture controls can be distinguished in two types.

Offline gestures: Those gestures that are processed after the user interaction with the object. E.g. A circle is drawn to activate a menu.

Online gestures: Direct manipulation gestures. They are used to scale or rotate a tangible object.

Gesture control is used in various industries like Automotive sector, consumer electronics sector, gaming sector, home automation, etc.

It can be combined with techniques like computer vision, image processing, machine learning, etc.

ABOUT MODEL

Types of Model: Whenever people hear about robots they think about a machine which looks and acts like a human. Robots bring productivity and assembly to the minds and there are different types of robots used for different applications such as Industrial robots, medical robots, security robots, etc. Some common types of robots used for different purposes are:

Cartesian robot: The most commonly used robot type for majority of industrial applications is cartesian. These are also called rectilinear or gantry robots. They have three joints that use the Cartesian coordinate system (X, Y, Z) of end-effectors with respect to base. The three prismatic joints deliver a linear motion along the axes. The kinematic description of this robot is the simplest of all configurations.

Fig. 3.1 Cartesian Robot

Cylindrical robot: This robot has at least one rotatory joint at the base while the other joints are prismatic to connect links. The rotatory joint uses a rotational motion along t6he axis, while the prismatic joint moves in a linear motion. These robots can be placed in the middle of a work space and due to its rotation element, it can work anywhere around it.

Fig 3.2 Cylindrical Robot

SCARA: SCARA for selective complaint articulate robot for assembly, which as the name suggests is commonly used in assembly operations. They are equipped with x, y, z and rotary motion because SCARAs have the easiest integration; they are the best solution for majority of application.

Fig 3.3 SCARA

Spherical robots: They are also called polar robots. In this configuration defines the position of the end-effectors with respect to the frame. The arm is connected to the base with a twisting joint and a combination of two rotary joints and one linear joint. The axes from a polar coordinate system and create a spherical shaped work envelope.

Fig 3.4 Spherical Robot

Articulated robots: This robot design features rotary joints and can range from simple to joint structure to 10 or more interacting joints and materials. The arm is connected to the base with a twisting joint. Each joint provides an addition degree of freedom or range to the mechanism.

Fig 3.5 Articulated Robot

Delta: There are spiders like robots having a dome-shaped work envelope which they can achieve very high speed. They are the fastest and most expensive robots. The main advantages of Delta robots are their speed and delicate precision with which they operate. They are heavily used in food, pharmaceuticals and electronic industries [5].

Fig 3.6 Delta Robot

Model: The aim of this research is to build a robotic arm from scratch. In this paper the authors worked on a Cartesian coordinate robot configuration carried out by gesture control The configuration is easy to use and program. The linear movements of the Cartesian elements provide the robot with a tube shaped which fits best with pick and place application and has a range from 100-1000 millimeters.[6]

The model is a robotic arm which is attached to a movable base to provide additional degree of freedom. The arm has two prismatic joints connected to a three-finger mechanism end effector.

The arm is made of aluminum rods of 20cm x 5cm. The joints are connected by servo motors which induce motion in the arm. The servo motors are connected with the nRF module for communication between the arm and the glove which contains the accelerometers and the flex sensors which is used to control the robot using hand gestures.

The end effector which is the three-finger claw is controlled by using servo motors and flex sensors. The fingers are connected to the servos using strings and the flex sensors control the motors which lead to the motion in the fingers

The base is connected with 4 wheels at the bottom so that the arm can move around increasing workspace of the arm. The wheels are synced together using a motor driver and the motion is controlled using accelerometer. The speed and direction can be controlled by the following code: #include

<MPU6050_tockn.h>

#include <Wire.h>

MPU6050 mpu6050(Wire);

void loop() { mpu6050.update();

Serial.print("accX : ");Serial.print(mpu6050.getAccX()); Serial.print("\taccY : ");Serial.print(mpu6050.getAccY()); Serial.print("\taccZ : ");Serial.println(mpu6050.getAccZ());

if (mpu6050.getAccX() >= 0.100)

{ //FORWARD

digitalWrite(4, HIGH); digitalWrite(5, LOW); digitalWrite(6, LOW); digitalWrite(7, HIGH);

}

else if(mpu6050.getAccX() <= -0.90)

{ //BACKWARD

digitalWrite(4, LOW); digitalWrite(5, HIGH); digitalWrite(6, HIGH); digitalWrite(7, LOW);

}

else if(mpu6050.getAccY() >= 0.90)

{ //LEFT

digitalWrite(4, LOW); digitalWrite(5, HIGH); digitalWrite(6, LOW); digitalWrite(7, HIGH);

}

else if(mpu6050.getAccY() <= -0.90)

{ //RIGHT

digitalWrite(4, HIGH); digitalWrite(5, LOW); digitalWrite(6, HIGH); digitalWrite(7, LOW);

}

else{

digitalWrite(4, LOW); digitalWrite(5, LOW); digitalWrite(6, LOW); digitalWrite(7, LOW);

long timer = 0; void setup() {

pinMode(4, OUTPUT); pinMode(5, OUTPUT); pinMode(6, OUTPUT); pinMode(7, OUTPUT);

Serial.begin(9600); Wire.begin(); mpu6050.begin();

}

}

}

This configuration can be used for many different purposes like pick and drop, Arc wielding, etc.

It can also be used for special purposes like defusing or detecting a bomb or other hazardous purposes where it is not safe for a human to infiltrate.

GESTURE RECOGNITION:

The transmitter sends the value of the accelerometer to the microcontroller and the receiver module receives it.[8]

There are two fixed values assigned to the x and y axis and when these values change the micro controller act upon it and

perform the specified action shows the gestures to control the movement of the robot.

When the user tilts his hand forward, the gesture is recognized as the forward movement, and the robot moves in the forward direction. The angle of the tilt or the difference

RECEIVING THE DATA:

The data is transmitted and received using HT12D microcontroller and corresponding to the data the power is sent to the corresponding wheels. [9]In transmitter, an accelerometer (mpu 6050) and a RF transmitter is used. The accelerometer gives an analog output so we need to convert this analog data in to digital. For this purpose, we have used 4 channel comparator circuits in place of any ADC. By setting reference voltage we get a digital signal and then apply this signal to HT12E encoder to encode data or converting it into serial form and then send this data by using RF transmitter into the microcontroller.

At the receiver end we have used RF receiver to receive data and then applied to HT12D decoder. This decoder IC converts received serial data to parallel and then read by using Arduino.

The robot is controlled by using this data.

CONCLUSION AND FUTURE WORK

The paper has successfully designed and built a gesture controlled robotic arm and explained the methods and solutions to various problems. The arm can be controlled with good accuracy and can place the end effector at a distinct location. It can be used to perform the pick and place application with full accuracy.

In future as the model advances it can be used to perform various hazardous tasks which can be proved lethal to a human with absolute safety. Many changes can be made to the model such as increasing the size, adding various tools to the end effector connecting various sensors to expand the range of application performed by the robot arm strengthen the precision of the model and increase the frequency range of the model by using better transmitters and receivers.

REFERENCES

1. International Journal of Advanced research in computer science and software engineering; Premangshu Chanda, Pallab Kanti Mukherjee,

   Subrata Modak, Asoke Nath

2. Robotics: Control, Sensing, Vision and Intelligence; K. S. Fu, R. C.

   Gonzalez, C. S. G. Lee

3. Monika Jain, Aditi, Ashwani Lohiya, Mohammad Fahad Khan, Abhishek Maurya Wireless Gesture Control Robot

4. John J.Craig – Introduction to Robotics Mechanics and Control 3rd edition-Pearson \ Education Inc. 2005

5. What are the main types of robots, RobotWorx

6. Five types of industrial robots and how to choose best fit, Valin

7. Dietrich Kammer, Mandy Keck, Georg Freitag, Markus Wacker, Taxonomy and Overview of Multi-touch Frameworks: Architecture, Scope and Features Archived 2011-01-25 at the Wayback Machine

8. Gesture Controlled Robot using Arduino and Android

9. Premangshu Chanda, Pallab Kanti Mukherjee, Subrata Modak, Asoke Nath

10. Accelerometer based hand gesture controlled robot

11. Lipkin, Harvey (2005). "A Note on DenavitHartenberg Notation in Robotics". Volume 7: 29th Mechanisms and Robotics Conference, Parts a and B.