Leap Motion Controlled Robotic Arm

Leap Motion Controlled Robotic Arm

Anchit Srivastava, Jaspreet Singh, Gurpreet Singh, Sachin Thakur

Department of Electronics Engineering KCCEMSR, Thane,India anchit.srivastava@ymail.com

Abstract-The Leap Motion Controller is a new device for hand gesture controlled user interfaces with declared sub-millimetre

below shows the basic block diagram of leap motion controlled robotic arm.

accuracy. However, up to this point its

capabilities in real environments have not been analysed. Therefore this paper presents the field

of study to control a robotic arm using leap

motion sensor. Robotic arm has many industrial as well as domestic applications which compelled us to pursue research in this field of study. Interfacing leap motion technology with a robotic arm will lead us to some cutting edge applications of robotic arm mimicking hand gesture.Using the conclusion of this analysis can improve the development of applica ions for the Leap Motion controller in the field of Human- Computer Interaction.

1. INTRODUCTION

   In the last few years, different optical sensors, which allow the acquisition of 3D objects, have been developed. Concurrently with the appearance

   Fig.1: Leap Motion Controlled Robotic Arm Block Diagram

2. LEAP MOTION TECHNOLOGY

   A Leap Motion System tracks and detects hands, fingers and finger-like objects. The device operates with high level of precision and high tracking frame rate. The leap motion software analyzes the object in the devices field of view. It recognizes hands, fingers, and tools, reporting both discrete positions, gestures, and motion. The Leap Motion field of view is an inverted pyramid centered on the device. The effective range of the Leap Motion Controller extends from approximately 25 to 600

   of the new sensors, the number applications vastly increases. These

   of potential applications

   millimetres above the device (1 inch to 2 feet).

   have different requirements in terms of resolution, speed, distance and target characteristics. Particularly with regard to gesture-based user interfaces, the accuracy of the sensor is a challenging task.The Leap Motion controller introduces a new gesture and position tracking

   system. In contrast to standard multi-touch

   solutions, this above-surface sensor is discussed for use in realistic stereo 3D interaction systems, especially concerning direct selection of

   stereoscopically displayed objects. Our main

   objective in this paper is to utilize the 3D mapping of the human gesture to control the robotic arm. With the help of the leap motion technology we can easily map positions and movements of our hand and fingers. Acquiring this 3D data will be utilized

   For controlling of servo motors in the robotics arm. The robotic arm is moving in accordance with the

   Fig1: Leap Sensor

   The Leap Motion system employs a right-handed Cartesian coordinate system. The origin is centered at the top of the Leap Motion Controller. The x- and z-axes lie in the horizontal plane, with the x- axis running parallel to the long edge of the device.

   Movement of the hand above the leap motion sensor.The data from the sensor is routed to the

   controller and interfaced with the motors and other

   The y-axis is vertical, with positive values

   increasing upwards (in contrast to the downward orientation of most computer graphics coordinate

   devices connected to it.All these devices are controlled through the sole movement of the hand and fingers above the leap motion sensor. Fig.1

   systems). The z-axis has positive values increasing

   toward the user.The Leap Motion API measures physical quantities such distance, time, speed and

   angle.

   Fig.2: GUI displaying the human hand gesture.

3. ROBOTIC ARM

   A robotic arm is a mechanical device resembling a human arm in shape and definition and designed

   Fig.3: Illustration of a robotic arm.

   Our design of arm consists of 3 degrees of freedom.

   for performing various functions in a controlled

   and precise manner. It basically has a fixed base on which other parts are mounted with an end effector which works like a human hand. Robotic arm is designed for various industrial applications like welding, painting; placing objects etc. it has to be arranged in such a way so as to accomplish the

   The main controller which we have used for controlling the servo motors and actuators is arduino. Arduinouno provides 6 pwm pins which can be used for controlling servo motors. We are using 3D printing technology for developing our model. With the help of cad design software we have designed our robotic arm chassis.

   desired tasks. The robotic arm can be

   programmable and hence can perform a single action repeatedly. For applications such as packaging and painting etc. precision is required and hence robotic arm is used. The design of our robotic arm is simple, consisting of the base which is on a fixed place and the joints for movements. At the end is the end manipulator which is used for actual environment interaction as the end

4. GUI

   Processing is used for basic GUI interface which is used for controlling the robotic arm. Processing is an open source programming language, development environment. Initially was created as software sketchbook and to teach computer programming fundamentals within a visual context.

   manipulator is in contact with the performing the desired task.

   object while

   Using the processing library leap motion is easily integrated. Serial functions are directly available in the processing for pushing the serial data through

   In constructing our arm we used three servo motors

   Bluetooth. The Processing app sends servo data to

   and gears. The servo motor at the base allows the

   circular movement of the whole structure. Another servo motors allows the forward and downward movement of the arm. The last servo motor at the wrist allows to pick up objects using gripper.

   the Arduino compatible board in format.

5. INTERFACING

   a very simple

   Processing is used for communicating with arduino and leap motion sensor. Arduino has inbuilt processing library which is used for communicating with processing. This library allows you to control an Arduino board from Processing without writing

   code for the Arduino. Instead, standard firmware (program) to

   you upload a the board and

   communicate with it using the library. The firmware is called Firmata, and is included in the Arduino software. The robotic arm is moving in accordance with the Movement of the hand above the leap motion sensor

   The leap motion sensor is connected to the computer system. GUI is made using the processing software which accepts the data from

   the leap sensor which is in co-ordinate form. The GUI has sliders which controls the servos. The data is send to theArduino via Bluetooth. Arduino receives the serial data which is interfaced with processing. Thus the data from leap sensor is send via Bluetooth to the arduino board. We have written the aduino code for controlling the motors. Using the hand and finger movement we can control the robotic arm. This is an example of leap motion in human computer interaction.

   1. Applications

   The control of robotic arm using leap is a concept driven by intrigued mind to figure out and derive new ways of doing things. The leap motion technology although is relatively new but is constantly being explored and hence we have devised a system keeping in mind he ever evolving field of technology. The leap motion sensor controlled robotic arm system is designed to be used in areas where humans cannot reach. Hence the robotic hand is used to perform certain tasks and the control is achieved using leap motion sensor. This allows us to perform tasks using natural human movements and achieve desired results. This makes for an alternate way to be completing the action. Robotic arm could be used to perform tasks in areas such as nuclear affected, deep sea, high pressure and humidity, extremely hostile climates and places such as space stations etc. All the tasks that need to done can be completed using natural human movements. Thus we can achieve all this using leap motion based devices for better control.

6. CONCLUSIONS

The main aim of making this paper is to make technology that is already available to make robots mimic humans. The first step in that is control of the robotic arm through human gestures. Thus the ongoing research suggests that it is very much possible to make a robot mimic human movements in the near future. Our paper gives an early study of this technology which could one day revolutionize the way humans and machine could interact. The control of robotic arm through the leap motion sensor is still in the developmental stages but is a promising new technology. It is quite possible that the robots of the future will be able to mimic humans and the applications of this feature are limited only to ones imagination.

REFERENCE

1. Frank Weichert, Daniel Bachmann, BartholomausRudak and Denis Fisseler, Analysis of the Accuracy and Robustness of the Leap Motion Controller,Sensors 2013, ISSN 1424-8220.

2. Khoshelham, K.; Elberink, S.O. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 2012, 12, 14371454.

3. Biswas, K.K.; Basu, S. Gesture Recognition using Microsoft Kinect. In Proceedings of the IEEE International Conference on Automation, Robotics and Applications (ICARA), Delhi, India, 68 December 2011.

4. Ambrosch, K.; Kubinger,W. Accurate hardware-based stereo vision. Comput. Vis. Image Underst.2010, 114, 1303 1316.

5. R. Dillmann, Teaching and learning of robot tasks via observation of human performance, in Robotics and Autonomous Systems, vol. 47,no. 2-3, pp. 109-116, 2004.

6. S. Perrin, A. Cassinelli, and M. Ishikawa, Gesture recognition using laser-based tracking system, in Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 541- 546, 2004.

7. K. Murakami, and H. Taguchi, Gesture Recognition using Recurrent Neural Networks, in Proceedings of ACM CHI91Conference on Human Factors in Computing Systems, pp. 237-242, New Orleans, USA, 1991

8. Chung-HsienKuo, Yu-Wei Lai, Kuo-Wei Chiu, Shih-TsengLee. Motion Planning and Control of Interactive Humanoid Robotic Arms IEEE International Conference on Advanced Robotics and its Social Impacts Taipei, Taiwan, Aug. 23- 25, 2008.

9. Robotics and automation by Robert schilling.

10. www.processing.org

11. https://developer.leapmotion.com/docume ntation/csharp/devguide/Leap_Overview.h tml