Real- Time Object Tracking, Semi-Autonomous, Catch/Pick and Place Robot with Pneumatic Subsystems

Real- Time Object Tracking, Semi-Autonomous, Catch/Pick and Place Robot with Pneumatic Subsystems

Vol. 9 Issue 07, July-2020

Vol. 9 Issue 07, July-2020

Real- Time Object Tracking, Semi-Autonomous, Catch/Pick and Place Robot with Pneumatic Subsystems

Ms. Manisha Gupta

BE EXTC Engg, DBIT,

Mumbai

Ms. Sayli Tandel

BE EXTC Engg, DBIT,

Mumbai

Mr. Nilesh Sharma

BE EXTC Engg, DBIT,

Mumbai

Mr. Pransu Kushwaha

BE EXTC Engg, DBIT,

Mumbai

Mr. Rajesh Maharana

SE MECH Engg, DBIT,

Mumbai

Prof. Yogesh Gholap

EXTC Engg, DBIT,

Mumbai

Abstract:- This paper describes a robotic system that is capable of real time object tracking, catching and placing an object with successfully achieved speed and accuracy. The former is a semi- autonomous robot which catches a rugby ball thrown and places it at the spot. The robot has a completely integrated system which includes sensors and mechanical parts thus, making it semi- autonomous. Pick and place mechanisms, both are performed by the same subsystem using pneumatics and thus reduce the robots weight as well as complexity in design. The robot also has a safety feature to breakdown the power supply to subsystems in case of any emergency. Adding to this, a power distribution board is designed which ensures proper voltage and current supply to each component.

Keywords:- Real time Object tracking, rake, serial communication, power distribution board.

1. INTRODUCTION

   During the last decades, there has been a tremendous growth in automation which include robotics. Daily activities are characterized by an increasing interaction with smart machines that present a certain level of autonomy. Robots ensure that a task can be done more accurately, can speed up the given task and can perform the repetitive task without any difficulty. Every emerging field includes some or other type of automation involved which makes life simpler. Robots contribution to industries, automobiles, medicine and various other fields have resulted in a promising technology. This technology is where the system gets updated depending upon its application, design and usage.

   Robots can be driven autonomously, semi-autonomously or manually. Remote controlled robots are easier to have control over and perform the actions as directed with manual precision. This control can be wireless or wired. Here, communication synchronization between devices plays an important role. Besides the situation of interaction, the implementation of human-like behaviors in a robots way of

   working also seems to benefit an operator that has to control a machine.[1]

   Though remote-controlled robots make them behave as human-like, autonomous robots perform those tasks in more efficient way. In recent years, the development of robotic movement has grown rapidly and the visual techniques have also been applied widely from the simple image processing techniques to complicated image recognition skills. The latest technology is to imitate the stereo vision of human eyes and to track or operate on fast moving objects.[2]

   Our aim is to build a similar type of bot which has features such as catching an object, picking and placing it to the desired location. The bot can be driven manually as well as with some automatic features which results in a semi-autonomous bot. The use of real time object tracking gives the bot a capability to locate any object with the help image processing which is configured using Rpi and camera, resembling a human eye to detect and track the trajectory of the object.

2. SYSTEM DESIGN

   Fig.1 describes the entire system of the robot. Place, pick and catch are the three mechanisms involved. Arduino microcontroller controls the overall system. Manually, the robot is driven by the PS3 controller using a bluetooth interface. Semi-autonomous nature of the robot is achieved by triggering the button on PS3. This includes line following sensors functionality, image processing through Rpi camera and proximity sensors feedback mechanism.

   Dual shaft wireless PS3 controller is useVdotlo. 9drIisvseueth0e7r,oJbuolyt-i2n0t2o0 the commanded direction. Different buttons present on PS3 have different defined functions.

   Fig.1 Block diagram of system

   Aluminium extrusion profiles (AISI 6063-T5) (20mm x 20mm) having tensile strength of 186MPa are used to make the base chassis of the bot. The chassis is designed in square configuration to keep the moment of inertia along the X and Y axis equal for better mobility. All the profiles are fastened together in a secure position with L-connectors using M4 screws and T-nuts, thus making it compact. Support members of 620mm and 3 others of 300mm are placed in Y configuration holding the centers of the base profiles. 4 Encoder motors (12V- 470rpm, 25.9N-cm), each are placed on the four corners of the chassis. Four parallel support members of 300mm are placed at 70mm away from the base chassis side members to provide assistance to the motors (Fig.2). All the motors are mounted on chassis using C clamps. Rubber material is filled between C clamp and motor to reduce the vibration of the motors. Adhesive (lock tight) is applied between all the nuts and screws to prevent loosening due to vibrations.

   Overall dimension of the robot at home position is 765mm x 890mm x 1000mm (l x b x h). Total estimated weight is 18 kg. During the game, the dimension is 1000mm x 890mm x 1000mm (l x b x h).

   1. Motion: The bots motion is specified by the joystick present on the controller. The joysticks analog values varying from 0 to 255, follows the cartesian coordinate system and are mapped with the robots motion. These values give the robot a particular direction to move. Therefore, the robot is capable of moving in right, left, front, back, diagonal and also of spot turns. DC geared planetary motors of 470RPM are used to drive four mecanum wheels (Ø 100mm) giving us required linear velocity for omnidirectional motion. Load is distributed evenly amongst four motors that are coupled to wheels using a flanged coupler which provides adequate support to both wheel and motor, making the motor body rigid and thus reducing vibrations.

   2. Catching Mechanism: Catching mechanism is one of the important mechanisms in the system. The chassis is mounted with a bucket (620 x 700 x 450mm) made up of PVC sheets mounted on a frame of aluminium profile (Fig.3). The rear two profiles are extended further by 550mm, covered by a Nylon net to damp the velocity of the object and to prevent rebound. Nylon net being flexible and strong, makes the object to fall at the desired receiving region of the robot. The bottom sheet of the bucket is at 20° angle in reference with the base chassis which channels the ball towards the rake system. Support members of 620mm for the base of the bucket are placed 100mm away from the base chassis on the 430mm support which is at the center of base chassis side profiles for extra rigidness of the bot. This system is tested for the object such as a rugby ball (in our case) which weighed around 460gm and had multiple trajectories with a range of around 4m.

      1. Manual

         Fig.2 Base chassis

         Fig.3 Bucket design

   3. Placing Mechanism: The ball being caught, should stop inside the bucket itself. For this, a rake system is designed. A pneumatic piston rod end (ø15x150mm) is attached to an aluminium square tube end (20mm x 20mm). The center of the tube is hinged between two aluminium extrusion profiles

   Manual motion is achieved by bluetooth commuication

   carried by bluetooth dongle and USB host shield. This host shield is connected with the arduino mega microcontroller.

   (20mm x 20mm) while the other end is hinged to the bucket frame and also to the piston's end cap. Rake (350mm x 195mm) is fabricated using acrylic sheet and is connected to the bottom

   of the square tube. The rake assembly mounted on the profile is

   1. Semi-Automatic

      Vol. 9 Issue 07, July-2020

      connected to two aluminium pulleys via nylon strings (tensile strength>25N). Tension generated in the string is calculated and is found to be of 23.8762N (Eqn. [1]). These pulleys are mounted on a 5mm shaft which is connected to an encoder motor (12V 460rpm 25.9N-cm) via a flexible shaft coupler. Encoder motors motion is controlled by PS3, while the opening/closing of the rake is controlled by the piston. The closed rake acts as a barrier and makes the ball stationary inside the bucket. After the ball is caught, the robot moves towards the placing spot carrying the ball. Hence, the inclined surface is so designed that during motion, the ball doesnt get displaced and always remains close to the rake. The rake is designed in such a way that while placing, it ensures that the ball doesnt touch the boundaries. The rake system is lifted with a pulley system and the ball is placed at the provided space named as try spot. (Fig.4).

      Since, the designed robot has a specific task, it can also be programmed to carry out all the tasks. This automation is done using sensors.

      1. Sensors: Sensors are used to estimate a robot's condition and environment. These signals are passed to a micro-controller to enable appropriate behavior. Following are the sensors used in the system:

   1. Tiny Lidar Sensor: VL53L0X tiny lidar sensor based on TOF (Time Of Flight) is connected at the bottoms left and right side of the robot. This sensor is a proximity sensor which uses I2C communication. I2C has two-wires named SDA (serial data line) and SCL (serial clock line), the bidirectional serial bus that provides a simple and efficient communication between devices. It is a multi-master and multi-slave protocol but single-master and the multi-slave combination are mostly used. Master is a device which initiates transactions and generates a clock signal. The slave is a device which is being addressed by the master.[4] These sensors detect the boundaries and signal the micro-controller when the robot is just a few distance away from hitting the boundary. Hence, these sensors act as preventive measure on both the sides by moving the robot in opposite direction of the boundary.

      Calculations:

      Fig. 4 Stages of placing mechanism

   2. TF Mini Sensor: For the perfect reception of the ball, it is important to have linear alignment with the throwing device. This sensor has the maximum range of about 12m with very less blind length of 0.3m. It has a very narrow beam and thus proves useful to detect the throwing device and align the

      = angle between nylon string & Al profile = 32

      T = tension in string

      m = mass of the rake assembly = 1.5kg Therefore, Tsin = mgEqn. (1) Hence, T = 23.8762N

      Therefore, the minimum required tensile strength according to the designed assembly is 23.8762N.

      R = radius of pulley = 23mm Therefore,

      Torque acting on the motor = T x R = 0.54 Nm

      4) Picking Mechanism: Picking mechanism is designed so as to overcome the possibility of the ball being missed while catching. This mechanism is carried by the same rake system. To pick the ball from ground, the robot is moved closer to the ball manually. Initially, the rake is in an open position i.e. piston in open state. When the robot is closer enough to the ball, the rake moves down and opens up i.e. the piston closes. Gradually, the piston opens up and pushes the ball inside. The ball rolls up due to the force provided by the rake and due to down inclination, it rolls down and there, the rake acts as a barrier. Gradual open and close of the piston is achieved using a flow control valve.

      robot accordingly. It uses UART communication technique.[5]

   3. Ultrasonic Sensor: This is another type of proximity sensor which use ultrasonic waves for detecting the distance. This sensor is attached at the internal side of the bucket. Automatically, to know whether the ball is caught and whether the ball has reached towards the rake system of the robot, this sensor is used. The ball caught, rolls down the ramp and is detected by this sensor and triggers the next movement.

   Line Follower: Line follower system is used to automate the motion of the robot. Four LSA08 sensors are mounted at center of each sides of base chassis of the robot. Initially, each sensor is being tested by connecting them at four different serial ports. For final testing, we shifted from four ports to two ports, which would speed up the communication process and hence decreases delay. PID tuning is followed for making it smoother and more precise.

   1. Image Processing: Image Processing system is implemented to detect and track the rugby ball thrown towards the robot using the parameters obtained from the projection path followed and use parameters such as centroid for side- ways motion of bot and velocity of the ball for forward/backward motion of the bot. Motion compensation method is used based on image processing, which estimates the

      location of the ball by calculating the motion vector. An algorithm based on YOLO (You Only Look Once) is developed to describe the relationship between Try ball (Rugby ball) and TRs (Try Robot) motion to catch the Try ball inside the bucket. Using the Yolo algorithm, the input image was first divided into S x S grid (e.g. 14 x14 grid on image). If the center of the ball falls into a grid cell, that grid cell is responsible for detecting the presence of the ball. Bounding box is formed around the ball consisting of 5 predictions: x, y, w, h and confidence. The (x, y) represent the center coordinate of the box relative to the grid, w and h are the width and height predictive relative to the image and confidence predicts the IOU (intersection of union) of predicted box and original box. The data set of rugby ball is created using darknet where we trained our own model on last convolutional layer. The data set was only trained for rugby ball so that the detection process is faster while dealing with real time projectile tracking. Fig.5 is the training result of our data set in which 100 images were used and 246 iterations were performed.

      Fig. 5 Final result of object detection

      Motion of bot in forward and backward direction:

      At what velocity the ball is ejected initially is known. Keeping that as the reference velocity, and considering the throwing system is based on pneumatics, if the pressures changes, the velocity of ejection will change and so the range of the ball. To overcome, the robot can move forward and backward depending upon at what velocity the ball is traveling and at what range it will cover. The velocity of the ball is thus calculated. Euclidean method is used to calculate the distance between two frames, as the center of the ball is tracked and found using the yolo algorithm. The distance obtained is divided with the frame per second (FPS) and multiplying with how many frames are taken in one second will result in velocity calculation.

      The calculated velocity is compared with the reference velocity. If the calculated velocity is greater than the reference velocity, it means that the ball will cover larger range and the bot has to move backward (Fig. 6) and if calculated velocity is less than the reference, the bot will move forward (Fig. 7) as the range covered by ball will be less. The motion of the bot is controlled using rotary encoder and how much distance the bot

      should have travel in one rotation iVs ocl.a9lcIuslsauteed07,frJoumly-2t0pe0

      circumference of wheel.

      Fig. 6 Backward motion

      Fig. 7 Forwrd motion

      Motion of the bot in right or left sideways:

      The center coordinate of the image is defined as origin. If the ball is detected to the left of the origin, the bot will move towards the left and the amount of rotation of wheels (or distance to be covered) will depend on the coordinate values detected. Same concept is applied for the right motion. For sideways motion by multiple trial and errors, it is found that its working results same but with maximum error of 5cm. (Fig. 8 & Fig. 9)

      Fig. 8 Right motion

      5) Pneumatic Sub-System: FollowVinolg. a9rIessthuee c0a7l,cJuullayt-i2o0n2s0 for the amount of pressure bar needed.

      Volume of 1 bottle = 3100cm3

      No. of strokes = 1 = 103

      1

      Volume of 1 piston = 30.14cm3

      No. of cycles = .

      .

      = 103 50

      2

      Fig. 9 Left motion

   2. Rake System: Rake system to work automatically, the number of rotations to the pulley motor is programmed. By the repetitive trials, number of rotations are selected so that the encoder motor perfectly moves rake up and down along with opening and closing of the piston.

   3. Power Distribution Board: Fig.10 shows the power distribution board implemented in the system. There are a number of different electronic components used for building the system with different current and voltage rating. If voltages and current ratings are not met properly for each system and if more current or voltage to the component are supplied than their ratings, then such situation could damage the components. Hence, a power distribution board is designed to prevent any damage to the components and make them function properly. As there are two types of components, one of 5V and another of 12V, so we have used two different LiPo batteries to power the entire bot. Those batteries are rated for 11.1V, current rating of 5500mAh and nominal discharge rate of 30C. 5V is obtained from 11.1V by using the DC-DC converter LM2596. LM2596 is set at 6V instead of 5V taking into consideration the losses due to resistance offered by track and other. This boards provides isolation between high and low power components. An emergency switch (mushroom) is used to cut off the power totally in the system, if some failure occurs. When mushroom switch is released, the ground connection breaks off and no power will be supplied to the system. Other safety features included are: reverse polarity protection, fuse to limit the excess current flow on any component and LiPo tester to monitor battery level.

   Fig. 10 Power distribution flow of the system

   One bottle provides 50 stroke cycles for a piston. Thus, considering the leakage losses and for better system margin, total two bottles are used.

   (one stroke cycle = open and close cycle of piston)

   According to the method that we are using to complete all the tasks, maximum 2 strokes are needed. As this task will be repeated many times, also considering the leakage in the bottle, two bottles are enough. Fig. 12 shows the pneumatic connection of piston, flow controller and solenoid valve using Fluid SIM.

   Fig. 12 Pneumatic connections

3. FINAL IMPLEMENTATION

   Fig.13 shows the final CAD model designed on Solid Works. This model comprises entirely the mechanical as well as the electronic components. Fig.14 shows the actual final implementation of the robot. The aimed mechanisms are tested applying different conditions.

   Fig. 13 CAD model of robot

   REFERENCEVSol. 9 Issue 07, July-2020

   Fig. 14 Final implementation of the robot

4. CONCLUSION

This paper presents our project work what we aim for. The goal of our project is to design a system which could track the object in real time. The YOLO algorithm was proven to be suitable for detecting and tracking the objects trajectory. In our system, we wanted only one object to be detected by image processing as so we trained or modeled a dataset for that object. We used darknet for training purposes and we had many results. As we tried training our model on a low-end machine, the first dataset gave false detection. So, we created a second dataset which included more images with different environmental conditions. This dataset gave us a better result in which only the target object was detected in the frame. Also, we came across a solution where instead of having two different systems for picking and placing mechanisms, only one rake system served the purpose. System was mechanically made robust and was tested under all conditions. Bucket for catching was designed perfectly to catch the rugby ball and was tested up to 4m of distance with successful catches. Entire robot was given electronic support so that all the tasks complete in less time using sensors which resulted in making the system autonomous. Power distribution board was designed as a safety feature to avoid damage to all the components.

1. Yves Rybarczyk and Diodo Gil Carvalho, Bioinspired Implementation and Assessment of a Remote-Controlled Robot, Applied Bionics and Biomechanics,11 Sep, 2019.

2. J. C. Titarmare, M. D. Katkar and A. J. Agarwal, Robotic Arm-The Ball Catcher, International Journal of Engineering Trends and Technology, Feb 2014.

3. Automation and Robotics (Robot Store), Robot Store (HK), Retrieved 23 January 2015.

4. Vivek Pandey, Sparsh Kumar, Vimal Kumar and Pankaj Goel, A Review Paper on I2C Communication Protocol, International Journal of Advanced Research, Ideas and Innovation in Technology, vol.4, issue.2.

5. Benewake (Beijing) Co. Ltd., TF mini Infrared Module Specification, SJ-GU-TFMINI-01 datasheet.

6. Lucky Gautam, Rahul Kr. Verma and Chinoy Sharma, Developing Manual Control for a Line Follower Robot, Advance in Electronics and Electric Engineering, vol.3, no.3 (2013), pp, 305-310.

7. Cytron Technologies, Advanced Line Following Sensor Bar LSA08, v1.1, June 2012.

8. Shraddha V. Kothiya and Ms. Kinjal B. Mistree. A review on real time object tracking in video sequences. March 2019.

9. Mr. P. RAVI Dr. A. ASHOKKUMAR. Analysis of various image processing techniques, 2017.

10. Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unied, real-time object detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.