A Review on Robot Hand/Arm That Is Controlled by Teleportation System

A Review on Robot Hand/Arm That Is Controlled by Teleportation System

Phongsavanh Sengaphone Mechanical Engineering Department, De La Salle University

Abstract: In this paper, we present a teleportation technique for commanding the robot hand/arm. The system interacts with the user using a novel data glove developed specifically for the purpose of evaluating the effectiveness of telepresence in telerobotics applications. The creation of a basic autonomous grab system with parallel joint torque/position control.

Keywords: Robotics Arm, Teleportation System, Novel data glove

1. INTRODUCTION

   Teleportation is the method by which a robot completes a job while being remotely controlled by a human operator. Over the years, teleportation has acquired favor in a number of professions, including the military [1]. Area of space [2]. Surgical procedures [3, 4]. Actuality [5, 6]. Exploration of the seabed [7]. In some of these systems, the robot arms are controlled through joysticks or a space ball [8, 9]. Despite the widespread use of the manipulator in a number of domains, most notably the industrial realm, executing physical in contact activities such as manipulating deformable materials remains challenging [10, 11, 12]. Cooperative work with people [13, 14]. Working in unstructured and unfamiliar situations [15]. As seen in Figures 1 and 2, A teleportation system with telepresence for a robot arm or hand: this teleportation system is comprised of three basic components: a human operating interface, a local network communication system, and a telepresence system. The telerobot system includes an arm/hand robot, a table, parallel hand-eye cameras, and global cameras. The dexterous hand is a HIT/DLR, while the robot arm is a Staubli RX60 [16]. Teleportation has been a driving factor in robotics research, motivated by the practical need of performing tasks in faraway places [17, 18].

   Fig. 2. HIT/DLR hand with data glove and CyberGrasp

   By increasing the transparency of the teleportation process, the operator's performance in such a system may be enhanced [19]. At the moment, multimodal interfaces such as virtual reality (VR)/augmented reality (AR) helmets [20], joysticks [21], contact force sensors [22], and biosignal sensors [23,24] have been developed and integrated into the teleportation system with the goal of providing immersive teleportation and increasing overall human performance. The remote robot is designed to enhance operator comfort and robot performance [19].

   This article conducts a detailed assessment of the major technology, applications, and issues associated with robot control using teleportation systems. The next parts of this paper are structured in the following manner. Section 2 is a review of the literature. Specifically, in Section 3, Materials and Methods. Section 4, Various methods and the use of the teleportation system to control the robot. Section 5 illustrates some common uses. Finally, Section 6 examines future prospects for robotic hands and arms that can be controlled by teleportation.

   Fig. 1. A robot arm/hand teleportation system with telepresence

2. LITERATURE REVIEW

   Yupei Wu, Bin Fang, Di Guo, Fuchun Sun, Huaping Liu. During the month of December (2015) [25] A robotic hand- arm teleportation device utilizes a revolutionary data glove and a human arm/hand. They provide an investigation of a robotic arm-hand teleportation system that makes use of a human arm-hand and a data glove. To detect movement during robotic hand teleportation, fifteen devices are attached to the operator's fingers. Three devices are individually connected to the palm, upper arm, and forearm to record motion for robotic arm teleportation. They used the MPU9250 [26], a System in Container device that combines nine-axis inertial and magnetic sensors in a small compact.

   Liarokapis, Minas V., Artemiadis, Panagiotis K., and Kyriakopoulos, Kostas J. Juin (2013) [27] Telemanipulation Using a Dataglove and a Low-Cost Force Feedback Device with the DLR/HIT II Robot Hand. They are examining the DLR/HIT II five-finger robot hand, which has a total of fifteen degrees of freedom (DoFs), three for each finger (two DoFs for finger flexion-extension and one DoF for finger abduction adduction). Each finger's last two joints are connected by a mechanical connection made of steel wire (with a transition ratio of 1:1). The Cyberglove II collects data at a 100Hz rate. To enable the Linux operating system to manage the Cyberglove II, necessary data collection software was written in C++ (Ubuntu 12.04 x86).

   Shuang Li, Xiaojian Ma, Hongzhuo Liang, Michael G. Orner, Philipp Ruppel, Bin Fang, Fuchun Sun, and Jianwei Zhang Teleoperation of a Shadow Dexterous Hand through a Vision-based Deep Neural Network. 18 February (2019) [28] To begin, they propose TechNet, a teacher-student network that is capable of learning the kinematic mappings between the robot and the human hand. Second, they build a paired human-robot hand dataset, which consists of pairs of depth pictures taken during the same move, as well as the robot hand's corresponding joint angles. Third, they provide an optimum mapping strategy that accounts for probable self- collisions while matching the shadow hand's Cartesian position and link direction relative to a human hand posture.

   H.F. Machiel Van der Loos, Waleed Uddin, Maram Sakr, Camilo Perez Quintero, and Camilo Perez Quintero. Orthographic Vision-based Interface for Teleportation of Robot Arms. 11 (2018) [29] They enable direct unilateral Cartesian control of a 6-DOF robot in real-time. A joystick, keyboard, or Leap Motion may be used to control the arm [30]. This Leap Motion controller is connected to a local computer through a serial connection. An external camera provides a view of the distant environment, which transmits the picture to the local computer. They use computer vision methods to augment the camera picture with depth information from the distant site, making teleoperating the robot simpler for the user.

   N. Mavridis, E. Machado, N. Giakoumidis, N. Batalas, I. Shebli, E. Ameri, F. Neyadi, and A. Neyadi Teleoperation of an Industrial Robotic Arm in Real Time through Imitation of Human Arm Movements (2010) [31] The motion capture subsystem is composed of VGA-resolution cameras (640 x 480 pixels) capable of a frame rate of up to 200 frames per second (Standard Deviation brand). The cameras are encircled by infrared LED rings and placed at a height of 2.62 meters on the corners and short-side midpoints of a rectangle measuring 6 by 4.80 meters. As a consequence, the effective capturing area has a footprint of 3m in diameter. The person is outfitted in unique clothing that has 19 2.5cm diameter luminous ball markers. The software API for the mobcap system includes a range of C++ functions that allow near-real-time reading of the tracked markers' 3D coordinates.

   Brennan T. Phillips, Kaitlyn P. Becker, Shunichi Kurumaya, Griffin Whittredge, Daniel M. Vogt, Clark B. Teeple, Michelle H. Rosen, Vincent A. Pieribone, David F. Gruber, and Robert J. Wood. A Dexterous and Low-Power Teleoperable Soft Robotic Arm for Delicate Deep-Sea Biological Exploration October 3rd (2018) [32] (A) A

   sectorized wireless glove regulates actuators by coordinating the management of separate proportional valves that provide pressure to the arm and end-effector actuators. (B) Hydraulic pressure to separate ports is managed by a unique open-circuit seawater engine capable of operating at depths of at least 2500

   m. (C) The soft arm, which is composed of modules for bending, turning, and grasping, may be employed alone or in combination with an existing manipulator system.

   Fumio Kojima, Futoshi Kobayashi, George Ikai, Wataru Fukui, and Futoshi Kobayashi. Haptic Device with Two Fingers for Robot Hand Teleoperation. September 27 (2011)

   [33] They created ExoPhalanx, a two-finger body-mounted haptic device. The ExoPhalanx supplies force to the distal portions of the human operator's thumb and middle finger, as well as the middle finger's basipodite. Due to the ExoPhalanx's tiny size and low weight, it may be worn on the human hand. As a result, the human operator receives just one-directional force. To test the haptic device's performance, a two-finger grasping teleoperation experiment was conducted utilizing the Universal Robot Hand II and the haptic device ExoPhalanx.

3. MATERIALS AND METHODS

   1. Materials

      There are several materials that might be used to create equipment for human-machine interaction, including the following: (a) Leap controller, (b) Teach Net, (c) Cyber glove II, (D) Cyber Grasp, (E) Motion Capture, (F) Soft sensors, and

      (G) ExoPhalanx on Cyber Glove. As seen in Fig 3.

      Fig. 3. Types of equipment for human machine interaction (a) Leap controller [29], (b) Teach Net [28], (c) Cyber glove II [27], (D) Cyber Grasp [16], (E) Motion Capture [31], (F) Soft sensors [32] and (G) Exophalanx on

      Cyber Glove [33]

   2. Equations

   A personal computer is also included in the suggested data glove system. Following calibration, the data glove's MCU analyses and estimates the measurements wraps them in a packet and transmits them over Bluetooth to the PC. The baud rate for data transmission is 115200 bits per second. The virtual model on the PC may be used to demonstrate the motion capture process instantly [16]. C# is used to write the interface. The system's flow diagram is seen in Fig. 4.

   Fig. 4. The flow diagram of the data glove system

4. MATERIALS AND METHODS

   There are several methods for controlling a robot arm/hand through a teleportation system. Below here is a table that summarizes the many methods for using the teleportation system.

   Table.1 Different methodologies and using the teleportation System

5. PROVIDES SEVERAL TYPICAL APPLICATIONS

   1. Exploration of Deep-Sea

      They exhibit the world's first self-contained soft robotic manipulator system designed specifically for deep-sea applications (Figure. 5). Their multi-degree-of-freedom arm is composed of modular bending, twisting, and grasping modules that are propelled by ambient low-pressure saltwater. Additionally, a ground-breaking hydraulic engine with a power need of less than 50 W is shown. The arm is teleported using a wireless glove equipped with flexible soft sensors [36]. Field trials of the manipulator system were conducted aboard a manned submersible and an unmanned remotely controlled vehicle at hydrostatic pressures equivalent to 2300 meters of ocean depth. Figure 6 illustrates earlier work on customizable soft grippers for deep-sea biological sampling (Figure 6). At a depth of 100 meters in Israel's Gulf of Eilat,

      1. a fiber-reinforced "Boa" style actuator holds a whip coral;

      2. a four-finger bellows-like actuator holds a brittle scleractinian coral (C) A two-finger bellows actuator grasps a glass sponge 300 meters underwater at Carandolet Reef, Phoenix Islands [38]. The green laser dots on the image's left side are spaced 10 cm apart. (D) A three-finger gripper with bellows-type actuators grasps a holothurian at 1800 m in the Channel Islands National Marine Sanctuary in California.

      Fig. 5. Overview of the deep-sea soft robotic arm system

      Fig. 7. Two finger grasping experiment

6. CONCLUSION

We focused on a human-operated teleportation mechanism in our review. The multimodal teleportation system, multimodal interfaces, remote robots, communication module, robotic control module, and remote perception module are all firsts for the human demonstrator. We discussed the remaining issues and future work in skill modeling and multimodal teleportation for usage with robotic arms. Finally, we show how to operate the robot's arm through teleportation using control devices such as a CyberGrasp, motion capture, and Leap controller.

REFERENCES

Fig. 6. Examples of prior work on versatile soft grippers for deep-sea

biological sampling

1. Motion Capture Subsystem with CyberGlove

This subsystem utilizes CyberGlove to identify the operator's location. The network transmits the measured operator's posture to the robot hand subsystem. The CyberGlove has three flexion sensors per finger, four abduction sensors, a palm-arch sensor, and sensors to measure wrist flexion and abduction. Because the universal robot hand's DIP and PIP joints synchronize similarly to a human finger [33], this subsystem utilizes sensor data except for the DIP joints. A two-finger gripping experiment is used to test the ExoPhalanx's haptic feedback capability (see Figure 7). Approximately the size of a baseball, a polystyrene ball has been grasped.

1. T. Kot and P. Nov´ak. Application of virtual reality in teleportation of the military mobile robotic system taros. International Journal of Advanced Robotic Systems, 15(1):1729881417751545, 2018.

2. Park, I.W., et al.: Developing a 3dof compliant perching arm for a freeflying robot on the international space station. In: 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 11351141.IEEE (2017).

3. Su, H., et al.: Improved humanrobot collaborative control of redundant robot for teleoperated minimally invasive surgery. IEEE Robot. Autom. Lett. 4(2), 14471453, (2019).

4. Zhang, D., et al.: A microsurgical robot research platform for robotassisted microsurgery research and training. Int. J. Comput. Assist. Radiol. Surg. 15(1), 1525 (2020).

5. Katsura, S., et al.: Section focused on new horizons in telerobotics for reallife applications. Adv. Robot. 32(13), 681682 (2018).

6. Leidner, D., et al.: Cognitionenabled robotic wiping: representation, planning, execution, and interpretation. Robot. Autonom. Syst. 114, 199216 (2019).

7. R. Saltaren, R. Aracil, et al. Field and service applications-exploring deep sea by teleoperated robot-an underwater parallel robot with high navigation capabilities. IEEE Robotics & Automation Magazine, 14(3):6575, 2007.

8. Ch. Borst, M. Fischer, S. Haidacher, H. Liu,G. Hirzinger. DLR Hand II: Experiments and Experiences with an Anthropomorphic Hand. Proceedings of the 2003 IEEE International Conference on Robotics & Automation. Taipei, Taiwan, September 14-19, 2003. 702-707.

9. You Song, Wang Tianmiao, Wei Jun, Yang Fenglei, Zhang Qixian. Share control in Intelligent Arm/Hand Teleoperated System. Proceeding of the 1999 IEEE International Conference on Robotics and Automation. May, 1999. p2489-2494.

10. Leidner, D., et al.: Cognitionenabled robotic wiping: representation, planning, execution, and interpretation. Robot. Autonom. Syst. 114, 199216 (2019).

11. Gao, J., Zhou, Y., Asfour, T.: Learning compliance adaptation in contact rich manipulation. arXiv preprint arXiv:200500227 (2020).

12. Leidner, D.: On cognitive reasoning for compliant manipulation tasks in smart production environments. KIKünstliche Intelligenz. 33(2), 197200 (2019).

13. Magrini, E., Flacco, F., De Luca, A.: Control of generalized contact motion and force in physical humanrobot interaction. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 22982304 (2015).

14. Kronander, K., Billard, A.: Learning compliant manipulation through kinesthetic and tactile humanrobot interaction. IEEE Trans. Haptics. 7(3), 367380 (2014).

15. Guan, C., Vega.Brown, W., Roy, N.: Efficient plannin for nearoptimal compliant manipulation leveraging environmental contact. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 215222.IEEE (2018).

16. A Robot Arm/Hand Teleoperation System with Telepresence and Shared Control, Haiying Hu, Jiawei Li, Zongwu Xie, Bin Wang, Hong Liu, Gerd Hirzinger. International Conference on Advanced Intelligent Mechatronics. Monterey, California, USA, 24-28 July, 1312 (2005).

17. Hokayem, P.F., Spong, M.W.: Bilateral teleoperation: an historical survey. Automatica. 42(12), 20352057 (2006).

18. Lichiardopol, S.: A survey on teleoperation, vol. 20, pp. 40 60.Technische Universitat Eindhoven, DCT report (2007).

19. Triantafyllidis, E., et al.: Study of multimodal interfaces and the improvements on teleoperation. IEEE Access. 8, 7821378227 (2020).

20. Stotko, P., et al.: A VR system for immersive teleoperation and live exploration with a mobile robot. arXiv preprint arXiv:190802949 (2019).

21. Luo, J., et al.: A task learning mechanism for the telerobots. Int. J. Humanoid Rob. 16(02), 1950009 (2019).

22. El Saddik, A.: The potential of haptics technologies. IEEE Instrum. Meas. Mag. 10(1), 1017 (2007).

23. Luo, J., et al.: A method of motion recognition based on electromyographic signals. Adv. Robot., 19 (2020).

24. Yang, C., et al.: Interface design of a physical humanrobot interaction system for human impedance adaptive skill transfer. IEEE Trans. Autom. Sci. Eng. 15(1), 329340 (2017).

25. Bin Fang, Di Guo, Fuchun Sun, Huaping Liu, Yupei Wu. A robotic hand-arm teleoperation system using human arm/hand with a novel data glove. December 2015 DOI: 10.1109/ROBIO.2015.7419712.

26. J. M. Lambrecht, R. F. Kirsch, Miniature low-power inertial sensors: promising technology for implantable motion capture systems, IEEE Trans. Neural systems and rehabilitation engineering, vol.22, no.6, 2014, pp.1138-1147.

27. Minas V. Liarokapis, Panagiotis K. Artemiadis and Kostas J. Kyriakopoulos. Telemanipulation with the DLR/HIT II Robot Hand Using a Dataglove and a Low Cost Force Feedback Device. 25-28 June 2013. DOI: 10.1109/MED.2013.6608758.

28. Shuang Li, Xiaojian Ma, Hongzhuo Liang, Michael G¨orner, Philipp Ruppel, Bin Fang, Fuchun Sun, Jianwei Zhang. Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network. 18 Feb 2019. arXiv:1809.06268v3.

29. Waleed Uddin, Maram Sakr, Camilo Perez Quintero and H.F. Machiel Van der Loos. Orthographic Vision-based Interface for Robot Arm Teleoperation. 11 2018.

30. L. Motion. Leap motion controller. URl: https://www. leapmotion. com, 2015.

31. Mavridis N., Machado E., Giakoumidis N., Batalas N., Shebli I., Ameri E., Neyadi F., Neyadi A. Real-time Teleoperation of an Industrial Robotic Arm Through Human Arm Movement Imitation. (2010), Page 288-293.

32. Brennan T. Phillips, Kaitlyn P. Becker, Shunichi Kurumaya, Kevin C. Galloway, Griffin Whittredge, Daniel M. Vogt, Clark B. Teeple, Michelle H. Rosen, Vincent A. Pieribone, David F. Gruber & Robert J. Wood. A Dexterous, Glove-Based Teleoperable Low-Power Soft Robotic Arm for Delicate Deep-Sea Biological Exploration 03 October (2018).

33. Futoshi Kobayashi, George Ikai,Wataru Fukui, and Fumio Kojima. Two-Fingered Haptic Device for Robot Hand Teleoperation. 27 September (2011).

34. Bin Fang, Di Guo, Fuchun Sun, Huaping Liu, Yupei Wu. A Robot ic hand arm teleoperation system using human arm/hand with a novel data glove. 6-9 Dec. 2015. DOI: 10.1109/ROBIO.2015.7419712.

35. J. M. Lambrecht, R. F. Kirsch, Miniature low-power inertial sensors: promising technology for implantable motion capture systems, IEEE Trans. Neural systems and rehabilitation engineering, vol.22, no.6, 2014, pp.1138-1147.

36. Vogt, D. M., & Wood, R. J. Wrist angle measurements using soft sensors. Ieee Sensors 16311634 (2014).

37. Galloway, K. C. et al. Soft robotic grippers for biological sampling on deep reefs. Soft Robotics 3(1), 2333 (2016).

38. Vogt, D. M. et al. Shipboard design and fabrication of custom 3D- printed soft robotic manipulators for the investigation of delicate deep- sea organisms. PLOS One 13(8), e0200386 (2018).