Augmented reality (AR) for enabling human-robot collaboration with ROS robots

Abstract

With current industrialization trends on the reintroduction of human into the manufacturing process and development of augmented reality I propose the interface, which uses Augmented reality to allow the operator to interact with robotic systems, such as manipulators and mobile robots. 2 interfaces were created: one for manipulators and one for mobile robots. Both of them are developed to work on Microsoft Hololens 2 augmented reality glasses and robots, running ROS. Interface for manipulators allows user to control the robot by sending end-effector goals and previewing the goal joint states of the manipulator. Interface for mobile robots allows the user to send navigation goals and preview the robot’s movement trajectory. Interfaces were developed using Unity game engine. Developed interfaces were tested with UR5e manipulator and Robotont mobile robot.