Teleoperation of Remote Controlled Toy Cars in VR

Abstract

Teleoperation, also known as remote operation, is a process enabling a vehicle, robot, or machine to be operated remotely. So that it establishes a two-way communication channel between the operator and the remote-controlled machine, which is called a teleoperator. Teleoperation is linked with several technologies, concepts, and terms such as telerobotics, telepresence, Virtual Reality (VR), and Augmented Reality (AR) and consists of several local and remote components [1]. Many areas benefit from teleoperation. The prominent examples are Autonomous Vehicles (AVs), Remotely Operated Vehicles (ROVs), drones, and entertainment. As a result, humans are able to explore environments and perform tasks via remote machines, without having to be there physically. In this thesis, while the focus is to provide a minimum viable product, a practical solution to teleoperate toy cars (Donkey Car S1[2]) in VR is introduced. VR is, in fact, a technology enabling users to immerse into an environment simulated with computers and interact with it[3]. As VR users take the advantage of immersive display and feel completely involved in the activity[3], the decision has been made to render the feedback (camera images) from the car in the VR glasses (Oculus Quest 2 - rebranded as Meta Quest 2 in November 2021[4]). As a result, the operator feels like being inside the car and will experience driving with the use of joysticks. The provided solution has been successful in providing a teleoperation opportunity, while it is open for improvement in some areas. The result shows with around 40 milliseconds of delay, the operator connects to the remote car successfully and is able to operate it. The camera images with a frame rate of 30 fps and a resolution of 320X240 are rendered in the Meta Quest successfully and the operator is able to send the commands to the remote car via joysticks in real-time. These prove that the solution is practical, but there are some areas open for further study and improvement such as camera streaming, network structure, and telepresence. The provided solution and topics discussed in this thesis yield a basic understanding of teleoperation. Getting to know the concepts and challenges in teleoperation help to step into some areas, where teleoperation is inevitable such as autonomous driving and telerobotics. The thesis will also motivate students and give them a direction in some areas such as teleoperation and VR.