Development of a testing framework for monocular depth estimation models on targeted hardware

Abstract

This thesis presents the development of a testing framework designed to evaluate monocular depth estimation (MDE) models on targeted embedded hardware. The work is motivated by the KuupKulgur lunar microrover project, where traditional depth sensing methods are impractical due to size, weight, and power limitations. MDE models have demonstrated strong accuracy. However, their real-time performance on targeted hardware requires validation. A modular pipeline was implemented to benchmark various MDE models on the NVIDIA Jetson Orin Nano platform. The framework supports multiple inference backends and weight precisions, enabling comparison between PyTorch and Open Neural Network Exchange (ONNX) Runtime using Floating Point (FP)32 and FP16 precision weights. The framework gathers time and hardware usage data during benchmarking. The developed testing framework provides a valuable tool for evaluating MDE models in robotic systems, including future lunar rovers. Four MDE models were tested: Depth Anything V2, Metric3Dv2, Depth Pro and SPIdepth. Once optimisation with NVIDIA TensorRT (TRT) was applied, all models showed improved inference times over PyTorch when using FP16 precision weights. When using FP32 values, PyTorch outperformed TRT, with a few exceptions. The best-performing model was Depth Anything V2’s smallest relative model when using TRT optimisation level 2 and FP16 precision weights.