Leveraging neural models for data processing and analysis automation

Abstract

Unmanned Ground Vehicles (UGVs) are a staple in some industries and are entering the market in others. Development of these UGVs and their automation is resource intensive and timeconsuming work. Specifically the job of processing and analysing data collected by the various sensors and cameras has so far been done by human workers. In recent years however, it has become possible to propose the automation of these tasks. This thesis describes the development of a pipeline application aimed at reducing the workload of the workers doing these jobs by leveraging neural models such as CLIPSeg, capable of zero-shot text-prompt image segmentation, to extract data from video frames based on specified classes of interest. A proof of concept demo was developed and presented to potential users, leading to the extraction of requirements for a minimum viable product (MVP). The MVP requirements included avoiding image resizing distortion, a command-line interface, and additional post-inference data analysis. The CLIPSeg model was evaluated alongside CLIPSurgery, another zero-shot image segmentation model, using a testing dataset. CLIPSeg demonstrated higher viability for the selected classes and was further evaluated using an 80% model score and 0.05% image area threshold to eliminate false positive results with great success. The final MVP application fulfilled all presented requirements and proved the viability of the CLIPSeg model for the use-case