Collecting and Using a Labeled Dataset of NATO Mission Task Symbols to Improve and Benchmark Detection Models

Abstract

Neural networks are commonly used for object detection tasks but require immense amounts of data to train. For the task of North Atlantic Treaty Organization (NATO) mission task symbol detection using object detection neural networks, it is not possible to meet the data requirements. Additionally, labeling mission task symbols is very time-consuming and costly. This thesis aims to collect and label a dataset of NATO mission task symbols, propose a part of it as a benchmark for our solutions and future solutions, and finally propose different methods to use a part of the scarce collected data to improve the performance of our object detection models. YOLOv5 neural network is selected and used to experiment with different ways of using the scarce collected data. As a result, 113 images were collected and labeled. Five performance metrics are proposed for the benchmark. Finally, it was discovered that when dataset size is limited, extracting information from the dataset and using it to generate artificial data improves performance compared to directly introducing the scarce dataset to symbol detection models.