Anomaly Detection and Imputation for Tartu Traffic Sensors

Abstract

The city of Tartu has 16 highway traffic sensors with many gaps of missing data. We analyzed the state of the sensors’ data and evaluated different anomaly detection and imputation solutions to better its quality. The best anomaly detection approach was deemed to be daily clustering with local outlier factor (LOF) used as the clustering algorithm. For imputation we utilized linear interpolation with a combination of seasonal decomposition and seasonal splitting. The chosen solutions were integrated into a service that processes CSV files of traffic data and uploads the results to Cumulocity, an IoT data aggregation platform. We processed and uploaded the historical data of 2019-04-29 to 2023-06-01 of every highway sensor. Finally, we also tested our solution on light traffic data.

Tartu linnal on 16 maantee liiklussensorit, mille andmetes esineb mitmeid auke. Me andsime ülevaate andmete olukorrast ning hindasime mitmeid anomaaliatuvastus- ning andmeparandus-lahendusi. Anomaaliaid leidsime kõige paremini päevase klasterdamise abil kasutades LOF algoritmi. Imputeerimislahenduseks valisime lineaarse interpoleerimise kombineerides ajaandmetes leitud hooajalisi mustreid. Me integreerisime valitud meetodid teenusesse, mis töötleb CSV andmeid ning laeb tulemid üles Cumulocitysse, IoT andmete agregeerimisplatvormile. Me töötlesime ning laadisime teenuse abil üles sensorite ajaloolised andmed vahemikus 2019-04-29 kuni 2023-06-01. Lõpetuseks katsetasime oma lahendust ka kergliiklusandmetel.