Anomaaliate tuvastamiseks sügav õpe kasutavate äriliste intsidentide klassifitseerimine ja prognoosimine
Date
2019
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Tarkvarasüsteemid katavad tänapäeva äriettevõtete jaoks elutähtsaid funktsioone ja on tihti ka äritegevuse läbiviimiseks primaarse tähtsusega. Taolised süsteemid võivad koosneda väga suurest hulgast komponentides, mis on arendatud erinevate meeskondade või ettevõtete poolt ning enamasti ka kasutades erinevaid tehnoloogiaid. Erinevate osade koostoime tagamine taolistest süsteemides on keerukas. Tüüpilistest tarkvaralahendustes võivad probleemid tekkida nii füüsiliste seadmete riketest, võrguühenduste katkestustest, seadistuste tegemisel, erinevate tarkvarakomponentide uuendamisel tekkivatest suhtlusprotokollide ebakõladest, samuti ka välistest faktoritest nagu kasutajate kasutusmustrite muutused või kasvõi kavatsetud rünnakud tarkvara süsteemide vastu. Seetõttu on äritegevuse tagamiseks vajalik tagada ka tarkvarasüsteemide pidev toimimine ning veaolukordade puhul reageerida ning viga kõrvaldada võimalikult kiiresti. Esimene samm vigade kõrvaldamisel on veaolukorra tuvastamine. Kiire reageerimise saavutamiseks peab tuvastamine toimuma samuti kiiresti, oluline on oskus eristada veaolukordadele viitavaid anomaaliaid normaalsest olukorrast. Playtech kasutab taoliseks automaatseks tuvastamiseks ja häirete tõstatamiseks tüüpilist anomaaliate tuvastamise lähenemist: reeglitel põhinevat tuvastamist. Kuid lisaks anomaalia tuvastamisele on oluline ka hinnata anomaalia tähendust ning võimalikku ärikriitilisust. Kui tuvastatud anomaalia registreeritakse intsidendina, on vajalik kategoriseerida see hinnates potentsiaalset intsidendi tõsidust, et tagada efektiivne reageerimine, vea tuvastamine ja kõrvaldamine. Antud töö eesmärk ongi masinõppel põhineva mudeli treenimine ja väljatöötamine, mis tuvastaks ja kategoriseeriks taolisi intsidente. Töö kirjeldab detailselt kuidas kasutatakse anomaaliate tuvastamise ja masinõppe tehnikaid parandamaks olemasolevat lahendust ning arendada seda edasi klassifitseerimaks intsidente. Välja pakutud lahendus ennustab tõenäosuslikult, millisesse kategooriasse võiks tuvastatud intsident kuuluda, kasutades selleks intsidendi tuvastamisel talletatud mõõdikute ajaseeriate informatsiooni.
Companies today, use a number of different software products and systems to carry out various business activities. These systems consist of a large number of components usually developed by different teams or companies using various technologies. With such complex systems, there can be issues both with the network or the application. There can be problems with changes in configurations that can cause unexpected behaviour in business flows or various version upgrades can introduce compatibility problems. These can lead to business and financial losses. Therefore, it is necessary for businesses to take proactive steps to manage such business incidents before they adversely affect other related components.This brings us to the need for an analytics platform which can identify and differentiate normal behaviour of a metric from anomalous behaviour (degradation or surge in metrics). The business anomaly detection and alert system in Playtech do the same thing using the most common anomaly detection technique i.e. rules matching engine. This system although doesn’t give the capability to automatically categorise incidents based on severity. Thus, it is required to train a machine learning model to detect such incidents and categorise them according to a severity which is the goal of this work. This work describes in detail the use of anomaly detection and machine learning approaches to improve the state-of-the-art and design and develop such a system for anomaly detection and classification of business incidents. The proposed solution is able to predict probabilities for categories of business incidents based on the available metrics information.
Companies today, use a number of different software products and systems to carry out various business activities. These systems consist of a large number of components usually developed by different teams or companies using various technologies. With such complex systems, there can be issues both with the network or the application. There can be problems with changes in configurations that can cause unexpected behaviour in business flows or various version upgrades can introduce compatibility problems. These can lead to business and financial losses. Therefore, it is necessary for businesses to take proactive steps to manage such business incidents before they adversely affect other related components.This brings us to the need for an analytics platform which can identify and differentiate normal behaviour of a metric from anomalous behaviour (degradation or surge in metrics). The business anomaly detection and alert system in Playtech do the same thing using the most common anomaly detection technique i.e. rules matching engine. This system although doesn’t give the capability to automatically categorise incidents based on severity. Thus, it is required to train a machine learning model to detect such incidents and categorise them according to a severity which is the goal of this work. This work describes in detail the use of anomaly detection and machine learning approaches to improve the state-of-the-art and design and develop such a system for anomaly detection and classification of business incidents. The proposed solution is able to predict probabilities for categories of business incidents based on the available metrics information.