Gaussi protsesside usaldusvahemik
Files
Date
2018
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Masinõpe on arvutiteaduse valdkond, mis tegeleb arvutisüsteemide oskusega iseseisvalt õppida. Masinõppemeetodeid kasutatakse nii andmete kirjeldamiseks kui ka tunnustele väärtuste ennustamiseks. Kui masinõppemudelit kasutatakse reaalarvulise väärtuse ennustamiseks, siis nimetatakse seda regressiooniks. Praktikas on reaalarvulist väärtust ennustades tihti tarvis arvestada, et vääral ennustusel võivad olla kallid tagajärjed. Väärade ennustuste kahju aitab vähendada see, kui mudel oskab ise hinnata, kui täpne tema ennustus on. Üheks näiteks sellisest hinnangust on tagastada vahemik, kuhu mudel 95% tõenäosusega hindab olevat õige väärtuse. Selline lähenemine on Gaussi protsessidel põhineva regressioonimudeli eriliseks omaduseks ning seda vahemikku nimetatakse usaldusvahemikuks. On oluline, et mudeli hinnang enda täpsuse kohta vastaks tegelikkusele ning et mudel ei hindaks end liiga enesekindlalt. Masinõppemudelite usaldusväärsuse hindamine on oluline, sest selliste mudelitega tarkvara kätte on tänapäeval usaldatud üha vastutusrikkamate otsuste langetamine. Antud bakalaureusetöö keskendub Gaussi protsessidel põhineva regressioonimudeli enesekindluse uurimisele. Antud töös uuritakse, kui tihti satuvad ennustatavate väärtuste tegelikud väärtused vahemikku, kuhu mudel hindab nende sattumise 95% tõenäosusega. Mõõtmised 6651 mudelil näitavad, et suurem osa päris märgendeid satuvad usaldusvahemikku oluliselt harvem kui 95% juhtudest ehk et Gaussi protsesside mudel on liigselt enesekindel. Keskmiseks usaldusvahemikku kuulunud osakaaluks on 0,93. Töö peamine tulemus on, et 73% mõõtetulemustest on madalamad kui võiks olla eelnevalt nimetatud tõenäosuse järgi. Ühtlasi on märkimisväärne see, et kõige väiksemate ja kõige suuremate väärtustega sisendväärtuste puhul on mudel rohkem liigselt enesekindel. Gaussi protsesside usaldusvahemiku uurimise näol on tegemist millegagi, midaei ole varem uuritud. Tänu käesolevale tööle on olemas hinnang Gaussi protsesside regressioonimudeli usaldusväärsusele ning selle töö tulemus aitab Gaussi protsesside kasutajatel võtta arvesse antud meetodi liigset enesekindlust.
Machine learning is a field in computer science that provides computer systems with the ability to learn independently. Machine learning methods are used for both descriptive and predictive purposes. When a machine learning model is used to predict a real valued number it is called regression. In practice, it is often important in regression to take into account that false predictions might have severe consequences. To avoid such false predictions, it is helpful if the model is able to rate how accurate its prediction is. An example of this is for the model to provide an interval where it predicts the true value with 95% certainty. This approach is unique to Gaussian process regression model and thisinterval is called confidence interval. It is important that the model rates itself accurately and not overly confidently. Evaluating confidence of machine learning models is important since software solutions equipped with machine learning algorithms are becoming more common and are being trusted with decisions that require more responsibility. This Bachelor’s thesis focuses on the confidence of Gaussian process regression models. This research examines how often are true values contained in the intervals where model predicts them with 95% probability. Measurement results on 6651 models show that the majority of true labels are included in the confidence interval in less than 95% of cases, which means that Gaussian process regression model is overconfident. Mean ratio of true labels in confidence intervals per model was 0.93. Main result of the research is that for 73% of the models the confidence intervalcontained less true labels than was expected by the probability. It is noteworthy that for input values that had smallest or largest values the model was more often overconfident.Confidence of Gaussian processes has not been researched before and this research provides evaluation on how reliable are Gaussian processes. The results of this thesis enable users of Gaussian processes models to consider overconfidence of models.
Machine learning is a field in computer science that provides computer systems with the ability to learn independently. Machine learning methods are used for both descriptive and predictive purposes. When a machine learning model is used to predict a real valued number it is called regression. In practice, it is often important in regression to take into account that false predictions might have severe consequences. To avoid such false predictions, it is helpful if the model is able to rate how accurate its prediction is. An example of this is for the model to provide an interval where it predicts the true value with 95% certainty. This approach is unique to Gaussian process regression model and thisinterval is called confidence interval. It is important that the model rates itself accurately and not overly confidently. Evaluating confidence of machine learning models is important since software solutions equipped with machine learning algorithms are becoming more common and are being trusted with decisions that require more responsibility. This Bachelor’s thesis focuses on the confidence of Gaussian process regression models. This research examines how often are true values contained in the intervals where model predicts them with 95% probability. Measurement results on 6651 models show that the majority of true labels are included in the confidence interval in less than 95% of cases, which means that Gaussian process regression model is overconfident. Mean ratio of true labels in confidence intervals per model was 0.93. Main result of the research is that for 73% of the models the confidence intervalcontained less true labels than was expected by the probability. It is noteworthy that for input values that had smallest or largest values the model was more often overconfident.Confidence of Gaussian processes has not been researched before and this research provides evaluation on how reliable are Gaussian processes. The results of this thesis enable users of Gaussian processes models to consider overconfidence of models.