Solving the “Databases” SQL Assignments with Language Models in Comparison with Students’ Results

Abstract

In recent years, the use of AI-based language models such as ChatGPT and Microsoft Copilot has become common in education. However, it remains unclear how well these tools can independently solve more complex tasks. The aim of this bachelor’s thesis was to evaluate the ability of these language models to solve homework assignments from the University of Tartu’s “Databases” course and compare the results with those of students from the 2025 spring semester. The assignments were completed using the language models in two ways: first without any prior information about the database structure, and then with the relevant structural information provided. The results were evaluated using automatic grading, and the types and accuracy of errors were analyzed. The models performed better when they had access to the database structure, but overall still fell short of student performance, especially in complex and ambiguous tasks.

Viimastel aastatel on tehisintellektil põhinevate keelemudelite, nagu ChatGPT ja Microsoft Copilot, kasutamine hariduses muutunud tavapäraseks. Samas pole selge, kui hästi suudavad need tööriistad keerukamaid ülesandeid iseseisvalt lahendada. Bakalaureusetöö eesmärk oli hinnata keelemudelite võimekust lahendada Tartu Ülikooli kursuse „Andmebaasid“ kodutöid ning võrrelda tulemusi 2025. aasta kevadsemestri üliõpilaste omadega. Kodutööd lahendati keelemudelitega kahel viisil: esmalt andmebaasi struktuuri eelinfota ja seejärel koos vastava teabega. Tulemusi hinnati automaatkontrolliga ning analüüsiti veatüüpe ja täpsust. Mudelid saavutasid paremaid tulemusi, kui neil oli juurdepääs andmebaasi struktuurile, kuid jäid üldiselt üliõpilastele alla, eriti keerukate ja mitmeti mõistetavate ülesannete puhul.