Sirvi Autor "Arnardóttir, Þórunn" järgi

Nüüd näidatakse 1 - 2 2

listelement.badge.access-status Avatud juurdepääs ,
Evaluating a Universal Dependencies Conversion Pipeline for Icelandic
(University of Tartu Library, 2023-05) Arnardóttir, Þórunn; Hafsteinsson, Hinrik; Jasonarson, Atli; Ingason, Anton Karl; Steingrímsson, Steinþór
listelement.badge.access-status Avatud juurdepääs ,
WikiQA-IS: Assisted Benchmark Generation and Automated Evaluation of Icelandic Cultural Knowledge in LLMs
(University of Tartu Library, 2025-03) Arnardóttir, Þórunn; Einarsson, Elías Bjartur; Ingvarsson Juto, Garðar; Helgason, Þorvaldur Páll; Einarsson, Hafsteinn; Tudor, Crina Madalina; Debess, Iben Nyholm; Bruton, Micaella; Scalvini, Barbara; Ilinykh, Nikolai; Holdt, Špela Arhar
This paper presents WikiQA-IS, a novel question-answering dataset focusing on Icelandic culture and history, along with an automated pipeline for dataset generation and evaluation. Leveraging GPT-4 to create questions and answers based on Icelandic Wikipedia articles and news sources, we produced a high-quality corpus of 2,000 question-answer pairs. We introduce an automatic evaluation method using GPT-4o as a judge, which shows strong agreement with human evaluations. Our benchmark reveals varying performances across different language models, with closed-source models generally outperforming open-weights alternatives. This work contributes a resource for evaluating language models' knowledge of Icelandic culture and offers a replicable framework for creating similar datasets in other cultural contexts.