Samuel, DavidMikhailov, VladislavVelldal, ErikØvrelid, LiljaCharpentier, Lucas Georges GabrielKutuzov, AndreyOepen, StephanJohansson, RichardStymne, Sara2025-02-182025-02-182025-03https://hdl.handle.net/10062/107253Training large language models requires vast amounts of data, posing a challenge for less widely spoken languages like Norwegian and even more so for truly low-resource languages like Northern Sámi. To address this issue, we present a novel three-stage continual training approach that substantially improves the downstream performance together with the inference efficiency for the target languages. Based on our findings, we train, evaluate, and openly release a new generative language model for Norwegian Bokmål, Nynorsk, and Northern Sámi with 11.4 billion parameters: NorMistral-11B.enAttribution-NonCommercial-NoDerivatives 4.0 Internationalhttps://creativecommons.org/licenses/by-nc-nd/4.0/Article