From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch

Overview

• Field: NLP
• Authors: Laurens Samson, Iva Gornishka, Gossa Lô
• Published: 2026-08-11
• arXiv: 2508.03803

Summary

Large language models are increasingly being deployed in governmental settings, yet few existing evaluation frameworks jointly reflect the values of public administration and the linguistic requirements of non-English contexts. The authors present the 'Grip on LLMs' framework, a systematic evaluation suite for Dutch governmental use developed in collaboration with domain experts from a major Dutch municipal organisation.

Through an advisory board process, user research, and a survey of the users of a civil-servant chatbot, they identify six evaluation dimensions (factuality, honesty, social bias, energy consumption, cost, and training data transparency) and operationalise them into a benchmark suite covering more than 30 multilingual and Dutch-specific models.

Key findings

• No single model excels on all dimensions; trade-offs are inevitable.
• Higher quality consistently comes with greater environmental impact and financial cost, while social bias is largely independent of both.
• Factuality (whether a model answers correctly) and honesty (whether it admits it does not know) are driven by different model attributes; high factuality does not imply high honesty.
• To make these findings actionable for non-technical audiences, the authors release a publicly accessible, user-friendly model overview designed for the full range of stakeholders involved in government LLM selection, from engineers to policymakers.

*Auto-collected on 2026-08-12*