VERA: An Open-Source Dashboard for Assessing Responsible LLM Deployments in Regulated Settings

Forum topic · 小凯 · 2026-10-25

Summary

VERA is an open-source, self-hosted dashboard designed to help institutions deploying large language models in regulated environments produce technical evidence of responsible AI practices. It builds on COMPL-AI, a technical interpretation of the EU AI Act that organizes evidence into six principle groups and eighteen technical requirements, twelve of which are measurable by existing benchmarks. Deployed at a large European bank, VERA lets a reviewer select a model, run public benchmarks, and obtain scores for the twelve measurable requirements, while also instrumenting energy consumption to assess environmental impact. The tool reduced average assessment time from 56 seconds (using raw files) to 30 seconds, and an internal survey gave it a usefulness score of 4.36/5. VERA and its accompanying learning materials are publicly released to support reproducibility across different organizational settings. Authored by Robin Quériaux, Solveig Bachellery, Léa Déléris, and colleagues, the paper was published on arXiv (2610.31588) in October 2026.

Overview

Field: Machine Learning
Authors: Robin Quériaux, Solveig Bachellery, Léa Déléris, Saurabhsingh Rajput, Mariam Barry, Tushar Sharma, Jérôme Lebecq
Published: 2026-10-22
arXiv: 2610.31588

Abstract

Institutions deploying LLMs in regulated settings need technical evidence of responsible AI practices before approving deployment. COMPL-AI provides a technical interpretation of the EU AI Act organizing evidence under six principle groups and eighteen technical requirements, twelve measurable by current benchmarks. We present VERA, an open-source self-hosted dashboard deployed at a large European bank. A reviewer selects a model and runs public benchmarks; VERA scores the twelve requirements and instruments energy consumption for environmental impact. The dashboard reduced assessment time from 56 seconds using raw files to 30 seconds on average, and scored 4.36/5 on usefulness.

Key Points

Built on the COMPL-AI framework, a technical interpretation of the EU AI Act with six principle groups and eighteen technical requirements.
Twelve of the eighteen requirements are measurable with current public benchmarks; VERA scores these automatically.
Open-source and self-hosted, deployed in production at a large European bank.
Tracks energy consumption per evaluation to report environmental impact.
Reduced average assessment time from 56s to 30s; internal usefulness rating of 4.36/5.
Dashboard and learning materials released publicly for reproducibility in other organizations.