Paper: Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring Authors: Indraneil Paul, Glavaš Glavas, Iryna Gurevych arXiv: 2605.00754 | 2026-04-30
The Problem: AI Judges That Only Look at Outcomes
Imagine a programming contest judge who only checks whether a program runs and produces the right output — ignoring readability, naming conventions, algorithmic efficiency, security risks, and best practices.
This reflects the current limitation of code reward models: most focus almost exclusively on execution feedback — whether the program runs.
The Single-Dimension Dilemma of Code Reward Models
Reward models (RMs) are central to post-training of language models. They determine what a model generates, what counts as a "good" output, and how to pick among candidates. But code-domain RMs suffer from serious limitations:
1. Functional correctness only — runnable code with correct output is deemed "good," ignoring other quality dimensions. 2. Self-contained executable code only — they cannot evaluate snippets, pseudocode, or code requiring specific environments, limiting post-training applications. 3. Single-language focus — mostly Python, ignoring real-world multilingual programming needs. 4. Single criterion — one score: right or wrong, with no flexible multi-criteria assessment.
Themis: A Multilingual, Multi-Criteria Code Reward Model
Themis introduces four core innovations:
1. Multilingual support — beyond Python, it covers Java, C++, JavaScript, and more, reflecting real-world programming diversity. 2. Multi-criteria evaluation — functional correctness, readability, efficiency, security, style conventions, and maintainability. 3. Flexible scoring — evaluation criteria can be selected per application scenario, with customizable weight combinations. One model, many uses. 4. Beyond execution feedback — even non-executable code can be assessed, including snippets, comments, and documentation.
This is an upgrade from a "results-only judge" to a "full-spectrum code review expert" — evaluating not just whether code is correct, but whether it is good, fast, and secure.
Themis-CodeRewardBench: A New Benchmark
The paper also introduces Themis-CodeRewardBench, a benchmark built specifically for evaluating code reward models. It covers multilingual, multi-criteria scenarios and provides fine-grained evaluation metrics — a comprehensive, fair, and rigorous test for the field.
Why It Matters: Good Code Is More Than Correct
As the forum post argues in a Feynman-inspired framing:
> "A program that only gives correct answers but that no one can understand has not fulfilled its mission. Science is shared, and code should be too."
In software engineering terms: correct code is merely the passing grade. Excellent code balances correctness, readability, efficiency, and security. A reward model that focuses only on "is it right" incentivizes models to produce "correct but bad" code.
Themis teaches AI something like taste — judging not just whether code "can be eaten," but whether it "tastes good."
Takeaways
If you train or use code generation models, ask:
1. Does my reward model focus only on functional correctness? 2. Are other quality dimensions (readability, security, efficiency) being ignored? 3. Is multilingual support important for my use case? 4. Do I need flexible, configurable evaluation criteria?
The bottom line: the ultimate goal of AI-assisted programming is not "generating code that runs" but "generating excellent code." Reward models define what AI pursues — if we only reward "correct," AI will only chase "correct"; if we reward "good," AI will chase "good."
In the AI era, taste is no longer exclusively human. AI can — and should — learn to appreciate code quality.