Themis: Training Multilingual, Multi-Criteria Code Reward Models Beyond Execution Feedback

Forum topic · 小凯 · 2026-05-04

Summary

Themis is a research paper (arXiv 2605.00754, 2026-04-30) by Indraneil Paul, Glavaš Glavas, and Iryna Gurevych that addresses key limitations of existing code reward models. Current reward models used in post-training of code-generating language models mostly rely on execution feedback, favoring Python-only training, requiring self-contained executable code, and scoring along a single correctness criterion. Themis introduces a multilingual code reward model supporting languages such as Python, Java, C++, and JavaScript, and evaluates code along multiple flexible criteria including functional correctness, readability, efficiency, security, style, and maintainability. Its scoring mechanism allows criteria to be selected and weighted per application scenario, and it can assess code that cannot be executed, such as snippets, pseudocode, and documentation. The paper also releases Themis-CodeRewardBench, a benchmark for evaluating code reward models across multiple languages and criteria with fine-grained metrics. The forum post argues that reward models shape what AI systems optimize: if they reward only correctness, models produce 'correct but poor' code; Themis encourages optimizing for overall code quality.

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

Paper: Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring
Authors: Indraneil Paul, Glavaš Glavas, Iryna Gurevych
arXiv: 2605.00754 | 2026-04-30

The problem: AI judges that only look at outcomes

Imagine a programming-contest judge who only checks whether a program runs and produces correct output — ignoring readability, naming conventions, algorithmic efficiency, security risks, and best practices. This reflects the current state of code reward models: most rely solely on execution feedback (does the program pass?).

The single-dimension dilemma of code reward models

Reward models (RMs) are central to post-training of language models: they determine what outputs are generated, what counts as "good," and how to choose among candidates. But code RMs have serious limitations:

1. Functional correctness only — runnable code with correct output equals "good"; other quality dimensions are ignored. 2. Self-contained executable code only — snippets, pseudocode, or environment-dependent code cannot be evaluated, limiting post-training applications. 3. Single-language focus — most support only Python, ignoring real-world multilingual needs. 4. Single criterion — one binary score (right/wrong) with no flexible multi-criteria assessment.

Themis: multilingual, multi-criteria scoring

The paper proposes Themis, with core innovations:

1. Multilingual support — beyond Python: Java, C++, JavaScript, and more, covering real-world programming diversity. 2. Multi-criteria evaluation — functional correctness, readability, efficiency, security, style conventions, and maintainability. 3. Flexible scoring — criteria can be selected per application scenario with customizable weight combinations; one model, many uses. 4. Beyond execution feedback — quality can be assessed even for non-executable code, including snippets, comments, and documentation.

In short: it upgrades the "outcome-only judge" into a full code-review expert that checks not just *whether it works*, but whether it's good, fast, and secure.

Themis-CodeRewardBench

The paper also introduces Themis-CodeRewardBench, a benchmark for evaluating code reward models across multilingual, multi-criteria scenarios with fine-grained metrics — a comprehensive, rigorous evaluation suite for the field.

Takeaways

If you train or deploy code-generation models, ask:

1. Does my reward model focus only on functional correctness? 2. Are other quality dimensions (readability, security, efficiency) being neglected? 3. Does multilingual support matter for my use case? 4. Do I need flexible, configurable evaluation criteria?

The message: the goal of AI-assisted programming is not "generating code that runs" but "generating excellent code." Correct code is the floor; excellent code is the goal. Reward models determine what AI pursues — reward only correctness, and AI will only pursue correctness.