A Counterintuitive Finding
When you give a large language model a multiple-choice question, its usual approach is to generate a passage of text that mentions "the answer is B." Doing this well typically requires fine-tuning.
But if you change how you ask—instead of requiring text generation, directly read the model's next-token probabilities for options A, B, C, and D, then pick the one with the highest probability—you find that LLMs could make decisions all along; you've just been asking them the wrong way.
This is the core finding of LLM2Jev: an LLM is already a Jev-style decision model, no fine-tuning required.
What Is a Jev-Style Decision Model
First, let's define "Jev-style decision model." These models:
1. Take a set of predefined options (e.g., A/B/C/D) 2. Output a classification probability distribution (e.g., A: 0.1, B: 0.7, C: 0.15, D: 0.05) 3. Don't generate free text—no explanations, no reasoning traces, just probabilities
This paradigm is extremely common in industrial decision scenarios: credit assessment outputs an "approve/reject" probability, medical diagnosis outputs a "positive/negative" probability, recommender systems output a "click/no-click" probability.
The traditional approach: take an LLM, fine-tune it on decision data so it learns to output probability distributions. That requires labeled data, training cost, and deployment maintenance.
LLM2Jev says: skip the fine-tuning. Just use the raw LLM's next-token probabilities.
How It Works
The method is remarkably simple. Using a multiple-choice question as an example:
Question: Which of the following is the capital of France? A. London B. Paris C. Berlin D. Madrid
Traditional approach: Have the LLM generate the text "the answer is B. Paris," then parse B out of the text.
LLM2Jev approach: 1. Construct a prompt so that each option A/B/C/D corresponds to a specific next token 2. Read the LLM's probability distribution over those four tokens 3. Use this distribution directly as the decision result
No generation, no parsing, no fine-tuning. The LLM's next-token probabilities themselves already contain decision information good enough.
Key Numbers: Training-Free, Yet Competitive
LLM2Jev's performance on JevBench (a decision benchmark):
- Qwen3.5-4B (untrained): 81.4% accuracy (listwise readout)
- Community fine-tuned model Plumb (same base): 89.6%
- Community fine-tuned model SemIf (frozen version): 82.3%
- jev-local (frozen Qwen3.5-9B): only 74.9%
- Tool selection (choose one of N tools) → LLM2Jev works
- Routing decisions (choose one of N paths) → LLM2Jev works
- Classification (choose one of N categories) → LLM2Jev works
- Risk assessment (choose one of N risk levels) → LLM2Jev works
Key takeaways:
1. The untrained Qwen3.5-4B (81.4%) beats the frozen Qwen3.5-9B (74.9%)—a smaller model with a better method wins. Decision ability doesn't come from model scale; it comes from how you extract it.
2. The gap to fine-tuned models is only 8 percentage points—given that fine-tuning requires labeled data and training cost while LLM2Jev is zero-cost, the cost-effectiveness is outstanding.
3. Listwise readout beats pointwise readout—scoring options one at a time is inferior to ranking all options together. This matches human decision intuition: comparison is easier than absolute judgment.
Why It Works
LLM2Jev works because next-token probabilities are already a calibrated probability distribution.
When you ask an LLM "which option is most likely the capital of France?", the model doesn't need to generate the word "Paris"—it just computes the probability of the token "Paris" internally. That probability is the LLM's confidence that "Paris is the correct answer."
This exposes something: the LLM's decision ability was always there; it was just masked by the "generation" task.
Traditional usage requires the LLM to first generate text, then extract the answer from that text. This introduces two noise sources: 1. Randomness in generation (temperature, sampling) 2. Ambiguity in parsing (how do you distinguish "I think the answer is B" from "the answer might be B"?)
LLM2Jev bypasses both noise layers and reads the probability distribution directly. Don't ask the LLM to tell you the answer—ask the LLM's internal probabilities to tell you.
Multimodal Too
LLM2Jev isn't limited to text. The paper also tests multimodal scenarios—give the model an image plus several description options, and have it pick the best-matching description.
The method is identical: construct a prompt mapping each option to a token, read the probability distribution. No fine-tuning of the multimodal model, no extra training data.
This means any model that can output next-token probabilities—text, image, audio—can become a Jev-style decision model directly. Decision ability is a byproduct of next-token prediction, not a capability that needs separate training.
The Deeper Implication: Generation and Decision Are Two Different Things
LLM2Jev exposes an overlooked fact: generation and decision-making are two different tasks, but LLMs have been designed to excel only at the former.
Generation requires the model to output coherent, logical text—language ability, reasoning, style control. Decision requires choosing among given options—only probability comparison is needed.
When we do "decisions" via "generation," we're doing something unnecessary: making the model produce an explanation first, then extracting the answer from it. Slow and inaccurate.
LLM2Jev's approach: acknowledge that decisions don't need generation, and read the decision probabilities directly.
It's like ordering at a restaurant. You don't need someone to write a 500-word food review to decide what to eat—just ask "steak or fish?" and see what they pick. The decision itself is simple; generating explanations is the complex part.
Implications for Agent Systems
A widespread assumption in current agent systems: LLMs must generate text to make decisions. "Think, then choose" is the standard flow.
But LLM2Jev points out: for structured decisions (multiple-choice among given options), generation is redundant. Reading probabilities directly is faster, more accurate, and cheaper.
Direct implications for agent design:
A Philosophical Question
LLM2Jev raises an interesting question: what does an LLM actually "know" how to do?
Traditional view: LLMs generate text; decision-making is a special case of generation. LLM2Jev view: LLMs compute probabilities; generation and decision are two different ways of reading the same probability computation.
If the latter is true, our evaluation methodology for LLMs may need rethinking. Leaderboards measure generation accuracy, but an LLM's decision ability may far exceed its generation ability—it just hasn't been measured.
Perhaps LLMs are not "generators that can answer multiple-choice questions," but "decision engines that can also generate."
---