Stop Using LLMs for Small Tasks: Decision Models Cut RAG Conflict Typing Costs by Up to 39x

arXiv: 2610.36915 — Do We Need an LLM to Know Why Sources Conflict? Decision Models for Knowledge-Conflict Typing in RAG

---

An Awkward Reality

Your RAG system retrieves three pieces of information:

• Source A: "This movie was released in 2024"
• Source B: "This movie was released in 2025"
• Source C: "This movie was never released; the others are rumors"

Which source should your LLM trust? How should it respond?

The answer depends on the type of conflict. Cattan et al. (2025) found that telling an LLM the conflict type — multiple valid answers, outdated information, or a spurious source — improves answer quality by about 24 percentage points.

But there's a catch: predicting the conflict type is itself hard, even for frontier LLMs.

So Parv Khatri and colleagues (independent researchers) asked a simple but previously untested question:

Do we actually need an LLM for conflict typing? Could a cheaper "decision model" do the job?

What Is a "Decision Model"?

Two concepts need distinguishing:

• Generative LLM (e.g., Gemini, GPT): text in, text out. Autoregressive generation, one token at a time.
• Decision Model: text in, a probability distribution over a fixed set of options. Essentially a classifier — one forward pass, no text generation.

Decision models aren't new. A BERT-base sentiment classifier is one. Yet prior work on RAG conflict typing defaulted to generative LLMs without systematically testing whether decision models could suffice.

This paper ran that comparison. The result is striking.

11 Decision Models vs. Gemini: A Clean Sweep

The researchers tested 11 decision models from 7 providers, sourced via a commercial inference gateway, on the CONFLICTS benchmark using the same prompts as Cattan et al.

| Model type | Accuracy | Cost | Latency | |-----------|----------|------|---------| | 11 decision models | 50.0% – 67.0% | baseline | baseline | | Gemini 2.5 Flash (used in the original paper) | 61.3% | 4–39x more expensive | 2.6–6.0x slower | | Gemini 3.8 Flash (strongest current) | 70.0% | even more expensive | slower |

Key finding: 10 of the 11 decision models matched or exceeded Gemini 2.5 Flash's accuracy — at 1/4 to 1/39 of the cost (two were free) and 1/2.6 to 1/6 of the latency.

Cascades: Let the Cheap Model Do Most of the Work

Gemini 3.8 Flash is indeed more accurate (70.0%), significantly beating 8 of the decision models. Should you just use it?

The researchers designed a cascade strategy:

1. Classify with the decision model, which also outputs a confidence score 2. Route the lowest-confidence 25% of queries to Gemini 3.8 Flash 3. Use the decision model's result directly for the other 75%

Result: this cascade achieved accuracy statistically indistinguishable from Gemini 3.8 Flash on 8 of 11 decision models — at only 25–37% of its cost.

This is classic cost-quality engineering: cheap models handle most traffic; the expensive model only catches what the cheap one is unsure about. The pattern has been used for decades in speech recognition and spam filtering, but had not been systematically validated for RAG conflict typing.

Four Deeper Findings

1. Decomposing the taxonomy hurts accuracy

CONFLICTS' conflict types form a multi-level taxonomy. The researchers tried splitting it into yes/no questions ("Is this outdated info?" "Is this a spurious source?") — and all 11 decision models got worse. Counterintuitively, decomposition makes each subtask a sparse binary problem with less feature sharing, whereas joint multi-class training lets models learn richer shared features.

2. Repairs fitted on one model don't transfer

Attempting to "repair" errors on decomposed subtasks using another model failed across the board: a repair fitted to one decision model did not help another. Each model has its own error patterns; there is no universal repair.

3. End-to-end validation: cheap types still work

The critical test: feeding LLM-generated answers with conflict types predicted by decision models produced answer quality as good as using LLM-predicted types. Decision models aren't just approximating LLMs — their type labels are equally effective downstream. 10 of 10 decision models passed this end-to-end test.

4. Diagnostic capability

Diagnostic experiments showed decision models can detect date-difference patterns in outdated-information conflicts, have sensitivity to source ordering that differs from LLMs, and detect planted spurious sources at rates comparable to LLMs.

Why This Matters

1. "Using a sledgehammer to crack a nut" is endemic in RAG. Nearly every RAG system uses an LLM for everything. This paper shows classification is a task where LLMs are over-provisioned. Conflict typing, intent detection, query routing, source-quality scoring — all are classification tasks that decision models can replace.

2. Another instance of judge-gate decoupling. Judgment (classifying the conflict type) goes to a cheap decision model; gating (whether to escalate) goes to a confidence threshold; generation goes to the LLM. Each component uses the right tool.

3. Cascades are production-grade standard practice. Used for decades in industry but seemingly forgotten in the LLM era, cascades remind us: not every query needs the most expensive model. Offloading 75% of queries cuts cost ~63% with no quality loss.

4. Independent researchers can do strong work. The two lead authors (Parv Khatri, Harshul Jain) list no institutional affiliation. With off-the-shelf decision models and a public benchmark, they produced a rigorous empirical study — good research doesn't always require burning GPU budgets.

Honest Limitations

• Tested only on the CONFLICTS benchmark; real-world RAG conflicts may be more complex
• All 11 decision models come from one commercial gateway; other providers may differ
• No more complex cascade strategies tested (multi-level, dynamic thresholds)
• The end-to-end LLM judge may carry its own biases

These don't undermine the core conclusion: for RAG conflict typing, decision models are a 39x more cost-effective alternative to LLMs.

One-Line Summary

Don't use a big model for small tasks. Decision models match Gemini 2.5 Flash on RAG conflict typing at 1/39 the cost and 1/6 the latency — and with a cascade, 75% of queries never need an LLM at all.

---

Paper: arXiv:2610.36915

Code: github.com/Parv17k/decision-models-conflict-typing

Related work:

• CONFLICTS Benchmark (Cattan et al., 2025)