English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Training a 4B Model to Beat Postgres Query Plans for $1,200

Forum topic · QianXun · 2026-09-18

Summary

A personal project by engineer Rohan Bansal fine-tuned a 4B open-source model (Qwen3 distill, actually 4.66B parameters) to generate pg_hint_plan hints that outperform Postgres's native query plans on the JOB benchmark (113 IMDb queries). Using 420 teacher traces distilled from a larger model, followed by GRPO reinforcement learning with EXPLAIN ANALYZE as a free reward signal, the model achieved a 1.81x geometric-mean speedup under a 15-try pick-the-best protocol — dropping to 1.40x in single-shot use. Total cost: ~95 hours on two rented H100s (~$800) plus ~$400 in teacher-trace API fees. Notable caveats include a possibly index-sparse baseline, unmeasured inference latency, and the author's own "fooling rate" metric showing 20-30% of no-op plans register phantom speedups due to measurement noise. The key insight: domains with free, automatic reward signals (like query execution time) allow small models with tiny budgets to deliver real gains — with zero model inference on the production path.

It's 2 AM. A query takes four minutes. You open EXPLAIN and the join order is clearly wrong — stale statistics, or a cost model that just doesn't fit this data. You write one hint, and four minutes becomes four seconds. The database hasn't changed, the data hasn't changed; only the route has.

That door hides a question the database world has argued about for over thirty years: who should decide the query plan?

On September 16, a Hacker News post gathered 381 upvotes with a blunt title: training a 4B model to produce query plans 81% faster than Postgres. The author, Rohan Bansal — a former Google, Traba, and Crosby engineer — built it as a personal project during a Recurse Center sabbatical. No company, no funding. The code is open source on GitHub, in a repository called qorl.

In one sentence: he took a 4-billion-parameter open-source model and taught it to write execution hints for Postgres.

The pipeline

The base model is the open-source Qwen3-4B distill — nominally 4B, actually 4.66B parameters. The training material is 420 traces from a much larger teacher model: the big model prototypes, the small model learns. First supervised distillation, then reinforcement learning with a refreshingly plain reward: hand the hint to pg_hint_plan, actually run EXPLAIN ANALYZE in Postgres, and fast is fast, slow is slow. The SQL is never modified — correctness is guaranteed by the database itself throughout.

The exam matters too. Training used CEB (13,600 IMDb queries); the exam used the JOB benchmark, 113 queries from Leis et al.'s famous 2015 paper — all on the same 8.5GB IMDb database. The author states outright in his blog: this is about *learning this one database*.

The results

  • Under a 15-tries-pick-the-best protocol: 1.81x geometric-mean speedup — the 81% in the headline
  • Total workload latency dropped 44.7%
  • Of 113 queries: 68 improved by more than 5%, zero regressions
The bill is also on the table: two rented H100s on Lambda ran 95 hours (~$800), plus ~$400 in API fees for generating teacher traces — just over $1,200 total. The infrastructure is a hybrid: training in the cloud, measurement on the author's home machine (nicknamed "FLOPper") running 4 Postgres containers, connected back to cloud rollout workers via Tailscale at 20-way concurrency. Measurement discipline was strict: warm up each configuration, then take the median of 20 runs.

The fine print

The numbers look great; the small print is more interesting. Three lines of it:

Line one: the 1.81x assumes you run the plan 15 times and pick the best one, with the picking itself based on a pre-evaluation. No production system gets 15 tries. Single-shot use drops the speedup to 1.40x. The ablation table opens the books: distillation alone yields only 1.16x; full RL at 1,200 steps reaches 1.41x. The model's improvement is real — but the headline gain is padded by the selection protocol.

Line two: model inference time is excluded from comparisons, and training costs aren't counted either. "Plans execute 81% faster" and "the planner is 81% faster" are two different ledgers.

Line three: the baseline itself is suspect. HN commenters dug out that the test tables appear to have only primary-key indexes — Postgres is fighting with one hand tied. The 8.5GB dataset sits entirely in warm cache, purely read-only, single database. And there's no head-to-head against lightweight specialized tuners like Bao; the author admits that's next on the list.

Line three also hides an easter egg: the author invented a metric called the fooling rate to quantify measurement noise. A no-op plan that changes nothing was "measured" as 10–26% faster some 20–30% of the time. Only raising the cache from 128MB to 2GB cut this misjudgment rate by four times. Anyone doing performance comparisons should feel a chill reading that — have you checked your comparisons for phantoms?

Is it still worth watching? Yes — precisely because of this.

The real point is the paradigm. The model appears exactly once, during development. Its entire output is a few lines of SQL comments that go into git with the code and ship with the release. Nothing model-shaped lives on the database's critical path; there's no new failure surface. When workload drift invalidates the hints — a pitfall the author honestly documents — you retrain.

Deeper still: the $1,200. Reinforcement learning is expensive because rewards are hard to define. Here the reward is free: run EXPLAIN ANALYZE and the speed speaks for itself. In domains with a free referee, a small model plus a small budget can deliver. That lesson transfers broadly — compiler IRs, cache eviction policies, scheduler parameters: any corner where you can score automatically and cheaply could be the next $1,200 experiment.

Postgres's planner descends from System R, a 1979 skeleton still on active duty, guessing routes via statistics and cost formulas. When it guesses wrong, the fix has always been a hand-written hint. Now there's another option: let a 4B model write those comment lines for you, at twelve hundred dollars a course.

How many corners of your system are hiding a free referee that nobody has thought to ask?

---

Sources: author's blog rohanbansal.com/qorl (2026-09-16); HN discussion 49731285; code at github.com/polyphilz/qorl; JOB benchmark from Leis et al. 2015.

Tags

#postgres#query-optimization#reinforcement-learning#llm#fine-tuning#database-performance#job-benchmark#open-source

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178634933