> Prerequisite navigation: You need 056 (logical qubit check-up), 057 (the competition landscape), 077 (compilation quality), 070 (how to read roadmap charts). > Next episode preview: 079, "The 2^n Wall: Inside Simulators and Tensor Networks."
Grading Quantum Machines: Quantum Volume and Benchmarks
Camera ads always boil down to one line: "200 megapixels!" People who know better then ask about sensor size, lens quality, algorithm tuning—pixels are just the number most easily printed on a poster.
For quantum machines, the "pixels" are the qubit count—the industry's top marketing word. This post covers the real measuring sticks: why any single number is bound to deceive, how the overall-fitness score (Quantum Volume, jargon: QV) is computed, and how to read any score you're handed.
Why quantum machines are especially hard to grade
Classical computers at least have established benchmarking conventions: run benchmark software, compare numbers. Quantum machine performance is a multi-dimensional vector, and every dimension can be hyped separately:
- Qubit count (the poster favorite);
- Two-qubit gate error rate (the core item in 056's check-up table);
- Gate speed (fast, per 061), connectivity (topology determines how many SWAPs 077 must insert), readout fidelity (the lying readings of 096).
- Speed metrics (machine-level throughput: how fast one job round-trips)—quantifies 075's latency bill, but says nothing about quality;
- Logical qubits and logical error rates (056/057)—the true ruler of the fault-tolerant era: one logical qubit with error rate 10⁻⁶ beats any physical-qubit score;
- Application-level benchmarks: run fragments of real problems (chemistry, optimization)—the most honest, hardest to standardize (whoever picks the problem is biased).
- Dissimilarity one: pixel count genuinely affects a floor on image quality (even if everything else is bad); qubit count, when error rates are bad, barely affects any usable performance—a thousand qubits at 99% fidelity can't beat fifty at 99.9% (056's exponential bill). The number is less relevant and more deceptive.
- Dissimilarity two: cameras' "good" has consensus sample photos for comparison (same scene shot by every machine); quantum machines have no universal sample set—"good" depends entirely on the specific application, so benchmark comparability is inherently crippled.
Judged on any single item, you'll be fooled: a thousand low-quality qubits versus fifty high-quality ones—the former may lose comprehensively on every serious metric (056 did the exponential math). The right way to read "world's largest qubit count" headlines: then ask about error rates and connectivity—ads print pixels, not sensors.
QV: an "overall fitness" exam
The industry has built many rulers; the most famous is Quantum Volume (QV)—its idea deserves a close look:
The exam question is a square circuit: a random circuit with equal width and depth (n qubits, n layers, gates random but nasty). How deep a square circuit can this machine run while producing correct outputs distinguishable from the ideal distribution? Call that depth d; then QV = 2^d—the deeper the runnable square, the higher the score.
The beauty is that the "square" shape is deliberately troublesome: width forces scale, depth forces noise endurance, and the random structure tests your compiler's real skill (077's one-click compilation quality feeds directly into the score)—qubit count, error rate, connectivity, and compiler quality all caught in one net. One-sided specialization is nearly impossible.
The limitations must also be stated plainly: square circuits are not your application. High QV doesn't mean your chemistry simulation runs fast—it's a physical-exam total score, not a specialty result. The score's magnitude is also easy to misread (2^d grows exponentially; doubling the score may correspond to just one extra layer of depth).
Other rulers
A methodology for reading scores
This column's reading tradition (a cousin of 070's roadmap reading):
1. Any single number demands companion items: qubit count with error rate, QV with square depth, logical qubits with error rate—treat scores without companions as advertisements; 2. Look at the slope, not the snapshot (070's reading, applied verbatim); 3. No substitute for the ultimate test: have the machine run a fragment of your problem—other people's scores are other people's applications.
Industry status (as of 2026-11): the QV race is cooling while logical-qubit metrics heat up (057's competition has changed its scoreboard)—benchmark confusion is itself a sign the industry is young; the rulers are still growing, and today's score is not a permanent ranking.
Dismantling the analogy
Where the camera-pixel analogy breaks down:
Luggage to take home
1. Performance is a vector, not a scalar: check all five dimensions—qubit count, error rate, gate speed, connectivity, readout. Ads print only pixels (qubit count); you must ask about the sensor (error rate) yourself. 2. What QV can and cannot do: the square exam covers all four essentials (including compiler quality), but squares ≠ your application—a physical-exam total can't replace a specialty result; logical error rate is the true ruler of the fault-tolerant era. 3. Three rules for reading scores (2026-11 view): treat unaccompanied scores as ads, watch slopes not snapshots, and the ultimate test is running your own problem—the rulers are growing; scores are not rankings.
Hook for the next stop: real machines are expensive, scarce, and queue-gated, so most of most people's experiments actually run on simulators—but simulators have their own ceiling: 2^n. Where exactly is that wall built, and why can tensor networks climb over a corner of it? 079: "The 2^n Wall: Inside Simulators and Tensor Networks."
Hashtags: quantum-computing, from-coins-to-error-correction, Quantum Volume, benchmarks