> A Feynman-style reading of *Scaling Coding Agents via Atomic Skills*
---
The 3 A.M. Bug-Fixing Expert
Imagine this scene: it's 3 a.m., and you're staring at a cryptic error on screen. Your AI assistant — the one that autocompletes in GitHub Copilot at lightning speed — suddenly acts helpless. It tries a dozen fixes, each proposed with confidence, each failing worse than the last.
"This code should fix the memory leak," it says.
You run the tests. Segmentation fault.
"Let me try again, changing the pointer deallocation order."
Still failing.
You sigh and spend ten minutes locating the real problem — not in memory management at all, but in completely unrelated config-file parsing logic. The AI assistant is like a student who only memorizes: it has "seen" many memory-leak fixes but never truly learned how to locate a problem.
That's today's AI coding assistants: lopsided specialists. Great at bug fixing, lost at refactoring; able to write unit tests, unable to judge whether a change is actually correct. Researchers from HKUST, NUS, Peking University, Shanghai Jiao Tong University, and BUPT give the answer: we've been teaching AI to do exercises, not to solve problems.
---
Whole-Task Training vs. Atomic Operations
A simple analogy: to teach a child to cook, the traditional approach is to have them repeatedly practice one complete dish — say, scrambled eggs with tomatoes. After a hundred reps, they make a decent version of that dish. But ask them to make shredded pork with green peppers, and they're lost. Why? Because they never learned knife skills, seasoning, or heat control — just a fixed recipe.
The researchers call this task-specific overfitting. We train models on "whole problems" — complete bug-fix or code-generation tasks — rather than the atomic operations of problem-solving.
The result: a model like a student who memorized the question bank. It aces SWE-bench bug fixes, but swap the task to code refactoring — which is also modifying code, just with a different goal — and it's confused, because refactoring requires a different distribution of atomic operations.
This recalls Feynman's father teaching him about birds: "You can name the bird in all languages, but knowing nothing whatsoever about the bird — you only know about humans and what they call things."
---
Five Lego Bricks
The researchers dissect software engineering tasks into five foundational, universal skills:
1. Code Localization — Where to look given a bug report like "occasional timeouts during login"? Humans navigate by architectural intuition; untrained AI wanders.
2. Code Editing — Modifying code while understanding intent, preserving style, and not breaking other functionality. Like surgery: precise repair, not just cutting.
3. Unit-Test Generation — Tests define expected behavior. Writing a test that actually catches bugs is harder than writing code itself.
4. Issue Reproduction — The essence of debugging: constructing the conditions under which a reported bug reproduces locally — understanding the causal chain, not just "it's broken."
5. Code Review — Judging whether a change truly solves the problem, introduces new bugs, or follows best practices — understanding both the original problem and the modification.
These are the atomic skills: bug fixing = localization + editing + review; refactoring = localization + editing + test generation; security fixes = reproduction + editing + review. Like Lego bricks — limited base blocks, infinite builds.
---
The Secret of Joint Training
Instead of training each skill separately, the researchers use joint reinforcement learning — one policy, one model, all five skills at once.
Counterintuitively, simultaneously training related skills helps rather than interferes. Like a chef apprentice practicing cutting, seasoning, and heat control together, the model learns cross-skill relationships: where you localize a problem should shape your editing strategy. This transfer cannot come from isolated training.
They use GRPO (Group-based Relative Policy Optimization), the method behind DeepSeek-R1: generate multiple candidate solutions per problem and learn from their relative quality, rather than from absolute scores — which invites overfitting.
Joint training improved every atomic skill:
- Code localization: 66.5% → 71.2%
- Code editing: 45.8% → 61.1%
- Issue reproduction: 54.2% → 60.5%
- Unit-test generation: 35.9% → 47.2%
- Code review: 56.3% → 62.2%
- SWE-bench Verified — real-world bug fixes
- SWE-bench Multilingual — multilingual bug fixing
- Terminal-Bench — pure command-line tasks
- Code refactoring — a brand-new benchmark
- SEC-Bench — security tasks requiring vulnerability reproduction and PoC writing
- Finer-grained skill decomposition — "sub-atomic" skills like understanding variable scope, analyzing control flow, recognizing design patterns.
- More complex skill compositions — architecture design, performance optimization, technical debt management, with long-horizon planning.
- New human-AI collaboration paradigms — humans no longer say "write this code" but "help me decompose this requirement into skill steps and complete it together."
- Continual learning — real engineers improve with every problem solved; making AI learn continually remains an open problem.
An average improvement of 18.7% — and crucially, these gains generalize to composite tasks the model never saw.
---
The Magic of Generalization
The paper's most exciting result: after training on the five atomic skills, the model was tested on five entirely different, unseen composite tasks:
The model improved on all of them: SWE-bench Verified 50.7% → 58.5%, Multilingual 30% → 38.9%, refactoring 14.6% → 17.1%.
This shows the model learned how to solve problems, not just memorized solutions to specific ones. Like the child who truly learns knife work, seasoning, and heat control — they can cook dishes they were never taught. That's the power of composability: five atomic skills are five notes that can play endless music.
---
Critique: Not a Panacea
First, are these five skills really "atomic"? The researchers admit the decomposition is somewhat subjective. Localization could be split into "understanding the issue," "searching the codebase," "judging relevance." Finer splits mean less training data per skill and possibly worse generalization; coarser splits lose atomicity. The balance is empirical, not theoretically derived.
Second, why does joint training work? Is it a deep relationship between skills, or just data diversification — a cleverer form of "data augmentation"? Unclear.
Third, is GLM-4.5-Air-Base a strong baseline? It's a 106B-parameter model (12B active) designed for complex coding and agentic tasks. An 18.7% gain on it is impressive — but does the method work for smaller models or closed-source giants like GPT-4? The paper doesn't say.
Fourth, are the metrics robust? Localization accuracy requires exact set match with the ground-truth modified files — missing or adding one file counts as total failure. In practice, locating most relevant files may already be useful.
Fifth, reward hacking risk. Any RL system can find shortcuts to high scores without real skill. The researchers use sandboxes, disabled network access, and removed .git history to prevent cheating — but whether that's enough in complex real settings is open.
---
The Future: From Exercise-Doers to Engineers
Today's coding assistants are essentially advanced autocomplete — predicting the next token from local context. But real software engineering is understanding problems, designing solutions, implementing, testing, and iterating.
The atomic-skills framework is a step toward a true AI software engineer. Possible future directions:
Closing: The Art of Naming
Knowing the name of something and understanding it are entirely different things.
"Atomic skills" is a catchy name — it makes us feel we've grasped something essential. But a name is just a name. What matters: is this the right decomposition? Can these skills truly compose into infinite possibilities? Did joint training genuinely discover deep skill relationships?
The paper gives preliminary, encouraging answers — and every answer is the starting point of the next question.
Next time an AI sets a new SWE-bench record, don't just marvel. Ask: did it really learn to program, or just memorize more solutions?
As Feynman said: The first principle is that you must not fool yourself — and you are the easiest person to fool.