From Plan to Action: Why AI Agents Don't Follow Your Plans
A Counterintuitive Finding
You write a perfect execution plan for an AI agent, but it refuses to follow it. Worse — the more you force it to comply, the worse it performs. This isn't a bug in some open-source project; it's the conclusion of an IBM and UIUC team after analyzing 16,991 real trajectories.
The paper, *From Plan to Action: How Well Do Agents Follow the Plan?*, raises a question everyone building agents should heed: how much of the 'plan' we write in System Prompts does the AI actually absorb?
What Is 'Plan Compliance'
Existing coding agents (like SWE-agent) typically follow this workflow: navigate the code (N), reproduce the bug (R), patch it (P), and verify (V). This four-step plan is written into the system prompt as a suggestion.
The problem: nobody had verified to what extent the AI actually follows this plan.
The authors propose three quantitative dimensions, turning 'obedience' into measurable engineering metrics:
| Metric | Meaning | Why It Matters | |--------|---------|----------------| | PPC (Phase Compliance) | Coverage of planned phases | Are steps being skipped? | | POC (Order Compliance) | Order fidelity | Are steps executed in the right order? | | PPF (Phase Fidelity) | Phase faithfulness | Is the agent doing things outside the plan? |
The geometric mean of the three forms the final Plan Compliance (PC) Score.
Report Card for Four Models
The study tested GPT-5 mini, DeepSeek-V3, DeepSeek-R1, and Devstral-small.
- Devstral-small is the most 'obedient' — strictly following plan order, but often doing things outside the plan. Like a compliant but imaginative student.
- DeepSeek-V3 rarely deviates from the plan (PPF ≈ 0.99) but frequently skips phases or executes out of order — boxed into the plan yet executing it in fragments.
- DeepSeek-R1 performs worst: low phase coverage, chaotic ordering, and many out-of-plan behaviors. As the strongest reasoning model, it's the least compliant.
- GPT-5 mini adapts strategy to problem difficulty: skipping reproduction and going straight to patching on easy problems, while following plans more closely on hard ones. Adaptive, but inconsistent.
- Without a plan: the agent falls back on strategies internalized during training — incomplete and inconsistent, but at least free.
- With a bad plan: the agent is forced down a flawed pipeline, performing *worse* than with no plan at all.
- Writing steps in System Prompts is no longer enough. Context limits, myopic local reasoning, and training overfitting all erode prompt effectiveness.
- Fine-tuning for plan compliance is the way forward: teach the model 'how to follow plans' at the weight level, not stuff the plan into every inference.
- Quantification is step one — without PPC/POC/PPF, you're judging agent behavior by feel.
- Alignment matters more than instruction — plans must match the model's internalized strategies, or you get cyber-bureaucracy.
- Training outlasts prompting — truly teaching agents to follow plans requires work at the weight level.
Finding 1: The DeepSeek-R1 Paradox
Why does the strongest reasoning model show the lowest compliance? The paper suggests:
1. The short-term reward trap of RL: reinforcement learning optimizes for immediate feedback rather than long-term plan execution. If a tool call earns a reward, the model tends to 'cut corners'. 2. Context window pressure: as trajectories grow, the original plan gets buried under errors, file contents, and history, diluting its influence. 3. Data contamination and overfitting: the LLM may have internalized successful workflows from training data, overriding your plan with its own 'experience'.
Most ironically, DeepSeek-R1 shows *lower* compliance on resolved instances — meaning it solved problems through some 'back door' such as memorization or overfitting, rather than plan-aligned reasoning.
Finding 2: Cyber-Bureaucracy — A Bad Plan Is Worse Than No Plan
Across eight experiments (removing the plan, adding extra phases, reordering steps, etc.), the most counterintuitive result:
It's like corporate bureaucracy — the process stops being about solving problems and becomes about following the process. The agent deadlocks under a wrong plan instead of adapting.
Adding extra phases early (like regression testing) can even *degrade* performance, especially when misaligned with the model's internal strategy. A plan's power depends on how well it matches the model's internalized workflows.
Finding 3: Periodic Reminders Help — But Only Symptomatically
Re-injecting the plan into the prompt periodically does reduce violations and improve success rates.
But it's like reminding a goldfish every five minutes not to forget how to swim — it works, yet exposes the root problem: the agent never truly 'internalizes' the plan; it merely holds it temporarily in the context window. Once other information fills the context, the plan is forgotten.
Finding 4: Prompt Engineering Is Failing
The paper's conclusion hits an industry pain point:
> "Future research should focus on fine-tuning paradigms that teach models to follow instructed plans, rather than encoding task-specific plans into prompts."
Implications:
If agent reliability ultimately depends on fine-tuning, vendors with high-quality training data and RL capability (OpenAI, DeepSeek, Google) will dig deeper moats.
Experiment Data at a Glance
| Setting | Key Finding | |---------|-------------| | Standard plan | Compliance varies by model; successful instances usually show higher compliance | | No plan | Lower success; agents fall back on incomplete/overfit internalized strategies | | Remove reproduction phase | Negative impact even when the agent usually ignores it | | Remove verification phase | Also negative — global plans affect local reasoning holistically | | Add regression testing | Degrades performance when added early, unless aligned with model strategy | | Reorder steps | Reproduction after patching → inefficiency and higher failure rates | | Periodic reminders | Fewer violations, higher success — but reveals context dependence |
Compliance on SWE-bench Pro is 13% lower than on Verified — harder, less-contaminated problems are less sensitive to plan guidance.
A Feynman-Style Breakdown: Where Is the Real Problem?
We assume writing a plan for AI is like writing a to-do list for a human — but the information processing differs fundamentally.
Humans store the plan in working memory and continuously reference it, consciously correcting deviations.
LLM agents: the plan is just a text block in the prompt. Each reasoning step depends only on the current context; the model doesn't 'actively remember' the plan it saw minutes ago. As tokens accumulate, the plan gets diluted. Worse, RL training teaches models to 'bypass rules for rewards' — the deep reason DeepSeek-R1 misbehaves.
The paper's essence: it's not saying 'plans are useless' — it's saying 'plans must become part of the model's capability, not part of the prompt'.
Industry Implications
1. Agent product teams: stop betting everything on System Prompts. Add 'plan-following' samples to training data, or do targeted RL alignment. 2. Open-source agent developers: strong reasoning models like DeepSeek-R1 being non-compliant is a feature, not a bug. You need extra scaffolding (periodic reminders, tool-call checks) to compensate. 3. Agent evaluators: don't just look at final success rates. Use process metrics like PPC/POC/PPF to determine whether the agent solved the problem through correct reasoning or through memorization/contamination/shortcuts.
Conclusion
The paper's value: it turns a vague feeling ('the AI seems disobedient') into a quantifiable engineering problem. It tells us:
---
arXiv: 2604.12147