32B Beats 671B: How OpenHands LM Proves Model Size Isn't Everything

> In March 2025, the OpenHands team released an open-source coding model based on Qwen2.5-Coder-32B. Fifteen months later, its successors—also 32B parameters—achieved a 62.2% resolution rate on SWE-Bench Verified. Meanwhile, the 671B-parameter DeepSeek V3 only managed 38.8%. This isn't magic; it's evolution across three layers: data quality, training methods, and agent-framework synergy. While the industry chases parameter scale, the OpenHands ecosystem has proven with hard numbers that software engineering capability depends more on *how you train* and *how you deploy* than *how big you are*.

• Published: 2026-06-05
• Source: OpenHands Dev Team / All Hands AI, 2025–2026
• Papers: Multiple arXiv papers (see references)
• Core model: OpenHands LM 32B (based on Qwen2.5-Coder-32B-Instruct)

1. A Counterintuitive Fact: 32B Can Beat 671B

In March 2025, the OpenHands team released a seemingly unremarkable model: OpenHands LM 32B. Based on Qwen2.5-Coder-32B-Instruct, its fine-tuning data came from successful trajectories of OpenHands agents on real open-source repositories.

Its SWE-Bench Verified score: 37.2%.

That score alone isn't spectacular. But the comparison makes it striking:

• DeepSeek V3 (671B): 38.8%
• Qwen2.5-Coder-32B-Instruct (original, no fine-tuning): 6.2%
• OpenHands LM 32B (fine-tuned): 37.2%

32B parameters, nearly matching a 671B model. 1/20th the parameters, 95% of the performance.

But that was just the beginning.

2. From 37.2% to 70.8%: A 15-Month Evolution Arc

OpenHands LM was not an isolated model but the starting point of an ecosystem. Over the next 15 months, multiple teams shipped improved versions on the same or similar 32B backbones.

| Model | Date | Framework | Training | SWE-Bench Verified | Key Innovation | |-------|------|-----------|----------|--------------------|----------------| | OpenHands LM 32B | 2025-03 | OpenHands | SFT (SWE-Gym) | 37.2% | First open-source 32B coding agent model | | SWE-agent-LM-32B | 2025-05 | SWE-Agent | SFT | 40.2% | Different agent framework | | SWE-Mirror-LM-32B | 2025-09 | MOpenHands | SFT (mirrored dataset) | 52.2% | Large-scale mirroring of real issues | | Skywork-SWE-32B | 2025-06 | OpenHands | SFT + TTS | 38.0% → 47.0% | Data scaling laws | | SWE-Compressor | 2025-12 | OpenHands | SFT | 57.6% | Trajectory compression | | SWE-Swiss-32B | 2025-12 | Agentless | SFT + RL | 58.0% | RL optimization | | DeepSWE-32B | 2025-10 | OpenHands | RL | 42.2% → 59.0% (+TTS) | Pure RL training | | daVinci-Dev-32B | 2026-04 | SWE-Agent | MT + SFT | 56.1% | Multi-task training | | SWE-Zero-32B | 2026-02 | OpenHands | SFT | 57.5% | Data built from scratch | | SWE-Master-32B-SFT | 2026-02 | R2E-Gym | SFT | 57.8% → 70.2% (+TTS) | Best SFT + test-time scaling | | SWE-Master-32B-RL | 2026-02 | R2E-Gym | SFT + RL | 61.4% → 70.8% (+TTS) | RL + test-time scaling | | SWE-Hero-32B | 2026-01 | OpenHands | SFT | 62.2% | Current best on OpenHands framework | | OpenSWE-32B | 2026-02 | SWE-Agent | SFT | 62.4% | Best on SWE-Agent framework |

Key observations:

1. 37.2% → 62.2%: same 32B parameters, same OpenHands framework, different training data and methods — a 67% relative improvement 2. Test-time scaling (TTS) is remarkably effective: SWE-Master-32B went from 57.8% → 70.2% just by increasing inference-time attempts 3. Diminishing returns for RL: SFT + RL beats pure SFT by roughly 3–5 points, but at significantly higher cost

3. Why Can 32B Do This? A Three-Layer Breakdown

3.1 Layer One: Data — Quality Beats Quantity, but Quantity Matters Too

OpenHands LM's starting success came from a shift in the data generation paradigm.

Traditional approach: manually collect codebases and annotate issue–solution pairs. OpenHands approach: let the agent run by itself and keep only successful trajectories.

The concrete pipeline (SWE-Gym framework): 1. Pick a real GitHub issue 2. Let an OpenHands agent (possibly Claude/GPT) attempt to solve it 3. If it passes the tests → keep the trajectory as training data 4. If it fails → discard

Core advantages of this bootstrapped data generation:

• Real distribution: trajectories come from real open-source projects, not synthetic data
• Naturally executable: each trajectory contains concrete file edits, command executions, and test verification
• High quality: only successful trajectories are kept — the model only learns "doing things right"

SWE-Mirror-LM's innovation was scaling up data volume: mirroring large numbers of real issues so training data covers a broader range of problem types. This explains its jump from 37.2% to 52.2%.

Skywork-SWE's research further found data scaling laws: on SWE tasks, performance keeps rising as data volume grows past a threshold, with no saturation.

3.2 Layer Two: Training — SFT Is the Foundation, RL Is Icing, TTS Is the Secret Weapon

Supervised fine-tuning (SFT) is the common foundation for all 32B models. The key difference is how training data is produced:

• OpenHands LM: trajectories generated via SWE-Gym, kept if successful
• SWE-Master: uses R2E-Gym (a new training environment), building more complex trajectories from scratch
• SWE-Swiss: adds RL optimization so the model learns to "verify its own output"

Reinforcement learning (RL) results:

• SWE-Master-32B-SFT: 57.8%
• SWE-Master-32B-RL: 61.4%
• Gain: 3.6 percentage points

The gain is real, but the costs are:

• RL training requires substantial compute
• Reward functions must be carefully designed (tests passing ≠ correct fix)
• Risk of overfitting to specific test sets

Test-time scaling (TTS) is the biggest performance lever:

• SWE-Master-32B-SFT: 57.8% → 70.2% (TTS@8)
• SWE-Master-32B-RL: 61.4% → 70.8% (TTS@8)

TTS means: at inference, generate multiple candidate solutions and use a verifier to pick the best one. This works especially well on SWE tasks because:

• Test suites provide explicit verifiers
• Candidates can be verified in parallel
• Marginal cost of more attempts decreases

3.3 Layer Three: Agent Framework — The Model Is the Engine, the Framework Is the Chassis

The most overlooked point: the same 32B model performs very differently across agent frameworks.

Qwen2.5-Coder-32B-Instruct (original, no fine-tuning):

• On OpenHands: 6.2%
• On SWE-Agent (after fine-tuning): 40.2%

The model is not the only variable. The agent framework determines what the model can do and how.

Core design of the OpenHands framework:

1. Event-driven architecture: every agent step (observation, thought, action) is an event that can be recorded, replayed, and analyzed 2. Sandboxed environment: all operations run in isolated containers 3. Multi-tool support:

• File editing (str_replace_editor)
• Shell command execution (execute_bash)
• Browser operations (for docs, testing web pages)
• Task submission (finish)
4. Persistent state: agents remember prior actions/observations, supporting long tasks 5. 128K context window: handles large codebases

OpenHands' unique advantages (per the Skywork paper):

• Across multiple evaluations, over half of adopters chose OpenHands
• The same model usually performs better on OpenHands than other frameworks
• Different OpenHands versions' system prompts and execution pipelines cause significant performance differences

4. Comparison with Closed Models: The Gap Is Narrowing but Persists

Closed-model state of the art (early 2026):

| Model | Framework | SWE-Bench Verified | Notes | |-------|-----------|--------------------|-------| | Claude-4.5-Sonnet | OpenHands | 77.2% | Current SOTA | | Claude-3.7-Sonnet | OpenHands | 53–72% | Depends on configuration | | GPT-4.1 | OpenHands | 54.6% | Strong multilingual | | GPT-4o | Multiple | ~50% | General capability |

Best open-source 32B results:

• SWE-Master-32B + TTS: 70.2% (near Claude-3.7 level)
• SWE-Hero-32B: 62.2% (comparable to Claude-3.5)
• OpenHands LM 32B: 37.2% (comparable to DeepSeek V3 671B)

Key conclusions:

• Closed models still lead, but the gap is shrinking
• 32B open models + TTS can reach ~70% of Claude-3.7's level
• The remaining gap is mainly in general reasoning and complex multi-step tasks, not code understanding itself

5. Deployment: A Single RTX 3090 Is Enough

One of OpenHands LM 32B's original selling points was local runnability.

Hardware requirements:

• Single RTX 3090 (24GB VRAM): can run a 4-bit quantized 32B model
• Consumer GPU: no A100/H100 cluster needed
• Via Ollama/SGLang/vLLM: OpenAI-compatible API support

This changes the deployment paradigm for open-source agents:

• Before: call Claude/GPT APIs at $1–5 per task
• Now: download once, run locally, zero marginal cost

What this means for developers:

• Privacy: code never leaves the machine
• Cost: high-frequency tasks approach zero marginal cost
• Latency: local inference beats API round-trips (especially multi-turn)
• Customization: fine-tune on private codebases

6. Key Limitations: Benchmarks Aren't Reality

Before celebrating these numbers, some cold water.

6.1 SWE-Bench's Limits

SWE-Bench tests: given a known GitHub issue and the relevant codebase, can the model produce a correct fix patch?

Real software engineering involves much more:

• Requirements understanding (vague, incomplete specs)
• Architecture design (building systems from scratch)
• Code review (understanding others' code and advising)
• Refactoring decisions (when to refactor, when not to)
• Technical debt management (balancing delivery and maintenance)

SWE-Bench covers only the subset of "fixing known bugs."

6.2 Data Contamination Risk

SWE-Bench's test set comes from public GitHub issues, which may have been "seen" in training data.

The Skywork paper explicitly notes: "We fixed a git log operation issue to prevent potential data leakage" — and the corrected score was slightly below the official baseline. This shows: data contamination is a real risk requiring strict safeguards.

6.3 Practicality of Test-Time Scaling

TTS lifts 57.8% to 70.2%, but at the cost of:

• 8× inference time (TTS@8 means 8 parallel attempts)
• 8× compute cost
• Strong verifiers (test suites) needed to filter correct solutions

In real settings, if a bug takes 8 attempts at 5 minutes each, that's 40 minutes total — unacceptable for urgent fixes.

6.4 Model Limitations

The OpenHands team itself acknowledges:

• The model is best at resolving GitHub issues; it underperforms on other software engineering tasks
• It sometimes produces repeated steps (loop behavior)
• Stability degrades on long sequences (>50 interaction turns)

7. Industry Impact: An Inflection Point for the Open-Source Agent Ecosystem

The emergence of OpenHands LM 32B and its successors marks several trends:

7.1 From "API Calls" to "Local Execution"

Open-source coding agents no longer depend on closed APIs. This reduces:

• Cost (especially for high-frequency usage)
• Latency (local inference is faster)
• Privacy risk (code stays local)
• Vendor lock-in (switch models freely)

7.2 From "Model Competition" to "Data Competition"

When multiple teams, on the same 32B backbone, reach wildly different performance through different training data, data quality becomes the key differentiator.

This explains why:

• Data generation frameworks like SWE-Gym, R2E-Gym, and SWE-Flow have become research hotspots
• Generation strategies (bootstrapping, mirroring, multilingual, multi-turn) are the core innovation points
• Data scaling laws deserve more attention than model scaling laws

7.3 From "Single Model" to "Model + Framework + Verifier" Systems Engineering

Best SWE performance comes not from a single model but from:

• Model: generates candidate solutions
• Framework: determines environment interaction
• Verifier: judges correctness (test suites, static analysis, human review)

This triad — model + framework + verifier — is defining the new paradigm of AI software engineering.

8. The Questions That Really Matter

1. Where is the 32B ceiling? 62.2% is near Claude-3.5 level, but Claude-4.5 sits at 77.2%. Is that gap a data problem, an architecture problem, or a fundamental capability gap? 2. TTS's practical boundary: 8× inference time for 12 points — is that trade-off worth it in real scenarios? 3. From bug fixing to full development: SWE-Bench only tests fixes for known issues. How do we make 32B models handle architecture design, code review, and refactoring decisions? 4. Multilingual challenges: SWE-Bench is mostly Python. Multi-SWE-Bench is emerging, but 32B performance on non-Python languages remains limited. 5. Automating data generation: current data generation still depends on expensive agent runs (Claude/GPT). Can cheaper models (7B/14B) produce high-quality training data? 6. RL's potential: RL currently adds only 3–5 points. Is that poor reward design, or is SFT already near the ceiling?

9. Summary

The evolution of OpenHands LM 32B and its ecosystem proves that on software engineering tasks: model size is not the only deciding factor — co-optimizing data quality, training methods, and agent frameworks lets mid-size models approach large-model performance.

Key data recap:

• OpenHands LM 32B (2025-03): 37.2% (on par with 671B DeepSeek V3's 38.8%)
• SWE-Hero-32B (2026-01): 62.2% (best on OpenHands framework)
• SWE-Master-32B + TTS (2026-02): 70.2% (near Claude-3.7 level)
• Original Qwen2.5-Coder-32B: 6.2% (fine-tuning delivers 6–10× improvement)

Key insights:

• Data generation strategy (bootstrapping, mirroring, filtering) matters more than model architecture
• Agent framework choice (OpenHands/SWE-Agent/R2E-Gym) significantly affects performance
• Test-time scaling is the most cost-effective performance lever
• RL's marginal gains are limited but there's optimization room
• Local deployability (single RTX 3090) changes the business model of open-source agents

The final open question: when 32B models run locally, cost near zero, and perform near Claude-3.7, what pressure does that place on closed-source API pricing and business models? That inflection point may arrive faster than most expect.

References

• OpenHands Dev Team (2025). Introducing OpenHands LM 32B. https://www.openhands.dev/blog/introducing-openhands-lm-32b
• Wang et al. (2025). SWE-Mirror-LM: Training on Mirrored Real-World Issues. arXiv:2509.08724
• Song et al. (2026). SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training. arXiv:2602.03411
• Luo et al. (2025). DeepSWE-32B-Preview. arXiv:2505.xxxxx
• He et al. (2025). SWE-Swiss-32B. Agentless framework with SFT+RL.
• Fu et al. (2026). OpenSWE-32B. SWE-Agent framework.
• Zeng et al. (2026). daVinci-Dev-32B. SWE-Agent with MT+SFT.
• Pan et al. (2025). SWE-Gym: Training SWE Agents with RL. arXiv:2512.21103
• Skywork AI (2025). Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs. arXiv:2506.19290
• Wang et al. (2024). OpenHands: An Open Platform for AI Software Developers as Generalist Agents. arXiv:2407.16741