Overview
SIA (Self Improving AI with Harness & Weight Updates) tackles a core bottleneck: improving AI still relies on humans. Prior research split into two isolated camps — Harness-Update methods (e.g., Darwin Gödel Machine, Meta-Harness) that rewrite agent scaffolding but never touch weights, and Test-Time Training methods (e.g., TTRL) that update weights through fixed human-engineered pipelines. SIA combines both levers in one closed loop.
Architecture
- Meta-Agent (M) — Claude Sonnet 4.6; generates the initial scaffold A₁ from the task spec.
- Task Agent (A_g) — gpt-oss-120b plus a scaffold: system prompt, tool routing, answer extraction. The grader is external infrastructure.
- Feedback-Agent (F) — Claude Sonnet 4.6; receives the complete execution trajectory and metrics, diagnoses specific failure modes, then decides whether the next step updates the harness (H), weights (W), or triggers an RL run.
- Base model: openai/gpt-oss-120b
- LoRA rank r = 32, learning rate 4×10⁻⁵
- Training on H100 GPUs via Modal
- Harness can fix search strategies and plumbing but cannot teach legal nuances, GPU microarchitecture intuition, or data statistics.
- Weights can encode domain priors but are wasted on a broken pipeline.
- The effect is multiplicative, and it held across three unrelated domains, suggesting a general architecture principle.
- Paper: Prannay Hebbar, Yogendra Manawat, Samuel Verboomen, Alesia Ivanova, Selvam Palanimalai, Kunal Bhatia, Vignesh Baskaran. "SIA: Self Improving AI with Harness & Weight Updates". arXiv:2605.27276
- https://arxiv.org/abs/2605.27276
The decision sequence is dynamic, not a fixed schedule, e.g. A₁ →H→ A₂ →H→ A₃ →W→ θ₁ →H→ A₄ →W→ θ₂.
Dynamic RL algorithm selection
The Feedback-Agent picks among six algorithms based on observed reward structure:
| Algorithm | Best when | |---|---| | PPO + GAE | dense step-level rewards, stability constrained | | GRPO | cheap rollouts, verifier at episode end; group-normalized advantages, no value net | | Entropic Advantage Weighting | right-skewed rewards, rare but strong successes | | REINFORCE + KL-to-Base | dense rewards, risk of capability regression | | Best-of-N behavior cloning | extremely sparse rewards (E[r] ≈ 0) | | DPO | verifier can rank but not score |
Weight update setup
Domains and results
| Task | Baseline | Prior SOTA | SIA-H only | SIA (H+W) | Margin over SOTA | |---|---|---|---|---|---| | LawBench (top-1 acc, 191 classes) | 13.5% | 45.0% | 50.0% | 70.1% | +25.1 pp | | TriMul (CUDA kernel, reward) | 0.105 | 1.292 | 0.120 | 1.475 (1,017 μs) | +14.2% reward, 12.4% faster | | MAGIC scRNA-seq denoising (mse_norm) | 0.048 | 0.240 | 0.241 | 0.289 | +20.4% |
Notable harness evolutions include a TF-IDF + LinearSVC reranker for LawBench, CUDA compiler-error parsers for TriMul, and batch configuration drivers for MAGIC. Notable weight-learned behaviors include fine-grained charge discrimination, H100 tiling/register intuition, and a np.clip + np.rint post-processing transform for RNA counts that no scaffold iteration ever produced.
Why combination wins
> Harness updates make the model agentic, shaping how it searches and acts, while weight updates build the domain intuition that no prompt or scaffold can instil.
Limitations
1. Compute cost — 7–10 iterations per domain with Claude API calls and H100 LoRA training. 2. Base-model dependence — LoRA cannot create latent knowledge that isn't there. 3. Verifier requirement — tasks need deterministic reward signals. 4. Feedback-Agent bottleneck — human effort shifts from implementation to designing the feedback system and graders. 5. No cross-domain transfer tested — harness improvements are domain-specific; only the meta-pattern may transfer.
Significance
SIA demonstrates an operational self-improvement framework: dual levers beat either alone, closed-loop trajectory-driven decisions beat fixed schedules, and the framework generalizes. LawBench went from 13.5% to 70.1% in a few iterations, surpassing SOTA that took humans years to reach.