Executive Summary
AGEL-Comp (arXiv:2604.26522, IntelliSys 2026) is a neuro-symbolic agent framework that pairs an LLM (divergent idea generator) with a Neural Theorem Prover NTP (strict verifier) operating on a Causal Program Graph CPG (world model as growing Horn-clause hypergraph), where ILP adds new clauses through a two-stage grounding loop: minimal-contrast causal attribution followed by meta-interpretive induction. On the authors' Retro Quest (10 quests, 5 difficulty levels, 3 seeds), baseline First-Try aggregate is 3.33% and AGEL-Comp reaches 100% Task Success / 60% First-Try across four models, with a 6.8× sample-efficiency gain.
Key Points
Architecture: deduction–abduction–neural-adaptation loop
- LLM Core (Planner): emits a pool of candidate sub-goals, framed by the paper as "a pool of plausible but unverified ideas."
- NTP Verifier: differentiable relaxation of backward-chaining theorem proving (Rocktäschel & Riedel 2017 lineage) that scores each candidate against the current CPG; σ≈0 candidates are rejected before execution.
- CPG (World Model): a directed hypergraph whose edges are Horn clauses
h ← b₁,…,b_n; supports incremental rule addition and concept embedding initialization for new predicates. - ILP Learning (two stages): 1. MCS (minimal-contrast causal attribution): only when ΔS between failing and nearest successful experience is a single literal, that literal is flagged as the cause (e.g.,
- Neural adaptation: periodic joint training of NTP and embeddings.
- w/o NTP: 91.7% Task Success, 22.5% First-Try — "learns but cannot verify"
- w/o ILP: 76.7% Task Success, 8.3% First-Try — "verifies but cannot learn," essentially overlapping the naked baseline (75.0% / 3.33%)
- Crisis evidence cited: SCAN (Lake & Baroni 2018), COGS (Kim & Linzen 2020), CFQ (Keysers 2020), Faith and Fate (Dziri 2023), CompWoB (NeurIPS 2023: GPT agents 94.0% basic → 24.9% compositional).
- Closest neighbors: Voyager (code skill library), ExpeL/Reflexion (natural-language reflection), CLIN (verbalized causal rules); AGEL-Comp occupies the "dynamic formal rule library + theorem-proving verification" quadrant — the most "symbol-heavy" branch.
- Distinguishing trio: Horn-clause knowledge with decidable consistency; prediction-error-driven minimal-contrast learning that resists LLM-prior confounds; strict planner–verifier separation.
- OSF preregistration (osf.io/a6j4c) and GenAI-use disclosure
- Full open-source release of framework, environment, and scripts:
github.com/Place-Beyond-Bytes/AGEL-Comp - Honest multi-metric reporting (First-Try 50–67% is not hidden)
- A meaningful ablation whose w/o-ILP curve overlaps the baseline
causes_damage(fire)).
2. MIL (meta-interpretive induction): produces a Horn clause covering E⁺ consistent with B∪H ⊭ E⁻.
Six-claim fact-check vs popular videos
| # | Popular claim | Verdict | Evidence | |---|---|---|---| | V1 | "Baseline First-Try only 3.3%" | True but misframed: 3.33% is the First-Try aggregate across four models; GPT-4o baseline Task Success is 86.67% | Table 3, §5.5.1 | | V2 | "Small local model hits 100%" | True for Task Success with retries; LLaVA First-Try is only 50% | Table 3 | | V3 | "LLaVA matches GPT-4o 100%, gap erased" | Exaggerated: First-Try 50% vs 66.67%, latency 96.81s vs 36.40s (~2.7×), iterations 17.22 vs 11.38 | Table 3 | | V4 | "Fire-breathing dragon crash" | Fabricated: no such monster; agent *does not know* fire is harmful until it steps on it (−50 HP) | §4, §5.2 | | V5 | "Local small model with billions of params" | True: LLaVA-1.6-Mistral-7B and DeepSeek-VL-7B | Footnotes 4/5 | | V6 | "60% improvement" | Misphrased: 60% is the post-improvement absolute First-Try mean; real gain is +56.7 pp / ~18× | §5.5.1 |Reconstructed Table 3 (30 attempts per cell)
| Model | Config | Task Success % | First-Try % | Avg Iter | Latency s | Sample Eff. | |---|---|---|---|---|---|---| | GPT-4o | Baseline | 86.67±5.77 | 6.67±5.77 | 18.87 | 347.88 | 146.97 | | GPT-4o | AGEL-Comp | 100±0 | 66.67 | 11.38 | 36.40 | 25.03 | | Gemini-2.5-Pro | Baseline | 83.33±5.77 | 3.33±5.77 | 17.73 | 302.02 | 159.30 | | Gemini-2.5-Pro | AGEL-Comp | 100±0 | 66.67 | 10.81 | 28.06 | 23.27 | | DeepSeek-VL-7B | Baseline | 66.67±5.77 | 3.33±5.77 | 20.36 | 739.81 | 243.00 | | DeepSeek-VL-7B | AGEL-Comp | 100±0 | 56.67 | 15.10 | 74.36 | 40.43 | | LLaVA-1.6-7B | Baseline | 63.33±5.77 | 0.00±0 | 22.95 | 1028.72 | 258.10 | | LLaVA-1.6-7B | AGEL-Comp | 100±0 | 50.00 | 17.22 | 96.81 | 41.70 |Aggregates: Baseline 75.0% / First-Try 3.33%; Full System 100% / First-Try 60.0%; w/o NTP 91.67% / 22.5%; w/o ILP 76.7% / 8.3%.
Statistical thinness
±5.77 = 10/√3 arises from three seeds whose successes differ by one quest out of 10, so the smallest discriminable step is ~3.3 pp. The paper reports no significance tests or confidence intervals; GPT-4o main comparison 26/30 vs 30/30 yields Fisher exact p ≈ 0.11. There is no independent train/test split or held-out set.Counter-intuitive ablation
The ILP contribution exceeds the NTP contribution: verification without learning is no verification. The w/o-ILP curve overlapping the baseline is the strongest evidence that gains come from the learning mechanism itself.
Sister paper (arXiv:2604.26521, AAAI MAKE 2026)
On controlled visual logic puzzles, iiLTN (iterative logic tensor network) reaches 51.2% compositional accuracy vs 11.3% for grounding-only — a 4.5× gain — and the reasoning-only ablation is worse, supporting the thesis that compositional reasoning does not emerge from grounding and must be an explicit learning objective. The main paper then answers "how to learn it explicitly" with NTP verification + ILP induction.Lineage positioning
Strongest Critiques (Devil's Advocate)
1. Author-built evaluation. All evidence comes from the authors' own 10-quest RPG; predicate vocabulary is small and unreported; the MCS single-literal trigger is essentially a self-confession of environment sparsity; no third-party benchmarks (TextWorld / ALFWorld / BabyAI) are evaluated. 2. Perception is spoon-fed. The environment API returns structured ground literals with zero noise or cost; the paper itself acknowledges noisy perception as an open challenge (§5.6.1 Outlook ii). The learning loop may simply not trigger in open-world noise. A Phase-1 NTP/embedding pretraining also injects an inductive bias absent from the ReAct baseline. 3. 100% is a ceiling effect. The LLM is reduced to a "generative hypothesis engine"; when tasks are simple enough for the symbolic side to backstop, any passable LLM reaches 100%. Improvement magnitude is strictly negatively correlated with base strength (GPT-4o +13.3 pp, DeepSeek +33.3 pp, LLaVA +36.7 pp).
Defenses (Fair Record)
Conclusion
AGEL-Comp is a clear-mechanism, procedurally rigorous workshop paper whose evidence strength trails its rhetorical scope. On a small, closed, symbol-friendly world it demonstrates that the loop *LLM proposes → NTP verifies → prediction error drives ILP legislation* significantly beats a naked LLM (First-Try 3.33%→60%, ~18×, 6.8× sample efficiency). The 100% number is a Task-Success ceiling under retries, not an LLM capability breakthrough. The Fire-breathing-dragon anecdote in popular coverage is invented and directionally inverted. The three most transferable ideas for agent builders are planner–verifier separation, prediction-error-triggered minimal-contrast rule induction, and the sister paper's empirical warning: compositional reasoning does not emerge from grounding for free.
Sources
| Category | Reference | |---|---| | Main paper | arXiv:2604.26522 | | Sister paper | arXiv:2604.26521 | | Code & preregistration | github.com/Place-Beyond-Bytes/AGEL-Comp ; osf.io/a6j4c | | Compositionality crisis | Lake & Baroni 2018 (SCAN); Kim & Linzen 2020 (COGS); Keysers 2020 (CFQ); Dziri 2023 (Faith and Fate); CompWoB (arXiv:2311.18751) | | NTP / ILP lineage | Rocktäschel & Riedel 2017; Minervini 2018/2020; Muggleton 2014/2015; Cropper & Dumančić 2021/2022; Evans & Grefenstette 2018 | | Peer agents | Voyager (arXiv:2305.16291); ExpeL (ICLR 2024); CLIN (ACL 2024); Reflexion 2023 |