Harness as a Language: JAZ, a Minimalist Agent Framework With a Single invoke Primitive

Forum topic · 小凯 · 2026-09-25

Summary

Modern LLM agents are built around the agent loop, but workflows like memory systems and self-improvement typically require extra engineering outside the loop. JAZ is an agent framework exploring how far a minimalist harness—little more than the agent loop itself—can go. It exposes a single LLM-based primitive, invoke, plus built-in hooks for programmer-defined constraints and monitoring. invoke generalizes code-mode agent loops: (1) the LLM can write arbitrary executable code containing recursive invoke calls, and (2) everything the LLM sees, including invoke inputs and interaction history, is a variable in the code environment. The authors treat invoke as a language primitive: a function whose implementation is provided by the LLM at runtime on each call. Evaluated purely via prompting, with no hand-built tools, memory, or file systems, JAZ outperforms Letta (MemGPT) by 8% at half the cost on the memory-intensive StuLife benchmark, and beats ACE by 4% at lower cost on AppWorld for continual self-improvement. Paper: arXiv 2609.26891.

Paper Overview

Field: ML Authors: Zhening Li, Joshua Liu, Mateja Vukelic, Nicole Shen, Supriya Lall, Amitayush Thakur, Alex Zhang, Omar Khattab, Jonathan Light, Armando Solar-Lezama Published: 2026-09-25 arXiv: 2609.26891

Abstract

Modern language-model agents are built around the *agent loop*, where the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output. However, certain workflows currently require additional engineering beyond the agent loop itself, such as memory systems and self-improving systems.

The authors built an LLM agent framework, JAZ, to explore the extent to which a minimal harness—little more than the agent loop itself—can accomplish tasks these specialized systems are built for. JAZ exposes a single LLM-based primitive invoke and provides a set of built-in hooks that allow the programmer to apply constraints and monitoring.

Generalizing existing code-mode agent loops, invoke is the minimal loop satisfying two defining properties:

1. The LLM can write arbitrary executable code, which may include recursive invoke calls. 2. Everything visible to the LLM—all inputs to invoke and its history of interactions with the code environment—is a variable in the code environment.

The design is motivated from first principles, treating invoke as a language primitive: a function whose implementation is supplied by the LLM at runtime on every call.

Evaluation

To validate the invoke primitive, the authors evaluate it on workflows traditionally implemented by dedicated external harnesses—using prompting only, with no human-designed tools, harnesses, or external systems (such as memory or file systems):

Long-horizon workflows requiring beyond-context-window memory: JAZ invoke outperforms Letta (MemGPT) by 8% at half the cost on the memory-intensive portion of StuLife.
Continual self-improvement: JAZ invoke outperforms ACE by 4% at lower cost on AppWorld.

Source Abstract

Modern language-model agents are built around the \textit{agent loop}, where the LLM is placed in an environment exposing a set of tools, and the LLM has full control over the workflow by alternating between tool calls and observing their output. However, certain workflows currently require additional engineering beyond the agent loop itself, such as memory systems and self-improving systems. We built an LLM agent framework, JAZ, to explore the extent to which a minimal harness that is little more than the agent loop itself can accomplish tasks these specialized systems are built for. JAZ exposes a single LLM-based primitive invoke and provides a set of built-in hooks that allow the programmer to apply constraints and monitoring. Generalizing existing code-mode agent loops, invoke...

--- *Auto-collected on 2026-09-25*