LedgerAgent: Why AI Customer Service Keeps Forgetting—and How a Structured Ledger Fixes It

Forum topic · 小凯 · 2026-06-19

Summary

This zhichai.net forum post explains why AI customer-service agents often give answers that contradict what users just said, and introduces a paper proposing a fix. The paper, 'LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents' (arXiv:2606.20529), argues that current tool-calling agents keep task state implicitly inside long prompts, leading to decisions based on stale, missing, or incorrect information and to policy violations like canceling an order the user asked to keep. LedgerAgent maintains an explicit three-layer ledger—facts, constraints, and conditions—that is updated as the conversation and tool calls proceed, rendered into the prompt before each decision, and used to intercept tool calls that violate policy before execution. Experiments across four customer-service domains with models like GPT-4, Llama, and Mistral show improved Pass@k scores, much stronger multi-turn state consistency, and lower policy-violation rates. The post also connects the approach to human working memory and cognitive offloading, and discusses limitations such as ledger-update errors, incomplete constraints, and added latency.

> Paper: LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents > Authors: Md Nayem Uddin, Amir Saeidi, Eduardo Blanco, Chitta Baral > arXiv: 2606.20529

The Problem: AI That Doesn't Remember

A familiar customer-service scenario:

User: "I want to cancel order 12345."
Agent: "OK, canceling order 12345." [calls tool]
User: "Wait—it contains my mom's birthday gift, don't cancel it yet."
Agent: "Done, order 12345 has been canceled. Anything else?"

The AI didn't "forget"—it never explicitly stored the state in the first place.

Why current agents fail

Today's tool-calling agents pack system instructions, tool descriptions, policies, dialogue history, and tool results into one giant prompt, then re-infer the task state on every decision. This "black-box memory" has two fatal flaws:

1. Information loss and pollution: context windows truncate or drown key facts; the agent may see "order 12345" but miss "it's a gift," or let old info override new. The paper calls this grounding decisions in *stale, missing, or incorrect information*. 2. Hard-to-check policies: constraints like "gift orders need supervisor approval" require the full task state, which is implicit and scattered. A syntactically correct tool call can still violate policy.

The Solution: An Explicit Ledger

LedgerAgent separates task state from the prompt, maintaining it in a structured ledger with three layers:

Facts: user intent, order number, amount, gift tag, delivery deadline, order status
Constraints: policy rules, system rules, user-stated requirements, implied rules (e.g., $150 > $100 triggers double confirmation)
Conditions: what's confirmed, pending, or on hold

Update: each user message, tool result, and policy check appends to the ledger. Render: before each decision, the ledger is structured into the prompt, highlighting unmet constraints.

Policy interception: after the agent generates a tool call but *before* execution, the system checks the ledger. In the gift example, it finds "gift order requires supervisor approval" → not obtained → blocks the call and returns an explanation, so the agent instead contacts a supervisor.

Experimental Results

Tested in four customer-service domains (e-commerce orders, flight changes, hotel booking changes, tech-support tickets) with a mixed panel of open-source (Llama, Mistral) and closed-source (GPT-4) models:

Higher task completion: average Pass@k improvements over standard prompting, largest on complex multi-turn tasks
Much better strict multi-turn consistency: explicit state management resolves the "forgetting" problem
Lower policy-violation rates: critical for legal risk and compliance

Why Structured State Is Inevitable

Explainability: the ledger is a white box—you can trace why a call was blocked, which matters for debugging, auditing, and compliance.
Long-term relationships: ledgers can persist across sessions, capturing preferences and history.
Structure over text: plain text is lossy; structured state avoids precision loss from repeated text extraction.

The post draws an analogy to cognitive science: the prompt is like limited, ephemeral human *working memory*, while the ledger acts as an external notebook—AI's version of *cognitive offloading*.

Limitations and Future Work

Limitations: ledger updates can still misclassify facts (e.g., detecting "gift"), incomplete policy constraints leave gaps, and per-call constraint checks add latency.

Future directions: automatically extracting constraints from policy documents, probabilistic state, multi-agent shared ledgers, and user-editable ledgers.

Conclusion

AI reliability depends not only on model scale but on system architecture. A ledger gives the agent structure (organized storage), persistence (no cross-session forgetting), auditability (transparent decision bases), and enforceable constraints (automatic policy compliance). Someday, when you ask "are you sure?", the agent may open its ledger and cite the exact line—rather than saying "I'm not sure I understand."

References

Uddin, M. N., et al. (2026). LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents. arXiv:2606.20529.
Schick, T., et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv:2302.04761.
Qin, Y., et al. (2023). Tool Learning with Foundation Models. arXiv:2304.08354.
Karpas, E., et al. (2022). MRKL Systems. AI21 Labs.