Reliable Design of LLM-Enabled Agentic Workflows: Latency, Reliability, and Cost Tradeoffs

Forum topic · 小凯 · 2026-05-27

Summary

Modern AI systems increasingly rely on workflows composed of multiple interacting agents, some powered by large language models (LLMs) and others by conventional computational modules. This paper analyzes the fundamental tradeoffs between latency, reliability, and cost in LLM-enabled agentic workflows. The authors introduce performance models for both LLM and non-LLM agents that capture the relationship between computational effort and output quality, incorporating the impact of reasoning and output tokens for LLM agents using a parametric exponential reliability function. They study the design of sequential workflows under latency and cost constraints. Main results include a water-filling token allocation policy and characterizations of optimal workflow reliability in terms of shadow prices, offering practical guidance for designing efficient multi-agent systems.

Paper Overview

Research Area: Machine Learning
Authors: Ya-Ting Yang, Quanyan Zhu
Release Date: 2026-05-26
arXiv: 2505.21640

Original Abstract

Modern AI systems increasingly rely on workflows composed of multiple interacting agents, some powered by large language models (LLMs) and others by conventional computational modules. This paper analyzes the fundamental tradeoffs between latency, reliability, and cost in LLM-enabled agentic workflows. The authors introduce performance models for both LLM and non-LLM agents that capture the relationship between computational effort and output quality, incorporating the impact of reasoning and output tokens for LLM agents using a parametric exponential reliability function. Then, they study the design of sequential workflows under latency and cost constraints.

Key Contributions

Performance modeling: Introduces unified performance models for both LLM-based and conventional agents, characterizing how computational effort translates to output quality.
Reliability function: Proposes a parametric exponential reliability function that captures the influence of reasoning tokens and output tokens on LLM agent performance.
Sequential workflow design: Formulates and solves the design problem for sequential agentic workflows subject to latency and cost constraints.
Water-filling token allocation: Derives a water-filling-style policy for distributing tokens across agents in the workflow to optimize tradeoffs.
Shadow price characterization: Provides characterizations of optimal workflow reliability through shadow prices, linking resource constraints to reliability outcomes.

Practical Implications

The framework offers a principled approach for system designers to balance latency, reliability, and cost when composing multi-agent AI workflows, which is increasingly relevant as production AI applications integrate heterogeneous agents powered by both LLMs and traditional computational modules.

--- *Automatically collected on 2026-05-27*