Overview
This post explains Conversation Routines (CR), a prompt engineering framework from Giorgio Robino's paper *"Conversation Routines: A Prompt Engineering Framework for Task-Oriented Dialog Systems"* (arXiv:2501.11613v7). The core idea: instead of hard-coding dialog logic, domain experts write business processes as structured natural-language "scripts" that LLMs read, interpret, and execute—like actors improvising within a precise script.
Key points
From hard-coded flows to agentic systems
- Pre-LLM dialog systems were rigid, scripted pipelines; every path had to be pre-programmed.
- LLM-based Conversational Agentic Systems (CAS) invert this: as LangChain founder Harrison Chase puts it, "AI agents are systems that use LLMs to decide the control flow of an application."
- Instruction tuning (e.g., InstructGPT) gave models the ability to follow instructions, bridging natural language and program logic.
- OpenAI's SWARM framework inspired multi-agent designs with dynamic agent switching and context-aware handoffs.
- Implemented with GPT-4o-mini (128k context, function calling, low latency).
- Key design insight: stateless functions — page state for station search is tracked by the LLM in context, e.g.
search_railway_station(query, page), avoiding transaction-ID synchronization nightmares. - Dialogs handle fuzzy queries ("a station in Genova, I forget the name"), pagination, flexible time expressions, and mandatory YES/NO confirmation before booking to prevent hallucinated decisions.
- In 50+ test runs, booking succeeded except with deliberately uncooperative users; one full session took 3m41s and ~36,208 tokens.
- Reuses an existing human-readable 18-step conveyor-belt troubleshooting manual—CR lets the LLM execute the manual directly.
- Two-agent architecture: a troubleshooter (step-by-step guidance, part queries, four-phase dialog flow) and a report agent (structured intervention reports), connected via
handoff_report(). - Rigid safety rules: mandatory warnings before hazardous steps, explicit confirmation, never skipping preconditions (e.g., zero-voltage verification), and structured response formats.
- Handles incoherent input, mid-flow interruptions with seamless state resumption, and generates compliance-friendly reports.
The CR framework: eight structural elements
1. Role & goals — agent identity and user persona 2. Function integration protocol — which tools exist and when to use them 3. Workflow control patterns — sequential steps, conditional branching (IF/ELSE), iteration loops, and user confirmation checkpoints, expressed via systematic indentation similar to Markdown/YAML 4. Output format & tone — presentation and style guidelines"Soft compliance" lets the LLM handle digressions and fuzzy input while steering back to the workflow.
Case study 1: train ticket booking
Case study 2: industrial troubleshooting assistant
Comparison with other frameworks
| Framework | Approach | Trade-off vs CR | |---|---|---| | RASA CALM | LLM for intent understanding; deterministic DSL executes logic | Cheaper, deterministic, but YAML hard-coding excludes business experts | | LangGraph | Graph of Python nodes/edges with shared state | Fine-grained control and persistence, but logic lives in code | | Smolagents | LLM generates executable Python workflows | Lower overhead, but debugging generated code is hard |CR keeps business logic in prompts and deterministic execution in pre-built functions—clear role separation, easy editing—but sacrifices determinism, latency, and debuggability.
Future directions (from the paper)
1. Goal-based evaluation — two-stage LLM-judge scoring plus an evaluate–improve–validate loop (CI/CD for prompts) 2. Joint prompt/tool optimization — optimizing function signatures alongside CRs; higher-order CRs as "source code" for future CR compilers 3. Small LLMs for resource-constrained settings — e.g., Mistral Small 3 (23.6B) matching GPT-4o-mini on tool-selection quality (0.832) at lower cost, enabling on-premise deployment 4. Adaptive dialog design — intent recovery for off-topic queries, dynamic tone/expertise adaptation, risk-scaled confirmation granularity 5. Modular multi-agent CR engineering — role-based authoring pipelines, plus challenges like context drift, tone inconsistency, and inter-agent conflict resolutionConclusion
CR represents a move toward "natural language as code"—an executable specification readable by both humans and machines. Its legacy: accessibility for domain experts, flexible soft compliance, rapid iteration, and natural human-machine alignment. Its challenges: non-determinism, token/latency overhead, and debugging complexity—open research problems rather than disqualifying flaws.