1flowbase Deep Dive: A Virtual Model Gateway for Local AI Agents with Observable Workflow Orchestration

Forum topic · 小凯 · 2026-06-14

Summary

1flowbase is an open-source virtual model gateway for local AI agent clients, built in Rust (Axum) with a TypeScript frontend and PostgreSQL storage, licensed under Apache-2.0. Unlike model routers such as LiteLLM, it composes multiple models, tools, verifiers, and formatters into a single workflow, then publishes that workflow as a standard OpenAI- or Claude-compatible API endpoint. Clients simply call a virtual model name, while developers get node-level tracing of inputs, outputs, tokens, latency, and costs. Typical use cases include vision-then-reasoning cascades that cut token costs by an estimated 50-70%, model cascades balancing quality and cost, programmable upstream models for coding agents like Claude Code and Cursor, and JSON Schema-validated output pipelines. The architecture follows a clean domain-driven design with separate control-plane, orchestration-runtime, publish-gateway, plugin framework (QuickJS sandbox via RquickJS), storage, and observability crates. Self-hosted by default with Docker Compose, the project remains early-stage (~110 stars, 36 open issues) and is not positioned as an MCP gateway yet. It fills a gap between model routing and AI app building: packaging workflows as transparent, auditable, cost-optimized virtual model endpoints.

Overview

1flowbase (https://github.com/taichuy/1flowbase) is an open-source, self-hosted virtual model gateway for local AI agent clients. Created by the taichuy organization, it is written in Rust (Axum + Tokio) with a pnpm/Turbo frontend and PostgreSQL persistence, licensed under Apache-2.0. As of the article it had ~110 stars, 10 forks, and 36 open issues.

Its core thesis: LiteLLM routes models; 1flowbase composes models into workflow-backed virtual model endpoints.

Why It Exists

A trivial user input can balloon once system prompts, tool definitions, chat history, and intermediate calls stack up. 1flowbase's motto: *"The first step to harness is to see the Agent's execution path clearly."* It surfaces the full execution path so teams can optimize on real runtime data rather than guesses.

The Five-Step Loop

1. Build a workflow (visual editor): model nodes, tool nodes, verifiers, formatters, routers 2. Publish as a model endpoint — compatible with:

/v1/responses (OpenAI Responses API)
/v1/chat/completions (OpenAI Chat Completions)
/v1/messages (Claude Messages API)

Call from clients

gpt-4o-virtual

Inspect execution

Optimize and reuse

Architecture

A Cargo workspace with clean domain-driven layering:

domain — core workflow/node/execution entities
control-plane — workflow CRUD, versioning, configuration
orchestration-runtime + runtime-core — execution engine, node execution, payload building
runtime-profile — token stats and latency tracing
publish-gateway — OpenAI/Claude protocol adaptation and virtual endpoint exposure
storage-durable / storage-ephemeral / storage-object — PostgreSQL, cache, S3-compatible storage
observability — logging, tracing, metrics

Notable tech choices: SQLx 0.8 (compile-time checked SQL), Utoipa 5 (OpenAPI 3.1), Moka 0.12 (caching), and RquickJS — an embedded QuickJS engine powering a sandboxed JS/TS plugin system with HostExtension interfaces to the Rust host.

Representative Use Cases

Vision-first, then text reasoning: small OCR/vision node → strong text model → verifier. Estimated 50–70% token cost reduction via cascading.
Model cascades: tiny models (e.g., Qwen3-0.6B) for classification, mid-size for analysis, large models only for complex reasoning, plus a verifier.
Programmable upstream for coding agents (Claude Code, Cursor, Windsurf): wrap generate → test → review → fix into a single virtual model with per-node debugging.
Schema-guaranteed output: LLM → JSON Schema validator → formatter, with retry on invalid output.

How It Differs from Competitors

| Competitor | Difference | |---|---| | LiteLLM | Routing vs. orchestration; request-level vs. node-level observability. Complementary — LiteLLM can route underneath. | | Dify / Flowise | App platform vs. infrastructure gateway; 1flowbase exposes full OpenAI/Claude API compatibility. | | LangGraph / CrewAI | Code-defined agents vs. visual orchestration published as a service. |

Maturity and Risks

Working today: visual editor, multi-node workflows, three protocol endpoints, streaming, execution logs, tool-call tracing, app-level token stats, prompt/model config version history.

In development: Token Bill of Materials (node-level cost attribution), session search/replay, anomaly cost detection, Claude Code/Codex templates, MCP-aware plugin nodes.

Risks:

Very young project (3 months, small community, 36 open issues) — cautious evaluation needed for production
Rust + modern TS stack raises operational bar (Node.js >= 24, latest stable Rust, Docker)
Plugin ecosystem immature; no marketplace yet
Explicitly not an MCP gateway today, while MCP is becoming a standard

Strengths: self-hosted-first (data stays local), no stealthy model replacement (everything auditable), clean modular architecture.

Verdict

1flowbase fills a real gap between model routing and AI app building: packaging multi-model workflows into standard, transparent, cost-observable virtual model endpoints. Worth watching for teams running local agents (Claude Code, Cursor, etc.) who need execution-path visibility, cost optimization through cascades, or self-hosted compliance.

Project: https://github.com/taichuy/1flowbase