Generative UI: When AI Agents Start Drawing Interfaces

*Structured English summary of a deep-dive forum post analyzing CopilotKit's generative-ui project and the emerging GenUI protocol ecosystem.*

Core idea

Traditional UIs are static and pre-defined — like a fixed restaurant menu. Generative UI flips this: the AI agent can generate, select, or control parts or all of the user interface in real time, sending UI state, structured UI specs, or complete interactive UI blocks that the frontend renders live. CopilotKit's open-source project (https://github.com/CopilotKit/generative-ui) frames it as "Build apps that adapt to your users."

Key points

Three modes on a control spectrum

1. Controlled Generative UI ("Lego mode") — Developers pre-build all components (weather cards, stock charts, approval forms). The agent only *selects* which component to show and passes data, e.g. via useFrontendTool with a render callback switching between WeatherLoadingState, WeatherCard, and WeatherError. Highest control, brand consistency, security, and performance; lowest freedom. Runs over AG-UI.

2. Declarative Generative UI ("3D-print mode") — The agent emits a structured JSON UI description rendered by local component libraries. Key protocols:

• A2UI (Google, JSONL-based): a component vocabulary (Card, Text, Button, TextField, List) describing *what* to show, not *how* it looks — the same spec renders natively on React, Flutter, or SwiftUI.
• Open-JSON-UI: standardized version of OpenAI's internal declarative UI pattern.
• Balanced flexibility, cross-platform, no code-execution risk, LLM-friendly; requires a shared component catalog.

3. Open-ended Generative UI ("free-sculpture mode") — The agent or a service returns complete UI surfaces (HTML or an iframe URL), rendered in a sandboxed iframe. Built on MCP Apps, an extension of Anthropic's Model Context Protocol where tools can declare UI resources via ui:// URIs and communicate with the host over JSON-RPC. Maximum freedom and server-side control; higher security and performance costs.

A fourth variant, Open Generative UI (useComponent hook), has the agent directly generate raw HTML/SVG/Canvas strings with no server round-trip.

The protocol stack

| Protocol | Layer | Role | |---|---|---| | AG-UI (CopilotKit) | Interaction | Real-time bidirectional Agent ↔ UI event stream (~16 event types: MESSAGE, TOOL_CALL_START/COMPLETE, STATE_SNAPSHOT/DELTA, RENDER, ACTION); transport-agnostic (SSE, WebSockets, webhooks) | | A2UI | Presentation | Declarative UI spec (the "score") | | MCP | Tools | Standardized tool/resource calls; MCP Apps adds UI resources | | A2A | Orchestration | Multi-agent coordination |

Key distinction: A2UI defines what UI to show; AG-UI defines how it flows between backend and frontend. AG-UI events can carry A2UI payloads. The post argues these protocols complement rather than compete — like subways, buses, and taxis in a city — and traces industry convergence: Oracle (Open Agent Specification), Google (A2UI), CopilotKit (AG-UI), Anthropic (MCP), OpenAI, Microsoft, and AWS all cooperating on plug-and-play portability.

Case studies

• Excalidraw MCP Apps: user says "draw a client-server architecture diagram"; the agent calls the Excalidraw MCP server's create_view tool, which returns an interactive editor loaded in a sandboxed iframe — editable in-chat, pushable to Excalidraw.
• OpenGenerativeUI: a LangGraph agent directly generates HTML/SVG/Canvas (algorithm visualizations, Three.js 3D scenes, D3 force graphs) rendered via a widgetRenderer component registered with useComponent, in a sandboxed iframe.

Developer guidance

• Mode selection: strict brand control → Controlled; dynamic cross-platform forms/lists → Declarative (A2UI); external professional services or highly custom interactions → Open-ended/MCP Apps.
• Security: sandbox iframes (allow-scripts, avoid allow-top-navigation), apply CSP, explicit user-consented permissions, sanitize generated HTML (DOMPurify), block eval()/new Function().
• Performance: Controlled renders fastest (pre-compiled components); Open-ended is slowest (iframe loading, independent context). Mitigate with preloading, lazy loading, caching MCP UI resources, fallback/degraded UI, and streaming A2UI rendering.
• Debugging: AG-UI DevTools for event streams, MCP Inspector for tool calls, trace IDs across the full request chain.

Outlook

Generative UI is early-stage but directionally clear: smarter (multimodal, pixel-accurate) generation, deeper personalization (accessibility-aware interfaces), seamless cross-platform rendering from one spec, and multi-agent collaboration via A2A producing unified interfaces.

Reference resources

• CopilotKit generative-ui: https://github.com/CopilotKit/generative-ui
• AG-UI docs: https://docs.ag-ui.com
• A2UI spec: https://a2ui.org
• MCP Apps: https://modelcontextprotocol.io/extensions/apps/overview
• OpenGenerativeUI: https://github.com/CopilotKit/OpenGenerativeUI
• Excalidraw MCP integration: https://github.com/CopilotKit/excalidraw-studio
• A2UI + Agent Spec example: https://github.com/CopilotKit/with-agent-spec
• Oracle Open Agent Specification: https://blogs.oracle.com/ai-and-datascience/post/open-agent-specification