When AI Learns Comedy: How the COMIC System from University of Washington Generates Sketch Comedy Videos

Forum topic · 小凯 · 2026-03-12

Summary

This article explains COMIC (Agentic Sketch Comedy Generation), a system proposed by four University of Washington researchers (Susung Hong, Brian Curless, Ira Kemelmacher-Shlizerman, Steve Seitz) that enables AI to write, direct, and produce complete sketch comedy videos. The system's core innovations include: multi-agent collaboration, where separate AI agents act as screenwriter, director, performer, and judge; evolutionary algorithms that generate large populations of creative candidates and iteratively select the funniest; and a key alignment technique—an LLM-based judge trained on YouTube sketch comedy data paired with real audience signals (views, likes, comments) so the system learns what genuinely makes humans laugh. COMIC also integrates state-of-the-art video generation models, maintains character reference images for consistency across scenes, and uses lip-sync technology for natural dialogue. According to the paper, human evaluations show COMIC's output approaches professionally produced comedy quality and achieves state-of-the-art video generation performance. The article discusses why humor is hard for AI, the implications of agentic creative automation, personalized content generation, and human-AI collaboration. Note: some details in this forum post are explanatory interpretations by the author rather than verified paper content.

When AI Learns Comedy: A Guide to the COMIC Sketch Comedy Generation System

This is an English translation of a Chinese tech forum explainer post about the COMIC system for AI-generated sketch comedy.

Why Is Humor So Hard for AI?

Humor is one of the most complex human cognitive abilities. Laughing at a comedy sketch requires:

Understanding surface meaning of dialogue
Catching puns and subtext
Predicting where the story is going
Experiencing cognitive conflict when the punchline subverts expectations
Resolving that conflict pleasurably — the laugh

Traditional large language models are essentially powerful text predictors. When asked to tell a joke, they fill in the statistically likely pattern of a joke, without a genuine comedic core. Worse, humor is deeply subjective and culture-dependent, shifting over time. And producing comedy *video* is harder still, requiring scriptwriting, performance timing, camera language, sound design, and character consistency — normally a whole professional team's work.

The COMIC System: An AI Comedy Studio

COMIC (Agentic Sketch Comedy Generation) was proposed by four University of Washington researchers: Susung Hong, Brian Curless, Ira Kemelmacher-Shlizerman, and Steve Seitz.

Multi-Agent Collaboration

COMIC mirrors a real comedy production company with specialized AI agents:

Screenwriter agent: writes scripts, dialogue, and plot
Director agent: controls pacing, shots, and scene transitions
Performer agent: generates character actions and performance
Judge agent: evaluates whether the material is funny

These agents debate, propose alternatives, critique each other, and iterate — like a human creative team.

Competition and Evolution

COMIC also applies evolutionary ideas: it generates a large "population" of creative candidates, the judge agent filters them (eliminating stale jokes, keeping good twists), and surviving ideas combine to produce new variants. Only the funniest material survives multiple rounds of selection.

Iterative Refinement

The system repeatedly refines its output based on judge feedback — strengthening opening hooks, tightening comedic rhythm, and improving punchlines across successive rounds.

Key Innovation: Teaching AI What Humans Find Funny

The most subtle component is the judge agent. Rather than having AI guess at humor in the abstract, the researchers trained an LLM-based judge on:

1. A large collection of YouTube sketch comedy videos 2. Extracted features: script structure, dialogue patterns, performance style, pacing 3. Real audience signals: view counts, likes, comment sentiment, watch-through rates

This learned judge captures not a theoretical definition of humor but which feature combinations actually make human audiences laugh — an example of preference alignment, calibrating the AI's "sense of humor" to real viewers. Without this, the system would produce text that is fluent and structurally sound but simply not funny to humans.

From Script to Full Video

COMIC aims at complete comedy videos, integrating state-of-the-art video generation models (the post mentions systems like Sora, Runway Gen-4, and Kling AI). Key technical challenges addressed include:

Character consistency: maintaining character reference images so the same character looks identical across scenes
Narrative coherence: the director agent plans storyboards, and scene generation references adjacent scenes for smooth transitions
Voice and lip sync: speech synthesis with synchronized lip movement so characters appear to genuinely speak

Results

According to the paper's experiments:

COMIC-generated sketches approach professionally produced quality, judged by human evaluators watching the output
The system achieves state-of-the-art performance in video generation on quality, coherence, and expressiveness
Evaluations used human judgments rather than AI self-scoring

> Note: the interpretive framing in this post (e.g., details of the agent roles and training pipeline) is the forum author's popular-science explanation; consult the original paper for exact specifications.

What This Means

Creative automation: humor, long considered a human stronghold, is now partially reachable by well-engineered AI systems
Multi-agent paradigm: collaborative agent teams may extend to novel writing, architectural design, and scientific research
Preference alignment: training evaluators on real human behavioral data (like YouTube reactions) is a practical alignment strategy applicable to music, design, and decision-making

Future Outlook

Personalized comedy: on-demand sketches tailored to individual tastes
Democratized creation: anyone with an idea could produce professional-grade video comedy
Human-AI collaboration: humans supply creative direction and emotional core; AI handles execution and iteration

References

1. Hong, S., Curless, B., Kemelmacher-Shlizerman, I., & Seitz, S. (2026). COMIC: Agentic Sketch Comedy Generation. *arXiv preprint arXiv:2603.11048*. 2. OpenAI. (2025). Sora: A generative video model. *OpenAI Technical Report*. 3. Qian, C., et al. (2024). ChatDev: Communicative Agents for Software Development. *ACL*. 4. Chan, C. M., et al. (2023). ChatEval: Towards better LLM-based evaluators through multi-agent debate. *NeurIPS*. 5. Fernando, C., et al. (2023). Promptbreeder: Self-referential self-improvement via prompt evolution. *ICLR*.