PAW (Program-as-Weights): Turning "Undescribable" Tasks into One Runnable Line of Code
> arXiv:2607.02512 | University of Waterloo / Cornell / Harvard > Authors: Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng
This post is a structured English summary of a detailed Chinese-language review of the PAW paper.
The Problem: Fuzzy Functions
Consider writing a function that decides whether a log line is "important." You might try rules — ERROR level, keywords like "failed" or "critical," frequency thresholds — but edge cases always break them: a typo like eror slips through, a benign INFO: critical service heartbeat triggers false alarms, and a 3 a.m. backup-failure alert turns out not to matter. The paper calls these fuzzy functions: tasks humans understand intuitively but that cannot be completely captured with precise symbols. The usual alternatives — brittle rules, or calling a remote LLM API every time (expensive, slow, vulnerable to silent model updates) — both fall short. PAW's insight: rather than querying a large model per call, have it build the capability once, then run it locally forever.
Core Idea: Program as Weights
- Traditional programming: you write Python; the interpreter executes line by line.
- LLM API: you re-teach the model the task on every request.
- PAW: describe the task once in natural language; a compiler (an LLM) translates it into tuned parameters. A tiny 0.6B interpreter model then executes the task locally — no network, no API, no code.
- Stage 1 — Pseudo compiler: a Qwen3-4B model, prompt-only, no training, rewrites the user's request into a normalized spec with examples. The authors tried RL training but found the model converges to the same format anyway.
- Stage 2 — LoRA compiler: a trained Qwen3-4B that reads the spec plus 64 learnable prefix tokens, extracts hidden states, maps them via a shallow MLP into LoRA mixing coefficients, and produces the final LoRA weights. The interpreter stays frozen; only the compiler and mapper are trained.
- 0.6B model running PAW beats a 32B model with direct prompting: 73.78% vs 68.70% accuracy, at ~1.2GB vs ~60GB memory (1/50 the memory).
- The same 0.6B model with direct prompting scores only 9.84% — capability depends on *how* the model is used, not just its size.
- GPT-5.2 API reaches 96.09% but requires remote inference.
- Fully local: 30 tokens/s on a MacBook M3; quantized system is a 430MB base model plus a 23MB per-task LoRA; runs in the browser, fully offline.
- Cross-modal: swapping the compiler for a vision-language model (Qwen3-VL-4B) keeps the same 0.6B text interpreter; image conditions are entirely encoded in the LoRA. PAW beats 32B direct prompting on 4 of 6 image-conditioned fuzzy tasks (e.g., portrait emotion detection, retouching detection).
- Direct prompting forces a model to simultaneously understand the task, the solution strategy, and the input. PAW pre-compiles task understanding into weights, so the interpreter only executes.
- LoRA's modularity means one interpreter serves unlimited PAW programs — swap the score, not the piano.
- The authors are candid about limits: on conventional NLP classification tasks (YouTube, SMS, Yelp, IMDB), PAW underperforms larger directly-prompted models, because those tasks have clear rules and abundant labels. PAW's advantage is precisely where rules are incomplete and labels scarce.
- Zhang, W., Hotsko, L., Kim, W., Nie, P., Shieber, S., & Deng, Y. (2026). Program-as-Weights: A Programming Paradigm for Fuzzy Functions. arXiv:2607.02512 [cs.LG]. https://arxiv.org/abs/2607.02512
- Code and demo: https://github.com/programasweights / https://programasweights.com
One-line summary: *natural-language descriptions are source code, compiled LoRA weights are the executable, and the 0.6B model is the CPU. Compile once, run millions of times.*
Hybrid Programs: Specification + Muscle Memory
A PAW program has two halves:
1. Discrete pseudo-program (the "manual"): a normalized task description plus input/output examples. It lets users inspect and debug the program, and tells the interpreter what task to perform. 2. Continuous parameters (LoRA weights, ~38.5M params, 23MB quantized): injected into the 0.6B interpreter's attention and MLP layers, encoding fine-grained behavior that text cannot express.
LoRA keeps things modular: instead of modifying multi-GB base weights per task, small swappable adapters plug into one shared interpreter.
Two-Stage Compilation
Key Results
Software Engineering Perspective
A compiled PAW program is a single 23MB file that can be version-controlled with Git, cached on CDNs, imported in Python, shipped in Docker images, or published on PyPI — the paper's own words: "A PAW program is a single file that can be cached, version-controlled, and distributed like any other Python module." Models become compilers; weights become programs.