PAW: Turning Fuzzy Tasks into One-Line Runnable Code with Program-as-Weights

Key points

• Problem addressed: Fuzzy Functions. Many real-world tasks (log severity classification, image-editing detection, sentiment triage) resist precise rule-based programming; humans grasp them intuitively but cannot fully specify them in code. Traditional solutions either hard-code brittle rules or call a remote LLM API on every invocation, which is expensive, slow, and unstable across model updates.
• PAW's core idea. Treat a natural-language task description as *source code* and a compiler-produced LoRA weight file as an *executable*. Compile once with a large model, then run the artifact indefinitely on a tiny local interpreter without any API calls.
• Hybrid program structure. Each PAW program has two halves:
• *Discrete pseudo-program*: a normalized natural-language specification plus a few input-output examples. This makes the program human-readable, inspectable, and editable, and it conditions the interpreter on what task to perform.
• *Continuous parameters*: a LoRA adapter (around 38.5M parameters, roughly 23MB after quantization) injected into the attention and MLP layers of the 0.6B interpreter. LoRA provides modular, low-cost task adaptation without retraining the base model.
• Two-stage compiler.
• *Stage 1 — Pseudo compiler*: a 4B Qwen3 model used with hand-crafted prompting (no training) produces the discrete pseudo-program. The authors found reinforcement-learning training converged to the same format.
• *Stage 2 — LoRA compiler*: a trainable 4B Qwen3 consumes the user spec, pseudo-program, and 64 learnable prefix tokens, extracts hidden states, and maps them through a shallow MLP into LoRA mixing coefficients that produce the final adapter weights. The 0.6B interpreter is frozen during training; only the compiler and mapper are trained.
• Empirical results. On a representative fuzzy task, PAW with a 0.6B interpreter reached 73.78% accuracy, compared with:
• GPT-5.2 API: 96.09% (remote).
• Qwen3-32B direct prompting: 68.70% (about 60GB memory).
• Qwen3-0.6B direct prompting: 9.84% (about 1.2GB memory).
PAW therefore beats a 32B direct-prompting baseline while using roughly 1/50 of the memory. On an M3 MacBook the system runs at about 30 tokens/sec; a fully quantized stack needs only about 430MB of base model plus 23MB per task.
• Cross-modal extension. Swapping the text compiler for a vision-language compiler (Qwen3-VL-4B) while keeping the same 0.6B text interpreter lets image-conditioned tasks be encoded entirely inside the LoRA. Across six image fuzzy tasks, PAW exceeded 32B direct prompting on four and was close on the other two.
• Software-engineering fit. A compiled PAW program is a single 23MB file that can be version-controlled in Git, cached on a CDN, imported in Python, baked into a Docker image, or published to PyPI like any normal package. The paper frames this as AI-native software development: the model is the compiler, the weights are the program.
• Limitations. On classical NLP benchmarks with clear labels and abundant data (YouTube, SMS, Yelp, IMDB), PAW underperforms larger directly prompted models. The intended sweet spot is tasks where rule-based code is incomplete and labeled data is scarce.
• Takeaway. The expensive part of an AI application is not inference compute but repeatedly re-understanding the task. Compile once, run forever, fully offline, fully private.

Reference

• Zhang, W., Hotsko, L., Kim, W., Nie, P., Shieber, S., & Deng, Y. (2026). *Program-as-Weights: A Programming Paradigm for Fuzzy Functions*. arXiv:2607.02512 [cs.LG]. https://arxiv.org/abs/2607.02512
• Code and demos: https://github.com/programasweights and https://programasweights.com