The chip design industry has an open secret: AI models that generate RTL code perform well in academic benchmarks but see almost no real-world adoption in industry. The reason is not that the models write bad code — it is a fundamental mismatch between real industrial workflows and academic evaluation setups.
Academic benchmarks assume a perfect golden testbench exists *before* the model writes code. In reality, testbenches are developed alongside the design, and no golden reference exists until the code is done. Academic evaluation assumes the model can call closed-source APIs. In reality, chip vendors' servers are physically isolated from the outside world, with no outbound data allowed. Academic evaluation assumes the model cannot be fine-tuned on a vendor's proprietary codebase. In reality, every chip company has decades of accumulated proprietary RTL, but that data can neither be uploaded to the cloud nor used for training.
Yu, Lin, and team's ChipMATE changes this paradigm from the ground up. The core design uses two agents: one writes Verilog (the hardware description language), the other writes a Python reference model (a software behavioral description). After the Verilog agent writes a piece of RTL, the Python agent independently writes a corresponding reference model. The two are compared against each other — not against an external golden reference, but serving as each other's validation standard. This mirrors a common industrial practice: two engineers independently design the same module, then cross-verify.
Training Pipeline
Training has two stages, both driven by reinforcement learning:
1. Stage 1 — Independent training: each agent's code-generation capability is saturated to its limit. 2. Stage 2 — Joint training: the team learns to collaborate — one agent anticipates the other's needs, writing code that is easy to verify and reference models that are easy to understand.
At inference time there is also a backtracking mechanism: if the first agent's output causes the second agent's verification to fail, the workflow does not proceed — instead it backtracks to the previous decision point and regenerates. This prevents errors from propagating across turns.
Results
- 75.0% pass@1 on VerilogEval V2 with a 4B base model
- 80.1% pass@1 with a 9B model
- All records for self-trained models broken
- Notably, the 9B ChipMATE even surpasses DeepSeek V4, a general-purpose 1600B-parameter model — two specialized agents totaling under 20B parameters beating a 1600B general-purpose giant.
Open Questions
The paper's training data comes from 64.4K samples generated by the framework itself — how is their quality guaranteed? If the reference-model agent has itself learned wrong patterns, cross-validation could degenerate into cross-confirmation. Additionally, the training stability of multi-agent RL has been disputed in similar systems — could collapse modes emerge, where the two agents settle into a low-quality mutual understanding?
---
References
1. Yu, Z., Lin, Y., Zhou, C., et al. (2026). *ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation*. arXiv:2605.12857 [cs.MA]. 2. Thakur, S., et al. (2024). *VerilogEval: Evaluating Large Language Models for Verilog Code Generation*. DAC. 3. Wu, M., et al. (2025). *RTLCoder: Fully Open-Source and Efficient LLM for RTL Code Generation*. DATE.