[论文] StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanc...

论文概要

研究领域: NLP 作者: Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi 发布时间: 2025-06-01 arXiv: 2506.00631

中文摘要

数字硬件设计的RTL代码自动生成仍具挑战性，原因在于长时程推理、多步依赖关系以及Verilog和VHDL中的严格正确性约束。本文提出StepPRM-RTL，一种新颖框架，结合步进轨迹建模、过程奖励模型（PRM）和检索增强微调（RAFT），以提升基于LLM的RTL代码生成的功能正确性和推理保真度。StepPRM-RTL从规范解构建步进推理轨迹，每步包含原理说明和增量代码修改。过程奖励模型评估中间步骤，提供密集反馈以指导RAFT微调中的强化式更新。蒙特卡洛树搜索（MCTS）探索替代推理路径，用高质量轨迹丰富训练数据集。这种步进与结果感知奖励的整合，使模型既学习如何构建正确RTL，也理解为何如此构建，从而超越标准监督或基于结果的训练。在基准Verilog和VHDL数据集上的实验评估表明，StepPRM-RTL在功能正确性和推理保真度指标上优于最佳先前方法超过10%。消融研究确认PRM引导奖励与步进轨迹探索的结合是其性能的关键。StepPRM-RTL跨RTL语言泛化，为高保真、可解释代码生成提供可扩展框架，为LLM辅助硬件设计自动化建立新标准。

原文摘要

Automatic generation of RTL code for digital hardware designs remains challenging due to long-horizon reasoning, multi-step dependencies, and strict correctness constraints in Verilog and VHDL. We present StepPRM-RTL, a novel framework that combines stepwise trajectory modeling, process-reward modeling (PRM), and retrieval-augmented fine-tuning (RAFT) to enhance both the functional correctness and reasoning fidelity of LLM-based RTL code generation. StepPRM-RTL constructs stepwise reasoning trajectories from canonical solutions, where each step contains a rationale and incremental code modification. A Process Reward Model (PRM) evaluates intermediate steps, providing dense feedback that guides reinforcement-style updates during RAFT fine-tuning. Monte Carlo Tree Search (MCTS) explores alte...

--- *自动采集于 2026-06-05*

#论文 #arXiv #NLP #小凯