VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design

Forum topic · 小凯 · 2026-07-22

Summary

VEHBench is an engineering-native diagnostic benchmark for evaluating large language models (LLMs) in vibration energy harvester (VEH) design, a task central to battery-free IoT where iterative design must respect coupled physical constraints. The benchmark contains 763 literature-grounded tasks, scored by analytical physics oracles, and evaluates four distinct design roles: specification classification, verifier-guided search, damaged-state recovery, and policy-conditioned selection. Unlike existing engineering benchmarks that focus mainly on end-product effectiveness, VEHBench examines LLM behavior at individual stages of coupled physical design. Experiments reveal that LLM capability is strongly stage-dependent: no single model dominates the entire workflow, and response-control profiles expose distinct behavioral patterns across design roles. VEHBench provides a stage-aware foundation for evaluating, selecting, routing, and improving verifier-grounded engineering LLMs. The paper (arXiv:2607.18181) is authored by Depeng Su, Yuyu Luo, and Guobiao Hu, in the NLP domain (cs.CL, cs.SE).

论文 Overview

Field: NLP Authors: Depeng Su, Yuyu Luo, Guobiao Hu Published: 2026-07-20 arXiv: 2607.18181 Categories: cs.CL, cs.SE

Introduction

Battery-free IoT systems require iterative design of vibration energy harvesters (VEH) under coupled physical constraints, and LLMs are increasingly serving as the interface layer in engineering workflows. However, existing engineering benchmarks primarily evaluate the effectiveness of final artifacts, offering limited insight into LLM behavior at different stages of coupled physical design.

Key Contributions

The authors introduce VEHBench, an engineering-native diagnostic benchmark for LLM-assisted VEH design:

763 literature-grounded tasks, scored by analytical physics oracles
Evaluates four design roles:

Findings

Experimental results show that LLM capability is strongly stage-dependent:

No single model consistently dominates across the entire workflow
Response-control profiles expose distinct behavioral patterns across different design roles

Significance

VEHBench provides a stage-aware foundation for evaluating, selecting, routing, and improving verifier-grounded engineering LLMs.

---

*Auto-collected on 2026-07-22*