[论文] What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Deg...

论文概要

研究领域: NLP 作者: Congjing Zhang, Vashishtha Patil, Henning Lange, Usman Aleem 发布时间: 2026-09-15 arXiv: 2609.17515

中文摘要

剪枝可以降低大语言模型（LLM）的部署成本，但其对上下文感知工具调用的影响仍知之甚少。我们系统地研究了智能家居工具调用中剪枝引起的性能退化，涵盖4个LLM，横跨稠密Transformer、稠密混合架构和专家混合（MoE）架构，以及深度、宽度、混合和专家剪枝方法。在剪枝后监督微调（SFT）之后，我们评估了来自三个智能家居数据集的超过19,500个实例。除了总体任务准确率外，我们还沿两个维度刻画退化：动作组件（即操作、设备、参数和值）和任务复杂度。结果表明，稠密模型的安全剪枝区间较窄，随后急剧退化，而MoE模型能容忍更多的剪枝。剪枝首先破坏具体性（specificity），然后才是模式级别的意图，激进稠密剪枝甚至可能引发系统性过度拒绝。这些发现强调了在为可靠的工具执行选择剪枝LLM时，超越总体准确率进行评估的重要性。

原文摘要

Pruning can reduce the deployment cost of large language models (LLMs), but its impact on context-grounded tool calling remains poorly understood. We systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts (MoE) architectures, together with depth, width, hybrid, and expert pruning methods. After post-pruning supervised fine-tuning (SFT), we evaluate more than 19,500 instances from three smart-home datasets. Beyond aggregate task accuracy, we characterize degradation along two dimensions: action components (i.e., operation, device, argument, and value) and task complexity. Our results show that dense models have narrow safe pruning regions followed by sharp degradation, while MoE models tol...

#论文 #arXiv #NLP #小凯