[论文] What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Deg...

论文概要

研究领域: NLP
作者: Congjing Zhang, Vashishtha Patil, Henning Lange, Usman Aleem
发布时间: 2026-09-15
arXiv: 2609.17515

中文摘要

剪枝可以降低大语言模型（LLM）的部署成本，但其对上下文感知工具调用的影响仍知之甚少。我们系统地研究了智能家居工具调用中剪枝引起的性能退化，涵盖4个LLM，横跨稠密Transformer、稠密混合架构和专家混合（MoE）架构，以及深度、宽度、混合和专家剪枝方法。在剪枝后监督微调（SFT）之后，我们评估了来自三个智能家居数据集的超过19,500个实例。除了总体任务准确率外，我们还沿两个维度刻画退化：动作组件（即操作、设备、参数和值）和任务复杂度。结果表明，稠密模型的安全剪枝区间较窄，随后急剧退化，而MoE模型能容忍更多的剪枝。剪枝首先破坏具体性（specificity），然后才是模式级别的意图，激进稠密剪枝甚至可能引发系统性过度拒绝。这些发现强调了在为可靠的工具执行选择剪枝LLM时，超越总体准确率进行评估的重要性。

原文摘要

Pruning can reduce the deployment cost of large language models (LLMs), but its impact on context-grounded tool calling remains poorly understood. We systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts (MoE) architectures, together with depth, width, hybrid, and expert pruning methods. After post-pruning supervised fine-tuning (SFT), we evaluate more than 19,500 instances from three smart-home datasets. Beyond aggregate task accuracy, we characterize degradation along two dimensions: action components (i.e., operation, device, argument, and value) and task complexity. Our results show that dense models have narrow safe pruning regions followed by sharp degradation, while MoE models tol...

自动采集于 2026-09-17

#论文 #arXiv #NLP #小凯