[论文] Relaxation-Informed Training of Neural Network Surrogate Models

论文概要

研究领域: ML 作者: Calvin Tsay 发布时间: 2025-04-28 arXiv: 2504.19770

中文摘要

作为替代模型训练的ReLU神经网络可以被精确嵌入混合整数线性规划（MILP），从而对所学函数进行全局优化。所得MILP的可解性取决于网络的结构特性。本研究针对下游MILP可解性设计了训练正则化项。我们提出了基于简单边界的正则化项，惩罚MILP公式中的big-M常数和/或不稳定神经元的数量。此外，我们引入了一种LP松弛间隙正则化项，显式惩罚训练点处连续松弛的逐样本间隙。实验表明，所提出的正则化项可将MILP求解时间比无正则化基线减少高达四个数量级，同时保持具有竞争力的替代模型精度。

原文摘要

ReLU neural networks trained as surrogate models can be embedded exactly in mixed-integer linear programs (MILPs), enabling global optimization over the learned function. The tractability of the resulting MILP depends on structural properties of the network. This work studies training regularizers that directly target downstream MILP tractability. We propose simple bound-based regularizers that penalize the big-M constants of MILP formulations and/or the number of unstable neurons. Moreover, we introduce an LP relaxation gap regularizer that explicitly penalizes the per-sample gap of the continuous relaxation at training points. Experiments demonstrate that the proposed regularizers can reduce MILP solve times by up to four orders of magnitude relative to an unregularized baseline, while m...

--- *自动采集于 2026-04-28*

#论文 #arXiv #ML #小凯