TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning

Forum topic · 小凯 · 2026-07-08

Summary

TabPack is a new approach for efficient MLP ensembles in tabular deep learning, proposed by Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, and Artem Babenko (arXiv:2607.05380). While ensembles of MLPs have become an effective and practical architecture for tabular data, existing methods use identical hyperparameters for all member models and rely on hyperparameter tuning for best performance. TabPack instead delivers strong out-of-the-box performance with reduced dependence on traditional tuning. In a single run, it efficiently samples and trains multiple MLPs with different hyperparameters in parallel, dynamically selecting ensemble members during training. Experiments show that TabPack with default settings matches extensively tuned prior methods, substantially reducing the effort and compute needed to achieve competitive results on tabular tasks. Notably, the default TabPack configuration runs on a modern MacBook faster than tuning certain baselines on industrial-grade GPUs. The paper is available at https://arxiv.org/abs/2607.05380.

Paper Overview

Research Area: ML Authors: Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, Artem Babenko Published: 2026-07-06 arXiv: 2607.05380

Abstract

In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as an effective and practical architecture. Existing approaches apply the same hyperparameters to all underlying MLPs and require hyperparameter tuning to achieve the best performance.

This paper proposes TabPack, an efficient MLP ensemble with strong out-of-the-box performance that reduces reliance on conventional tuning. In a single run, TabPack efficiently samples and trains multiple MLPs with different hyperparameters in parallel, and dynamically selects ensemble members during training.

Key Findings

TabPack with default settings performs on par with extensively tuned prior methods.
It significantly reduces the effort and computational resources required to achieve competitive results on tabular tasks.
Remarkably, the default TabPack configuration runs on a modern MacBook faster than tuning certain baselines requires on industrial-grade GPUs.

---

*Auto-collected on 2026-07-06*