Paper Overview
Research Area: Machine Learning
Authors: Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, Artem Babenko
Published: 2026-07-06
arXiv: 2607.05380
Summary
In tabular deep learning, efficient ensembles of multi-layer perceptrons (MLPs) have recently emerged as an effective and practical architecture. Existing methods apply the same hyperparameters to all underlying MLPs and require hyperparameter tuning to achieve optimal performance.
This paper proposes TabPack, an efficient MLP ensemble with strong out-of-the-box performance that reduces reliance on conventional tuning. In a single run, TabPack efficiently samples and trains multiple MLPs with different hyperparameters in parallel, and dynamically selects ensemble members during training.
Experiments demonstrate that TabPack with default settings performs on par with extensively tuned prior methods, significantly reducing the effort and computational resources needed to achieve competitive results on tabular tasks. Notably, running the default TabPack configuration on a modern MacBook takes less time than tuning some baselines on industrial-grade GPUs.
Key Points
- Introduces TabPack, an efficient MLP ensemble method for tabular deep learning
- Unlike prior ensembles that share one hyperparameter setting, TabPack trains MLPs with diverse hyperparameters
- Sampling, parallel training, and dynamic member selection all happen in a single run
- Default configuration matches heavily tuned baselines
- Runs on a MacBook faster than tuning some baselines on industrial GPUs
*Auto-collected on 2026-07-06*