Paper Overview
- Field: Machine Learning
- Authors: Yury Gorishniy, Akim Kotelnikov, Ivan Rubachev, Artem Babenko
- Posted: 2026-07-06
- arXiv: 2607.05380
- Problem with prior MLP ensembles: all member models share identical hyperparameters, so competitive performance requires costly per-dataset hyperparameter search.
- TabPack approach: in one run, it samples and trains many MLPs with varied hyperparameters in parallel, then dynamically picks ensemble members as training progresses.
- Out-of-the-box performance: default TabPack achieves results comparable to previously published methods that were extensively tuned, while needing far less compute and engineering effort on tabular tasks.
- Compute efficiency: running the default TabPack configuration on a modern MacBook can take less wall-clock time than tuning certain baselines on an industrial-grade GPU.
- Practical impact: lowers the barrier to deploying strong deep tabular models by combining ensembling, hyperparameter diversity, and dynamic member selection in a single efficient framework.
- Paper: https://arxiv.org/abs/2607.05380
Summary
In tabular deep learning, efficient ensembles of multilayer perceptrons (MLPs) have recently become a practical and effective architecture. Existing approaches use the same hyperparameters for all underlying MLPs, so reaching peak performance requires dedicated hyperparameter tuning.
This paper proposes TabPack, an efficient MLP ensemble with strong out-of-the-box performance that reduces dependence on traditional tuning. Within a single run, TabPack efficiently samples and trains multiple MLPs with diverse hyperparameters in parallel and dynamically selects ensemble members during training.