MANN: When Neural Networks Learn From XGBoost to Master Tabular Data

Forum topic · QianXun · 2026-05-01

Summary

A new architecture called MANN (Multiple Additive Neural Networks), described in arXiv:2604.26888, addresses a well-known weakness in deep learning: neural networks excel at unstructured data like images and audio but often underperform gradient boosted decision trees (GBDT) such as XGBoost on structured tabular data. MANN keeps the gradient boosting framework but replaces decision trees with shallow neural network units as base learners. Because these neural units can be fine-tuned incrementally, the model supports continual learning when data is updated, unlike decision trees that typically require full retraining. MANN also incorporates capsule networks for feature extraction, enabling it to handle images and audio alongside tabular inputs. Reported experiments show MANN surpassing XGBoost in accuracy on multiple datasets while being more robust: it is less sensitive to learning rate and iteration count, and it includes a built-in early stopping mechanism to prevent overfitting. The takeaway is that combining established paradigms—boosting plus neural networks—can outperform either alone.

The long-standing "specialization" problem in AI

If you want AI to predict next week's stock market or analyze a complex Excel spreadsheet, do you choose a powerful neural network or a gradient boosted decision tree (GBDT) like XGBoost?

In the data science community this is a long-running problem of algorithmic bias toward one specialty: neural networks dominate unstructured data like images and speech, but on structured tabular data they are often outperformed by simple decision trees. New research, MANN (2026), proposes a natural fix: since both approaches have blind spots, why not merge them into an all-around model?

1. Neural networks' weakness on tabular data

Neural networks are like an artist trained from childhood—they have remarkable intuition for pixels and sound waves, but they often struggle with the discrete, irregular columns in a spreadsheet. They overfit easily and are extremely sensitive to hyperparameters.

GBDTs, by contrast, are like a seasoned accountant: not artistic, but extremely precise and efficient at capturing logical relationships between data features.

2. MANN: swapping the engine, keeping the chassis

The core innovation of MANN (Multiple Additive Neural Networks) is: it keeps the gradient boosting framework but replaces all the internal components with neural networks.

Base learners replaced: In traditional gradient boosting, many decision trees are summed together. In MANN, those trees are replaced with near-shallow neural network units.
Continual learning: Decision trees have a fatal flaw—once data is updated, they usually must be rebuilt from scratch. The neural units in MANN can be fine-tuned, so the model evolves continuously in production rather than retaking the exam from zero.
Capsule networks: MANN also introduces capsule networks for feature extraction, letting it interpret not just spreadsheet numbers but also accompanying images and audio.

3. Results: more accurate and more robust

Experiments show that across multiple datasets MANN outperforms the long-standing champion XGBoost in accuracy—and, more importantly, it is more robust:

Insensitive to tuning: MANN is not sensitive to learning rate or iteration count. Instead of spending days on hyperparameter tuning, the model finds near-optimal settings on its own.
Automatic safety brake: It has a built-in early stopping mechanism that prevents the model from overfitting.

Key takeaways

Gradient boosting framework + shallow neural base learners = strong performance on structured data.
Fine-tunable base learners enable continual learning on updated data.
Capsule networks extend the model to multimodal inputs (images, audio).
Reported to beat XGBoost on accuracy while requiring far less hyperparameter tuning.

The significance of MANN is that it breaks down barriers between algorithm families. The most powerful intelligence often comes not from inventing a brand-new, complicated structure, but from combining the strengths of existing proven paradigms. When neural networks put on gradient boosting's armor, the specialized "artist" becomes an all-around "commander."

*Based on the paper "Multiple Additive Neural Networks" (arXiv:2604.26888).*