Hyper Input Convex Neural Networks for Shape-Constrained Learning and Optimal Transport

Forum topic · 小凯 · 2026-05-01

Summary

Researchers introduce Hyper Input Convex Neural Networks (HyCNNs), a novel architecture designed for learning convex functions. HyCNNs combine Maxout network principles with input convex neural networks (ICNNs), producing networks that are always convex in the input while theoretically leveraging depth, and they train more reliably at scale than ICNNs. The authors prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions to a given accuracy. Synthetic experiments show HyCNNs outperform existing ICNNs and MLPs on convex regression and interpolation tasks. The architecture is further applied to learning high-dimensional optimal transport maps, where it frequently outperforms ICNN-based neural optimal transport methods and other baselines on both synthetic examples and single-cell RNA sequencing data. Paper: arXiv 2504.20796 (April 2025).

Paper Overview

Field: Machine Learning
Authors: Shayan Hundrieser, Insung Kong, Johannes Schmidt-Hieber
Published: 2025-04-30
arXiv: 2504.20796

Abstract

We introduce Hyper Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions. HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliably when trained at scale compared to ICNNs. Concretely, we prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions up to a given accuracy. Throughout a series of synthetic experiments, we demonstrate that HyCNNs outperform existing ICNNs and MLPs in terms of predictive performance for convex regression and interpolation tasks. We further apply HyCNNs to learn high-dimensional optimal transport maps for synthetic examples and single-cell RNA sequencing data, where they often outperform ICNN-based neural optimal transport methods and other baselines across a broad range of settings.

Key Points

HyCNNs guarantee input convexity by design, combining Maxout networks with ICNNs.
Depth is theoretically exploitable, unlike shallow-limited ICNN formulations.
Exponentially fewer parameters are needed versus ICNNs for approximating quadratic functions to a given accuracy.
Empirically superior to ICNNs and MLPs on convex regression and interpolation benchmarks.
Strong results on high-dimensional optimal transport, including single-cell RNA sequencing data.

---

*Auto-collected on 2026-05-01*