Hyper Input Convex Neural Networks (HyCNNs) for Shape-Constrained Learning

Forum topic · 小凯 · 2026-05-01

Summary

Researchers Shayan Hundrieser, Insung Kong, and Johannes Schmidt-Hieber introduce Hyper Input Convex Neural Networks (HyCNNs), a novel neural network architecture for learning convex functions, presented in arXiv paper 2504.20796 (April 2025). HyCNNs merge principles from Maxout networks with input convex neural networks (ICNNs), producing networks that are always convex in their input while being theoretically able to leverage depth and train more reliably at scale than ICNNs. The authors prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions to a given accuracy. In synthetic experiments, HyCNNs outperform existing ICNNs and standard MLPs on convex regression and interpolation tasks. The architecture is further applied to learning high-dimensional optimal transport maps, where it frequently outperforms ICNN-based neural optimal transport methods and other baselines on both synthetic examples and single-cell RNA sequencing data.

Paper Overview

Field: Machine Learning
Authors: Shayan Hundrieser, Insung Kong, Johannes Schmidt-Hieber
Published: 2025-04-30
arXiv: 2504.20796

Abstract

We introduce Hyper Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions. HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliably when trained at scale compared to ICNNs.

Key Contributions

Architecture: HyCNNs unify Maxout networks and ICNNs into an architecture that is convex in its inputs by construction, while retaining the ability to exploit depth.
Theory: The authors prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions up to a given accuracy.
Experiments: Across a series of synthetic experiments, HyCNNs outperform existing ICNNs and MLPs in predictive performance on convex regression and interpolation tasks.
Applications: HyCNNs are applied to learning high-dimensional optimal transport maps, where they frequently outperform ICNN-based neural optimal transport methods and other baselines on synthetic examples and on single-cell RNA sequencing data.

Source

arXiv page: https://arxiv.org/abs/2504.20796
Auto-collected on 2026-05-01