Summary
A 2025 arXiv paper (2509.00146) by Nan Zheng, Hoi Yiu Cheung, and Vibhu Sharma introduces a general framework for implementing neural network mixed-effects models (NMMs) using Template Model Builder (TMB). NMMs combine the representation and predictive power of artificial neural networks with mixed-effects modeling's ability to capture complex correlation structures, but existing estimation methods rely heavily on manual derivations of objective functions and gradients, forcing simplifying approximations that limit model complexity and accuracy. The proposed TMB-based framework leverages automatic differentiation and the Laplace approximation: users need only specify the negative joint log-likelihood and any regularization terms, while the framework automatically integrates out random effects and evaluates the marginal objective function with exact gradients. The authors demonstrate efficiency, flexibility, and statistical performance through two numerical examples, including an application of monotone NMMs, and provide reproducible code to encourage adoption.
Paper Overview
- Field: Machine Learning
- Authors: Nan Zheng, Hoi Yiu Cheung, Vibhu Sharma
- Published: 2025-09-01
- arXiv: 2509.00146
Summary
Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs.
This work introduces a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function along with its exact gradients, eliminating the need for manual derivations or ad-hoc approximations.
Highlights
- General TMB-based framework for fitting neural network mixed-effects models
- Automatic differentiation + Laplace approximation replaces manual gradient derivation
- Users specify only the negative joint log-likelihood and regularization terms
- Random effects are integrated out automatically; exact gradients of the marginal objective are computed
- Demonstrated on two numerical examples, including an application of monotone NMMs
- Reproducible code is provided by the authors
> Original abstract: "Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs. In this work, we introduce a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and e..."
Full paper: https://arxiv.org/abs/2509.00146
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178634402