Paper Overview
Field: Machine Learning Authors: Nan Zheng, Hoi Yiu Cheung, Vibhu Sharma Published: 2025-09-01 arXiv: 2509.00146
Summary
Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs.
This work introduces a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function together with its exact gradients, eliminating the need for manual derivations or ad-hoc approximations.
The authors demonstrate the efficiency, flexibility, and statistical performance of TMB-based NMMs in two numerical examples, including an application of monotone NMMs. Reproducible code is provided to facilitate broader adoption.
Key Takeaways
- TMB removes the burden of manually deriving objectives and gradients for NMMs.
- Laplace approximation handles integration over random effects automatically.
- Validated on two numerical examples, including a monotone NMM case.
- Reproducible code is available to support adoption.