Implementing Neural Network Mixed-Effects Models in Template Model Builder (TMB): arXiv 2509.00146

Forum topic · 小凯 · 2026-09-02

Summary

This forum post shares arXiv paper 2509.00146, "Implementing neural network mixed-effects models in Template Model Builder" by Nan Zheng, Hoi Yiu Cheung, and Vibhu Sharma (September 2025). Neural network mixed-effects models (NMMs) combine the representation and predictive power of artificial neural networks with mixed-effects modeling's ability to capture complex correlation structures. Existing estimation approaches rely heavily on manual derivation of objective functions and gradients, forcing simplifying approximations that limit model complexity and accuracy. The paper introduces a general framework implementing NMMs in Template Model Builder (TMB), which leverages automatic differentiation and Laplace approximation. Users only need to specify the negative joint log-likelihood and regularization terms; the framework automatically integrates out random effects and evaluates the marginal objective and its exact gradients. Efficiency, flexibility, and statistical performance are demonstrated on two numerical examples, including a monotonic NMM application, with reproducible code provided.

论文概要

Research Area: Machine Learning Authors: Nan Zheng, Hoi Yiu Cheung, Vibhu Sharma Published: 2025-09-01 arXiv: 2509.00146

English Translation (Abstract)

Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs.

In this work, the authors introduce a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function and its exact gradients, eliminating the need for manual derivations or ad-hoc approximations.

The efficiency, flexibility, and statistical performance of the TMB-based NMM framework are demonstrated in two numerical examples, including an application of monotonic NMMs. Reproducible code is provided to facilitate broader adoption.

Key Points

NMMs blend neural network representation power with mixed-effects correlation modeling
Prior methods require manual objective/gradient derivations, forcing simplifying approximations
The proposed TMB framework needs only the negative joint log-likelihood and regularization terms from the user
Automatic differentiation plus Laplace approximation handles random-effect integration and exact gradients automatically
Validated on two numerical examples, including a monotonic NMM application, with reproducible code released

*Source: arXiv:2509.00146*