Implementing neural network mixed-effects models in Template Model Builder (TMB)
Nan Zheng, Hoi Yiu Cheung, Vibhu Sharma, James T. Thorson, Noel G. Cadigan
stat.ML · cs.LG
Quick verdict
Uses TMB+Laplace to fit neural mixed models without manual derivations; promising, but gains rest on digitized baselines and real-data edge is tiny.
Summary
This method paper shows that Template Model Builder (TMB) can fit neural network mixed-effects models (NMMs) with users supplying only the negative joint log-likelihood and L2 regularization; TMB's automatic differentiation and Laplace approximation then integrate out random effects and deliver exact gradients, avoiding the manual quasi-likelihood derivations and omitted Hessian-determinant term of Mandel et al. (2023). The authors replicate Mandel et al.'s nonlinear XOR simulation (100 subjects, six binary predictors, two-hidden-layer network with 10 and 5 units, 500 replicates) for continuous and binary responses. TMB achieves MSPE of about 0.003 (near the 0.0025 noise floor) versus roughly 0.45-0.55 reported for GNMM-2, and AUC 0.89-0.91 versus 0.78-0.88, though baseline numbers were read from published figures rather than re-run. A real application to American plaice maturity data (58 cohorts, about 78,900 fish) demonstrates monotonic NMMs via KKT conditions with AR(1) cohort random effects; test BCE is 0.249 versus 0.250 for a GLMM - a practically negligible edge, and no timing benchmarks substantiate the claimed computational efficiency. Main limitations: unverified figure-digitized comparisons, no uncertainty on performance gaps, and heuristic KKT multiplier selection. Public GitHub code supports reproducibility.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- TMB fits NMMs from only the negative joint log-likelihood plus regularization, computing exact Laplace-approximated marginal objectives and gradients with no omitted terms.
- This removes the accuracy, scalability (two-layer cap), and exponential-family restrictions of the manual quasi-likelihood approach of Mandel et al. (2023).
- In the replicated XOR simulation, TMB-NMM achieves MSPE ~0.003 (noise floor 0.0025) and AUC 0.89-0.91, markedly better than figures reported for GNMM-2 (MSPE ~0.45-0.55, AUC ~0.78-0.88).
- Monotonic NMMs with AR(1) cohort random effects are straightforward to implement in TMB via KKT conditions; on plaice maturity data the NMM edges a GLMM (BCEtest 0.249 vs 0.250).
Strengths
Eliminates the manual-derivation barrier for NMMs
Users specify only the negative joint log-likelihood plus L2 penalty (Eqs. 1, 7); TMB's AD + Laplace yields exact marginal gradients including the RE-derivative and determinant terms Mandel et al. (2023) omitted (Sec. 1, citing Kristensen et al. 2016 Eq. 7).
Carefully matched simulation replication
Sec. 3 reuses Mandel et al.'s exact design: GNMM-2 with 10 and 5 hidden units, sigmoid activation, lambda=0.005, same tau grids, 500 replicates; resulting MSPE ~0.003 sits at the theoretical noise floor sigma2=0.0025, an internally coherent result indicating near-exact XOR recovery.
General observational model beyond exponential families
Constrained modeling demonstrated on real data
Reproducibility and honest scoping
Weaknesses
Headline gains rest on digitized baselines
Real-data improvement is negligible and unquantified
Efficiency claims lack timing evidence
Under-documented tuning of lambda and the KKT multiplier
No alternative-estimator comparison or SEs
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a methods paper (stat.ML) proposing Template Model Builder (TMB) as the estimation engine for neural network mixed-effects models (NMMs), mu_i = g{f_N(x_i) + Z_i^T b_i}. The central claim: a user supplies only the negative joint log-likelihood (Eq. 1) plus an L2 weight penalty; TMB's automatic differentiation and Laplace approximation then integrate out the random effects (Eq. 6) and return exact marginal gradients. This is pitched against Mandel et al. (2023), whose manual quasi-likelihood derivation dropped a Hessian-determinant term and the derivative-of-RE-estimators term, capped architectures at two hidden layers, and restricted the observation model to exponential families. Two evidence pillars follow: a replication of Mandel et al.'s XOR simulation (Section 3) and an American plaice maturity application featuring a KKT-constrained monotonic NMM (Section 4).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TMB and why use it for neural network mixed-effects models?
Template Model Builder (TMB) is an R/C++ tool that uses automatic differentiation and the Laplace approximation to fit models with many random effects. The user writes only the joint likelihood; TMB automatically integrates out random effects and computes exact gradients. For neural network mixed-effects models this replaces error-prone manual derivations of objective functions and gradients.
How much better is the TMB approach than earlier NMM methods?
In the paper's replicated XOR simulation, the TMB implementation reached MSPE of about 0.003 (near the 0.0025 irreducible noise floor) versus roughly 0.45-0.55 reported for the earlier GNMM-2 method, and AUC 0.89-0.91 versus 0.78-0.88. However, those baseline numbers were read from the earlier paper's published figures rather than re-run, and on the real plaice dataset the NMM beat a GLMM only marginally (test BCE 0.249 vs 0.250).
What is a monotonic neural network in this paper?
For the fish maturity application, the predicted maturation probability is constrained to be non-decreasing with age. This is enforced by a Karush-Kuhn-Tucker (KKT) multiplier added to the objective that penalizes any negative age-derivative of the network output across ages and cohorts, all computed inside TMB's differentiable objective.
Is code available for this paper?
Yes. The authors state that the TMB code is publicly available in the GitHub repository ZhengNan18/Implementing-neural-network-mixed-effects-models-in-Template-Model-Builder-TMB-.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are present only as captions in the extracted text; curve values attributed to Mandel et al. (e.g., MSPE ~0.45-0.55) are the authors' own figure readings and could not be independently checked.
- The extracted text runs complete through the reference list; no end-of-text truncation was detected.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.