Day 14 Software Implementation of LMMs
14.2 Model review
In general, we can write LLMs as
\[\mathbf{y} \vert \boldsymbol{b} \sim N(\mathbf{X}\boldsymbol{\beta} + \mathbf{Z}\boldsymbol{b}, \sigma^2 \mathbf{W}^{-1})\]
\(\boldsymbol{b} \sim N(\boldsymbol{0}, \boldsymbol{\Sigma})\), where\(\boldsymbol{\Sigma}\) must be positive semidefinite and can be expressed also as \(\boldsymbol{\Sigma}_{\theta} = \sigma^2\boldsymbol{\Lambda}_\theta \boldsymbol{\Lambda}_\theta^T\), where \(\sigma\) is the scale factor (same as above) and \(\boldsymbol{\Lambda}_\theta\) is a relative covariance factor that depends on the variance component \(\theta\).
- Check out Bates et al. (2015)
14.3 Computational tricks
14.3.1 Improving computational stability
- For computational stability and efficiency, reformulate the model such that \(\theta\) appears only in the conditional distribution for the response.
- For ML, minimize -2 log-likelihood, \(-2 \mathcal{L}(\boldsymbol{\theta}, \boldsymbol{\beta}, \sigma^2\vert\mathbf{y})=\log \frac{\vert\boldsymbol{L}_\theta\vert^2}{\vert\boldsymbol{W}\vert} + n \log(2\pi\sigma^2)+ \frac{r^2(\boldsymbol{\theta})}{\sigma^2} + \frac{\vert\vert \mathbf{R}_X(\boldsymbol{\beta}-\hat{\boldsymbol{\beta}}_\theta) \vert\vert}{\sigma^2}\)
14.3.2 Nonlinear optimizing algorithms
- Bound Optimization by Quadratic Approximation (bobyqa)
- Used in
lme4(default)
- Used in
- Nelder Mead
- Alternative in
lme4 - Visualization
- Alternative in
- Template Model Builder (TMB)
nlminb()oroptim()under the hood- GLMMs