Week Learning Objectives

• Describe the major assumptions in basic multilevel models

• Conduct analyses to decide whether cluster means and random slopes should be included

• Use graphical tools to diagnose assumptions of linearity, homoscedasticity (equal variance), and normality

• Solve some basic convergence issues

• Report results of a multilevel analysis based on established guidelines

Two Components of a Parametric Model

Functional Form

Two Components of a Parametric Model

Random Component

I.e., distribution of random effects/errors

$$\left[\begin{array}{c}{u}_{0j}\\ u_{1j}\end{array}\right]\sim N(\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{cc}{\tau }_0^2& {\tau }_{01}\\ {\tau }_{01}& {\tau }_1^2\end{array}\right])$$

$$e_{ij}\sim N(0,\sigma )$$

Versus $e_{ij}\sim t_3(0,\sigma )$

Or $e_{ij}\sim N(0,{\sigma }_j)$, where different clusters $j$ have a different SD ${\sigma }_j$

Five Assumptions in Normal Linear Models

Linearity

Independence of errors (at the highest level)

Normality

Equal variance of errors (i.e., homoscedasticity)

Correct Specification of the model

‍Importance: S, L, I > E, N

Assumptions Are Important

Your result is only as good as the assumptions

• Garbage in, garbage out

Correct Specification

Fixed effects

• Cluster means should be included (unless between coefficient = within coefficient)

  • Otherwise, between and within coefficients are conflated

• Relevant predictors should be included to answer the target research question

  • E.g., Gender gap vs. gender gap adjusting for profession

Random effects

• If random slope variance is not zero, omitting it leads to inflated Type I error rates for fixed effects

  • Varying slopes could also be important information from the data

Linearity

Lack of linear association $\ne$ lack of association

Independence of Errors

We use MLM because students within the same school are more similar (i.e., not independent)

If schools are from different school districts, they may also not be independent

• Need a three-level model

Or, student A in school 1 is from the same neighborhood as student B in school 2

• Cross-classified model

Temporal dependence

• E.g., Repeated measures closer in time are more similar

  • Autoregressive model

Equal Variance of Errors (Homoscedasticity)

Residual plots

Normality

Examples of data for which a normal model is not good

• Binary/ordinal outcome with < 5 categories (including the homework)

• Count data (e.g., # binge drinking episodes; # of success in 5 trials)

• Bounded data with ceiling/floor effects (e.g., depressive symptoms)

• Reaction time

Additional Issues

• Outliers/influential observations
  • Check coding error
  • Don't drop outliers unless you adjust the standard errors accordingly, or use robust models

• Reliability (e.g., $\alpha$ coefficient)
  • Reliability may be high at one level but low at another level
  • See Lai (2021, doi: 10.1037/met0000287) for level-specific reliability
    • You can use the multilevel_alpha() function from https://github.com/marklhc/mcfa_reliability_supp/blob/master/multilevel_alpha.R
    • The procedure was recently implemented in the semTools::compRelSEM() function, thanks to Dr. Terrence D. Jorgensen.

References

• Chapter by McCoach (2019); Paper by Meteyard & Davies (2020)

Things to report: