Different R2 estimates from this package vs. MuMIn for non-correlated random slope-intercept

[adityac95]

Thanks for your great package. However, I'm having a tough time understanding why the marginal and conditional $R^2$ computed from this package are not the same as those from the MuMIn package. For instance, I have a linear mixed-effects model called cc_simple_model <- lmer(response <- item_order + (1|stimulus) + (item_order||participant_private_id) which has the following marginal and conditional $R^2$ (as per Nakagawa & Schielzeth, 2013) estimates from the two packages:

$R^2$ type	MuMIn (using function r.squaredGLMM)	performance (using function model_performance)
marginal	.01	.01
conditional	.49	.22

Every other linear mixed effects model I've tested also has discrepancies. Could you explain what's going on?

[strengejacke]

Here's a minimal reprex. It seems to be when correlation between slope+intercepts is suppressed:

library(lme4)
#> Loading required package: Matrix
library(performance)
library(MuMIn)

m1 <- lmer(Reaction ~ Days + (1 + Days | Subject), data = sleepstudy)
m2 <- lmer(Reaction ~ Days + (1 + Days || Subject), data = sleepstudy)

r2_nakagawa(m1)
#> # R2 for Mixed Models
#> 
#>   Conditional R2: 0.799
#>      Marginal R2: 0.279
r.squaredGLMM(m1)
#> Warning: 'r.squaredGLMM' now calculates a revised statistic. See the help page.
#>            R2m       R2c
#> [1,] 0.2786511 0.7992199

r2_nakagawa(m2)
#> # R2 for Mixed Models
#> 
#>   Conditional R2: 0.702
#>      Marginal R2: 0.415
r.squaredGLMM(m2)
#>            R2m       R2c
#> [1,] 0.2829806 0.7965226

^{Created on 2023-05-24 with reprex v2.0.2}

[strengejacke]

Currently, I can't say which package behaves wrong here. Will look into it.

[mattansb]

The error is in performance (this is a duplicate of #428 ).

If we compare prediction-$R^2$ with performance and MuMIn, we see that MuMIn's results are similar to the prediction-$R^2$:

library(lme4)
library(performance)
library(MuMIn)

r2_from_prediction <- function(model) {
  #' Function to get prediction r2 for a mixed model
  y <- insight::get_response(model)
  c(
    Marg. = 1 - var(predict(model, re.form = NA) - y) / var(y),
    Cond. = 1 - var(predict(model, re.form = NULL) - y) / var(y)
  ) |> as.matrix() |> t()
}

r2_to_matrix <- function(L) {
  #' Function to convert r2_nakagawa() outcput to a matrix
  as.matrix(rev(unlist(L))) |> t()
}


m1 <- lmer(Reaction ~ Days + (1 + Days | Subject), data = sleepstudy)
m2 <- lmer(Reaction ~ Days + (1 + Days || Subject), data = sleepstudy)

rbind(
  r2_from_prediction(m1),
  r2_nakagawa(m1) |> r2_to_matrix(),
  r.squaredGLMM(m1)
) |> `rownames<-`(c("predict", "performance", "MuMIn"))
#>                 Marg.     Cond.
#> predict     0.2864714 0.8258987
#> performance 0.2786511 0.7992199
#> MuMIn       0.2786511 0.7992199

rbind(
  r2_from_prediction(m2),
  r2_nakagawa(m2) |> r2_to_matrix(),
  r.squaredGLMM(m2)
) |> `rownames<-`(c("predict", "performance", "MuMIn"))
#>                 Marg.     Cond.
#> predict     0.2864714 0.8266442
#> performance 0.4150271 0.7015745
#> MuMIn       0.2829806 0.7965226

[strengejacke]

Closing in favor of #428