Skip to content

Default to r_eff=FALSE for loo() method #1091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion R/cpp_opts.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

# running and parsing exe info --------------------------------
# run <model> info command
#' @noRd
#' @example `.cmdstan/bin`
run_info_cli <- function(exe_file) {
withr::with_path(
Expand Down Expand Up @@ -52,7 +53,7 @@ parse_exe_info_string <- function(ret_stdout) {
model_compile_info <- function(exe_file, version) {
info <- NULL
if (version > "2.26.1") {

ret <- run_info_cli(exe_file)
if (ret$status == 0) {
info <- list()
Expand Down
17 changes: 10 additions & 7 deletions R/fit.R
Original file line number Diff line number Diff line change
Expand Up @@ -1501,12 +1501,15 @@ CmdStanMCMC <- R6::R6Class(
#' containing the pointwise log-likelihood. The default is to look for
#' `"log_lik"`. This argument is passed to the [`$draws()`][fit-method-draws]
#' method.
#' @param r_eff (multiple options) How to handle the `r_eff` argument for `loo()`:
#' * `TRUE` (the default) will automatically call [loo::relative_eff.array()]
#' to compute the `r_eff` argument to pass to [loo::loo.array()].
#' * `FALSE` or `NULL` will avoid computing `r_eff` (which can sometimes be slow),
#' but the reported ESS and MCSE estimates can be over-optimistic if the
#' posterior draws are not (near) independent.
#' @param r_eff (multiple options) How to handle the `r_eff` argument for
#' `loo()`. `r_eff` measures the amount of autocorrelation in MCMC draws, and
#' is used to compute more accurate ESS and MCSE estimates for pointwise and
#' total ELPDs.
#' * `TRUE` will call [loo::relative_eff.array()] to compute the `r_eff`
#' argument to pass to [loo::loo.array()].
#' * `FALSE` (the default) or `NULL` will avoid computing `r_eff`,
#' which can be very slow. The reported ESS and MCSE estimates may be
#' over-optimistic if the posterior draws are far from independent.
#' * If `r_eff` is anything else, that object will be passed as the `r_eff`
#' argument to [loo::loo.array()].
#' @param moment_match (logical) Whether to use a
Expand Down Expand Up @@ -1537,7 +1540,7 @@ CmdStanMCMC <- R6::R6Class(
#' print(loo_result)
#' }
#'
loo <- function(variables = "log_lik", r_eff = TRUE, moment_match = FALSE, ...) {
loo <- function(variables = "log_lik", r_eff = FALSE, moment_match = FALSE, ...) {
require_suggested_package("loo")
if (length(variables) != 1) {
stop("Only a single variable name is allowed for the 'variables' argument.", call. = FALSE)
Expand Down
17 changes: 10 additions & 7 deletions man/fit-method-loo.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.