Skip to content

Missing mappings for $obs_loss #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jemus42 opened this issue Mar 3, 2025 · 0 comments
Open

Missing mappings for $obs_loss #57

jemus42 opened this issue Mar 3, 2025 · 0 comments

Comments

@jemus42
Copy link
Member

jemus42 commented Mar 3, 2025

I noticed that some measures like bbrier and logloss don't have an associated obs_loss

library(mlr3)

measure = msr("classif.bbrier")
measure$obs_loss
#> NULL

measure = msr("classif.logloss")
measure$obs_loss
#> NULL

While spelunking the measure-related internals I ended up here:

# Ok as expected
mlr3measures:::measures$mse$obs_loss
#> [1] "se"

# Should be "se" as well
mlr3measures:::measures$bbrier$obs_loss
#> [1] NA

# NA but should not be?
mlr3measures:::measures$logloss$obs_loss
#> [1] NA

For logloss I assumed something like Metrics:ll():

#> Metrics::ll
function (actual, predicted) 
{
    score <- -(actual * log(predicted) + (1 - actual) * log(1 - 
        predicted))
    score[actual == predicted] <- 0
    score[is.nan(score)] <- Inf
    return(score)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant