Skip to content

Conversation

jemus42
Copy link
Member

@jemus42 jemus42 commented Aug 8, 2025

  • Had wrong man entry linking to the performance filter rather than the permutation filter
  • Standardization was performed only when standardize = FALSE, i.e. !isTRUE() instead of `isTRUE()

Reprex with behavior after fix:

library(mlr3learners)
#> Loading required package: mlr3
library(mlr3filters)
library(data.table)

set.seed(123)
task <- tgen("friedman1")$generate(n = 500)
learner <- lrn("regr.ranger", num.trees = 100)
resampling <- rsmp("holdout")
measure <- msr("regr.mse")

With standardize = FALSE

filter_pfi <- flt("permutation", 
                  learner = learner,
                  measure = measure,
                  resampling = resampling,
                  nmc = 1, 
                  standardize = FALSE)

filter_pfi$param_set$values
#> $standardize
#> [1] FALSE
#> 
#> $nmc
#> [1] 1
set.seed(123)
filter_pfi$calculate(task = task)
filter_pfi
#> <FilterPermutation:permutation>: Permutation Score
#> Task Types: regr
#> Properties: missings
#> Task Properties: -
#> Packages: mlr3, mlr3learners, ranger, mlr3measures
#> Feature types: logical, integer, numeric, character, factor, ordered
#>          feature         score
#>  1:   important2  6.5207463936
#>  2:   important4  6.4689073720
#>  3:   important1  3.9712012063
#>  4: unimportant2  0.9704977771
#>  5:   important3  0.4698455885
#>  6:   important5  0.3825112961
#>  7: unimportant3 -0.0008537673
#>  8: unimportant5 -0.7238706536
#>  9: unimportant1 -0.9496679245
#> 10: unimportant4 -1.1555031201

With standardize = TRUE

filter_pfi <- flt("permutation", 
                  learner = learner,
                  measure = measure,
                  resampling = resampling,
                  nmc = 1, 
                  standardize = TRUE)

filter_pfi$param_set$values
#> $standardize
#> [1] TRUE
#> 
#> $nmc
#> [1] 1
set.seed(123)
filter_pfi$calculate(task = task)
filter_pfi
#> <FilterPermutation:permutation>: Permutation Score
#> Task Types: regr
#> Properties: missings
#> Task Properties: -
#> Packages: mlr3, mlr3learners, ranger, mlr3measures
#> Feature types: logical, integer, numeric, character, factor, ordered
#>          feature         score
#>  1:   important2  1.0000000000
#>  2:   important4  0.9920501399
#>  3:   important1  0.6090102216
#>  4: unimportant2  0.1488323144
#>  5:   important3  0.0720539583
#>  6:   important5  0.0586606614
#>  7: unimportant3 -0.0001309309
#>  8: unimportant5 -0.1110103982
#>  9: unimportant1 -0.1456379174
#> 10: unimportant4 -0.1772041190

Created on 2025-08-08 with reprex v2.1.1

@be-marc
Copy link
Member

be-marc commented Sep 12, 2025

Thanks!

@be-marc be-marc merged commit 7a05f88 into mlr-org:main Sep 12, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants