Skip to content

Support composite observation space with proper min max in manager based env #2811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ooctipus
Copy link
Contributor

Description

This PR supports enables ManagerBasedRLEnv to properly support gymnaisum's composite observation gym.spaces.Dict at term-level with proper min, max specification. The benefit is that this will give rl-library a clearer signs how to pre-process the obseravtion data.

Before:
All terms are assigned with gym.Spaces.Box(min=-np.inf, max=np.inf), one problem with this is that from rl-library side, is that some rl-libraries network construction depends heavily on observations space details. RL-library (e.g. sb3) looks at gym.Spaces.Box.min and gym.Spaces.Box.max to determine if they need to normalize it at library side.

After:
this PR utilizes on obs_terms's clip to determine if gym.Spaces.Box should be stricter. For example, environment obs_term returning gym.Spaces.Box(min=0, max=255) will be automatically detected by sb3, and a proper scalling be applied automatically. whereas gym.Spaces.Box(min=-inf, max=inf) will fail.

while no special case is treated for gym.Spaces.Box(min=-1, max=1), but this will makes the rl-library easy to figure out that this term is already clipped, not more transformation needed.

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@ooctipus ooctipus force-pushed the feat/manager_based_env_composite_obs_space_support branch from 96908c4 to 1c92818 Compare June 29, 2025 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant