Support composite observation space with proper min max in manager based env #2811
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR supports enables ManagerBasedRLEnv to properly support gymnaisum's composite observation gym.spaces.Dict at term-level with proper min, max specification. The benefit is that this will give rl-library a clearer signs how to pre-process the obseravtion data.
Before:
All terms are assigned with
gym.Spaces.Box(min=-np.inf, max=np.inf)
, one problem with this is that from rl-library side, is that some rl-libraries network construction depends heavily on observations space details. RL-library (e.g. sb3) looks atgym.Spaces.Box.min
andgym.Spaces.Box.max
to determine if they need to normalize it at library side.After:
this PR utilizes on obs_terms's clip to determine if gym.Spaces.Box should be stricter. For example, environment obs_term returning gym.Spaces.Box(min=0, max=255) will be automatically detected by sb3, and a proper scalling be applied automatically. whereas gym.Spaces.Box(min=-inf, max=inf) will fail.
while no special case is treated for gym.Spaces.Box(min=-1, max=1), but this will makes the rl-library easy to figure out that this term is already clipped, not more transformation needed.
Type of change
Screenshots
Please attach before and after screenshots of the change if applicable.
Checklist
pre-commit
checks with./isaaclab.sh --format
config/extension.toml
fileCONTRIBUTORS.md
or my name already exists there