Feature that enable modify hydra configuration group through command line

[ooctipus]

Description

This PR introduce a new feature in interacting with hydra, bring group override feature to isaac lab

Demo Script

Demo 1: Overriding Event Managers with Pre-grouped Alternative Managers

Example 1: Overriding Event Manager

# Default
python train.py <default>


Output:
Event Terms: ["reset_robot_joint_position"]

# Override to include friction
python train.py env.events=rand_joint_pos_friction



Output:
Event Terms: ["reset_robot_joints", "reset_robot_joint_friction"]

# Override to include armature as well
python train.py env.events=rand_joint_pos_friction_amarture



Output:
Event Terms: ["reset_robot_joints", "reset_robot_joint_friction", "reset_robot_joint_amature"]

---

Example 2: Overriding Observation Manager

# Default
python train.py <default>



Output:
joint_pos.noise.n_min   -> -0.01
joint_pos.noise.n_max   ->  0.01
enable_corruption       ->  True

# Disable noise and corruption
python train.py env.observations=state_obs_no_noise



Output:
joint_pos.noise         -> None
enable_corruption       -> False

---

Demo 2: Overriding Terms Under a Manager

Example: Change Only arm_action Term Under ActionManager

# Default action manager
python train.py <default>



Output:
env.actions.arm_action  -> JointPositionActionCfg

# Switch to inverse-kinematics action
python train.py env.actions.arm_action=ik_abs_arm_action



Output:
env.actions.arm_action  -> DifferentialInverseKinematicsActionCfg

# Switch to operational-space control
python train.py env.actions.arm_action=osc_arm_action



Output:
env.actions.arm_action  -> OperationalSpaceControllerActionCfg

---

Demo 3: Overriding agent.policy Configuration

Demo Script:

# Default policy network
python.train.py <default>



Output:
Actor MLP: Sequential(
  (0): Linear(in_features=32, out_features=64, bias=True)
  (2): Linear(in_features=64, out_features=64, bias=True)
  (4): Linear(in_features=64, out_features=7, bias=True)
)

# Use a medium-sized network
python train.py agent.policy=medium_network



Output:
Actor MLP: Sequential(
  (0): Linear(in_features=32, out_features=256, bias=True)
  (2): Linear(in_features=256, out_features=128, bias=True)
  (4): Linear(in_features=128, out_features=64, bias=True)
  (6): Linear(in_features=64, out_features=7, bias=True)
)

Fixes # (issue)

Type of change

• New feature (non-breaking change which adds functionality)

Checklist

• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[kellyguo11]

how would this work for other RL libraries that don't use yaml?

[kellyguo11]

we can remove this blob

[kellyguo11]

is there a good way to test this in the unit tests?

[kellyguo11]

I think it'll be helpful to have some documentation explaining usage and the setup for this, to highlight what can be modified and how the configs should be set up to allow those modifications through cli

[ooctipus]

Maybe we should move the all of them to Hydra Group Configurables

instead of

gym.register(
    id="Isaac-Cartpole-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.cartpole_env_cfg:CartpoleEnvCfg",
        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_ppo_cfg.yaml",
        "rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:CartpolePPORunnerCfg",
        "skrl_cfg_entry_point": f"{agents.__name__}:skrl_ppo_cfg.yaml",
        "sb3_cfg_entry_point": f"{agents.__name__}:sb3_ppo_cfg.yaml",
    },
)

gym.register(
    id="Isaac-Cartpole-RGB-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.cartpole_camera_env_cfg:CartpoleRGBCameraEnvCfg",
        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_camera_ppo_cfg.yaml",
    },
)

gym.register(
    id="Isaac-Cartpole-Depth-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.cartpole_camera_env_cfg:CartpoleDepthCameraEnvCfg",
        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_camera_ppo_cfg.yaml",
    },
)

gym.register(
    id="Isaac-Cartpole-RGB-ResNet18-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.cartpole_camera_env_cfg:CartpoleResNet18CameraEnvCfg",
        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_feature_ppo_cfg.yaml",
    },
)

gym.register(
    id="Isaac-Cartpole-RGB-TheiaTiny-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.cartpole_camera_env_cfg:CartpoleTheiaTinyCameraEnvCfg",
        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_feature_ppo_cfg.yaml",
    },
)

do:

gym.register(
    id="Isaac-Cartpole-v0",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{__name__}.cartpole_camera_env_cfg:CartpoleRGBCameraEnvCfg",
        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_camera_ppo_cfg.yaml",
        "configurable_entry_point": f"{agents.__name__}.configurables:AgentConfigurables",
    },
)

python train.py --task Isaac-Cartpole-v0 env.observation=State
python train.py --task Isaac-Cartpole-v0 env.observation=RGB
python train.py --task Isaac-Cartpole-v0 env.observation=Depth
python train.py --task Isaac-Cartpole-v0 env.observation=ResNet18Feature
python train.py --task Isaac-Cartpole-v0 env.observation=TheiaTinyFeature

Motivation:

Incorporating observation space, (and potentialy action space) in task id can make one environment generates countless versions (combinatoric scaling). If we use configurables instead of task id, it makes sense that

1. Nothing else but just observation space has been changed
2. the scaling is hidden within configurable, and configurable scales linearly rather than combinatoric scaling