From 3df849476365aa1f74c2c88be84b41f8ce434b82 Mon Sep 17 00:00:00 2001 From: yangchen73 <122090643@link.cuhk.edu.cn> Date: Wed, 14 Jan 2026 12:47:20 +0800 Subject: [PATCH 1/4] Add Embodied Environments documentation --- docs/source/overview/gym/index.rst | 167 ++++++++++++++++++++++++++++- 1 file changed, 164 insertions(+), 3 deletions(-) diff --git a/docs/source/overview/gym/index.rst b/docs/source/overview/gym/index.rst index b0fe003..93206f5 100644 --- a/docs/source/overview/gym/index.rst +++ b/docs/source/overview/gym/index.rst @@ -1,6 +1,167 @@ -Embodied Environments -================== +Embodied Environments +===================== -*To be completed by adding a detailed description of the Embodied Environments.* +.. currentmodule:: embodichain.lab.gym +The :class:`envs.EmbodiedEnv` is the core environment class in EmbodiChain designed for complex Embodied AI tasks. It adopts a **configuration-driven** architecture, allowing users to define robots, sensors, objects, lighting, and automated behaviors (events) purely through configuration classes, minimizing the need for boilerplate code. +Core Architecture +----------------- + +Unlike the standard :class:`envs.BaseEnv`, the :class:`envs.EmbodiedEnv` integrates several manager systems to handle the complexity of simulation: + +- **Scene Management**: Automatically loads and manages robots, sensors, and scene objects defined in the configuration. +- **Event Manager**: Handles automated behaviors such as domain randomization, scene setup, and dynamic asset swapping. +- **Observation Manager**: Allows flexible extension of observation spaces without modifying the environment code. +- **Dataset Manager**: Built-in support for collecting demonstration data during simulation steps. +- **Action Bank**: Advanced action composition system supporting action graphs and configurable action primitives (optional, for complex manipulation sequences). + +Configuration System +-------------------- + +The environment is defined by inheriting from :class:`envs.EmbodiedEnvCfg`. This configuration class serves as the single source of truth for the scene description. + +.. code-block:: python + + from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg + from embodichain.utils import configclass + + @configclass + class MyTaskEnvCfg(EmbodiedEnvCfg): + # 1. Define Scene Components + robot = ... # Robot configuration + sensor = [...] # List of sensors (e.g., Cameras) + light = ... # Lighting configuration + + # 2. Define Objects + rigid_object = [...] # Dynamic objects (e.g., tools, debris) + rigid_object_group = [...] # Object groups (efficient for many similar objects) + articulation = [...] # Articulated objects (e.g., cabinets) + + # 3. Define Managers + events = ... # Event settings (Randomization, etc.) + observations = ... # Custom observation spec + dataset = ... # Data collection settings + + # 4. Task Extensions + extensions = { # Task-specific parameters + "episode_length": 500, + "obs_mode": "state", + } + +Key Components +~~~~~~~~~~~~~~ + +The :class:`envs.EmbodiedEnvCfg` exposes several key fields to compose your environment: + +- **robot** (:class:`embodichain.lab.sim.cfg.RobotCfg`): + Defines the agent in the scene. Supports loading robots from URDF/MJCF with specified initial state and control mode. + +- **sensor** (List[:class:`embodichain.lab.sim.cfg.SensorCfg`]): + A list of sensors attached to the scene or robot. Common sensors include :class:`embodichain.lab.sim.sensors.StereoCamera` for RGB-D and segmentation data. + +- **light** (:class:`envs.EmbodiedEnvCfg.EnvLightCfg`): + Configures the lighting environment. + + - ``direct``: Direct light sources (Point, Spot, Directional) affecting local illumination. + - ``indirect``: Global illumination settings (Ambient, IBL) - *planned for future release*. + +- **rigid_object** / **rigid_object_group** / **articulation**: + Defines the interactive elements. + + - ``rigid_object``: For dynamic or kinematic simple bodies. + - ``rigid_object_group``: Collections of rigid objects that can be managed together. + - ``articulation``: For complex mechanisms with joints (doors, drawers). + +- **background** (List[:class:`embodichain.lab.sim.cfg.RigidObjectCfg`]): + Static or kinematic objects serving as obstacles or landmarks in the scene. + +- **extensions** (Dict[str, Any]): + Task-specific extension parameters that are automatically bound to the environment instance. + This allows passing custom parameters (e.g., ``episode_length``, ``obs_mode``, ``action_scale``) + without modifying the base configuration class. These parameters are accessible via ``self.param_name`` + after environment initialization. + +- **filter_visual_rand** (bool): + Whether to filter out visual randomization functors. Useful for debugging motion and physics issues + when visual randomization interferes with the debugging process. Defaults to ``False``. + +Manager Systems +--------------- + +Event Manager +~~~~~~~~~~~~~ + +The Event Manager automates changes in the environment. Events can be triggered at different stages: + +- **startup**: Executed once when the environment initializes. +- **reset**: Executed every time ``env.reset()`` is called. +- **interval**: Executed periodically every N steps. + +Common use cases include **Domain Randomization** (randomizing friction, mass, visual textures) and **Asset Swapping** (changing object models on reset). + +Observation Manager +~~~~~~~~~~~~~~~~~~~ + +While :class:`envs.EmbodiedEnv` provides default observations (robot state and raw sensor data), the Observation Manager allows you to inject task-specific information. + +You can configure it to compute and add: + + - Object poses relative to the robot. + - Keypoint positions. + - Task-specific metrics (e.g., distance to target). + +Dataset Manager +~~~~~~~~~~~~~~~ + +For Imitation Learning (IL) tasks, the Dataset Manager automates data collection. It can: + + - Record full observation-action pairs. + - Save episodes automatically when environments terminate. + - Handle data formatting for training pipelines. + +Creating a Custom Task +---------------------- + +To create a new task, inherit from :class:`envs.EmbodiedEnv` and implement the task-specific logic. + +.. code-block:: python + + from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg + from embodichain.lab.gym.utils.registration import register_env + + @register_env("MyTask-v0", max_episode_steps=500) + class MyTaskEnv(EmbodiedEnv): + def __init__(self, cfg: MyTaskEnvCfg, **kwargs): + super().__init__(cfg, **kwargs) + + def create_demo_action_list(self, *args, **kwargs): + # Optional: Implement for scripted demonstrations + # Must set self.action_length = len(action_list) if returning actions + pass + + def is_task_success(self, **kwargs): + # Optional: Define success criteria (mainly for IL data collection) + # Returns: torch.Tensor of shape (num_envs,) with boolean values + return success_tensor + + def get_reward(self, obs, action, info): + # Optional: Override for RL tasks + # Returns: torch.Tensor of shape (num_envs,) + return super().get_reward(obs, action, info) + + def get_info(self, **kwargs): + # Optional: Override to add custom info fields + # Should include "success" and "fail" keys for termination + info = super().get_info(**kwargs) + info["custom_metric"] = ... + return info + +For a complete example of a modular environment setup, please refer to the :ref:`tutorial_modular_env` tutorial. + +See Also +-------- + +- :ref:`tutorial_create_basic_env` - Creating basic environments +- :ref:`tutorial_modular_env` - Advanced modular environment setup +- :doc:`/api_reference/embodichain/embodichain.lab.gym.envs` - Complete API reference for EmbodiedEnv and EmbodiedEnvCfg From 6e6bb4a7b6a14f49d303e17f0e0075a24ed45e22 Mon Sep 17 00:00:00 2001 From: yangchen73 <122090643@link.cuhk.edu.cn> Date: Wed, 14 Jan 2026 20:04:58 +0800 Subject: [PATCH 2/4] Enrich the content of Manager Systems and Action Bank --- docs/source/overview/gym/index.rst | 278 +++++++++++++++++++---------- 1 file changed, 182 insertions(+), 96 deletions(-) diff --git a/docs/source/overview/gym/index.rst b/docs/source/overview/gym/index.rst index 93206f5..6fe3290 100644 --- a/docs/source/overview/gym/index.rst +++ b/docs/source/overview/gym/index.rst @@ -10,11 +10,11 @@ Core Architecture Unlike the standard :class:`envs.BaseEnv`, the :class:`envs.EmbodiedEnv` integrates several manager systems to handle the complexity of simulation: -- **Scene Management**: Automatically loads and manages robots, sensors, and scene objects defined in the configuration. -- **Event Manager**: Handles automated behaviors such as domain randomization, scene setup, and dynamic asset swapping. -- **Observation Manager**: Allows flexible extension of observation spaces without modifying the environment code. -- **Dataset Manager**: Built-in support for collecting demonstration data during simulation steps. -- **Action Bank**: Advanced action composition system supporting action graphs and configurable action primitives (optional, for complex manipulation sequences). +* **Scene Management**: Automatically loads and manages robots, sensors, and scene objects defined in the configuration. +* **Event Manager**: Handles automated behaviors such as domain randomization, scene setup, and dynamic asset swapping. +* **Observation Manager**: Allows flexible extension of observation spaces without modifying the environment code. +* **Dataset Manager**: Built-in support for collecting demonstration data during simulation steps. +* **Action Bank**: Advanced action composition system supporting action graphs and configurable action primitives (optional, for complex manipulation sequences). Configuration System -------------------- @@ -23,102 +23,188 @@ The environment is defined by inheriting from :class:`envs.EmbodiedEnvCfg`. This .. code-block:: python - from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg - from embodichain.utils import configclass - - @configclass - class MyTaskEnvCfg(EmbodiedEnvCfg): - # 1. Define Scene Components - robot = ... # Robot configuration - sensor = [...] # List of sensors (e.g., Cameras) - light = ... # Lighting configuration - - # 2. Define Objects - rigid_object = [...] # Dynamic objects (e.g., tools, debris) - rigid_object_group = [...] # Object groups (efficient for many similar objects) - articulation = [...] # Articulated objects (e.g., cabinets) - - # 3. Define Managers - events = ... # Event settings (Randomization, etc.) - observations = ... # Custom observation spec - dataset = ... # Data collection settings - - # 4. Task Extensions - extensions = { # Task-specific parameters - "episode_length": 500, - "obs_mode": "state", - } + from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg + from embodichain.utils import configclass + + @configclass + class MyTaskEnvCfg(EmbodiedEnvCfg): + # 1. Define Scene Components + robot = ... # Robot configuration + sensor = [...] # List of sensors (e.g., Cameras) + light = ... # Lighting configuration + + # 2. Define Objects + rigid_object = [...] # Dynamic objects (e.g., tools, debris) + rigid_object_group = [...] # Object groups (efficient for many similar objects) + articulation = [...] # Articulated objects (e.g., cabinets) + + # 3. Define Managers + events = ... # Event settings (Randomization, etc.) + observations = ... # Custom observation spec + dataset = ... # Data collection settings + + # 4. Task Extensions + extensions = { # Task-specific parameters + "episode_length": 500, + "obs_mode": "state", + } Key Components ~~~~~~~~~~~~~~ The :class:`envs.EmbodiedEnvCfg` exposes several key fields to compose your environment: -- **robot** (:class:`embodichain.lab.sim.cfg.RobotCfg`): +* **robot** (:class:`embodichain.lab.sim.cfg.RobotCfg`): Defines the agent in the scene. Supports loading robots from URDF/MJCF with specified initial state and control mode. - -- **sensor** (List[:class:`embodichain.lab.sim.cfg.SensorCfg`]): - A list of sensors attached to the scene or robot. Common sensors include :class:`embodichain.lab.sim.sensors.StereoCamera` for RGB-D and segmentation data. -- **light** (:class:`envs.EmbodiedEnvCfg.EnvLightCfg`): +* **sensor** (List[:class:`embodichain.lab.sim.cfg.SensorCfg`]): + A list of sensors attached to the scene or robot. Common sensors include :class:`~embodichain.lab.sim.sensors.StereoCamera` for RGB-D and segmentation data. + +* **light** (:class:`envs.EmbodiedEnvCfg.EnvLightCfg`): Configures the lighting environment. - - ``direct``: Direct light sources (Point, Spot, Directional) affecting local illumination. - - ``indirect``: Global illumination settings (Ambient, IBL) - *planned for future release*. + * ``direct``: Direct light sources (Point, Spot, Directional) affecting local illumination. + * ``indirect``: Global illumination settings (Ambient, IBL) - *planned for future release*. -- **rigid_object** / **rigid_object_group** / **articulation**: +* **rigid_object** / **rigid_object_group** / **articulation**: Defines the interactive elements. - - ``rigid_object``: For dynamic or kinematic simple bodies. - - ``rigid_object_group``: Collections of rigid objects that can be managed together. - - ``articulation``: For complex mechanisms with joints (doors, drawers). + * ``rigid_object``: For dynamic or kinematic simple bodies. + * ``rigid_object_group``: Collections of rigid objects that can be managed together. + * ``articulation``: For complex mechanisms with joints (doors, drawers). -- **background** (List[:class:`embodichain.lab.sim.cfg.RigidObjectCfg`]): +* **background** (List[:class:`embodichain.lab.sim.cfg.RigidObjectCfg`]): Static or kinematic objects serving as obstacles or landmarks in the scene. -- **extensions** (Dict[str, Any]): - Task-specific extension parameters that are automatically bound to the environment instance. - This allows passing custom parameters (e.g., ``episode_length``, ``obs_mode``, ``action_scale``) - without modifying the base configuration class. These parameters are accessible via ``self.param_name`` - after environment initialization. +* **extensions** (Dict[str, Any]): + Task-specific extension parameters that are automatically bound to the environment instance. This allows passing custom parameters (e.g., ``episode_length``, ``obs_mode``, ``action_scale``) without modifying the base configuration class. These parameters are accessible as instance attributes after environment initialization. For example, if ``extensions = {"episode_length": 500}``, you can access it via ``self.episode_length``. -- **filter_visual_rand** (bool): - Whether to filter out visual randomization functors. Useful for debugging motion and physics issues - when visual randomization interferes with the debugging process. Defaults to ``False``. +* **filter_visual_rand** (bool): + Whether to filter out visual randomization functors. Useful for debugging motion and physics issues when visual randomization interferes with the debugging process. Defaults to ``False``. Manager Systems --------------- +The manager systems in :class:`envs.EmbodiedEnv` provide modular, configuration-driven functionality for handling complex simulation behaviors. Each manager uses a **functor-based** architecture, allowing you to compose behaviors through configuration without modifying environment code. Functors are reusable functions or classes (inheriting from :class:`envs.managers.Functor`) that operate on the environment state, configured through :class:`envs.managers.cfg.FunctorCfg`. + Event Manager ~~~~~~~~~~~~~ -The Event Manager automates changes in the environment. Events can be triggered at different stages: +The Event Manager automates changes in the environment through event functors. Events can be triggered at different stages: + +* **startup**: Executed once when the environment initializes. Useful for setting up initial scene properties that don’t change during episodes. +* **reset**: Executed every time ``env.reset()`` is called. Applied to specific environments that need resetting (via ``env_ids`` parameter). This is the most common mode for domain randomization. +* **interval**: Executed periodically every N steps (specified by ``interval_step``, defaults to 10). Can be configured per-environment (``is_global=False``) or globally synchronized (``is_global=True``). + +Event functors are configured using :class:`envs.managers.cfg.EventCfg`. Common event functors include: + +**Physics Randomization:** + * ``randomize_rigid_object_mass``: Randomize object masses within a range. + * ``randomize_rigid_object_friction``: Vary friction coefficients for more robust sim-to-real transfer. *Planned for future release.* + * ``randomize_rigid_object_restitution``: Adjust bounciness of objects. *Planned for future release.* -- **startup**: Executed once when the environment initializes. -- **reset**: Executed every time ``env.reset()`` is called. -- **interval**: Executed periodically every N steps. +**Visual Randomization:** + * ``randomize_visual_material``: Randomize textures, base colors, and material properties (implemented as a Functor class). + * ``randomize_light``: Vary light position, color, and intensity. + * ``randomize_camera_extrinsics``: Randomize camera poses for viewpoint diversity. + * ``randomize_camera_intrinsics``: Vary focal length and principal point. -Common use cases include **Domain Randomization** (randomizing friction, mass, visual textures) and **Asset Swapping** (changing object models on reset). +**Spatial Randomization:** + * ``randomize_rigid_object_pose``: Randomize object positions and orientations. + * ``randomize_robot_eef_pose``: Vary end-effector initial poses. + * ``randomize_robot_qpos``: Randomize robot joint configurations. + +**Asset Management:** + * ``replace_assets_from_group``: Swap object models from a folder on reset for visual diversity. + * ``prepare_extra_attr``: Set up additional object attributes dynamically. Observation Manager ~~~~~~~~~~~~~~~~~~~ -While :class:`envs.EmbodiedEnv` provides default observations (robot state and raw sensor data), the Observation Manager allows you to inject task-specific information. +While :class:`envs.EmbodiedEnv` provides default observations organized into two groups: + +* **robot**: Contains ``qpos`` (joint positions), ``qvel`` (joint velocities), and ``qf`` (joint forces). +* **sensor**: Contains raw sensor outputs (images, depth, segmentation masks, etc.). + +The Observation Manager allows you to extend the observation space with task-specific information. Observations are configured using :class:`envs.managers.cfg.ObservationCfg` with two operation modes: + +* **modify**: Update existing observations in-place. The observation must already exist in the observation dictionary. Useful for normalization, transformation, or filtering existing data. Example: Normalize joint positions to [0, 1] range based on joint limits. +* **add**: Compute and add new observations to the observation space. The observation name can use hierarchical keys separated by ``/`` (e.g., ``"object/fork/pose"``). + +Common observation functors include: -You can configure it to compute and add: +**Pose Computations:** + * ``get_rigid_object_pose``: Get world poses of objects (returns 4x4 transformation matrices). + * ``get_sensor_pose_in_robot_frame``: Transform sensor poses to robot coordinate frame. - - Object poses relative to the robot. - - Keypoint positions. - - Task-specific metrics (e.g., distance to target). +**Relative Measurements:** + * ``get_relative_pose``: Compute relative poses between objects or robot parts. *Planned for future release.* + * ``get_distance``: Calculate distances between entities. *Planned for future release.* + +.. note:: + To get robot end-effector poses, you can use the robot's ``compute_fk()`` method directly in your observation functors or task code. + +**Keypoint Projections:** + * ``compute_exteroception``: Project 3D keypoints (affordance poses, robot parts) onto camera image planes. Supports multiple sources: affordance poses from objects (e.g., grasp poses, place poses) and robot control part poses (e.g., end-effector positions). + +**Normalization:** + * ``normalize_robot_joint_data``: Normalize joint positions/velocities based on limits. Dataset Manager ~~~~~~~~~~~~~~~ -For Imitation Learning (IL) tasks, the Dataset Manager automates data collection. It can: +For Imitation Learning (IL) tasks, the Dataset Manager automates data collection through dataset functors. It currently supports: + +* **LeRobot Format** (via :class:`envs.managers.datasets.LeRobotRecorder`): + Standard format for LeRobot training pipelines. Includes support for task instructions, robot metadata, success flags, and optional video recording. + +.. note:: + Additional dataset formats (HDF5, Zarr) are planned for future releases. - - Record full observation-action pairs. - - Save episodes automatically when environments terminate. - - Handle data formatting for training pipelines. +The manager operates in a single mode ``"save"`` which handles both recording and auto-saving: + +* **Recording**: On each environment step, observation-action pairs are buffered in memory. +* **Auto-saving**: When ``dones=True`` (episode completion), completed episodes are automatically saved to disk with proper formatting. + +**Configuration options include:** + * ``save_path``: Root directory for saving datasets. + * ``robot_meta``: Robot metadata dictionary (required for LeRobot format). + * ``instruction``: Task instruction dictionary. + * ``use_videos``: Whether to save video recordings of episodes. + * ``export_success_only``: Filter to save only successful episodes (based on ``info["success"]``). + +The dataset manager is called automatically during ``env.step()``, ensuring all observation-action pairs are recorded without additional user code. + +Action Bank +~~~~~~~~~~~~ + +The Action Bank is an advanced action composition system designed for complex manipulation tasks. It enables you to define and compose complex action sequences through **action graphs** and **configurable action primitives**, making it particularly useful for generating expert demonstrations and handling multi-step manipulation sequences. + +**Key Concepts:** + +* **Action Graph**: A directed graph structure where nodes represent affordances (target poses/states) and edges represent action primitives (trajectory segments) that connect affordances. The graph defines the possible action sequences for completing a task. + +* **Action Primitives**: Reusable action functions that generate trajectories between affordances. Each primitive is configured with duration, source affordance, target affordance, and optional parameters. + +* **Scopes**: Independent action executors (e.g., ``left_arm``, ``right_arm``, ``left_eef``) that can operate in parallel. Each scope has its own action graph and dimension. + +**Usage:** + +Action Bank is optional and typically used for: + +* Generating expert demonstration trajectories for Imitation Learning +* Defining complex multi-step manipulation sequences +* Composing actions from reusable primitives + +To use Action Bank in your environment: + +1. Create a custom ActionBank class inheriting from :class:`envs.action_bank.configurable_action.ActionBank` +2. Define node and edge functions using function tags (``@tag_node`` and ``@tag_edge`` decorators) +3. Initialize the action bank in your environment's ``__init__`` method using ``_init_action_bank()`` +4. Use ``action_bank.create_action_list()`` to generate action sequences + +.. note:: + Action Bank is an advanced feature primarily used for complex manipulation tasks. For simple tasks, you can use standard action spaces without Action Bank. Creating a Custom Task ---------------------- @@ -127,41 +213,41 @@ To create a new task, inherit from :class:`envs.EmbodiedEnv` and implement the t .. code-block:: python - from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg - from embodichain.lab.gym.utils.registration import register_env - - @register_env("MyTask-v0", max_episode_steps=500) - class MyTaskEnv(EmbodiedEnv): - def __init__(self, cfg: MyTaskEnvCfg, **kwargs): - super().__init__(cfg, **kwargs) - - def create_demo_action_list(self, *args, **kwargs): - # Optional: Implement for scripted demonstrations - # Must set self.action_length = len(action_list) if returning actions - pass - - def is_task_success(self, **kwargs): - # Optional: Define success criteria (mainly for IL data collection) - # Returns: torch.Tensor of shape (num_envs,) with boolean values - return success_tensor - - def get_reward(self, obs, action, info): - # Optional: Override for RL tasks - # Returns: torch.Tensor of shape (num_envs,) - return super().get_reward(obs, action, info) - - def get_info(self, **kwargs): - # Optional: Override to add custom info fields - # Should include "success" and "fail" keys for termination - info = super().get_info(**kwargs) - info["custom_metric"] = ... - return info + from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg + from embodichain.lab.gym.utils.registration import register_env + + @register_env("MyTask-v0", max_episode_steps=500) + class MyTaskEnv(EmbodiedEnv): + def __init__(self, cfg: MyTaskEnvCfg, **kwargs): + super().__init__(cfg, **kwargs) + + def create_demo_action_list(self, *args, **kwargs): + # Optional: Implement for scripted demonstrations + # Must set self.action_length = len(action_list) if returning actions + pass + + def is_task_success(self, **kwargs): + # Optional: Define success criteria (mainly for IL data collection) + # Returns: torch.Tensor of shape (num_envs,) with boolean values + return success_tensor + + def get_reward(self, obs, action, info): + # Optional: Override for RL tasks + # Returns: torch.Tensor of shape (num_envs,) + return super().get_reward(obs, action, info) + + def get_info(self, **kwargs): + # Optional: Override to add custom info fields + # Should include "success" and "fail" keys for termination + info = super().get_info(**kwargs) + info["custom_metric"] = ... + return info For a complete example of a modular environment setup, please refer to the :ref:`tutorial_modular_env` tutorial. See Also -------- -- :ref:`tutorial_create_basic_env` - Creating basic environments -- :ref:`tutorial_modular_env` - Advanced modular environment setup -- :doc:`/api_reference/embodichain/embodichain.lab.gym.envs` - Complete API reference for EmbodiedEnv and EmbodiedEnvCfg +* :ref:`tutorial_create_basic_env` - Creating basic environments +* :ref:`tutorial_modular_env` - Advanced modular environment setup +* :mod:`embodichain.lab.gym.envs` - Complete API reference for EmbodiedEnv and EmbodiedEnvCfg From 4d67da17fe720ffa91af7ff7e83b78a9e026ce2d Mon Sep 17 00:00:00 2001 From: yangchen73 <122090643@link.cuhk.edu.cn> Date: Thu, 15 Jan 2026 11:05:04 +0800 Subject: [PATCH 3/4] Restructure the organization of Embodied Environments's docs --- docs/source/overview/gym/env.md | 201 ++++++++++++++ docs/source/overview/gym/event_functors.md | 85 ++++++ docs/source/overview/gym/index.rst | 259 ++---------------- .../overview/gym/observation_functors.md | 90 ++++++ 4 files changed, 392 insertions(+), 243 deletions(-) create mode 100644 docs/source/overview/gym/env.md create mode 100644 docs/source/overview/gym/event_functors.md create mode 100644 docs/source/overview/gym/observation_functors.md diff --git a/docs/source/overview/gym/env.md b/docs/source/overview/gym/env.md new file mode 100644 index 0000000..243b30f --- /dev/null +++ b/docs/source/overview/gym/env.md @@ -0,0 +1,201 @@ +# Embodied Environments + +```{currentmodule} embodichain.lab.gym +``` + +The {class}`envs.EmbodiedEnv` is the core environment class in EmbodiChain designed for complex Embodied AI tasks. It adopts a **configuration-driven** architecture, allowing users to define robots, sensors, objects, lighting, and automated behaviors (events) purely through configuration classes, minimizing the need for boilerplate code. + +## Core Architecture + +Unlike the standard {class}`envs.BaseEnv`, the {class}`envs.EmbodiedEnv` integrates several manager systems to handle the complexity of simulation: + +* **Scene Management**: Automatically loads and manages robots, sensors, and scene objects defined in the configuration. +* **Event Manager**: Handles automated behaviors such as domain randomization, scene setup, and dynamic asset swapping. +* **Observation Manager**: Allows flexible extension of observation spaces without modifying the environment code. +* **Dataset Manager**: Built-in support for collecting demonstration data during simulation steps. + +## Configuration System + +The environment is defined by inheriting from {class}`envs.EmbodiedEnvCfg`. This configuration class serves as the single source of truth for the scene description. + +### EmbodiedEnvCfg Parameters + +The {class}`envs.EmbodiedEnvCfg` class exposes the following parameters: + +* **robot** ({class}`embodichain.lab.sim.cfg.RobotCfg`): + Defines the agent in the scene. Supports loading robots from URDF/MJCF with specified initial state and control mode. This is a required field. + +* **sensor** (List[{class}`embodichain.lab.sim.cfg.SensorCfg`]): + A list of sensors attached to the scene or robot. Common sensors include {class}`~embodichain.lab.sim.sensors.StereoCamera` for RGB-D and segmentation data. Defaults to an empty list. + +* **light** ({class}`envs.EmbodiedEnvCfg.EnvLightCfg`): + Configures the lighting environment. The {class}`EnvLightCfg` class contains: + + * ``direct``: List of direct light sources (Point, Spot, Directional) affecting local illumination. Defaults to an empty list. + * ``indirect``: Global illumination settings (Ambient, IBL) - *planned for future release*. + +* **rigid_object** (List[{class}`embodichain.lab.sim.cfg.RigidObjectCfg`]): + List of dynamic or kinematic simple bodies. Defaults to an empty list. + +* **rigid_object_group** (List[{class}`embodichain.lab.sim.cfg.RigidObjectGroupCfg`]): + Collections of rigid objects that can be managed together. Efficient for many similar objects. Defaults to an empty list. + +* **articulation** (List[{class}`embodichain.lab.sim.cfg.ArticulationCfg`]): + List of complex mechanisms with joints (doors, drawers). Defaults to an empty list. + +* **background** (List[{class}`embodichain.lab.sim.cfg.RigidObjectCfg`]): + Static or kinematic objects serving as obstacles or landmarks in the scene. Defaults to an empty list. + +* **events** (Union[object, None]): + Event settings for domain randomization and automated behaviors. Defaults to None, in which case no events are applied through the event manager. Please refer to the {class}`embodichain.lab.gym.managers.EventManager` class for more details. + +* **observations** (Union[object, None]): + Custom observation specifications. Defaults to None, in which case no additional observations are applied through the observation manager. Please refer to the {class}`embodichain.lab.gym.managers.ObservationManager` class for more details. + +* **dataset** (Union[object, None]): + Dataset collection settings. Defaults to None, in which case no dataset collection is performed. Please refer to the {class}`embodichain.lab.gym.managers.DatasetManager` class for more details. + +* **extensions** (Union[Dict[str, Any], None]): + Task-specific extension parameters that are automatically bound to the environment instance. This allows passing custom parameters (e.g., ``episode_length``, ``obs_mode``, ``action_scale``) without modifying the base configuration class. These parameters are accessible as instance attributes after environment initialization. For example, if ``extensions = {"episode_length": 500}``, you can access it via ``self.episode_length``. Defaults to None. + +* **filter_visual_rand** (bool): + Whether to filter out visual randomization functors. Useful for debugging motion and physics issues when visual randomization interferes with the debugging process. Defaults to ``False``. + +### Example Configuration + +```python +from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg +from embodichain.utils import configclass + +@configclass +class MyTaskEnvCfg(EmbodiedEnvCfg): + # 1. Define Scene Components + robot = ... # Robot configuration + sensor = [...] # List of sensors (e.g., Cameras) + light = ... # Lighting configuration + + # 2. Define Objects + rigid_object = [...] # Dynamic objects (e.g., tools, debris) + rigid_object_group = [...] # Object groups (efficient for many similar objects) + articulation = [...] # Articulated objects (e.g., cabinets) + + # 3. Define Managers + events = ... # Event settings (Randomization, etc.) + observations = ... # Custom observation spec + dataset = ... # Data collection settings + + # 4. Task Extensions + extensions = { # Task-specific parameters + "episode_length": 500, + "obs_mode": "state", + } +``` + +## Manager Systems + +The manager systems in {class}`envs.EmbodiedEnv` provide modular, configuration-driven functionality for handling complex simulation behaviors. Each manager uses a **functor-based** architecture, allowing you to compose behaviors through configuration without modifying environment code. Functors are reusable functions or classes (inheriting from {class}`envs.managers.Functor`) that operate on the environment state, configured through {class}`envs.managers.cfg.FunctorCfg`. + +### Event Manager + +The Event Manager automates changes in the environment through event functors. Events can be triggered at different stages: + +* **startup**: Executed once when the environment initializes. Useful for setting up initial scene properties that don't change during episodes. +* **reset**: Executed every time ``env.reset()`` is called. Applied to specific environments that need resetting (via ``env_ids`` parameter). This is the most common mode for domain randomization. +* **interval**: Executed periodically every N steps (specified by ``interval_step``, defaults to 10). Can be configured per-environment (``is_global=False``) or globally synchronized (``is_global=True``). + +Event functors are configured using {class}`envs.managers.cfg.EventCfg`. For a complete list of available event functors, please refer to {doc}`event_functors`. + +### Observation Manager + +While {class}`envs.EmbodiedEnv` provides default observations organized into two groups: + +* **robot**: Contains ``qpos`` (joint positions), ``qvel`` (joint velocities), and ``qf`` (joint forces). +* **sensor**: Contains raw sensor outputs (images, depth, segmentation masks, etc.). + +The Observation Manager allows you to extend the observation space with task-specific information. Observations are configured using {class}`envs.managers.cfg.ObservationCfg` with two operation modes: + +* **modify**: Update existing observations in-place. The observation must already exist in the observation dictionary. Useful for normalization, transformation, or filtering existing data. Example: Normalize joint positions to [0, 1] range based on joint limits. +* **add**: Compute and add new observations to the observation space. The observation name can use hierarchical keys separated by ``/`` (e.g., ``"object/fork/pose"``). + +For a complete list of available observation functors, please refer to {doc}`observation_functors`. + +### Dataset Manager + +For Imitation Learning (IL) tasks, the Dataset Manager automates data collection through dataset functors. It currently supports: + +* **LeRobot Format** (via {class}`envs.managers.datasets.LeRobotRecorder`): + Standard format for LeRobot training pipelines. Includes support for task instructions, robot metadata, success flags, and optional video recording. + +```{note} +Additional dataset formats (HDF5, Zarr) are planned for future releases. +``` + +The manager operates in a single mode ``"save"`` which handles both recording and auto-saving: + +* **Recording**: On each environment step, observation-action pairs are buffered in memory. +* **Auto-saving**: When ``dones=True`` (episode completion), completed episodes are automatically saved to disk with proper formatting. + +**Configuration options include:** + * ``save_path``: Root directory for saving datasets. + * ``robot_meta``: Robot metadata dictionary (required for LeRobot format). + * ``instruction``: Task instruction dictionary. + * ``use_videos``: Whether to save video recordings of episodes. + * ``export_success_only``: Filter to save only successful episodes (based on ``info["success"]``). + +The dataset manager is called automatically during ``env.step()``, ensuring all observation-action pairs are recorded without additional user code. + +## Creating a Custom Task + +To create a new task, inherit from {class}`envs.EmbodiedEnv` and implement the task-specific logic. + +```python +from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg +from embodichain.lab.gym.utils.registration import register_env + +@register_env("MyTask-v0", max_episode_steps=500) +class MyTaskEnv(EmbodiedEnv): + def __init__(self, cfg: MyTaskEnvCfg, **kwargs): + super().__init__(cfg, **kwargs) + + def create_demo_action_list(self, *args, **kwargs): + # Optional: Implement for expert demonstration data generation (for Imitation Learning) + # This method is used to generate scripted demonstrations for IL data collection. + # Must set self.action_length = len(action_list) if returning actions + pass + + def is_task_success(self, **kwargs): + # Optional: Define success criteria (mainly for IL data collection) + # Returns: torch.Tensor of shape (num_envs,) with boolean values + return success_tensor + + def get_reward(self, obs, action, info): + # Optional: Override for RL tasks + # Returns: torch.Tensor of shape (num_envs,) + return super().get_reward(obs, action, info) + + def get_info(self, **kwargs): + # Optional: Override to add custom info fields + # Should include "success" and "fail" keys for termination + info = super().get_info(**kwargs) + info["custom_metric"] = ... + return info +``` + +```{note} +The ``create_demo_action_list`` method is specifically designed for expert demonstration data generation in Imitation Learning scenarios. For Reinforcement Learning tasks, you should override the ``get_reward`` method instead. +``` + +For a complete example of a modular environment setup, please refer to the {ref}`tutorial_modular_env` tutorial. + +## See Also + +- {ref}`tutorial_create_basic_env` - Creating basic environments +- {ref}`tutorial_modular_env` - Advanced modular environment setup +- {doc}`/api_reference/embodichain/embodichain.lab.gym.envs` - Complete API reference for EmbodiedEnv and EmbodiedEnvCfg + +```{toctree} +:maxdepth: 1 + +event_functors.md +observation_functors.md +``` diff --git a/docs/source/overview/gym/event_functors.md b/docs/source/overview/gym/event_functors.md new file mode 100644 index 0000000..e1f5a63 --- /dev/null +++ b/docs/source/overview/gym/event_functors.md @@ -0,0 +1,85 @@ +# Event Functors + +```{currentmodule} embodichain.lab.gym.envs.managers +``` + +This page lists all available event functors that can be used with the Event Manager. Event functors are configured using {class}`envs.managers.cfg.EventCfg` and can be triggered at different stages: ``startup``, ``reset``, or ``interval``. + +## Physics Randomization + +```{list-table} Physics Randomization Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``randomize_rigid_object_mass`` + - Randomize object masses within a specified range. Supports both absolute and relative mass randomization. +``` + +## Visual Randomization + +```{list-table} Visual Randomization Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``randomize_visual_material`` + - Randomize textures, base colors, and material properties (metallic, roughness, IOR). Implemented as a Functor class. Supports both RigidObject and Articulation assets. +* - ``randomize_light`` + - Vary light position, color, and intensity within specified ranges. +* - ``randomize_camera_extrinsics`` + - Randomize camera poses for viewpoint diversity. Supports both attach mode (pos/euler perturbation) and look_at mode (eye/target/up perturbation). +* - ``randomize_camera_intrinsics`` + - Vary focal length (fx, fy) and principal point (cx, cy) within specified ranges. +``` + +## Spatial Randomization + +```{list-table} Spatial Randomization Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``randomize_rigid_object_pose`` + - Randomize object positions and orientations. Supports both relative and absolute pose randomization. +* - ``randomize_robot_eef_pose`` + - Vary end-effector initial poses by solving inverse kinematics. The randomization is performed relative to the current end-effector pose. +* - ``randomize_robot_qpos`` + - Randomize robot joint configurations. Supports both relative and absolute joint position randomization, and can target specific joints. +``` + +## Asset Management + +```{list-table} Asset Management Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``replace_assets_from_group`` + - Swap object models from a folder on reset for visual diversity. Currently supports RigidObject assets with mesh-based shapes. +* - ``prepare_extra_attr`` + - Set up additional object attributes dynamically. Supports both static values and callable functions. Useful for setting up affordance data and other custom attributes. +``` + +## Usage Example + +```python +from embodichain.lab.gym.envs.managers.cfg import EventCfg, SceneEntityCfg + +# Example: Randomize object mass on reset +events = { + "randomize_mass": EventCfg( + func="randomize_rigid_object_mass", + mode="reset", + params={ + "entity_cfg": SceneEntityCfg(uid="cube"), + "mass_range": (0.1, 2.0), + "relative": False, + }, + ), +} +``` diff --git a/docs/source/overview/gym/index.rst b/docs/source/overview/gym/index.rst index 6fe3290..e72a20f 100644 --- a/docs/source/overview/gym/index.rst +++ b/docs/source/overview/gym/index.rst @@ -1,253 +1,26 @@ -Embodied Environments -===================== +Gym +=================== .. currentmodule:: embodichain.lab.gym -The :class:`envs.EmbodiedEnv` is the core environment class in EmbodiChain designed for complex Embodied AI tasks. It adopts a **configuration-driven** architecture, allowing users to define robots, sensors, objects, lighting, and automated behaviors (events) purely through configuration classes, minimizing the need for boilerplate code. - -Core Architecture ------------------ - -Unlike the standard :class:`envs.BaseEnv`, the :class:`envs.EmbodiedEnv` integrates several manager systems to handle the complexity of simulation: - -* **Scene Management**: Automatically loads and manages robots, sensors, and scene objects defined in the configuration. -* **Event Manager**: Handles automated behaviors such as domain randomization, scene setup, and dynamic asset swapping. -* **Observation Manager**: Allows flexible extension of observation spaces without modifying the environment code. -* **Dataset Manager**: Built-in support for collecting demonstration data during simulation steps. -* **Action Bank**: Advanced action composition system supporting action graphs and configurable action primitives (optional, for complex manipulation sequences). - -Configuration System --------------------- - -The environment is defined by inheriting from :class:`envs.EmbodiedEnvCfg`. This configuration class serves as the single source of truth for the scene description. - -.. code-block:: python - - from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg - from embodichain.utils import configclass - - @configclass - class MyTaskEnvCfg(EmbodiedEnvCfg): - # 1. Define Scene Components - robot = ... # Robot configuration - sensor = [...] # List of sensors (e.g., Cameras) - light = ... # Lighting configuration - - # 2. Define Objects - rigid_object = [...] # Dynamic objects (e.g., tools, debris) - rigid_object_group = [...] # Object groups (efficient for many similar objects) - articulation = [...] # Articulated objects (e.g., cabinets) - - # 3. Define Managers - events = ... # Event settings (Randomization, etc.) - observations = ... # Custom observation spec - dataset = ... # Data collection settings - - # 4. Task Extensions - extensions = { # Task-specific parameters - "episode_length": 500, - "obs_mode": "state", - } - -Key Components -~~~~~~~~~~~~~~ - -The :class:`envs.EmbodiedEnvCfg` exposes several key fields to compose your environment: - -* **robot** (:class:`embodichain.lab.sim.cfg.RobotCfg`): - Defines the agent in the scene. Supports loading robots from URDF/MJCF with specified initial state and control mode. - -* **sensor** (List[:class:`embodichain.lab.sim.cfg.SensorCfg`]): - A list of sensors attached to the scene or robot. Common sensors include :class:`~embodichain.lab.sim.sensors.StereoCamera` for RGB-D and segmentation data. - -* **light** (:class:`envs.EmbodiedEnvCfg.EnvLightCfg`): - Configures the lighting environment. - - * ``direct``: Direct light sources (Point, Spot, Directional) affecting local illumination. - * ``indirect``: Global illumination settings (Ambient, IBL) - *planned for future release*. - -* **rigid_object** / **rigid_object_group** / **articulation**: - Defines the interactive elements. - - * ``rigid_object``: For dynamic or kinematic simple bodies. - * ``rigid_object_group``: Collections of rigid objects that can be managed together. - * ``articulation``: For complex mechanisms with joints (doors, drawers). - -* **background** (List[:class:`embodichain.lab.sim.cfg.RigidObjectCfg`]): - Static or kinematic objects serving as obstacles or landmarks in the scene. - -* **extensions** (Dict[str, Any]): - Task-specific extension parameters that are automatically bound to the environment instance. This allows passing custom parameters (e.g., ``episode_length``, ``obs_mode``, ``action_scale``) without modifying the base configuration class. These parameters are accessible as instance attributes after environment initialization. For example, if ``extensions = {"episode_length": 500}``, you can access it via ``self.episode_length``. - -* **filter_visual_rand** (bool): - Whether to filter out visual randomization functors. Useful for debugging motion and physics issues when visual randomization interferes with the debugging process. Defaults to ``False``. - -Manager Systems ---------------- - -The manager systems in :class:`envs.EmbodiedEnv` provide modular, configuration-driven functionality for handling complex simulation behaviors. Each manager uses a **functor-based** architecture, allowing you to compose behaviors through configuration without modifying environment code. Functors are reusable functions or classes (inheriting from :class:`envs.managers.Functor`) that operate on the environment state, configured through :class:`envs.managers.cfg.FunctorCfg`. - -Event Manager -~~~~~~~~~~~~~ - -The Event Manager automates changes in the environment through event functors. Events can be triggered at different stages: - -* **startup**: Executed once when the environment initializes. Useful for setting up initial scene properties that don’t change during episodes. -* **reset**: Executed every time ``env.reset()`` is called. Applied to specific environments that need resetting (via ``env_ids`` parameter). This is the most common mode for domain randomization. -* **interval**: Executed periodically every N steps (specified by ``interval_step``, defaults to 10). Can be configured per-environment (``is_global=False``) or globally synchronized (``is_global=True``). - -Event functors are configured using :class:`envs.managers.cfg.EventCfg`. Common event functors include: - -**Physics Randomization:** - * ``randomize_rigid_object_mass``: Randomize object masses within a range. - * ``randomize_rigid_object_friction``: Vary friction coefficients for more robust sim-to-real transfer. *Planned for future release.* - * ``randomize_rigid_object_restitution``: Adjust bounciness of objects. *Planned for future release.* - -**Visual Randomization:** - * ``randomize_visual_material``: Randomize textures, base colors, and material properties (implemented as a Functor class). - * ``randomize_light``: Vary light position, color, and intensity. - * ``randomize_camera_extrinsics``: Randomize camera poses for viewpoint diversity. - * ``randomize_camera_intrinsics``: Vary focal length and principal point. - -**Spatial Randomization:** - * ``randomize_rigid_object_pose``: Randomize object positions and orientations. - * ``randomize_robot_eef_pose``: Vary end-effector initial poses. - * ``randomize_robot_qpos``: Randomize robot joint configurations. - -**Asset Management:** - * ``replace_assets_from_group``: Swap object models from a folder on reset for visual diversity. - * ``prepare_extra_attr``: Set up additional object attributes dynamically. - -Observation Manager -~~~~~~~~~~~~~~~~~~~ - -While :class:`envs.EmbodiedEnv` provides default observations organized into two groups: - -* **robot**: Contains ``qpos`` (joint positions), ``qvel`` (joint velocities), and ``qf`` (joint forces). -* **sensor**: Contains raw sensor outputs (images, depth, segmentation masks, etc.). +The ``gym`` module provides a comprehensive framework for creating robot learning environments. It extends the Gymnasium interface to support multi-environment parallel execution, custom observations, and robotic-specific functionality. -The Observation Manager allows you to extend the observation space with task-specific information. Observations are configured using :class:`envs.managers.cfg.ObservationCfg` with two operation modes: +Environment Classes +------------------- -* **modify**: Update existing observations in-place. The observation must already exist in the observation dictionary. Useful for normalization, transformation, or filtering existing data. Example: Normalize joint positions to [0, 1] range based on joint limits. -* **add**: Compute and add new observations to the observation space. The observation name can use hierarchical keys separated by ``/`` (e.g., ``"object/fork/pose"``). +Base Environments +~~~~~~~~~~~~~~~~~ -Common observation functors include: +- :class:`envs.BaseEnv` - Foundational environment class that provides core functionality for all EmbodiChain RL environments +- :class:`envs.EnvCfg` - Configuration class for basic environment settings -**Pose Computations:** - * ``get_rigid_object_pose``: Get world poses of objects (returns 4x4 transformation matrices). - * ``get_sensor_pose_in_robot_frame``: Transform sensor poses to robot coordinate frame. - -**Relative Measurements:** - * ``get_relative_pose``: Compute relative poses between objects or robot parts. *Planned for future release.* - * ``get_distance``: Calculate distances between entities. *Planned for future release.* - -.. note:: - To get robot end-effector poses, you can use the robot's ``compute_fk()`` method directly in your observation functors or task code. - -**Keypoint Projections:** - * ``compute_exteroception``: Project 3D keypoints (affordance poses, robot parts) onto camera image planes. Supports multiple sources: affordance poses from objects (e.g., grasp poses, place poses) and robot control part poses (e.g., end-effector positions). - -**Normalization:** - * ``normalize_robot_joint_data``: Normalize joint positions/velocities based on limits. - -Dataset Manager -~~~~~~~~~~~~~~~ - -For Imitation Learning (IL) tasks, the Dataset Manager automates data collection through dataset functors. It currently supports: - -* **LeRobot Format** (via :class:`envs.managers.datasets.LeRobotRecorder`): - Standard format for LeRobot training pipelines. Includes support for task instructions, robot metadata, success flags, and optional video recording. - -.. note:: - Additional dataset formats (HDF5, Zarr) are planned for future releases. - -The manager operates in a single mode ``"save"`` which handles both recording and auto-saving: - -* **Recording**: On each environment step, observation-action pairs are buffered in memory. -* **Auto-saving**: When ``dones=True`` (episode completion), completed episodes are automatically saved to disk with proper formatting. - -**Configuration options include:** - * ``save_path``: Root directory for saving datasets. - * ``robot_meta``: Robot metadata dictionary (required for LeRobot format). - * ``instruction``: Task instruction dictionary. - * ``use_videos``: Whether to save video recordings of episodes. - * ``export_success_only``: Filter to save only successful episodes (based on ``info["success"]``). - -The dataset manager is called automatically during ``env.step()``, ensuring all observation-action pairs are recorded without additional user code. - -Action Bank -~~~~~~~~~~~~ - -The Action Bank is an advanced action composition system designed for complex manipulation tasks. It enables you to define and compose complex action sequences through **action graphs** and **configurable action primitives**, making it particularly useful for generating expert demonstrations and handling multi-step manipulation sequences. - -**Key Concepts:** - -* **Action Graph**: A directed graph structure where nodes represent affordances (target poses/states) and edges represent action primitives (trajectory segments) that connect affordances. The graph defines the possible action sequences for completing a task. - -* **Action Primitives**: Reusable action functions that generate trajectories between affordances. Each primitive is configured with duration, source affordance, target affordance, and optional parameters. - -* **Scopes**: Independent action executors (e.g., ``left_arm``, ``right_arm``, ``left_eef``) that can operate in parallel. Each scope has its own action graph and dimension. - -**Usage:** - -Action Bank is optional and typically used for: - -* Generating expert demonstration trajectories for Imitation Learning -* Defining complex multi-step manipulation sequences -* Composing actions from reusable primitives - -To use Action Bank in your environment: - -1. Create a custom ActionBank class inheriting from :class:`envs.action_bank.configurable_action.ActionBank` -2. Define node and edge functions using function tags (``@tag_node`` and ``@tag_edge`` decorators) -3. Initialize the action bank in your environment's ``__init__`` method using ``_init_action_bank()`` -4. Use ``action_bank.create_action_list()`` to generate action sequences - -.. note:: - Action Bank is an advanced feature primarily used for complex manipulation tasks. For simple tasks, you can use standard action spaces without Action Bank. - -Creating a Custom Task ----------------------- - -To create a new task, inherit from :class:`envs.EmbodiedEnv` and implement the task-specific logic. - -.. code-block:: python - - from embodichain.lab.gym.envs import EmbodiedEnv, EmbodiedEnvCfg - from embodichain.lab.gym.utils.registration import register_env - - @register_env("MyTask-v0", max_episode_steps=500) - class MyTaskEnv(EmbodiedEnv): - def __init__(self, cfg: MyTaskEnvCfg, **kwargs): - super().__init__(cfg, **kwargs) - - def create_demo_action_list(self, *args, **kwargs): - # Optional: Implement for scripted demonstrations - # Must set self.action_length = len(action_list) if returning actions - pass - - def is_task_success(self, **kwargs): - # Optional: Define success criteria (mainly for IL data collection) - # Returns: torch.Tensor of shape (num_envs,) with boolean values - return success_tensor - - def get_reward(self, obs, action, info): - # Optional: Override for RL tasks - # Returns: torch.Tensor of shape (num_envs,) - return super().get_reward(obs, action, info) - - def get_info(self, **kwargs): - # Optional: Override to add custom info fields - # Should include "success" and "fail" keys for termination - info = super().get_info(**kwargs) - info["custom_metric"] = ... - return info +Embodied Environments +~~~~~~~~~~~~~~~~~~~~~ -For a complete example of a modular environment setup, please refer to the :ref:`tutorial_modular_env` tutorial. +- :class:`envs.EmbodiedEnv` - Advanced environment class for complex Embodied AI tasks with configuration-driven architecture +- :class:`envs.EmbodiedEnvCfg` - Configuration class for Embodied Environments -See Also --------- +.. toctree:: + :maxdepth: 1 -* :ref:`tutorial_create_basic_env` - Creating basic environments -* :ref:`tutorial_modular_env` - Advanced modular environment setup -* :mod:`embodichain.lab.gym.envs` - Complete API reference for EmbodiedEnv and EmbodiedEnvCfg + env.md \ No newline at end of file diff --git a/docs/source/overview/gym/observation_functors.md b/docs/source/overview/gym/observation_functors.md new file mode 100644 index 0000000..a3d6c26 --- /dev/null +++ b/docs/source/overview/gym/observation_functors.md @@ -0,0 +1,90 @@ +# Observation Functors + +```{currentmodule} embodichain.lab.gym.envs.managers +``` + +This page lists all available observation functors that can be used with the Observation Manager. Observation functors are configured using {class}`envs.managers.cfg.ObservationCfg` and can operate in two modes: ``modify`` (update existing observations) or ``add`` (add new observations). + +## Pose Computations + +```{list-table} Pose Computation Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``get_rigid_object_pose`` + - Get world poses of rigid objects. Returns 4x4 transformation matrices of shape (num_envs, 4, 4). If the object doesn't exist, returns a zero tensor. +* - ``get_sensor_pose_in_robot_frame`` + - Transform sensor poses to robot coordinate frame. Returns pose as [x, y, z, qw, qx, qy, qz] of shape (num_envs, 7). +``` + +## Sensor Information + +```{list-table} Sensor Information Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``get_sensor_intrinsics`` + - Get the intrinsic matrix of a camera sensor. Returns 3x3 intrinsic matrices of shape (num_envs, 3, 3). For stereo cameras, supports selecting left or right camera intrinsics. +* - ``compute_semantic_mask`` + - Compute semantic masks from camera segmentation masks. Returns masks of shape (num_envs, height, width, 3) with channels for robot, background, and foreground objects. +``` + +## Keypoint Projections + +```{list-table} Keypoint Projection Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``compute_exteroception`` + - Project 3D keypoints (affordance poses, robot parts) onto camera image planes. Supports multiple sources: affordance poses from objects (e.g., grasp poses, place poses) and robot control part poses (e.g., end-effector positions). Returns normalized 2D coordinates. Implemented as a Functor class. +``` + +## Normalization + +```{list-table} Normalization Functors +:header-rows: 1 +:widths: 30 70 + +* - Functor Name + - Description +* - ``normalize_robot_joint_data`` + - Normalize joint positions or velocities to [0, 1] range based on joint limits. Supports both ``qpos_limits`` and ``qvel_limits``. Operates in ``modify`` mode. +``` + +```{note} +To get robot end-effector poses, you can use the robot's ``compute_fk()`` method directly in your observation functors or task code. +``` + +## Usage Example + +```python +from embodichain.lab.gym.envs.managers.cfg import ObservationCfg, SceneEntityCfg + +# Example: Add object pose to observations +observations = { + "object_pose": ObservationCfg( + func="get_rigid_object_pose", + mode="add", + name="object/cube/pose", + params={ + "entity_cfg": SceneEntityCfg(uid="cube"), + }, + ), + # Example: Normalize joint positions + "normalized_qpos": ObservationCfg( + func="normalize_robot_joint_data", + mode="modify", + name="robot/qpos", + params={ + "joint_ids": list(range(7)), # First 7 joints + "limit": "qpos_limits", + }, + ), +} +``` From 0420116a5b35b7964bf54c760e9c205cdf26e123 Mon Sep 17 00:00:00 2001 From: yangchen73 <122090643@link.cuhk.edu.cn> Date: Thu, 15 Jan 2026 19:55:29 +0800 Subject: [PATCH 4/4] Add BaseEnvCfg parameters documentation --- docs/source/overview/gym/env.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/docs/source/overview/gym/env.md b/docs/source/overview/gym/env.md index 243b30f..e540a44 100644 --- a/docs/source/overview/gym/env.md +++ b/docs/source/overview/gym/env.md @@ -18,9 +18,30 @@ Unlike the standard {class}`envs.BaseEnv`, the {class}`envs.EmbodiedEnv` integra The environment is defined by inheriting from {class}`envs.EmbodiedEnvCfg`. This configuration class serves as the single source of truth for the scene description. +{class}`envs.EmbodiedEnvCfg` inherits from {class}`envs.EnvCfg` (the base environment configuration class, sometimes referred to as `BaseEnvCfg`), which provides fundamental environment parameters. The following sections describe both the base class parameters and the additional parameters specific to {class}`envs.EmbodiedEnvCfg`. + +### BaseEnvCfg Parameters + +Since {class}`envs.EmbodiedEnvCfg` inherits from {class}`envs.EnvCfg`, it includes the following base parameters: + +* **num_envs** (int): + The number of sub environments (arenas) to be simulated in parallel. Defaults to ``1``. + +* **sim_cfg** ({class}`embodichain.lab.sim.cfg.SimulationManagerCfg`): + Simulation configuration for the environment, including physics settings, device selection, and rendering options. Defaults to a basic configuration with headless mode enabled. + +* **seed** (int | None): + The seed for the random number generator. Defaults to ``None``, in which case the seed is not set. The seed is set at the beginning of the environment initialization to ensure deterministic behavior across different runs. + +* **sim_steps_per_control** (int): + Number of simulation steps per control (environment) step. This parameter determines the relationship between the simulation timestep and the control timestep. For instance, if the simulation dt is 0.01s and the control dt is 0.1s, then ``sim_steps_per_control`` should be 10. This means that the control action is updated every 10 simulation steps. Defaults to ``4``. + +* **ignore_terminations** (bool): + Whether to ignore terminations when deciding when to auto reset. Terminations can be caused by the task reaching a success or fail state as defined in a task's evaluation function. If set to ``False``, episodes will stop early when termination conditions are met. If set to ``True``, episodes will only stop due to the timelimit, which is useful for modeling tasks as infinite horizon. Defaults to ``False``. + ### EmbodiedEnvCfg Parameters -The {class}`envs.EmbodiedEnvCfg` class exposes the following parameters: +The {class}`envs.EmbodiedEnvCfg` class exposes the following additional parameters: * **robot** ({class}`embodichain.lab.sim.cfg.RobotCfg`): Defines the agent in the scene. Supports loading robots from URDF/MJCF with specified initial state and control mode. This is a required field.