Create a sim environment

This tutorial walks you through authoring a SimScene YAML — the on-disk (robot × scene × task) tuple — that openral sim run consumes together with an rSkill (--rskill rskills/<id>), so you can test a VLA / rSkill against a task in simulation without touching hardware. The runtime form the adapters see is the composed SimEnvironment (SimScene + RSkillManifest); the YAML on disk never carries a vla: block.

SimScene is the middle tier of the three-tier scene hierarchy:

DeployScene  ⊆  SimScene  ⊆  BenchmarkScene
  (deploy run)    (sim run)    (benchmark scene / benchmark run)

This page focuses on the SimScene tier — the ad-hoc, single-rollout shape consumed by openral sim run. Adding metadata: {paper, honest_scope} + non-None seed and n_episodes to a SimScene YAML turns it into a BenchmarkScene that openral benchmark scene accepts; dropping the task: block turns it into a DeployScene that openral deploy sim accepts. Per-tier loaders refuse wrong-tier YAMLs at parse time.

It covers six things, in increasing depth:

The openral sim run flag surface and what is registry-resolved (i.e. not hardcoded).
Authoring a SimScene YAML for an existing robot, scene, task, and pairing it with an rSkill.
Bringing a new robot manifest (robots/<id>/robot.yaml) into sim.
Writing a new scene adapter (a new task suite or simulator wrapper) in Python, for cases that need custom robot, task, or physics behavior.
Writing a new policy adapter (a new VLA backend) and matching it to an rSkill.

The companion cookbook is scenes/README.md.

1. What `openral sim run` accepts

openral sim run is defined in python/sim/src/openral_sim/cli.py. Every axis — robot, scene, task, VLA, physics backend, device — is resolved at runtime through one of three registries (no hardcoded IDs):

Registry	Defined in	Populated by
`SCENES`	`python/sim/src/openral_sim/registry.py`	`@SCENES.register("<id>")` decorators in `backends/*.py`
`POLICIES`	same	`@POLICIES.register("<id>")` decorators in `policies/*.py`
`ROBOTS`	same	Auto-discovered from `robots/<id>/robot.yaml` at import time

List everything that is currently registered on your install:

openral sim list

openral sim list is a sibling subcommand to openral sim run; it prints the three registries (scenes / policies / robots) and exits without touching OTel or the runtime path.

Flags (`openral sim run`)

--config PATH                Path to a SimScene YAML (REQUIRED; strict —
                             DeployScene / BenchmarkScene YAMLs are rejected
                             with a redirect message pointing at
                             `openral deploy sim` / `openral benchmark scene`).
--rskill <weights_uri>       rSkill reference: rskills/<id>, bare name,
                             or HF repo id  (REQUIRED).
--robot ID                   robot_id override for free-axis scenes
                             (rejected on scenes that hard-fix a robot,
                              and on YAMLs that already set robot_id:).
--task  ID                   Override task.id (e.g. libero_spatial/3).
--instruction TEXT           Override the natural-language task instruction.
                             Wins over a scene's per-episode language (a
                             RoboCasa sampled-object string) — see §4.
--max-steps  N               Override task.max_steps.
--n-episodes N               Override SimScene.n_episodes.
--seed       N               Override the global seed.
--device     {cpu, cuda:0, mps, auto}
                             Torch device for the policy.
--save-dir   DIR             Where to write the JSON summary.
--save-video [PATH]          Write a per-episode MP4 (also enables frame capture).
--video-style {debug,world}  debug (default) = 3-panel montage; world = clean
                             single-view world render named
                             <scene>_<rskill>_<success|fail>.mp4 + videos.json
                             (for website hero clips; overlays drawn by the page).
--video-size INT             Square edge (px) for --video-style world (default 1024).
--view / --no-view           Open a passive mujoco.viewer.
--verbose / -v               DEBUG logging.

Canonical invocation

openral sim run --config scenes/sim/libero_spatial.yaml \
            --rskill rskills/smolvla-libero

Both --config and --rskill are required — openral sim run always composes a runtime SimEnvironment from a SimScene (the YAML) and an RSkillManifest (the rSkill). There is no bare-CLI invocation path; supply the scene + task in the YAML.

Per-axis overrides on top of `--config`

Beyond the two required flags, the remaining options — --robot (only on free-axis scenes), --task, --instruction, --max-steps, --n-episodes, --seed, --device, --save-dir, --save-video — overlay the loaded config (see _load_or_build_env in cli.py), so a single YAML can drive an entire task suite:

# Same VLA + scene; iterate through tasks.
for i in 0 1 2 3; do
    openral sim run --config scenes/sim/libero_spatial.yaml --rskill smolvla-libero --task "libero_spatial/$i"
done

If you need to swap an axis that is baked in, copy the YAML and edit it — that is the supported pattern.

Startup performance (`activate()` parallelisation)

SimRunner.activate() builds the env (MuJoCo XML compile / dataset prefetch) and the policy (PaliGemma / NF4 quantization on π0.5, weights load on SmolVLA) concurrently on a 2-worker ThreadPoolExecutor. Both factories only read the immutable SimEnvironment and share no mutable state, so the wall-clock for activate collapses to max(env_ms, policy_ms) instead of their sum — a meaningful win on LIBERO / RoboCasa / GR1-tabletop where each side is 30–150 s.

On every invocation the runner logs a structured sim_init_parallel record with env_ms, policy_ms, total_ms, and saved_ms, so you can confirm the win for your specific (scene, VLA) combination.

To force the legacy sequential path (e.g. when interleaved logs would obscure a profiling investigation), set OPENRAL_SIM_SEQUENTIAL_INIT=1:

OPENRAL_SIM_SEQUENTIAL_INIT=1 openral sim run --config scenes/sim/libero_spatial.yaml --rskill pi05-libero-int8

See GH-134.

2. Author a SimScene YAML

The on-disk shape is a SimScene — (robot × scene × task) — defined at python/core/src/openral_core/schemas.py:6652. At runtime the CLI composes it with the rSkill manifest (--rskill) into a SimEnvironment (schemas.py:6425) that adapter factories consume. Loading a YAML that carries a vla: block raises ROSConfigError — policy always travels on the CLI, not in the YAML.

Take an existing config as your starting point — for example, scenes/sim/libero_spatial.yaml:

# SimScene = scene + task. Policy is supplied at the CLI via
#   --rskill rskills/<id>

# robot_id: free-axis scenes only (LIBERO/MetaWorld/RoboCasa hard-fix the
# robot in the registry; setting it here on those scenes is a ROSConfigError).
# robot_id: franka_panda

scene:
  id: libero_spatial              # key into SCENES
  backend: mujoco                 # PhysicsBackend enum
  observation_height: 256
  observation_width: 256
  cameras: ["top", "wrist"]        # optional; SceneSpec.cameras defaults to []

task:
  id: libero_spatial/0            # adapter splits on "/" to resolve
  scene_id: libero_spatial        # MUST equal scene.id (validated post-init)
  instruction: ""                 # LIBERO overrides from suite metadata
  max_steps: 100
  success_key: is_success         # which info[] key marks success

seed: 42                          # optional on SimScene; defaults to 0
n_episodes: 1                     # optional on SimScene; defaults to 1
record_video: false               # optional; defaults to false

To promote this same scene to paper-comparable form, add a metadata: BenchmarkMetadata block + set seed and n_episodes to canonical values, then move the file to scenes/benchmark/. The benchmark-tier sibling lives at scenes/benchmark/libero_spatial.yaml and adds:

seed: 0                           # required (no default)
n_episodes: 500                   # required (no default; paper protocol)
metadata:
  paper: "https://arxiv.org/abs/2309.11500"
  honest_scope: "..."             # honest scope statement; required

openral sim run loads the sim-tier sibling; openral benchmark scene loads the benchmark-tier sibling. The per-tier loader (load_scene_strict) rejects wrong-tier YAMLs — a BenchmarkScene YAML passed to openral sim run returns a redirect message pointing at openral benchmark scene, and vice versa.

The required blocks

Block	Schema	Required keys
`scene`	`SceneSpec` (`schemas.py:6109`)	`id`; `backend` defaults to `mujoco`
`task`	`TaskSpec` (`schemas.py:6327`)	`id`, `scene_id` (must equal `scene.id`)
`robot_id`	string, key into `ROBOTS`	Only on free-axis scenes — and only if you want to bake the robot into the YAML rather than passing `--robot`. Forbidden on `fixed_robot` scenes (LIBERO/MetaWorld/RoboCasa).

Policy is not a YAML block; it is supplied at the CLI as --rskill rskills/<id> → resolves to an RSkillManifest (schemas.py — search for class RSkillManifest) and is composed onto the runtime SimEnvironment.vla by _load_or_build_env in cli.py. The Pydantic registries reject unknown ids with a list of valid ones, so typos surface immediately. Run it:

openral sim run --config my_config.yaml --rskill rskills/<your_skill>

You will see a per-episode summary line and a 0 exit code on success. The same runtime pattern also drives openral benchmark run. openral deploy run is the hardware sibling; it consumes a DeployScene workcell YAML and lets the reasoner select policy at runtime.

3. Add a new robot manifest

Robots are auto-registered from robots/<id>/robot.yaml at import time — no Python edit required. The discovery loop lives at python/sim/src/openral_sim/policies/robots.py:71-125. The search path is, in order:

$OPENRAL_ROBOTS_DIR/<id>/robot.yaml (if the env var is set)
<repo_root>/robots/<id>/robot.yaml

Use robots/so100_follower/robot.yaml as a small (6-DoF + gripper) template; use robots/franka_panda/robot.yaml for a 7-DoF arm. The required top-level blocks are:

Block	Purpose
`name`, `embodiment_kind`, `base_frame`	Identity + URDF root frame
`joints[]`	Per-joint: name, type, parent/child links, axis, limits, actuator
`end_effectors[]`	Gripper(s) / hand(s); kind, DoF, force/payload limits
`sensors[]`	Cameras / IMUs / etc.; each maps to a `vla_feature_key`
`capabilities`	Control modes, embodiment tags, lift / dexterity flags
`safety`	Workspace box, speed/force/torque limits, deadman flag
`observation_spec`, `action_spec`	State / action shapes and representations
`assets` (optional)	URDF / MJCF / SRDF reference block — see below
`sim` (optional)	MuJoCo joint↔qpos wiring consumed by `MujocoArmHAL.from_description` — see below

The `assets:` block

The robot's URDF / MJCF / SRDF are named once, at the top level, via the unified assets: block. Every ref shares the openral_core.assets.resolve_asset grammar:

assets:
  # The MJCF that MujocoArmHAL loads. One of:
  #   rd:<module>          — robot_descriptions package (downloads on first use)
  #   gym_aloha:<scene>    — gym-aloha package asset
  #   openarm:bimanual     — Enactic OpenArm v2 (fetched on first use)
  #   file:<relpath>       — repo/manifest-relative explicit override
  mjcf: "rd:ur5e_mj_description"
  # Optional URDF (robot_state_publisher / collision lowering):
  # urdf:
  #   ref: "file:ur5e.urdf"          # or rd:<module> / ros2://robot_description
  #   root_frame: "base_link"         # robot_state_publisher root-frame wiring
  #   base_to_root_xyz_rpy: [0, 0, 0, 0, 0, 0]
  # Optional SRDF (seeds allowed_collision_pairs):
  # srdf: "file:ur5e.srdf"

The `sim:` block

For any robot that should drive a MuJoCo digital twin through the shared MujocoArmHAL base, declare a sim: block alongside assets.mjcf. The runner reads it directly; no per-robot Python file is required — pass the loaded manifest into MujocoArmHAL.from_description(desc).

sim:
  # The MJCF itself is named by `assets.mjcf` (above); this block carries
  # only the joint↔qpos/qvel/actuator plumbing.

  # Floating-base humanoids (G1, H1): qpos offset 7, qvel offset 6 are
  # derived automatically.  Single-arm robots can omit.
  floating_base: false

  # Explicit joint→qpos / →actuator overrides — only needed when the
  # MJCF declares joints in an order other than ``description.joints`` or
  # leaves passive follower qpos slots in between (OpenArm).
  # joint_qpos_addr:
  #   joint_a: 0
  #   joint_b: 1
  # actuator_index:
  #   joint_a: 0

  grippers:                                    # zero, one, or two entries
    - joint: "panda_gripper"                   # name from joints[]
      ctrl_range: [0.0, 255.0]
      qpos_addrs: [7, 8]                       # finger qpos indices
      qpos_scale: 0.08                         # 2 * 0.04 m max extent
      read_mode: "sum_over_scale"              # | "affine_low_high" | "passthrough"
      write_mode: "normalised"                 # | "passthrough"
      # actuator_index: 7                      # override; defaults to actuator_index map
      # mirror_actuator_index: 15              # Aloha: writes -ctrl to the negative finger

  # Connect-time hooks:
  # keyframe_index: 0          # mj_resetDataKeyframe(model, data, idx) — Aloha
  # seed_ctrl_from_qpos: true  # ctrl = qpos on connect — OpenArm v2

Single source of truth. When you add a <ROBOT>_DESCRIPTION Python constant (e.g. UR5e_DESCRIPTION in openral_hal/ur.py), mirror the sim block via sim=SimDescription(...) so the manifest-vs-YAML drift guard (tests/unit/test_robot_manifests_match_hal_constants.py, plus tests/sim/test_data_driven_mujoco_hal.py::test_python_description_matches_yaml) stays green.

Worked examples in tree (use as templates):

Robot	Pattern
`so100_follower`	single-arm + revolute Jaw (`read_mode: affine_low_high`)
`franka_panda`	single-arm + parallel gripper (`read_mode: sum_over_scale`)
`ur5e`, `ur10e`, `rizon4`	single-arm, no gripper, no overrides
`g1`, `h1`	floating-base humanoid (`floating_base: true`)
`aloha_bimanual`	bimanual + two passthrough grippers with `mirror_actuator_index` + `keyframe_index: 0`
`openarm`	bimanual + two passthrough grippers + explicit `joint_qpos_addr` skipping passive follower fingers + `seed_ctrl_from_qpos: true`

Drop a new robot in

mkdir -p robots/my_arm
$EDITOR robots/my_arm/robot.yaml        # copy & adapt so100_follower/robot.yaml
$EDITOR robots/my_arm/README.md         # pair the manifest with adapter notes

Verify it registered

openral sim list | grep "robots:"
# should now include `my_arm`

You can also confirm the manifest loads cleanly from Python:

from openral_sim import ROBOTS
robot = ROBOTS.get("my_arm")()         # invokes the cached factory
print(robot.name, len(robot.joints))

Match the manifest to a sim scene

Every scene adapter expects a specific embodiment. LIBERO assumes a 7-DoF arm with a parallel gripper; MetaWorld assumes the Sawyer. For your new robot to run end-to-end you also need either (a) a scene adapter that knows how to drive it, or (b) the mock scene, which accepts any action dimensionality (see §4).

4. When the existing scene adapters are not enough

Reach for the Python adapter path (Section 5) when the registered scene catalogue cannot express the task you need. Common reasons:

You want a completely different arena (no LIBERO floor / table).
You want a different robot than a benchmark suite hard-wires.
You want objects or fixtures the existing suite adapter cannot spawn.
You want physics independent of robosuite (e.g. a mock scene or a non-MuJoCo backend).

5. Write a custom scene adapter

A scene adapter is a function decorated with @SCENES.register("<id>") that returns an object satisfying the SimRollout Protocol:

class SimRollout(Protocol):
    scene: SceneSpec
    task:  TaskSpec
    def reset(self, seed: int | None = ...) -> Observation: ...
    def step(self, action: NDArray[np.float32]) -> StepResult: ...
    def render(self) -> NDArray[np.uint8] | None: ...
    def close(self) -> None: ...

Observation is a free-form dict; adapters SHOULD include "images" (dict of HWC uint8 RGB frames), "state" (1-D float32), and "task" (natural-language instruction). StepResult is a 5-tuple-shaped dataclass (observation, reward, terminated, truncated, info) — the runner reads success from info[task.success_key].

The smallest working scene adapter lives in python/sim/src/openral_sim/policies/mock.py. It is the recommended reference because it has no physics and no external dependencies. The realistic reference is backends/libero.py (wraps the LIBERO gymnasium env).

Skeleton

Place new adapters under python/sim/src/openral_sim/{policies,backends}/ so they are imported by the package's __init__.py (which is what triggers the @register side-effect):

# python/sim/src/openral_sim/{policies,backends}/my_scene.py
from __future__ import annotations
from dataclasses import dataclass

import numpy as np
from numpy.typing import NDArray

from openral_sim.registry import SCENES
from openral_sim.rollout import Observation, StepResult


@dataclass
class _MyScene:
    scene: "SceneSpec"
    task:  "TaskSpec"
    _step: int = 0

    def reset(self, seed: int | None = None) -> Observation:
        self._step = 0
        return self._observe()

    def step(self, action: NDArray[np.float32]) -> StepResult:
        self._step += 1
        done = self._step >= self.task.max_steps
        return StepResult(
            observation=self._observe(),
            reward=0.0,
            terminated=done,
            truncated=False,
            info={self.task.success_key: done},
        )

    def render(self) -> NDArray[np.uint8] | None:
        return np.zeros(
            (self.scene.observation_height, self.scene.observation_width, 3),
            dtype=np.uint8,
        )

    def close(self) -> None:
        return None

    def _observe(self) -> Observation:
        return {
            "images": {"camera1": self.render()},
            "state":  np.zeros(8, dtype=np.float32),
            "task":   self.task.instruction,
        }


@SCENES.register("my_scene")
def _build(env_cfg: "SimEnvironment") -> _MyScene:
    return _MyScene(scene=env_cfg.scene, task=env_cfg.task)

Then wire it into the package's import set so @register actually fires. The simplest way is a one-line import in python/sim/src/openral_sim/{policies,backends}/__init__.py:

from . import my_scene  # noqa: F401  # reason: register-by-import

Confirm it shows up in openral sim list under scenes:.

Optional: `mujoco_handles` for `openral sim run --view`

If your adapter wraps a MuJoCo model and exposes mujoco_handles(self) -> tuple[mujoco.MjModel, mujoco.MjData] | None, --view will open a passive viewer. The method is intentionally not part of the SimRollout Protocol — the runner uses getattr(env, "mujoco_handles", None), so non-MuJoCo adapters need not stub it.

Worked example — `so101_box` (custom arena + custom objects + non-Panda robot)

python/sim/src/openral_sim/backends/so101_box/ is a reference for the "custom arena + custom robot + custom task" shape this section targets. It registers so101_box (fixed_robot=so101_follower) and ships an SO-101 in a configurable box arena with an OAK-D Pro RGB-D overhead camera, a wrist camera parented to the gripper, a slotted target block + cylindrical tube as the task, and a geometric tube-insertion success check.

Two design points worth lifting into your own adapter:

Every scene parameter is YAML, no geometry is hard-coded. The composer is fed a single typed BoxSceneOptions dataclass that carries every dimension, pose and threshold. The CLI's scene.backend_options block populates it via _options_from_backend_options (which rejects unknown keys loudly). Once the adapter is registered, the next "SO-101 in a similar arena" scene is a pure YAML edit — no Python change. See scenes/sim/so101_tube_insertion.yaml for the full surface.
The MJCF is composed by reading the upstream robot MJCF, then rewriting + appending. compose_so101_box_mjcf reads robot_descriptions:so_arm101_mj_description (the SO-101 MJCF shipped with TheRobotStudio/SO-ARM100), re-anchors its <body name="base"> to the configured world pose via a regex rewrite, splices a <camera> into the gripper body, and appends the arena + objects + overhead camera to the worldbody just before </worldbody>. The result is written next to the upstream MJCF so meshdir="assets" resolves at compile time without copying any STLs. Same pattern as openarm_robosuite.

If your custom scene also needs RGB + depth from the same camera, the so101_box rollout shows the convention: two mujoco.Renderer instances on the same model, one with enable_depth_rendering() set, both updating against the same camera by name. The depth array is in metres directly (no normalisation) so the policy / dataset bridge sees the same units the real OAK-D Pro driver emits.

6. Write a custom policy adapter

A policy adapter is a function decorated with @POLICIES.register("<id>") that returns an object satisfying the PolicyAdapter Protocol:

class PolicyAdapter(Protocol):
    spec: VLASpec
    device: str
    def reset(self) -> None: ...
    def step(self, observation: Observation, instruction: str) -> NDArray[np.float32]: ...
    def close(self) -> None: ...

The simplest reference is _ZeroPolicy / _RandomPolicy in policies/mock.py (no weights, CPU-only, end-to-end test on any machine). The realistic reference is policies/smolvla.py (loads a SmolVLAPolicy, normalises the observation dict, caches action chunks).

Skeleton

# python/sim/src/openral_sim/{policies,backends}/my_policy.py
from __future__ import annotations
from dataclasses import dataclass

import numpy as np
from numpy.typing import NDArray

from openral_sim.registry import POLICIES
from openral_sim.rollout import Observation


@dataclass
class _MyPolicy:
    spec: "VLASpec"
    device: str
    action_dim: int = 7

    def reset(self) -> None:
        # load weights / reset action queue / re-seed RNG here
        return None

    def step(self, observation: Observation, instruction: str) -> NDArray[np.float32]:
        # adapter is responsible for mapping observation → its input format
        del observation, instruction
        return np.zeros(self.action_dim, dtype=np.float32)

    def close(self) -> None:
        return None


@POLICIES.register("my_policy")
def _build(env_cfg: "SimEnvironment") -> _MyPolicy:
    return _MyPolicy(spec=env_cfg.vla, device=env_cfg.vla.device)

Wire it in via the same policies/__init__.py import pattern as §4.

Pair it with an rSkill manifest

The runner enforces that vla.weights_uri is a valid skill reference and that the rSkill manifest's embodiment_tags overlap the robot's (see openral_sim.runner._check_rskill_compatibility). To use your policy with real weights, create rskills/<my-skill>/rskill.yaml declaring the target embodiment(s), then reference it as rskills/<my-skill>. The skills directory layout is documented under rskills/README.md; existing manifests (e.g. rskills/smolvla-libero/) are the practical templates.

If your rollout needs to write a LeRobotDataset v3 via the bridge (openral sim run --dataset-out), declare BOTH state_contract AND action_contract on the manifest:

# Per-checkpoint proprioception layout — bridge's observation.state shape.
state_contract:
  dim: 8        # e.g. 7 joint + 1 gripper for Franka+LIBERO
# Per-checkpoint action vector — bridge's action shape.
action_contract:
  dim: 7        # e.g. 7-D delta-EEF for pi05/smolvla/xvla/act on LIBERO

The dataset bridge reads these contracts (not the robot manifest) because the same physical robot can host many checkpoints with different I/O dims (Franka emits 7-D delta-EEF on LIBERO, 12-D for RoboCasa, 8-D joint+gripper on real hardware). The scene's observation_height/width flows through as the bridge's per-camera shape.

Optional: `last_input_frame` for video capture

Visuomotor adapters MAY also implement last_input_frame(self) -> NDArray[np.uint8] | None. When present and record_video is True, the runner stitches the frame the VLA actually saw into the debug MP4. Scripted / mock policies should omit it.

End-to-end smoketest

Compose your new scene and policy with an existing robot to confirm everything is wired:

openral sim run \
    --config  scenes/<your-config>.yaml \
    --rskill  rskills/<my-skill> \
    --n-episodes 1 \
    --max-steps  10

(Substitute <my-skill> for the rSkill manifest you wrote alongside your <my-config>.yaml; the runner refuses to load a manifest whose embodiment_tags do not overlap the resolved robot's, so the smoketest fails loud on mismatches.)

Level 6: a custom MuJoCo environment via RoboCasa

RoboCasa is a openral sim backend so you can run kitchen scenarios with custom robots, tasks, and rSkills against real MuJoCo physics.

One-time setup (auto-installed on first use)

openral_sim._deps.ensure_backend_deps handles the install chain for you. On the first openral sim run against a robocasa/<task> scene id, you see a Rich banner listing every subprocess step plus the license posture, then a typer.confirm() prompt:

openral sim run --config scenes/sim/robocasa_pnp.yaml \
            --rskill rskills/rldx1-ft-rc365-nf4 \
            --max-steps 200

Banner steps (kitchen variant):

uv sync --all-packages --group robocasa
mkdir -p ~/.cache/openral/repos
git clone https://github.com/ARISE-Initiative/robosuite.git ~/.cache/openral/repos/robosuite (idempotent, master branch — kitchen needs the master tip, not the 1.5.2 PyPI wheel. Master adds make mink optional (commit 95743f6, 3 commits past the v1.5.2 tag) so mink stays in extras_require; without that, mink==0.0.5's numpy<2 pin would wedge the workspace. The lockfile-honest [tool.uv.sources] robosuite = { git = "…", rev = "…" } entry in pyproject.toml pins the exact commit _robocasa_kitchen_plan reinstalls editable here.)
patch the clone: touch robosuite/examples/__init__.py, touch robosuite/examples/third_party_controller/__init__.py, touch robosuite/macros_private.py (upstream find_packages() drops the examples/ directory + the macros nag is silenced by an empty macros_private.py)
uv pip install --force-reinstall --no-deps -e ~/.cache/openral/repos/robosuite
uv pip install --no-deps "robosuite-models @ git+..."
uv pip install h5py>=3.16 lxml>=5 llvmlite numba qpsolvers pyopengl-accelerate
uv pip install --no-deps "mink==0.0.5" (newer mink relocated mink.tasks.exceptions.TargetNotSet away from where robosuite's mink_controller.py imports it; --no-deps avoids the numpy<2 downgrade)
uv pip install --no-deps "robocasa @ git+..."

Confirm with y and walk away — total install runs in ~30 s on fibre. Skip the prompt in CI / Dockerfiles with OPENRAL_AUTO_INSTALL_DEPS=1.

After the deps land, the same command auto-fetches the ~11 GB CC-BY-4.0 kitchen asset bundle into ~/.cache/openral/robocasa/ behind a second Rich license banner (gated by typer.confirm() or OPENRAL_ALLOW_ROBOCASA_ASSETS=1).

The libero extras-group conflict means you cannot share a venv with LIBERO. The workspace's [tool.uv].conflicts block declares this so uv sync refuses an impossible mix; if you need both backends, use two clones (or two venvs against the same clone).

The auto-installer also handles a number of upstream-bug workarounds transparently: robocasa/__init__.py's hardcoded mujoco==3.3.1 / numpy==2.2.5 / robosuite>=1.5.2 assertions (upstream kitchen 1.0.1) — or mujoco==3.2.6 / numpy in {1.23.x, 1.26.4} / robosuite in {1.5.0, 1.5.1} (robocasa-gr1-tabletop-tasks 0.2.0 fork) — are bypassed at adapter import-time via _spoof_robocasa_version_pins; the controller config's missing-actuator entries are stripped before they reach robosuite. See "Known constraints" below for the full set.

Running with a RoboCasa-365 checkpoint

rskills/rldx1-ft-rc365-nf4 wraps the verified RoboCasa-365 PandaMobile rSkill. It runs through the same three-camera, 16-D state, 12-D action contract as the RoboCasa scene.

OPENRAL_ALLOW_ROBOCASA_ASSETS=1 \
  uv run openral sim run --config scenes/sim/robocasa_pnp.yaml \
                    --rskill rskills/rldx1-ft-rc365-nf4 \
                    --view --max-steps 200

What that config wires together:

robot_id: panda_mobile -- the robots/panda_mobile/robot.yaml manifest (Franka Panda on a 3-DoF holonomic base).
scene.id: robocasa/PickPlaceCounterToCabinet -- one of the curated atomic tasks registered with SCENES.register(..., fixed_robot="panda_mobile") so an accidental --robot franka_panda fails fast with ROSConfigError.
scene.backend_options.mode: prebuilt -- validated through :class:openral_core.RoboCasaBackendOptions (prebuilt-vs-procedural XOR).
--rskill rskills/rldx1-ft-rc365-nf4 -- the verified RoboCasa-365 manifest that declares the embodiment tags / sensor requirements the runner validates.

The first invocation fetches the ~11 GB CC-BY-4.0 kitchen asset bundle under ~/.cache/openral/robocasa/. The download is gated by typer.confirm(); OPENRAL_ALLOW_ROBOCASA_ASSETS=1 is the CI bypass. The fetch script also needs the mujoco-version spoof, which the asset helper does in a subprocess -c wrapper.

Authoring a procedural kitchen

For free-axis authoring, pass --scene robocasa (no slash) plus a procedural backend_options block:

scene:
  id: robocasa
  backend: mujoco
  backend_options:
    mode: procedural
    kitchen_style: 3      # 0..9, one of robocasa's 10 aesthetic packs
    layout_id: 7          # 0..9, one of robocasa's 10 floor plans
    fixtures: ["sink", "stovetop", "microwave"]
    spawn_objects: ["coffee_cup", "apple"]
    task_verb: pnp        # pnp | open | close | press | navigate
    robots: ["PandaMobile"]
    controller: BASIC
    horizon: 500

task_verb resolves to the matching atomic env (e.g. pnp → PickPlaceCounterToCabinet); the remaining keys are validated by RoboCasaBackendOptions's model_validator to enforce the prebuilt-vs-procedural XOR.

Known constraints

mujoco / numpy / robosuite assertion. Both robocasa variants hard-assert exact micro versions of mujoco, numpy, and robosuite at import time even though all three work fine on newer versions in practice. The RoboCasa adapter (openral_sim/backends/robocasa.py: _spoof_robocasa_version_pins) monkey-patches the __version__ strings only across the robocasa import block and restores them immediately afterwards so the rest of the workspace is unaffected.
examples/ / macros_private.py patches. The auto-installer clones robosuite + the robocasa fork to ~/.cache/openral/repos/, drops the missing __init__.py files under robosuite/examples/, and writes empty macros_private.py stubs into both packages. Without these patches every run emits a Could not load the mink-based whole-body IK WARN (robosuite's own __init__.py:37 import) + 3 No private macro file found lines per package. See the _robocasa_*_plan install steps in openral_sim/_deps.py for the full list.
No LIBERO in the same venv. [tool.uv].conflicts enforces this cleanly when both groups would be active; swap venvs (or clones) when you need both backends.
One informational print remains. robocasa.__init__ does try: import mimicgen + print() unconditionally; installing mimicgen IS a real fix but mimicgen's own __init__ imports robosuite paths the current master has renamed and prints 2 new warning lines, so the net noise is worse with mimicgen than without. We leave the one print.

Level 7: NVIDIA GR-1 tabletop tasks (RoboCasa GR1 fork)

The RoboCasa GR1 Tabletop Tasks fork — a soft fork of robocasa that NVIDIA shipped alongside the GR00T N1 open foundation model (arXiv:2503.14734) — adds 24 PnP tabletop tasks on the Fourier GR-1 humanoid (the GR1ArmsAndWaistFourierHands composition: 7-DoF right arm + 7-DoF left arm + 3-DoF waist + two 6-DoF Fourier dex hands, leg and head actuation disabled). The bot-harness sim layer exposes them as robocasa/gr1/<TaskName> scene ids pinned to the gr1 robot manifest (robots/gr1/robot.yaml).

The two robocasa-named python packages (kitchen + GR1 fork) share the python package name, so a host installs ONE or the OTHER -- the auto-installer picks the variant matching the scene id you requested. The adapter still registers both task families regardless; the unavailable one fails at robosuite.make() with a clean "unknown env_name" rather than at import time.

One-shot run

Drive the GR1 tabletop scene with the in-tree RLDX-1-FT-GR1 nf4 rSkill via the auto-managed sidecar:

OPENRAL_AUTO_INSTALL_DEPS=1 \
OPENRAL_ALLOW_ROBOCASA_ASSETS=1 \
OPENRAL_ALLOW_NONCOMMERCIAL=1 \
  openral sim run --config scenes/sim/robocasa_gr1_pnp_cup_to_drawer.yaml \
              --rskill rskills/rldx1-ft-gr1-nf4 --max-steps 30

This drives a real robosuite.make(env_name="PnPCupToDrawerClose", robots=["GR1ArmsAndWaistFourierHands"]) rollout on the local GPU.

Drop the env-var bypasses on first run for the interactive prompts:

robocasa_gr1 deps banner → y (clones robosuite + the GR1 fork to ~/.cache/openral/repos/, installs both editable with the examples/__init__.py + macros_private.py patches dropped in).
RoboCasa GR1 tabletop assets license banner → y (~750 MB download to the cloned fork's robocasa/models/assets/).

Sweeping to the other 23 tasks

Swap scene.id + scene.backend_options.prebuilt_task + task.scene_id in scenes/sim/robocasa_gr1_pnp_cup_to_drawer.yaml to any of the 24 tabletop env class names (see _GR1_TABLETOP_TASKS in openral_sim/backends/robocasa.py):

PnPCupToDrawerClose / PnPPotatoToMicrowaveClose / PnPMilkToMicrowaveClose / PnPBottleToCabinetClose / PnPWineToCabinetClose / PnPCanToDrawerClose — the 6 canonical PnP atomic tasks.
18 Posttrain* variants (Cuttingboard → Basket / Cardboardbox / Pan / Pot / Tieredbasket / Plate / etc.) — the post-training task set from the GR00T-N1 paper.

Benchmark protocol

benchmarks/gr1_tabletop.yaml mirrors the paper's eval protocol (scaled to 10 episodes for a ~30 min single-GPU run; bump n_episodes + seeds together for the 50-ep paper reproduction):

openral benchmark run --suite gr1_tabletop --rskill rskills/<gr1-skill>

The auto-install prompts fire from the benchmark runner's path too — openral benchmark run and openral sim run share the same scene factory.

Where to go next

The full list of registered IDs on your machine: openral sim list.
The cookbook of existing configs and a per-backend ID table: scenes/README.md.
Design background: the original scene/eval design renamed SceneEnvironment to SimScene and later split it into the three-tier DeployScene ⊆ SimScene ⊆ BenchmarkScene hierarchy (with per-tier loader strictness) that also underlies the openral sim run vs openral benchmark run split. RoboCasa was added as a free-axis MuJoCo backend with custom robots + tasks — rolling out in five PRs per issue #88; the Pydantic RoboCasaBackendOptions validator and the [dependency-groups].robocasa extras group already ship today, the adapter and a Level-6 procedural-kitchen walkthrough land in later PRs.
The public-symbol inventory for the sim layer: docs/METHODS.md, section Eval (sim).

Create a sim environment

1. What openral sim run accepts

Flags (openral sim run)

Canonical invocation

Per-axis overrides on top of --config

Startup performance (activate() parallelisation)

2. Author a SimScene YAML

The required blocks

3. Add a new robot manifest

The assets: block

The sim: block

Drop a new robot in

Verify it registered

Match the manifest to a sim scene

4. When the existing scene adapters are not enough

5. Write a custom scene adapter

Skeleton

Optional: mujoco_handles for openral sim run --view

Worked example — so101_box (custom arena + custom objects + non-Panda robot)

6. Write a custom policy adapter

Skeleton

Pair it with an rSkill manifest

Optional: last_input_frame for video capture

End-to-end smoketest

Level 6: a custom MuJoCo environment via RoboCasa

One-time setup (auto-installed on first use)

Running with a RoboCasa-365 checkpoint

Authoring a procedural kitchen

Known constraints

Level 7: NVIDIA GR-1 tabletop tasks (RoboCasa GR1 fork)

One-shot run

Sweeping to the other 23 tasks

Benchmark protocol

Where to go next

1. What `openral sim run` accepts

Flags (`openral sim run`)

Per-axis overrides on top of `--config`

Startup performance (`activate()` parallelisation)

The `assets:` block

The `sim:` block

Optional: `mujoco_handles` for `openral sim run --view`

Worked example — `so101_box` (custom arena + custom objects + non-Panda robot)

Optional: `last_input_frame` for video capture