VLA × Robot × Simulation Compatibility Matrix

Canonical reference for which Vision-Language-Action models run on which robots under which simulators, derived from upstream model cards, papers, and checkpoint inspection. TBD entries are not locally verified; PRs that fill them must include checkpoint-inspection evidence.

The normative license matrix and the rSkill packaging format are in CLAUDE.md §3.

1. Robots (primary reference embodiments)

The full set of 21 integrated RobotDescription manifests, with HAL modules and per-robot status, is in robots.md. The rows below give the observation/action control contract for the two most-exercised sim embodiments; every other robot follows the same pattern.

Robot	Embodiment tags	DoF	Control mode	HAL module	Sim env
SO-100 / SO-101 (LeRobot)	`so100_follower`, `so101_follower`	6 arm + 1 gripper	`joint_position`	`openral_hal.so100_follower`	SO-100/SO-101 digital twin (MuJoCo, in-process)
Franka Panda (LIBERO sim only)	`libero`, `franka_panda`	7 + gripper	`cartesian_delta` (6-D EEF + axis-angle)	LiberoEnv (lerobot)	LIBERO (MuJoCo via robosuite)

Normalized OSC checkpoints declare action_contract.cartesian_delta_scale. RoboSuite/LIBERO policies use [0.05, 0.05, 0.05, 0.5, 0.5, 0.5]: the raw [-1, 1] command still reaches the native controller, while predictive safety multiplies it into metres/radians before Jacobian look-ahead.

Hardware-in-loop gates: - UR5e / UR10e / Franka Panda / Sawyer / ALOHA: tests/hil/ gates require the matching self-hosted lab runner and vendor controller environment; none has a recorded run (per-robot status in robots.md). - SO-101: tests/hil/test_so101_serial_live.py checks bench wiring and the live Feetech servo bus for the scenes/deploy/so101_bench.yaml bench (it issues no position commands).

2. Embodiment Tag Registry

Embodiment tags are short strings that appear in rskill.yaml under embodiment_tags and in RobotCapabilities.embodiment_tags. The skill loader refuses to activate a skill whose tags do not intersect the target robot's capability set.

Tag	Robot / Platform	DoF	Source dataset / paper	Notes
`so100_follower`	LeRobot SO-100 arm	6	lerobot/so100	Follower arm in leader-follower teleoperation setup
`so101_follower`	LeRobot SO-101 arm	6	lerobot/so101	Updated hardware revision of SO-100
`libero`	Franka Panda on LIBERO benchmark	7 + gripper	LIBERO (Yuke Zhu et al., NeurIPS 2023)	Simulation-only tag for LIBERO benchmark training
`franka_panda`	Franka Panda (real + sim)	7 + gripper	Standard industry robot; widespread in BridgeData / Open X	Broader tag; use `libero` when targeting LIBERO-specific checkpoints
`widowx`	WidowX 250s	6	BridgeData V2	Low-cost research arm; common in Open X-Embodiment
`r1pro`	Galaxea R1 Pro (BEHAVIOR simulation)	22 articulated joints + holonomic base	BEHAVIOR-1K 2026	Sim-only OpenRAL manifest; 61-D policy state and mixed 23-D action
`gr1`	Fourier GR-1 humanoid	17 joints (3 waist + 2 × 7 arm) + 2 × 6 hand DoF	NVIDIA Arena dataset	Legs and head disabled in the RoboCasa GR1 composition; eval-only
`aloha`	Aloha bimanual teleoperation setup	2 × 7	ACT paper (Stanford / Toyota)	Bimanual; two Viperx arms with overhead + wrist cameras
`aloha_agilex`	ALOHA-AgileX dual-arm (RoboTwin 2.0)	2 × 7	RoboTwin 2.0	Bimanual SAPIEN benchmark embodiment; targeted by `smolvla-robotwin`
`mobile_base`	Differential/omni mobile base (Nav2)	—	Nav2 stack	Navigation embodiment for `rskill-nav2-navigate-to-pose` (result-only, publishes `/cmd_vel`)
`koch`	Koch arm	6	lerobot/koch	Low-cost leader-follower arm
`piper`	Agilex Piper arm	6	ISdept dataset	Mid-range research arm from Agilex

3. VLA Compatibility Matrix

Column notes: Cameras = image inputs (resolution + pre-processing); Norm stats = whether normalisation statistics are bundled in the checkpoint; rSkill = local skill path, if any; License = SPDX expression for the weights (code license may differ). ✓ = locally verified.

3.1 LIBERO (Franka Panda, MuJoCo via robosuite)

The OpenRAL embodiment for LIBERO is franka_panda — see robots/franka_panda/. The sim-imposed observation/action contract (8-D EEF state, 7-D delta-EEF action, 180° image flip) lives in the LIBERO scene adapter.

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
`lerobot/smolvla_libero`	LIBERO	`libero`	8-D `eef_pos(3)+axisangle(3)+gripper_qpos(2)` ✓	`image`→`camera1` + `image2`→`camera2` (256×256, flip 180°) ✓	Yes — `step_5_normalizer_processor.safetensors` (state=[8], action=[7]) ✓	`rskills/smolvla-libero/`	Apache-2.0	Paper: Spatial 90% / Object 96% / Goal 92% / Long 71% (avg 87.3%). `scenes/benchmark/libero_spatial.yaml` (with `--rskill rskills/smolvla-libero`)
`HuggingFaceVLA/smolvla_libero`	LIBERO	`libero`	8-D (same as above)	same as above	Yes (assumed same as above)	—	Apache-2.0	Community mirror. Not locally verified.
`lerobot/pi05_libero_finetuned_v044`	LIBERO	`libero`, `franka_panda`	8-D same as smolvla ✓	`image`+`image2` (256×256, flip 180°) + `empty_camera_0` (224×224 zeros) ✓	Yes — `step_2_normalizer_processor.safetensors` (state=[8], action=[7]) ✓	`rskills/pi05-libero-int8/`	Permissive research (weights) / Apache-2.0 (code)	π0.5 (PaliGemma 3B backbone); requires ≥8 GB VRAM. `scenes/benchmark/libero_spatial.yaml` (with `--rskill rskills/pi05-libero-int8`). Permissive-research weights, not cleared for commercial use — see §5
`lerobot/pi0_libero_finetuned_v044`	LIBERO	`libero`, `franka_panda`	8-D (same format as pi05 — unverified)	same 3-camera format as pi05 (unverified)	Yes (assumed same format)	—	Permissive research (weights) / Apache-2.0 (code)	π0 (same license caveat). Not locally verified.
`lerobot/xvla-libero`	LIBERO	`libero`, `franka_panda`	8-D same `eef_pos+axisangle+gripper_qpos`; padded to max_state_dim=20 internally ✓	`image`+`image2` (224×224, flip 180°) + `empty_camera_0` (224×224 zeros) ✓	IDENTITY norm (no stats file) ✓; action output [20] (first 7 elements = LIBERO 7-D) ✓	`rskills/xvla-libero/`	Apache-2.0	xVLA (Florence-2 backbone, flow-matching). `scenes/benchmark/libero_spatial.yaml` (with `--rskill rskills/xvla-libero`)
`ar0s/groot_libero`	LIBERO	`libero`, `franka_panda`	TBD	TBD	TBD	—	Apache-2.0 (fine-tune)	GR00T on LIBERO; base model is NVIDIA AI Foundation non-commercial — guard required
GR00T N1.7 (Isaac, 3B)	LIBERO	`franka_panda`	8-D (LIBERO EEF)	`image`+`wrist_image` (two positional views)	Backbone-only NF4	`rskills/gr00t-n17-libero/`	NVIDIA Open Model License (commercial OK)	In-process lerobot 0.6.0 `GrootPolicy`; live LIBERO-spatial 5/5. No ZMQ sidecar.
MolmoAct2 NF4 (~5.5 B)	LIBERO	`franka_panda`	8-D (LIBERO EEF)	`image`+`image2`	NF4 (Molmo2-ER VLM + flow-matching)	`rskills/molmoact2-libero-nf4/`	Apache-2.0	NF4 fits 8 GB via CUDA `expandable_segments`; `trust_remote_code` (`OPENRAL_ALLOW_REMOTE_CODE=1`)

3.2 RLBench (Franka Panda, CoppeliaSim/PyRep)

RLBench tasks are fixed to the Franka Panda in CoppeliaSim/PyRep. OpenRAL runs both the simulator and 3D keyframe policy out-of-process in an externally-provisioned py3.10 sidecar venv; CoppeliaSim is proprietary (free EDU) and is never vendored.

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in checkpoint	rSkill	License	Notes
`katefgroup/3d_diffuser_actor` (`diffuser_actor_peract.pth`)	RLBench PerAct subset	`franka_panda`	8-D `gripper_pose(7)+gripper_open(1)` history, policy emits an 8-D absolute EE keyframe	`left_shoulder`, `right_shoulder`, `wrist`, `front` RGB-D point clouds at 256×256	Precomputed CLIP instruction embeddings (`instructions.pkl`) + task bounds JSON	`rskills/3d-diffuser-actor-rlbench/`	MIT	Starter scene set: `rlbench_open_drawer.yaml`, `rlbench_meat_off_grill.yaml`, `rlbench_close_jar.yaml`; live-verified on an 8 GB Ada host.

3.3 MetaWorld (Sawyer, MuJoCo)

The OpenRAL embodiment for MetaWorld is sawyer — see robots/sawyer/. The MetaWorld benchmark simulates a Rethink Sawyer; some upstream checkpoints carry a franka_panda tag, but the actual robot is Sawyer.

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
`lerobot/smolvla_metaworld`	MetaWorld MT50	`franka_panda`, `manipulator`	4-D `agent_pos` (XYZ + gripper) ✓	`observation.image`→`camera1` (256×256, flip+resize from 480×480) ✓	Yes — `step_5_normalizer_processor.safetensors` (state=[4], action=[4]) ✓	`rskills/smolvla-metaworld/`	Apache-2.0	Action: 4-D delta (XYZ + gripper). Sawyer robot in MetaWorld (not Franka despite tag). `scenes/benchmark/metaworld_push.yaml` (with `--rskill rskills/smolvla-metaworld`)

3.4 RoboCasa (Panda mobile / GR1, MuJoCo)

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
`lerobot/smolvla_robocasa`	RoboCasa	`franka_panda`, `manipulator`	TBD	TBD	TBD	—	Apache-2.0	Kitchen manipulation; no rSkill stub yet
`RLWRLD/RLDX-1-FT-RC365`	RoboCasa-365	`panda_mobile`	3-camera layout (`state_contract.layout`)	3 RGB streams	Processor sidecars in rSkill	`rskills/rldx1-ft-rc365-nf4/`	RLWRLD non-commercial	Out-of-process ZMQ sidecar; mobile-manipulator kitchen fine-tune
`XiaomiRobotics/Xiaomi-Robotics-1-RoboCasa`	RoboCasa v0.2	`panda_mobile`	8-D arm joints(7)+gripper(1)	3 RGB 256x256 views	Action mean/std in plain `preprocessor_config.json`; state is raw	`rskills/xr1-robocasa/`	Apache-2.0	Custom-code MiBoT sidecar; 7-D delta-EEF output, 10-step replay. OpenRAL's current RoboCasa stack is an integration check, not upstream-score reproduction.
`XiaomiRobotics/Xiaomi-Robotics-1-RoboCasa365`	RoboCasa365	`panda_mobile`	14-D EE axis-angle + gripper + base, from 4-frame history	3 RGB videos, 4 frames sampled at interval 2; exact 256x256 policy resize	Action mean/std in checkpoint processor; state is raw	`rskills/xr1-robocasa365/`	Apache-2.0	Custom-code MiBoT sidecar; 12-D action, 16-step replay; executable `weights_uri` pinned to the reviewed HF revision. Deploy sim applies grouped actions atomically but deliberately does not consume the simulator success oracle or reset episodes.

Cross-policy contract audit: normalized Cartesian scales apply to the shipped RoboSuite/LIBERO delta policies; normalized joint input bounds additionally apply to the XR-1/RLDX RoboCasa365 mobile-base slots. Atomic application is generic for every multi-slot action contract. Exact voxel-cube geometry is a global safety-kernel concern, not policy metadata. An exact image resize is currently evidenced only for XR-1 RoboCasa365; other adapters keep their checkpoint-native preprocessing until a manifest provides evidence otherwise.

3.5 SO-100 / SO-101 (real robot or sim)

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
`chamborgir/smolvla_pickplace_20k`	SO-101 real	`so101_follower`	TBD	TBD	TBD	—	Apache-2.0	20k steps pick-and-place fine-tune
`TakuyaHiraoka/act_so101_pick_diverse_objects`	SO-101 real	`so101_follower`	TBD	TBD	TBD	—	Apache-2.0	ACT policy; diverse object pick task
`edge-inference/smolvla-so101-pick-orange`	Isaac Sim	`so101_follower`	TBD	TBD	TBD	—	Apache-2.0	Isaac Sim backend; requires Isaac Sim license for reproduction
`makermods/smolvla_makermods_eraser_place_unblurry_real_2026-07-31_17-35-54`	SO-101 real	`so101_follower`	6	`front` + `wrist`	✅ (processor safetensors)	`rskills/rskill-smolvla-so101-eraser_place-bf16/`	Apache-2.0	"place the erase on the blue square" (prompt verbatim, upstream typo); `joint_units: degrees`; 50-step chunk reproduces the training teleop to ≤1.4 MAE/joint; training envelope overshoots the so101 elbow/wrist_flex limits
MolmoAct2 NF4 SO-101	SO-101 real / sim	`so100_follower`, `so101_follower`	TBD	wrist + overhead	NF4	`rskills/molmoact2-so101-nf4/`	Apache-2.0	`trust_remote_code` (`OPENRAL_ALLOW_REMOTE_CODE=1`); NF4 fits 8 GB

3.6 SimplerEnv / ManiSkill3 Bridge (WidowX)

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
`RLinf/RLinf-OpenVLAOFT-PPO-ManiSkill3-25ood`	SimplerEnv `PutCarrotOnPlateInScene-v1` (ManiSkill3)	`widowx`	8-D `simpler_widowx` surfaced by env; checkpoint uses no proprio (`use_proprio=False`) ✓	single 224×224 RGB (`camera1` / 3rd-view) ✓	Yes — `config.json` `norm_stats.bridge_orig`, 7-D action, chunk 8 ✓	`rskills/openvla-oft-simpler-widowx-nf4/`	MIT	OpenVLA-OFT custom-code model; immutable `weights_uri` revision and `OPENRAL_ALLOW_REMOTE_CODE=1` are both mandatory. NF4 fits 8 GB. Requires RLinf eval path in manifest `policy_extras` (`generate_action_verl`, padding length 30, temperature 0.6, torch seed 0, action scale 2.0, binary gripper).
`RLWRLD/RLDX-1-FT-SIMPLER-WIDOWX`	SimplerEnv `PutCarrotOnPlateInScene-v1`	`widowx`	8-D `simpler_widowx` ✓	single RGB stream ✓	Processor sidecars in rSkill ✓	`rskills/rldx1-ft-simpler-widowx-nf4/`	RLWRLD non-commercial	Sidecar runtime; sibling Bridge baseline.

3.7 Other platforms

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
`nvidia/smolvla-arena-gr1-microwave`	NVIDIA Arena	`gr1`	TBD	TBD	TBD	—	Apache-2.0	Fourier GR-1 humanoid, microwave-opening task
`ISdept/smolvla-piper`	Piper real	`piper`	TBD	TBD	TBD	—	Apache-2.0	Agilex Piper arm; community fine-tune
`RLWRLD/RLDX-1-FT-GR1`	RoboCasa GR1	`gr1`	1-camera layout (`state_contract.layout`)	single RGB stream	Processor sidecars in rSkill	`rskills/rldx1-ft-gr1-nf4/`	RLWRLD non-commercial	Out-of-process ZMQ sidecar; GR1 humanoid bimanual

3.8 RoboTwin 2.0 (ALOHA-AgileX, SAPIEN)

RoboTwin 2.0 is a dual-arm SAPIEN benchmark on the 14-DoF ALOHA-AgileX. The simulator and policy run out-of-process in a py3.10 SAPIEN sidecar.

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
SmolVLA RoboTwin	RoboTwin 2.0 (50 tasks)	`aloha_agilex`	14-D bimanual	dual-arm camera set	Yes (in ckpt)	`rskills/smolvla-robotwin/`	Apache-2.0	py3.10 SAPIEN sidecar; scenes `scenes/benchmark/robotwin_*.yaml`

3.9 VLABench (Franka Panda)

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
SmolVLA VLABench	VLABench (97 tasks)	`franka_panda`	TBD	TBD	Yes (in ckpt)	`rskills/smolvla-vlabench/`	Apache-2.0	Integration baseline (0% on current primitives); scene `scenes/benchmark/vlabench_select_fruit.yaml`
`XiaomiRobotics/Xiaomi-Robotics-1-VLABench`	VLABench	`franka_panda`	7-D XYZ+Euler+gripper	front(raw 2)+base(raw 0)+wrist(raw 3), 480x480, no flip	Action mean/std in checkpoint processor; state is raw	`rskills/xr1-vlabench/`	Apache-2.0	MiBoT NF4 sidecar predicts 10 deltas; adapter integrates absolute targets and replans after 5. Live one-step validation: 3.66 GiB process VRAM, 0.81 s warm chunk on RTX 4070 Laptop 8 GB. Persistent local NF4 reload is bit-identical to runtime NF4.

The public XiaomiRobotics/Xiaomi-Robotics-1-5B/model_states.pt is not an inference checkpoint: it is a non-self-describing post-training seed. It is therefore not packaged as an rSkill and cannot be claimed as real-robot ready.

3.10 ALOHA (gym-aloha, MuJoCo)

VLA (HF ID)	Sim env	Robot tag	State dim	Cameras	Norm stats in ckpt	rSkill	License	Notes
ACT (transfer cube)	gym-aloha	`aloha`	2 × 7	overhead + wrist	Yes (in ckpt)	`rskills/act-aloha/`	MIT	Bimanual cube-transfer; `scenes/benchmark/aloha_transfer_cube.yaml`
ACT (insertion)	gym-aloha	`aloha`	2 × 7	overhead + wrist	Yes (in ckpt)	`rskills/act-aloha-insertion/`	MIT	Custom-example insertion checkpoint; `scenes/benchmark/aloha_insertion.yaml`

3.11 BEHAVIOR-1K 2026 Challenge (Galaxea R1 Pro)

VLA / source	Sim env	Robot tag	State dim	Cameras	Action	rSkill	License	Notes
Organizer GR00T N1.7 `turning_on_radio` checkpoint (baseline)	OmniGibson / Isaac Sim	`r1pro`	61-D official R1Pro proprio order	224² RGB under the official `DefaultWrapper` (`RGBDFullResWrapper` crashes at boot on the pinned OmniGibson build — it reads joint state before the physics views exist)	23-D base velocity (3) + torso (4) + arms (7+7) + symmetric grippers (1+1)	`rskills/gr00t-n17-b1k-turning-on-radio`	NVIDIA Open Model License, commercial OK — per the organizers' pinned runtime README (the Drive artifact itself carries no license file)	Runs through `openral behavior serve`, `openral sim run`, `openral benchmark run --suite behavior`, or the full `openral deploy sim` graph. Deploy preserves the 61-D state and commits all six safety-approved typed slots as one simulator step.

4. Sim Environment Reference

Sim env	Backend	Install	Robot(s)	Task suites	Camera setup
LIBERO	MuJoCo (robosuite)	`CC=/usr/bin/gcc just sync --group libero` + fix `~/.libero/config.yaml` to point at conda/pip libero data dirs	Franka Panda	libero_spatial, libero_object, libero_goal, libero_10 (= LIBERO-Long)	agentview + wrist 256×256; raw keys `image`/`image2` renamed to `camera1`/`camera2` by stored preprocessor; flip 180°
RLBench	CoppeliaSim/PyRep sidecar	`just sync --group rlbench` for the openral-side ZMQ wire; CoppeliaSim 4.1.0 + PyRep + RLBench@peract live in an external py3.10 venv	Franka Panda	RLBench PerAct starter subset (`open_drawer`, `meat_off_grill`, `close_jar`)	left/right shoulder + wrist + front RGB-D point clouds at 256×256
MetaWorld	MuJoCo	`uv run pip install metaworld==3.0.0 --no-deps`	Sawyer (MT50)	MT50 (50 tasks, v3)	1 camera `corner2` 480×480 → resize to 256×256; `observation.image` key renamed to `camera1`
RoboCasa	MuJoCo (robosuite fork)	`just sync --group robocasa`; the matching `robocasa` fork (editable) and its ~11 GB asset bundle are provisioned on first run	Panda mobile; GR1 (`robocasa/gr1/*`)	Kitchen manipulation (RoboCasa / RoboCasa365); GR1 tabletop	Panda mobile: `robot0_agentview_left` / `_right` + `robot0_eye_in_hand`; GR1: head `egoview`
SO-100 Digital Twin	MuJoCo (in-process, `python/sim/`)	`just sync --group sim`	SO-100	Smoke-test only (no task suite)	None — joint-space smoketest
SO-101 Box (`so101_box`)	MuJoCo (raw, `python/sim/src/openral_sim/backends/so101_box/`)	`just sync --group sim`	SO-101	tube-insertion (geometric success: tube vertical + lower tip ≥ 10 mm below the slotted-block hole top) — both block and tube spawn at random (x, y, yaw) on the floor each `reset()`	OAK-D Pro overhead (RGB + depth, default 640×480) + wrist RGB parented to the gripper body
SimplerEnv WidowX	ManiSkill3/SAPIEN via `simpler_env`	`just sync --group simpler-env` + `uv pip install "simpler-env @ git+https://github.com/simpler-env/SimplerEnv.git@maniskill3"`	WidowX 250s	carrot-on-plate (`simpler_env/widowx_carrot_on_plate`)	3rd-view RGB surfaced as `top`
NVIDIA Arena	Isaac Sim	Requires NVIDIA Isaac Sim license	GR1	microwave	TBD
BEHAVIOR-1K 2026	OmniGibson / Isaac Sim	Official `behavior` environment plus `just sync --group behavior-groot` for the OpenRAL-side wire	Galaxea R1 Pro	100 household tasks; packaged starter is `turning_on_radio`	head + dual wrist RGB; official full wrapper also exposes depth
ManiSkill3	SAPIEN via `mani_skill`	`just sync --group maniskill3`	Franka Panda	`PickCube-v1` (+ more)	single RGB `camera1`
RoboTwin 2.0	SAPIEN (py3.10 sidecar)	py3.10 SAPIEN sidecar (auto-provisioned)	ALOHA-AgileX	50 dual-arm tasks (`robotwin/*`)	dual-arm camera set
VLABench	MuJoCo	auto-installed on first env build (`vlabench` install plan, `OPENRAL_AUTO_INSTALL_DEPS`); the ~12 GB asset bundle is a manual download (`VLABENCH_ROOT`)	Franka Panda	97 tasks (`vlabench/*`)	TBD
PushT	gym-pusht (`pymunk`, 2-D)	`just sync --group sim`	PushT 2-D	`pusht/0`	2-D top view
gym-aloha	MuJoCo	`just sync --group sim`	ALOHA bimanual	transfer-cube, insertion	overhead + wrist
Isaac Sim	Omniverse Isaac Sim	Requires NVIDIA Isaac Sim license	any manifest robot with a URDF (Franka Panda, Panda mobile, OpenArm)	`isaac_sim` deploy scenes (e.g. the warehouse)	per-sensor (manifest-defined)

4.1 LIBERO eval CLI

The lerobot lerobot-eval CLI drives LIBERO natively. Verified against huggingface/lerobot main:

# Single suite
lerobot-eval \
  --policy.path=lerobot/smolvla_libero \
  --env.type=libero \
  --env.task=libero_spatial \
  --eval.n_episodes=10 \
  --eval.batch_size=10 \
  --eval.use_async_envs=true \
  --policy.device=cuda

# All four LIBERO suites
lerobot-eval \
  --policy.path=lerobot/smolvla_libero \
  --env.type=libero \
  --env.task=libero_spatial,libero_object,libero_goal,libero_10 \
  --eval.n_episodes=10 \
  --eval.batch_size=10 \
  --eval.use_async_envs=true \
  --policy.device=cuda

Suite max steps: libero_spatial 280, libero_object 280, libero_goal 300, libero_10 520.

Note: libero_10 is the lerobot/upstream name for LIBERO-Long. LiberoProcessorStep is injected automatically by lerobot.envs.LiberoEnv — no separate LIBERO gym install is required beyond the lerobot extras.

5. Known Limitations

Checkpoint normalisation requires snapshot_download: lerobot/smolvla_libero bundles normalisation statistics in policy_preprocessor_step_5_normalizer_processor.safetensors. A bare from_pretrained call that only fetches model.safetensors + config.json will fail at inference time. Use snapshot_download(repo_id="lerobot/smolvla_libero") or hf_hub_download for the preprocessor file explicitly.
GR00T weights — license is version-specific: N1 / N1.5 / N1.6 ship under the NVIDIA OneWay Noncommercial License, and any checkpoint built on them (e.g. ar0s/groot_libero) inherits it even if the fine-tune layer is Apache-2.0: the manifest sets license: nvidia_non_commercial and the loader requires OPENRAL_ALLOW_NONCOMMERCIAL=1 for a commercial deployment. N1.7+ ship under the NVIDIA Open Model License (commercial OK): manifests set license: nvidia_open_model (e.g. rskills/gr00t-n17-libero) and load without the guard. The older Python-3.10 GR00T ZMQ sidecar is deleted (runtime posture in §6). RLDX-1 (a GR00T-N1.5 finetune) still runs on its own ZMQ sidecar.
π0 / π0.5 weights are "permissive research", not full Apache-2.0: The code under lerobot/ is Apache-2.0; the weights for pi0 and pi05 checkpoints carry a Physical Intelligence permissive-research license that is not equivalent to Apache-2.0 for commercial deployment. The corresponding rSkill manifests set license: permissive_research, from which the loader derives is_commercial_use_allowed = False.
Reward monitor (rskills/robometer-4b) co-residency on 8 GB: The Robometer-4B reward monitor (kind: reward) runs in parallel with a VLA to score per-frame progress/success. At NF4 it is ~3.33 GB resident / 3.56 GB peak (8-frame window) on the 8 GB reference GPU, leaving ~4.4 GB — enough for a small NF4 VLA (e.g. SmolVLA ≈ 1.5–2 GB) but not a 3–4 GB π0.5/GR00T checkpoint simultaneously. When the VLA already saturates the card, run the reward monitor on CPU, a second GPU, or a cloud host, or shrink the reward frame_window_s / num_bins (activation peak scales with both). It is an S2-cadence monitor (~0.2–1 Hz over a frame window), not a per-control-step signal, and is advisory-only (never gates motors). In deploy-sim, the signal is only available on camera-rendering robots (the monitor needs sensor_msgs/Image frames). Apache-2.0; commercially usable.
Community RoboCasa and SO-101 entries are TBD: lerobot/smolvla_robocasa and the SO-101 community checkpoints have not been locally verified. See the ✓ markers in §3 for what has.
Isaac Sim entries require a separate license: edge-inference/smolvla-so101-pick-orange was trained in NVIDIA Isaac Sim. Reproducing its eval requires an Isaac Sim license and is not covered by the standard just sync --group sim environment.
Embodiment tag libero implies simulation only: The libero tag is defined for the LIBERO benchmark Franka Panda setup. Do not apply it to real Franka Panda deployments without verifying that action normalisation and camera geometry match your physical setup.
smolvla_libero state is 8-D, not 6-D: The checkpoint's normalizer safetensors has observation.state stats for shape [8] (eef_pos(3)+axisangle(3)+gripper_qpos(2)), not [6]. The earlier config.json entry of shape [6] was a documentation error in the checkpoint. Always verify against the safetensors file, not config.json.
xvla action output is 20-D (padded): xVLA pads actions to max_state_dim=20. LIBERO's env.step expects 7-D. Slice action_np = action_tensor.squeeze(0).cpu().numpy()[:7] to extract the real 7-D action.
xvla is LIBERO-engine-only: the xVLA adapter's env preprocessor (LiberoProcessorStep) consumes the nested LiberoEnv observation that the scene must expose as observation['raw']. Non-LIBERO scenes (e.g. the Isaac Sim Franka scenes) do not populate it, so xvla raises ROSCapabilityMismatch on the first step. Run xvla only on LIBERO scenes (libero_spatial, libero_object, libero_goal, libero_10, …).
GR00T reads a fixed camera set; the RLDX sidecar has no single-camera fallback: these checkpoints read a fixed number of distinct camera streams positionally — LIBERO=2 (agentview+wrist), RC365=3, GR1/Simpler=1 — set by the manifest's state_contract.layout. The rldx factory (out-of-process sidecar) rejects a scene that declares fewer cameras than the layout needs with an upfront ROSCapabilityMismatch (before the multi-minute sidecar boot). GR00T N1.7 is now in-process (no sidecar boot); its _GrootAdapter reads the state/action width from the rSkill (state_contract.dim / action_contract.dim) and the GR00T video modality keys from the slot-keyed image_preprocessing.aliases (camera1: image, camera2: wrist_image for libero_sim) — the same rename every VLA adapter uses. Both positional cameras must still be supplied.
RLBench requires a separately-provisioned CoppeliaSim/PyRep sidecar: just sync --group rlbench installs only the openral-side ZMQ/msgpack client. CoppeliaSim 4.1.0 (proprietary, free EDU), PyRep, the MohitShridhar/RLBench@peract fork, and 3D Diffuser Actor live in ~/.cache/openral/rlbench-policy/.venv (or OPENRAL_RLBENCH_SIDECAR_PYTHON). The adapter raises a typed ROSConfigError with the recipe when that venv or COPPELIASIM_ROOT is missing.
OpenVLA-OFT / RLinf needs a transformers<5 runtime: RLinf/RLinf-OpenVLAOFT-PPO-ManiSkill3-25ood loads through OpenVLA's custom AutoModelForVision2Seq code path, verified with transformers==4.40.1 / accelerate==0.33. The default OpenRAL VLA workspace pins transformers>=5.4.0,<5.14.0 for lerobot families, so do not sync OpenVLA into the same venv as LIBERO/π0.5/SmolVLA unless the upstream custom code is ported.
π0.5 requires ≥8 GB VRAM: The PaliGemma-3B backbone requires more memory than the 7-class GPU can provide in typical shared use. Use --device cpu for slow inference or a dedicated A100/H100 for production eval.
MetaWorld uses Sawyer, not Franka (see §3.3): the franka_panda tag in the lerobot metaworld dataset metadata refers to the broader manipulation skill class. Do not use smolvla_metaworld weights on a real Franka without re-training.
LIBERO ~/.libero/config.yaml must point at the data files: After installing hf-libero via pip, the config file at ~/.libero/config.yaml pins absolute paths computed at first import and is never refreshed when you switch venv / workspace path. The next just sim-libero / just sim-xvla-libero / just sim-pi05-libero run then crashes inside lerobot.envs.libero.get_task_init_states with a FileNotFoundError on <stale-path>/init_files/<task>.pruned_init. The _ensure-libero-config private recipe (chained off every libero just sim-* target) invokes tools/fix_libero_config.py to detect + rewrite the file when stale; idempotent. Run it manually any time with uv run --group libero python tools/fix_libero_config.py --verbose, or set LIBERO_CONFIG_PATH to a project-local dir to bypass ~/.libero entirely.

6. Runtime posture per VLA family

Which families run in the workspace interpreter and which run out-of-process in a pinned sidecar. Licenses are in the VLA license matrix in CLAUDE.md §3.

XR-1 (RoboCasa, RoboCasa365, VLABench checkpoints): xr1 Py3.12 sidecar pinned to torch 2.9.1 / transformers 4.57.1 / FlashAttention 2.8.3 (upstream validated against torch 2.8, but its cu128 build has no linux-aarch64 wheel — see aarch64-support.md). Manifests select bitsandbytes NF4 at load so the 5.44B MiBoT model targets 8 GB-class GPUs. The VLABench checkpoint is live-validated at 3.66 GiB process VRAM and 0.81 s per warmed chunk on an RTX 4070 Laptop 8 GB.
LingBot-VLA: auto-provisioned Py3.12 ZMQ sidecar, because lingbotvla pins torch==2.8.0 + custom kernels. v2 — 6B Qwen3-VL MoE on transformers==4.57.3, torch stack overridden to 2.9.1 / triton 3.5.1 so the sidecar provisions on aarch64. v1 — 4B Qwen2.5-VL dense expert on transformers==4.51.3 / lerobot==0.4.2 (the RoboTwin post-train), same sidecar via --variant v1, on torch 2.7.1 / torchvision 0.22.1 (cu128) / torchcodec 0.5 — the newest stack under lerobot 0.4.2's torch<2.8.0 cap. x86_64-only, because torch 2.7.x needs the x86-only triton 3.3.1.
GR00T N1.7: standard checkpoints run in-process under the workspace's Python 3.12 via lerobot 0.6.0's native GrootPolicy (backbone-only NF4). The official 2026 BEHAVIOR-1K organizer checkpoint is the exception: its pinned wensi-ai/Isaac-GR00T runtime stays in an external Python 3.10 sidecar; its deploy sim path preserves the official 61-D state and commits the six individually safety-approved action slots atomically as one 23-D OmniGibson step.
RLDX-1: out-of-process ZMQ sidecar through the rldx adapter.
3D Diffuser Actor (diffuser_actor): out-of-process RLBench/CoppeliaSim sidecar (§3.2).
LingBot-VA (lingbot_va_a1, Galaxea A1): driven over a local socket through the Galaxea A1 runtime's policy gateway.
In-process under the workspace Python 3.12: smolvla, pi05, xvla, act, diffusion, molmoact2 and openvla (the last two are transformers custom-code models).
InternVLA-N1 / DualVLN (arXiv:2512.08186): dual-system (Qwen2.5-VL-7B S2 planner + NavDP DiT S1); Py3.11 ZMQ sidecar (upstream pins transformers 4.51) through the internvla_n1 adapter; consumes RGB-D + instruction, emits BODY_TWIST.