API Reference

Auto-generated from docstrings on the workspace's public Python surface. Source of truth: each package's __init__.py (__all__) and the modules referenced below.

Core schemas and exceptions

openral core schemas and exceptions — Pydantic v2 contracts shared across all layers.

Public surface is everything imported here. Private modules use leading underscore.

BenchmarkName: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

Benchmark suite id (open; _REGISTRY_ID_PATTERN).

One per benchmarks/<id>.yaml suite in tree.

Membership is checked against the checkout (benchmarks/*.yaml plus scenes/benchmark/*.yaml stems, openral_rskill.loader.known_benchmark_ids) by the benchmark writeback (update_rskill_benchmarks), the openral benchmark scene guard and CI; rSkill.from_pretrained only WARNS on an unknown suite, since a Hub install may have no checkout to compare against.

Used as keys in RSkillManifest.benchmarks. Each value is the headline success rate [0.0, 1.0] the skill achieves on that suite. The full breakdown lives in the matching rskills/<id>/eval/<key>.json.

aloha_insertion / aloha_transfer_cube are retained as task-level ids (the act-aloha* manifests cite their per-task paper numbers) even though the two single-task suites were unified into aloha.yaml — the unified suite auto-filters per rSkill so a single run scores one ACT checkpoint's one task.

CANONICAL_MODEL_TOKENS: frozenset[str] = frozenset({'smolvla', 'pi05', 'xvla', 'xr1', 'act', 'diffusion', 'molmoact2', 'openvla', 'openvla_oft', 'gr00t_n17', 'rldx1_ft', '3d_diffuser_actor', 'lingbot_vla', 'lingbot_vla2', 'lingbot_va_a1', 'internvla_n1', 'sam2_1', 'omdet_turbo', 'rtdetr_coco_r18', 'rtdetr_v2_r50vd', 'robometer_4b', 'topreward_qwen3vl_4b', 'locateanything_3b', 'qwen35_4b', 'moveit', 'nav2'}) module-attribute

Canonical <model> name tokens — the versioned checkpoint vocabulary the repo-naming convention allows. Distinct from ModelFamily (a runner dispatch key): several tokens can share one family (openvla / openvla_oft) and non-VLA tool models (omdet_turbo, robometer_4b, moveit, …) have no family at all. Adding a checkpoint token here lets a new rSkill publish under it.

CANONICAL_QUANT_TOKENS: frozenset[str] = frozenset({'fp32', 'fp16', 'bf16', 'int8', 'nf4'}) module-attribute

Canonical <quant> name tokens for weight-bearing kinds. int4 → nf4 (bitsandbytes NF4). The ROS wrappers (_WEIGHTLESS_KINDS) carry no weights and OMIT the <quant> segment entirely (a 4-part name).

CANONICAL_ROBOT_NAME_TOKENS: frozenset[str] = frozenset({'any', 'multi'}) module-attribute

Reserved robot-agnostic <robot> name tokens: the "any" wildcard and the "multi" aggregate (a skill declaring >1 concrete robot). Concrete robot tokens are an open registry (robots/*/robot.yaml): repo_name_is_canonical checks them by shape only, and CI checks in-tree names against the robot tags (with the so10x_follower tags shortened to so100/so101 via _EMBODIMENT_TO_ROBOT_TOKEN) plus these reserved tokens.

CollisionShape: TypeAlias = Annotated[CapsuleShape | SphereShape | BoxShape, Field(discriminator='shape')] module-attribute

Discriminated union of convex collision primitives.

Discriminator field is shape. Used by LinkCollisionGeometry (robot links) and AttachedCollisionPrimitive (carried payloads). Mesh primitives are excluded — the allocation-free safety kernel checks only convex analytic shapes; mesh-accurate collision is a planning-layer concern.

Notes:

• Validating a mapping requires the shape key; an untagged dict (e.g. {"radius_m": ...}) is rejected with union_tag_not_found rather than structurally guessed. Every robots/ manifest already writes the tag.
• Constructing a member directly is unaffected: BoxShape(...) still fills shape="box" from its default.
• Never dump a shape with exclude_defaults=True — it drops the tag and the result no longer re-validates.

EmbodimentTag: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

Embodiment tag (open; _REGISTRY_ID_PATTERN).

One per robots/<id>/robot.yaml shipped in tree, plus "custom" as the explicit "I know what I'm doing" escape hatch and "any" as the explicit embodiment-agnostic wildcard.

"multi" is a repo-name aggregate token (CLAUDE.md §3 rSkill packaging): the <robot> segment a skill declaring >1 concrete robot uses (e.g. the moveit-* ROS wrappers). It is a member of the Literal so the name validator can accept it; a manifest normally lists the specific robots in embodiment_tags rather than the aggregate.

"any" is the declared way to say "this rSkill runs on every embodiment": perception kinds (detector / vlm / reward) and playbook decision procedures use embodiment_tags: ["any"]. An empty embodiment_tags is rejected by RSkillManifest._check_embodiment_tags_present — agnosticism must be declared, never derived from an empty list (CLAUDE.md §1.4). The rSkill↔robot gate (openral_rskill.loader.rSkill.check_embodiment_tags) treats "any" in a skill's tags as match-any; a robot never declares "any".

"mobile_base" is a CLASS tag (not a specific robot): any robot with a planar base + body_twist actuator declares it so base-only rSkills (Nav2 NavigateToPose, etc.) can target the whole class without naming each specific mobile platform. Robot-specific tags (e.g. "panda_mobile") coexist on the same RobotDescription for skills that DO depend on the specific composition.

The in-tree CI guard (tests/unit/test_manifest_registry_ids.py) restricts every rskills/*/rskill.yaml tag to {any, custom, multi} plus the tags some robots/*/robot.yaml declares, so a typo cannot land in a manifest where the loader's compat check would silently never match; openral_rskill.loader.intree_embodiment_tags is that set. When "custom" is used, the manifest MUST also populate embodiment_extra (see EmbodimentExtra); the cross-validator on RSkillManifest enforces this.

FailureEvidence: TypeAlias = TimeoutEvidence | ForceEvidence | WorkspaceEvidence | PerceptionStaleEvidence | CriticEvidence | ControllerEvidence | SelfVerifyEvidence | HumanEvidence | WamEvidence | ReasonerTimeoutEvidence | CollisionEvidence | SuppressedSummaryEvidence module-attribute

Discriminated union for FailureTrigger.evidence_json payloads.

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming evidence_json string with::

from pydantic import TypeAdapter
from openral_core import FailureEvidence

evidence = TypeAdapter(FailureEvidence).validate_json(msg.evidence_json)

Producers serialize via evidence.model_dump_json().

The union mirrors the KIND_* constants on openral_msgs/msg/FailureTrigger. Adding a new variant requires adding a new KIND_* constant to the IDL.

GripperConvention: TypeAlias = Literal['normalized_open_unit', 'normalized_open_symmetric', 'normalized_close_symmetric', 'binary_close_one', 'raw_joint_rad', 'width_meters'] module-attribute

How a gripper command value is encoded — on both sides of the wire.

An rSkill declares the encoding it emits on its GRIPPER_* actuator (ControlModeSemantics.gripper_convention); a robot declares the encoding each gripper end effector consumes (EndEffectorSpec.command_convention). rSkill.check_gripper_conventions refuses to load a skill whose encoding differs from the end effector it drives — silently mis-encoding the gripper between scenes / robots is a top observed mis-actuation failure — and the safety kernel bounds every GRIPPER_* chunk by that end effector's command_range.

Conventions:

• normalized_open_unit — 0.0 = fully closed, 1.0 = fully open. Most lerobot / SmolVLA / pi0.5 LIBERO checkpoints.
• normalized_open_symmetric — -1.0 = fully closed, +1.0 = fully open. Some MetaWorld checkpoints.
• normalized_close_symmetric — -1.0 = fully open, +1.0 = fully closed. robosuite's gripper action (RoboCasa).
• binary_close_one — 0.0 = open, 1.0 = close (single bit).
• raw_joint_rad — raw per-finger joint angle in radians (Fourier dexhands, some humanoid skills).
• width_meters — physical gripper width in metres (Franka FCI native).

JointRole: TypeAlias = Literal['arm', 'base', 'gripper', 'torso', 'leg', 'head', 'neck', 'wheel', 'unknown'] module-attribute

Structural classification of a JointSpec.

Carries the joint's purpose in the embodiment's morphology — what the runner, safety kernel, and dataset bridge need to identify a channel without relying on name-substring heuristics (e.g. "gripper" in name.lower() in rskill_runner_node._build_joint_permutation which silently misclassifies any joint with "gripper" in the name).

"unknown" is the default so legacy manifests load unchanged; the fleet annotates incrementally as this rolls out.

MemorySection: TypeAlias = Literal['home_map', 'preferences', 'lessons', 'object_locations', 'open_tasks'] module-attribute

The fixed sections of the self-maintained MEMORY.md core.

• home_map — stable places / region-connectivity (EDIT in place).
• preferences — distilled user-preference rules (TidyBot; EDIT in place).
• lessons — distilled human corrections / hard-won lessons (DROC; APPEND).
• object_locations — timestamped object→place log (SUPERSEDE on re-observation).
• open_tasks — recurring commitments / open items (EDIT; remove on completion).

ModelFamily: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

VLA / policy family the skill belongs to (open; _REGISTRY_ID_PATTERN).

Membership is checked against the policy registry openral_sim.POLICIES (the runner refuses an unregistered family at dispatch; CI checks every in-tree manifest). In-tree families: smolvla, pi05, xvla, xr1, act, diffusion, rldx, molmoact2, gr00t, diffuser_actor, openvla, lingbot_vla2, lingbot_vla, lingbot_va_a1, internvla_n1.

Used by the eval / runner adapters to dispatch to the right openral_sim.backends.<family> policy adapter without string-matching the skill name. Adding a family means registering the matching adapter in openral_sim.POLICIES — not editing this module.

xr1 (Xiaomi Robotics XR-1 / MiBoTForActionGeneration) runs out-of-process because its released checkpoints pin transformers 4.57.1 and FlashAttention 2.8.3 (the sidecar installs torch 2.9.1 rather than upstream's 2.8.0, which publishes no linux-aarch64 wheel). The adapter consumes the three public benchmark checkpoints; the bare 5B model_states.pt is a post-training seed, not an executable policy.

gr00t (NVIDIA Isaac GR00T N1.x / N2) runs out-of-process via a ZMQ sidecar in an isolated Python 3.10 venv, sharing the architecture of the rldx adapter (RLDX-1 is itself a GR00T-N1.5 finetune).

lingbot_vla2 (Robbyant LingBot-VLA 2.0, Qwen3-VL-4B backbone + sparse-MoE flow-matching action expert, Apache-2.0 code + weights) runs out-of-process via a ZMQ sidecar in its own Python 3.12 + torch-2.9.1 venv — the upstream lingbotvla package pins torch==2.8.0 / transformers==4.57.3 + custom Triton MoE kernels, incompatible with the workspace transformers>=5 (the torch half of that pin set is overridden to 2.9.1 because 2.8.0 has no linux-aarch64 wheel; adapter: openral_sim.policies.lingbot_vla2).

lingbot_vla (Robbyant LingBot-VLA 1.0, 4B Qwen2.5-VL-3B backbone + a dense Qwen2 flow-matching action expert; paper "A Pragmatic VLA Foundation Model", arXiv 2601.18692) is the RoboTwin post-train checkpoint. It shares the v2 sidecar transport + adapter but loads from the separate V1 upstream repo (github.com/robbyant/lingbot-vla) in its own transformers==4.51.3 / lerobot==0.4.2 venv, driven by tools/_lingbot_vla2_server.py --variant v1 (adapter: openral_sim.policies.lingbot_vla2 id lingbot_vla).

lingbot_va_a1 is the Galaxea A1 EEF deployment adapter for the LingBot-VA checkpoint. It connects to versioned local Camera Bridge and policy-gateway services owned by the separately maintained A1 Runtime, without importing its checkout. Runtime owns the exact model configuration, EEF transforms, cache semantics, and IK; OpenRAL validates the gateway identity and returns its joint/gripper proposals through the normal safety and HAL path.

openvla (OpenVLA / OpenVLA-OFT) is a transformers custom-code model loaded in-process (trust_remote_code, gated by OPENRAL_ALLOW_REMOTE_CODE=1); the adapter de-normalizes the policy's discrete action tokens with the checkpoint's embedded unnorm_key stats and replays the action chunk closed-loop.

internvla_n1 (InternRobotics InternVLA-N1 / DualVLN, arXiv:2512.08186) is a dual-system vision-language NAVIGATION policy (Qwen2.5-VL-7B System-2 + NavDP DiT System-1). Upstream InternNav pins transformers==4.51.0 — incompatible with the workspace — so it runs out-of-process via the same ZMQ sidecar architecture as rldx. The adapter consumes RGB-D + instruction and emits a 6-D BODY_TWIST velocity command for a mobile base.

PerceptionEventMetadata: TypeAlias = MotionMetadata | ObjectsMetadata | OcrMetadata | SceneChangeMetadata module-attribute

Discriminated union for PromptStamped.metadata_json payloads on the /openral/perception/<kind> topics.

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming metadata_json string with::

from pydantic import TypeAdapter
from openral_core import PerceptionEventMetadata

metadata = TypeAdapter(PerceptionEventMetadata).validate_json(msg.metadata_json)

Producers serialise via metadata.model_dump_json().

The four variants map 1:1 onto the four per-kind topics fixed in the perception-event contract (motion, objects, ocr, scene_change). Adding a new variant requires adding a new /openral/perception/<kind> topic to the contract — by design, new kinds get new topics, not a schema bump, so subscribers can subscribe to exactly the kinds they care about.

REASONER_TOOL_CHOICES: tuple[str, ...] = ('auto', 'required') module-attribute

Values OPENRAL_REASONER_TOOL_CHOICE accepts (OpenAI tool_choice modes).

ROBOT_UNIT_ENV = 'OPENRAL_ROBOT_UNIT' module-attribute

Env var naming the robots/<robot_id>/units/<unit>.yaml overlay for this host.

Wins over DeployScene.robot_unit so one committed scene runs on several cells.

RSkillKind: TypeAlias = Literal['vla', 'wam', 'ros_action', 'ros_service', 'detector', 'segmenter', 'vlm', 'reward', 'playbook'] module-attribute

Discriminator selecting how an rSkill is instantiated at the loader.

• "vla" — learnable Vision-Language-Action policy. Requires RSkillManifest.model_family and RSkillManifest.weights_uri; resolved by the policy adapter dispatch in openral_rskill (the pre-existing path used by every in-tree rSkill prior to this discriminator landing).
• "wam" — World Action Model (planning-layer mental-simulation / failure-anticipation component per CLAUDE.md §3). Reserved so the discriminator is forward-compatible; the loader / runner branch is not implemented yet and kind: wam manifests are rejected at resolve time with ROSConfigError.
• "ros_action" — wraps an existing ROS 2 action server. Requires a RosIntegration block. model_family and weights_uri are forbidden. chunk_size is pinned to 1 so the safety supervisor's per-row check sees every commanded position.
• "ros_service" — wraps an existing ROS 2 service. Same constraints as "ros_action".
• "detector" — perception producer that runs an exported detection model (RT-DETR / D-FINE ONNX) on the camera tee and publishes ObjectsMetadata; emits no Action. Requires a DetectorContract block and RSkillManifest.weights_uri (the exported ONNX / TensorRT engine). model_family and action_contract / state_contract are forbidden.
• "segmenter" — perception producer that answers a geometric prompt (a point, or a small set of positive/negative points, in pixel coordinates) with a binary mask over the current camera frame. Unlike "detector" it carries no label vocabulary and no score threshold: it does not say what the pixels are, only which pixels belong to the prompted thing. Its consumer is the HAL's vision attachment-evidence producer, which intersects the mask with the wrist depth frame to bound a grasped payload. Requires a SegmenterContract block and RSkillManifest.weights_uri; model_family, detector, reward, ros_integration, action_contract / state_contract, processors, image_preprocessing, n_action_steps and starting_pose are FORBIDDEN, and actuators_required MUST be empty — a segmenter actuates nothing.
• "vlm" — vision/video-language model used as a scene-understanding perception component (e.g. Qwen3.5-4B NF4). Accepts RGB image or video frames and a natural-language query; returns a text answer. Emits no actions or bounding boxes. Runs at S2 rate (role: "s2"), so it is surfaced to the reasoner as a read-only scene-query tool, never as an ExecuteRskill policy. weights_uri REQUIRED; actuators_required MUST be empty; action_contract, state_contract, detector, ros_integration, processors, image_preprocessing, n_action_steps, and starting_pose are FORBIDDEN. model_family is OPTIONAL metadata.
• "playbook" — a symbolic, human-authored decision procedure (a Markdown standard-operating-procedure) the S2 Reasoner reads, not code it executes. Carries no weights, no actuators, no ROS server, no Action contract — it is the symbolic counterpart to a vla policy. Requires a PlaybookContract block and role: "s2"; chunk_size MUST be 1 (required field, no Action rows); actuators_required MUST be empty; actions MUST include RSkillAction.PLAN. weights_uri, model_family, min_vram_gb, detector, reward, ros_integration, processors, image_preprocessing, action_contract, state_contract, n_action_steps, starting_pose are FORBIDDEN. Surfaced to the reasoner by injecting its PLAYBOOK.md body into the system prompt (or via a retrieval tool at scale), never as an ExecuteRskill policy.

ReasonerDialect = Literal['anthropic', 'openai'] module-attribute

Wire client an endpoint speaks — the only two the reasoner ships (ADR-0088).

ReasonerHosting = Literal['cloud', 'managed_local', 'byo_local'] module-attribute

Where a reasoner model runs: a vendor's cloud, a server OpenRAL spawns and manages locally, or a local server the operator brings up themselves.

ReasonerToolCall: TypeAlias = Annotated[ExecuteRskillTool | ReloadGstPipelineTool | LifecycleTransitionTool | EmitPromptTool | WaitTool | RecallObjectTool | ResolvePlaceTool | LocateInViewTool | QuerySceneTool | QueryTaskProgressTool | MemoryWriteTool | MemorySearchTool | DecomposeMissionTool, Field(discriminator='tool')] module-attribute

Discriminated union over the reasoner tool variants.

The discriminator field is tool (a string Literal on each variant), and it is declared as one: a malformed payload reports only the errors of the variant it tagged itself as, instead of every variant's. The caller always sets tool from the provider's function name before validating (openral_reasoner.tool_use), so an untagged payload is a bug in the caller, not a shape to guess at.

Consumers decode an LLM tool-use payload with::

from pydantic import TypeAdapter
from openral_core import ReasonerToolCall

call = TypeAdapter(ReasonerToolCall).validate_json(payload)

Producers (LLM clients) serialise via call.model_dump_json().

Actuation/effect palette: ExecuteRskillTool, ReloadGstPipelineTool, LifecycleTransitionTool, EmitPromptTool, WaitTool. Read-only query variants (no actuation authority): RecallObjectTool, ResolvePlaceTool, LocateInViewTool, QuerySceneTool, QueryTaskProgressTool, MemorySearchTool. Write-to-memory: MemoryWriteTool. S2-ledger-only: DecomposeMissionTool (populates/flat-splices MissionState's task queue, never actuation).

Extending the palette requires (a) a new variant here, (b) the corresponding ROS-side dispatch in openral_reasoner_ros.reasoner_node, (c) a CLAUDE.md §3 amendment if the new tool shifts the reasoner's authority surface. The query variants' dispatch + result-return path is wired separately; until then they are a typed contract not yet exposed in the live provider palette.

StateLayout: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

Per-checkpoint proprioception layout id (open; _REGISTRY_ID_PATTERN).

Membership is NOT checked here: a task-space layout without a registered assembler is dropped by the reasoner palette and refused by openral_state_adapter.assemble_state (ROSConfigError); CI checks every in-tree manifest layout against registered_layouts() plus the sim adapters' layout maps.

In-tree layouts:

• smolvla_9d, human300_16d, gr1, rc365.
• simpler_widowx matches RLWRLD/RLDX-1-FT-SIMPLER-WIDOWX's bridge_orig modality config (8 scalar state keys, single video.image_0 camera, Bridge-data orientation rotation). simpler_google matches RLWRLD/RLDX-1-FT-SIMPLER-GOOGLE's fractal20220817_data modality config (8 scalar state keys including a 4-D quaternion in state.r{x,y,z,w}, single video.image camera, sticky-gripper postprocessing).
• libero_eef8d — LIBERO 8-D task-space proprio eef_pos(3) ‖ eef_axisangle(3) ‖ gripper_qpos(2) in the world frame, what the lerobot/smolvla_libero, pi05-libero and xvla-libero checkpoints were trained on. The benchmark (openral sim run) supplies it directly; deploy assembles it from live TF (EE pose) + JointState (gripper) via the libero_eef8d assembler. Without it the runner would feed raw joint-space state to a task-space policy.

A layout names the SHAPE the checkpoint was trained on — field order, frame convention, gripper encoding, quaternion handedness. The per-robot SOURCE bindings (which TF frame is "the EE", which joint names are "the gripper") live on StateContractBindings. The openral_state_adapter registry maps each layout id to an assembler function that joins shape + bindings + live JointState + live TF.

WRAPPED_TASK_SPACE_LAYOUTS: frozenset[StateLayout] = frozenset({'rc365', 'human300_16d', 'libero_eef8d'}) module-attribute

Layouts that are TASK-space composites (Cartesian poses + gripper widths), NOT one-scalar-per-joint. These layouts REQUIRE StateContract.bindings to name the source TF frames + JointState entries — the cross-validator on StateContract enforces this at manifest load. The remaining layouts (smolvla_9d, gr1, simpler_*) are joint-space slices (potentially across multiple controller groups, as in GR1's 29-D waist+arms+hands composite) that the runner serves verbatim from observation.joint_state.position; a robot.yaml with the matching joint count dispatches them without an assembler.

Action

Bases: BaseModel

A single action step or chunk produced by a Skill.

Attributes:

Name	Type	Description
control_mode	ControlMode	Target action space.
horizon	int	Number of steps (1 = single step, H = chunk).
joint_targets	list[list[float]] | None	Joint position targets, shape (H, N).
joint_velocities	list[list[float]] | None	Joint velocity targets, shape (H, N).
joint_torques	list[list[float]] | None	Joint torque targets, shape (H, N).
cartesian_pose	list[Pose6D] | None	EE pose targets.
cartesian_delta	list[tuple[float, ...]] | None	EE pose deltas.
cartesian_twist	list[tuple[float, ...]] | None	EE velocity targets.
body_twist	list[tuple[float, float, float, float, float, float]] | None	Base twist targets.
foot_placements	list[dict[str, object]] | None	Discrete footstep targets.
gripper	list[float] | None	Gripper commands in [0, 1].
dex_hand_joints	list[list[float]] | None	Dexterous hand joint targets.
confidence	float	rSkill confidence in [0, 1].
stamp_ns	int	Action timestamp in nanoseconds.
ee_name	str | None	Target end-effector name.
frame_id	str | None	Reference frame for Cartesian actions.
cartesian_delta_scale	tuple[float, ...] | None	Per-axis physical range for a native normalized Cartesian-delta controller. Predictive safety applies clip(raw, -1, 1) * scale; the raw delta remains unchanged for the HAL. None means the policy already emits physical units.
joint_names	list[str] | None	Robot joint names this action's rows target, in the emitting slot's order (ADR-0102). None = a whole-vector action in RobotDescription.joints order. Set for JOINT_* slots, which are zero-padded to full dof and so cannot otherwise say which joints they own; authoritative over slot arrival order when present.
tick_index	int	Shared 1-based inference tick for every slot emitted by one policy step. Preserved across the ROS safety wire so a simulation HAL can commit a multi-surface action atomically after every slot passes.
tick_group_size	int	Number of non-discard slots emitted for this inference tick. 1 keeps single-surface actions unchanged.
runner_session_id	int	Random nonzero id of the rSkill runner process that emitted this action (ActionChunk.runner_session_id). A HAL keys its replay watermark on (runner_session_id, tick_index) because tick_index restarts with every runner process. 0 = unknown (session-less producer), which keeps the tick-only replay heuristic.
safety_overrides	dict[str, object]	Operator-approved safety override tokens.

ActionContract

Bases: BaseModel

Per-rSkill action-vector contract.

Mirrors StateContract for the action side. Carries the output dimensionality the checkpoint emits so the dataset bridge (and any downstream consumer) can bind the LeRobot v3 action feature shape without consulting the sim or hardware adapter.

The sim-specific action contract belongs on the per-checkpoint rSkill manifest, not on the physical RobotDescription (the same Franka emits 7-D delta-EEF on LIBERO vs 8-D joint pos on a hardware deploy).

Attributes:

Name	Type	Description
dim	int	Dimensionality of the action vector emitted by Skill.step() / PolicyAdapter.step(). Required when this contract is set.
representation	ActionRepresentation | None	Optional named representation (mirrors ActionSpec.representation). When set, downstream consumers can map between equivalent representations (e.g. joint_positions → delta_ee_6d_plus_gripper).
slots	list[ActionSlot] | None	Declarative slot layout. When set, every index in [0, dim) must be covered by exactly one ActionSlot (no gaps, no overlaps). The skill_runner reads this to dispatch slices of the policy vector onto typed Action objects. When None, the runner falls back to the legacy single-Action path (one implicit JOINT_POSITION slot covering the whole vector). Manifests carrying slots are exempt from the dim <= len(robot.joints) invariant because the slot decoder gives a typed contract per slice instead.
cartesian_delta_scale	tuple[float, ...] | None	Per-axis physical range for a normalized Cartesian controller. Predictive safety applies clip(raw, -1, 1) * scale. None means identity (the policy already emits physical units).
joint_names	list[str] | None	The checkpoint's joint order on the whole-vector joint path, spelled as RobotDescription.joints names (matched BY NAME by openral_rskill._policy_io.PolicyIOCodec). None means the codec derives the order from the checkpoint's own feature names, or identity. Forbidden together with slots (each JOINT_* slot carries its own joint_names).
gripper_scale	float	Policy-unit gripper value per robot-unit value. The HAL gripper contract is a normalized [0, 1] jaw fraction; a LeRobot SO-ARM checkpoint's gripper channel is [0, 100] → 100.0. Applied by the codec on BOTH the joint and the slot path.
control_freq_hz	float | None	The rate (Hz) the checkpoint was trained to be executed at — one action-chunk row per control period. None = undeclared (the runner uses the robot's action_spec.control_freq_hz unchecked). When declared, the skill runner refuses a goal whose tick rate differs: there is no temporal resampler, and executing a 15 Hz policy at 30 Hz doubles its commanded speed.

ActionRepresentation

Bases: str, Enum

Action vector representation format.

ActionSlot

Bases: BaseModel

One contiguous slice of an rSkill's action vector.

The skill_runner reads ActionContract.slots and emits one typed Action per non-discard slot per step. All actions inherit the parent step's trace_id so the safety supervisor and downstream telemetry can join them post-hoc.

Attributes:

Name	Type	Description
range	tuple[int, int]	Inclusive [start, end] indices into the flat policy action vector. range[0] must be ≤ range[1]; both must fall within [0, ActionContract.dim).
control_mode	ControlMode | None	The ControlMode the slice is routed to. The HAL whitelist on the target robot must include this mode (the palette filter rejects the rSkill at install time otherwise). None only when discard is True.
discard	bool	When True the slice is dropped silently — used for dataset artefacts like RoboCasa365's torso placeholder dim or paired gripper channels. The slot still occupies its range so coverage validation works; no Action is emitted.
ee	str | None	End-effector name from the robot's RobotDescription.end_effectors / RobotDescription.joints. REQUIRED for CARTESIAN_* modes (the pose is computed in the named EE's frame) and for GRIPPER_* modes (names the actuator). FORBIDDEN for BODY_TWIST and JOINT_POSITION.
frame	str | None	tf2 frame name. REQUIRED for cartesian + body-twist modes (the slice's bytes are expressed in this frame). FORBIDDEN for joint-position and gripper modes.
joint_names	list[str]	Robot joint names this slice targets, in slot order. REQUIRED for JOINT_POSITION / JOINT_VELOCITY / JOINT_TORQUE when the slot covers fewer than all robot joints; FORBIDDEN for non-joint modes. Length must equal range[1] - range[0] + 1.
input_bounds	tuple[float, float] | None	Optional inclusive controller-input bounds. The skill runner clips this slot before safety and HAL dispatch, matching native controllers that clip normalized policy outputs before applying physical scaling.

ActionSpec

Bases: BaseModel

VLA action configuration for this robot.

Attributes:

Name	Type	Description
dim	int | None	Dimensionality of the action vector. None = not declared: the dataset recorder then takes the width from the action itself. A robot whose policy contract is not committed declares only the control rate rather than a guessed width (issue #303).
representation	ActionRepresentation | None	How the action vector is encoded.
control_freq_hz	float | None	Control frequency the actions are executed at — the runner's tick, the HAL node's proprio publish rate, the real ros2_control HAL's trajectory deadline, and the recorder's fps.
chunk_size	int | None	Number of action steps per inference call (chunk size H).

ActuatorRequirement

Bases: BaseModel

One actuator slot an rSkill emits actions for.

Symmetric with SensorRequirement on the action side. The compatibility check resolves each entry against a RobotDescription's action_spec and rejects pairings whose declared ControlMode is not advertised by the robot.

For predefined embodiments (one of the 9 canonical robots/<id>/robot.yaml slugs), n_dof and vla_action_key are optional — the loader auto-fills them from the robot YAML at compatibility-check time. For the "custom" embodiment escape hatch they MUST be set on the manifest; the RSkillManifest cross-validator enforces this.

Attributes:

Name	Type	Description
kind	ControlMode	The action interface the skill emits (e.g. ControlMode.JOINT_POSITION, ControlMode.CARTESIAN_DELTA). Reuses the canonical ControlMode enum so the manifest, the safety kernel, Action.control_mode, and RobotDescription.action_spec all share one source of truth.
n_dof	int | None	Optional degrees of freedom the skill emits. Auto-filled from the robot YAML for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
vla_action_key	str | None	Optional slot name the policy emits, e.g. "action.joints.arm_left". Auto-filled for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
control_mode_semantics	ControlModeSemantics	Action-space semantics for this slot. REQUIRED — closes Gap 2 of the rSkill self-containment audit. Cross-validators enforce: gripper kinds require gripper_convention; cartesian kinds require reference_frame; other kinds forbid both.

Example

a = ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) a.kind is ControlMode.JOINT_POSITION True a.n_dof is None and a.vla_action_key is None True

ApproachViewpoint

Bases: BaseModel

A camera-facing standoff pose for viewing/manipulating an object.

Attributes:

Name	Type	Description
pose	Pose6D	Base/EE goal pose (map frame) at a standoff from the object, oriented so the gripper-mounted camera faces it.
standoff_m	float	Standoff distance from the object, in metres.
camera_frame_id	str	tf2 frame of the camera the viewpoint orients toward.

AssetRefs

Bases: BaseModel

The unified description-asset block on RobotDescription.

Replaces the scattered urdf_path / mjcf_uri / srdf_path (+ the earlier URDF-root fields) with one block whose refs share the openral_core.assets.resolve_asset grammar.

Attributes:

Name	Type	Description
urdf	UrdfAsset | None	URDF asset (with optional robot_state_publisher wiring), or None when the robot ships no URDF.
mjcf	str | None	MJCF asset ref (MuJoCo wiring), or None.
srdf	str | None	SRDF asset ref whose disable_collisions block seeds RobotDescription.allowed_collision_pairs, or None.

Example

AssetRefs(urdf=UrdfAsset(ref="rd:panda_description")).urdf.ref 'rd:panda_description'

AttachedCollisionObject

Bases: BaseModel

Collision geometry rigidly attached to a robot link.

The object is removed from world occupancy only when the same geometry is present here, so collision checking never loses the carried payload. The removal itself is the occupancy bridge's (openral_octomap_bridge clears the payload's own cells out of /openral/world_voxels while it appears here); the kernel keeps checking the payload as robot geometry.

Attributes:

Name	Type	Description
object_id	str	Stable perception/world identity.
attach_link	str	Robot link that owns the link-relative pose.
touch_links	list[str]	Explicit robot links allowed to contact the object.
primitives	list[AttachedCollisionPrimitive]	Bounded collision geometry in the object frame.
pose_in_link	Pose6D	Object pose whose frame_id equals attach_link.
mass_kg	float | None	Optional payload mass for real controller integration.
center_of_mass_m	tuple[float, float, float] | None	Optional payload CoG in the attached-object frame.
inertia_kg_m2	tuple[float, ...] | None	Optional row-major 3x3 inertia matrix.
confidence	float	Attachment confidence in [0, 1].
evidence_kind	AttachmentEvidenceKind	Source that confirmed the attachment.
evidence_ref	str | None	Optional trace/contact/track reference.
stamp_ns	int	Confirmation timestamp.
contact_force	ContactForceWitness | None	Optional attestation of a measured contact force on the declared place target (ADR-0100). None — the default, and what every producer that cannot measure force publishes — means the declaration-scoped force gate never arms and geometry decides alone. The gate it feeds only ever adds a refusal; it never licenses a contact the geometry refuses.
support_contact	SupportContactWitness | None	Optional attestation that this payload is in bounded support contact with a named environment surface (ADR-0092 D6). None — the honest value for any producer that cannot measure support contact, including the vision attachment producer — means the safety kernel exempts no payload-vs-world contact at all.

fill_idl(msg: object, *, primitive_factory: Callable[[], object]) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object, *, primitive_factory: Callable[[], object]) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS."""
    msg.object_id = self.object_id  # type: ignore[attr-defined]
    msg.attach_link = self.attach_link  # type: ignore[attr-defined]
    msg.touch_links = list(self.touch_links)  # type: ignore[attr-defined]
    msg.primitives = []  # type: ignore[attr-defined]
    for primitive in self.primitives:
        item = primitive_factory()
        primitive.fill_idl(item)
        msg.primitives.append(item)  # type: ignore[attr-defined]
    pose = msg.pose_in_link  # type: ignore[attr-defined]
    pose.position.x, pose.position.y, pose.position.z = self.pose_in_link.xyz
    (
        pose.orientation.x,
        pose.orientation.y,
        pose.orientation.z,
        pose.orientation.w,
    ) = self.pose_in_link.quat_xyzw
    msg.mass_valid = self.mass_kg is not None  # type: ignore[attr-defined]
    msg.mass_kg = float(self.mass_kg or 0.0)  # type: ignore[attr-defined]
    msg.center_of_mass_valid = self.center_of_mass_m is not None  # type: ignore[attr-defined]
    if self.center_of_mass_m is not None:
        center = msg.center_of_mass_m  # type: ignore[attr-defined]
        center.x, center.y, center.z = self.center_of_mass_m
    msg.inertia_valid = self.inertia_kg_m2 is not None  # type: ignore[attr-defined]
    msg.inertia_kg_m2 = (  # type: ignore[attr-defined]
        [float(value) for value in self.inertia_kg_m2]
        if self.inertia_kg_m2 is not None
        else [0.0] * 9
    )
    msg.confidence = float(self.confidence)  # type: ignore[attr-defined]
    msg.evidence_kind = self.evidence_kind.value  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref or ""  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.support_contact_valid = self.support_contact is not None  # type: ignore[attr-defined]
    if self.support_contact is not None:
        self.support_contact.fill_idl(msg.support_contact)  # type: ignore[attr-defined]
    msg.contact_force_valid = self.contact_force is not None  # type: ignore[attr-defined]
    if self.contact_force is not None:
        self.contact_force.fill_idl(msg.contact_force)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    center_of_mass = None
    if bool(msg.center_of_mass_valid):  # type: ignore[attr-defined]
        center = msg.center_of_mass_m  # type: ignore[attr-defined]
        center_of_mass = (float(center.x), float(center.y), float(center.z))
    inertia = (
        tuple(float(value) for value in msg.inertia_kg_m2)  # type: ignore[attr-defined]
        if bool(msg.inertia_valid)  # type: ignore[attr-defined]
        else None
    )
    pose = msg.pose_in_link  # type: ignore[attr-defined]
    attach_link = str(msg.attach_link)  # type: ignore[attr-defined]
    return cls(
        object_id=str(msg.object_id),  # type: ignore[attr-defined]
        attach_link=attach_link,
        touch_links=list(msg.touch_links),  # type: ignore[attr-defined]
        primitives=[
            AttachedCollisionPrimitive.from_idl(item, object_id=str(msg.object_id))  # type: ignore[attr-defined]
            for item in msg.primitives  # type: ignore[attr-defined]
        ],
        pose_in_link=Pose6D(
            xyz=(
                float(pose.position.x),
                float(pose.position.y),
                float(pose.position.z),
            ),
            quat_xyzw=(
                float(pose.orientation.x),
                float(pose.orientation.y),
                float(pose.orientation.z),
                float(pose.orientation.w),
            ),
            frame_id=attach_link,
        ),
        mass_kg=(
            float(msg.mass_kg)  # type: ignore[attr-defined]
            if bool(msg.mass_valid)  # type: ignore[attr-defined]
            else None
        ),
        center_of_mass_m=center_of_mass,
        inertia_kg_m2=inertia,
        confidence=float(msg.confidence),  # type: ignore[attr-defined]
        evidence_kind=AttachmentEvidenceKind(str(msg.evidence_kind)),  # type: ignore[attr-defined]
        evidence_ref=str(msg.evidence_ref) or None,  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        contact_force=(
            ContactForceWitness.from_idl(msg.contact_force)  # type: ignore[attr-defined]
            if bool(getattr(msg, "contact_force_valid", False))
            else None
        ),
        support_contact=(
            SupportContactWitness.from_idl(msg.support_contact)  # type: ignore[attr-defined]
            if bool(msg.support_contact_valid)  # type: ignore[attr-defined]
            else None
        ),
    )

AttachedCollisionPrimitive

Bases: BaseModel

One bounded payload primitive positioned in its object frame.

Attributes:

Name	Type	Description
shape	CollisionShape	The convex primitive (capsule, sphere, or box).
pose_in_object	Pose6D	Pose of the primitive in the owning object's frame.
tight_geometry	TightCollisionGeometry | None	Optional tighter convex geometry the safety kernel uses in place of shape in the payload-vs-world-voxel check — the same staged 26-DOP → exact-hull narrow phase a robot link gets from LinkCollisionGeometry.tight_geometry, and the same containment rule: only valid on a BoxShape, and only when it fits inside it. shape stays the broad-phase bound and the geometry every other check uses. None is not a defect — a sphere/box/capsule sim geom lowers exactly and has nothing to refine. It is only a mesh geom, whose lowering is its local AABB, that leaves a real gap: measured across the 2026-09-10 resolution A/B (424 samples, 4 scenes) a carried payload's box corners stand a median 50.78 mm proud of the mesh, 2.3-3.9x the entire world-voxel quantisation term, and 97 % of 15 mm-arm stops were payload-vs-world (issue #266).

Example

p = AttachedCollisionPrimitive( ... shape=BoxShape(half_extents_m=(0.03, 0.03, 0.09)), ... pose_in_object=Pose6D( ... xyz=(0.0, 0.0, 0.0), ... quat_xyzw=(0.0, 0.0, 0.0, 1.0), ... frame_id="sim:obj_main", ... ), ... ) p.tight_geometry is None True

fill_idl(msg: object) -> None

Populate one duck-typed attached primitive without importing ROS.

Parameters:

Name	Type	Description	Default
msg	object	A duck-typed openral_msgs/AttachedCollisionPrimitive to populate in place.	required

Raises:

Type	Description
ROSConfigError	The primitive is none of the three shapes the IDL can carry.

Fail-closed: an unrepresentable shape raises here rather than publishing shape_type=0 (the IDL default, indistinguishable from "no shape"). Mirrors the fail-closed pattern in openral_cli.collision.collision_primitive_envelope and this class's own from_idl, which already refuses an unknown shape_type.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate one duck-typed attached primitive without importing ROS.

    Args:
        msg: A duck-typed ``openral_msgs/AttachedCollisionPrimitive`` to
            populate in place.

    Raises:
        ROSConfigError: The primitive is none of the three shapes the IDL
            can carry.

    Fail-closed: an unrepresentable shape raises here rather than
    publishing ``shape_type=0`` (the IDL default, indistinguishable from
    "no shape"). Mirrors the fail-closed pattern in
    ``openral_cli.collision.collision_primitive_envelope`` and this
    class's own ``from_idl``, which already refuses an unknown
    ``shape_type``.
    """
    if isinstance(self.shape, SphereShape):
        msg.shape_type = msg.SHAPE_SPHERE  # type: ignore[attr-defined]
        msg.shape_dimensions = [float(self.shape.radius_m)]  # type: ignore[attr-defined]
    elif isinstance(self.shape, CapsuleShape):
        msg.shape_type = msg.SHAPE_CAPSULE  # type: ignore[attr-defined]
        msg.shape_dimensions = [  # type: ignore[attr-defined]
            float(self.shape.radius_m),
            float(self.shape.length_m),
        ]
    elif isinstance(self.shape, BoxShape):
        msg.shape_type = msg.SHAPE_BOX  # type: ignore[attr-defined]
        msg.shape_dimensions = [  # type: ignore[attr-defined]
            float(value) for value in self.shape.half_extents_m
        ]
    else:
        raise ROSConfigError(
            f"attached collision primitive {self.shape.shape!r} has no "
            "openral_msgs/AttachedCollisionPrimitive encoding. Add a SHAPE_* "
            "constant and a branch here (and in from_idl) before carrying it "
            "on a payload — it must never be published as an unset shape."
        )
    pose = msg.pose_in_object  # type: ignore[attr-defined]
    pose.position.x, pose.position.y, pose.position.z = self.pose_in_object.xyz
    (
        pose.orientation.x,
        pose.orientation.y,
        pose.orientation.z,
        pose.orientation.w,
    ) = self.pose_in_object.quat_xyzw
    tight = self.tight_geometry
    msg.tight_dop_lo = [] if tight is None else list(tight.dop_lo_m)  # type: ignore[attr-defined]
    msg.tight_dop_hi = [] if tight is None else list(tight.dop_hi_m)  # type: ignore[attr-defined]
    msg.tight_hull_vertices = (  # type: ignore[attr-defined]
        []
        if tight is None
        else [coordinate for vertex in tight.hull_vertices_m for coordinate in vertex]
    )

from_idl(msg: object, *, object_id: str) -> Self classmethod

Decode one duck-typed attached primitive without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object, *, object_id: str) -> Self:
    """Decode one duck-typed attached primitive without importing ROS."""
    dimensions = [float(value) for value in msg.shape_dimensions]  # type: ignore[attr-defined]
    shape_type = int(msg.shape_type)  # type: ignore[attr-defined]
    if shape_type == int(msg.SHAPE_SPHERE):  # type: ignore[attr-defined]
        if len(dimensions) != 1:
            raise ValueError("Attached sphere requires one shape dimension.")
        shape: CollisionShape = SphereShape(radius_m=dimensions[0])
    elif shape_type == int(msg.SHAPE_CAPSULE):  # type: ignore[attr-defined]
        if len(dimensions) != 2:  # noqa: PLR2004
            raise ValueError("Attached capsule requires two shape dimensions.")
        shape = CapsuleShape(radius_m=dimensions[0], length_m=dimensions[1])
    elif shape_type == int(msg.SHAPE_BOX):  # type: ignore[attr-defined]
        if len(dimensions) != 3:  # noqa: PLR2004
            raise ValueError("Attached box requires three shape dimensions.")
        shape = BoxShape(half_extents_m=(dimensions[0], dimensions[1], dimensions[2]))
    else:
        raise ValueError(f"Unknown attached collision shape type: {shape_type}")
    pose = msg.pose_in_object  # type: ignore[attr-defined]
    return cls(
        shape=shape,
        pose_in_object=Pose6D(
            xyz=(
                float(pose.position.x),
                float(pose.position.y),
                float(pose.position.z),
            ),
            quat_xyzw=(
                float(pose.orientation.x),
                float(pose.orientation.y),
                float(pose.orientation.z),
                float(pose.orientation.w),
            ),
            frame_id=object_id,
        ),
        tight_geometry=_tight_geometry_from_idl(msg),
    )

AttachmentEvidenceKind

Bases: str, Enum

Evidence source that confirmed a robot/object attachment or its support.

SIM_CONTACT and SIM_GEOM_DISTANCE are both simulator ground truth, and the distinction is load-bearing rather than cosmetic. A simulator's contact list is not a proximity oracle: MuJoCo's contype / conaffinity bitmasks suppress whole geom pairs, so an object flush on a counter can generate no contact record at all. SIM_GEOM_DISTANCE names the signed closest-distance probe (mj_geomDistance), which sees the pairs the contact list hides. A consumer reading a witness must not infer from SIM_CONTACT that a solver contact existed, nor from SIM_GEOM_DISTANCE that one did not.

BenchmarkMetadata

Bases: BaseModel

Provenance block required on every BenchmarkScene.

paper / honest_scope are the published-protocol citation + a one-sentence "what this eval actually measured" statement.

display_name / simulator are optional paper-comparison labels that surface into RSkillEvalResult.benchmark when present — display_name becomes benchmark.name and simulator becomes benchmark.simulator. Before the scene-hierarchy convergence these lived in a free-form dict on the deleted BenchmarkSpec; moving them per-scene keeps them with their provenance and lets the aggregator emit identical JSON whether driven by run_benchmark (suite) or run_benchmark_scene (single scene).

BenchmarkScene

Bases: SimScene

Full benchmark eval spec for openral benchmark.

n_episodes, seed, and metadata are required with no defaults — they must match the published evaluation protocol. The task must supply success_key and max_steps.

BoxShape

Bases: BaseModel

Oriented box (OBB) collision primitive — a rectangular convex block.

An axis-aligned box in the primitive's local frame (placed and oriented by the owning frame's origin_xyz_rpy / pose), spanning [-half_extents_m[k], +half_extents_m[k]] along each local axis k.

A box fits a blocky link (a near-cubic housing, e.g. the SO-ARM100/101 base) far tighter than a capsule: a capsule's circular cross-section must bulge past the block's flat faces, so it over-reports clearance at poses where the flat faces are what actually face a neighbour. That bulge is what makes the manifest capsule model false-E-stop the SO-101 at its home pose (base↔lower_arm / base↔wrist reported penetrating while the true mesh clearance is +0.16 m / +0.24 m). See issue #84.

Attributes:

Name	Type	Description
shape	Literal['box']	Discriminator (always "box").
half_extents_m	tuple[PositiveFloat, PositiveFloat, PositiveFloat]	Half-sizes (hx, hy, hz) along the local x/y/z axes in metres (the full box is 2*hx x 2*hy x 2*hz).

Example

BoxShape(half_extents_m=(0.055, 0.048, 0.036)).shape 'box'

CameraSimPlacement

Bases: BaseModel

Where a camera sensor's (RGB or depth) camera sits in the sim MJCF.

Lets the generic HAL camera rig (openral_hal._camera_rig) splice a manifest camera into a bare-arm MJCF that ships no <camera> elements, so a deploy sim digital twin renders the robot's declared cameras without a per-robot scene composer. A camera is a property of the robot's sensor suite (the wrist camera is bolted to the gripper; the overhead camera is a declared sensor), so its sim pose lives on the SensorSpec, not in a scene.

The rig only splices a camera that is absent from the loaded MJCF, so this is a no-op for scene-attached or already-composed models (those carry their own cameras).

Attributes:

Name	Type	Description
parent_body	str | None	MJCF body the camera is rigidly mounted to (tracks that body's pose — e.g. the gripper/Fixed_Jaw for a wrist camera). None mounts it world-fixed in <worldbody> (an overhead / third-person camera).
pos	tuple[float, float, float]	Camera position (x, y, z) in metres, in parent_body's frame (or world when parent_body is None).
target	tuple[float, float, float]	Look-at point (x, y, z) in the same frame; the camera's MuJoCo -z view axis is oriented from pos toward it.
up	tuple[float, float, float]	World-up hint used to level the camera roll. Defaults to (0, 0, 1). A camera physically mounted upside-down (e.g. a wrist phone bolted inverted on the gripper) uses (0, 0, -1) to roll the image 180°.
fovy_deg	float | None	Vertical field of view in degrees. None derives it from the sensor's pinhole intrinsics (2·atan(height / (2·fy))) so the rendered FoV matches the declared camera model.

Example

CameraSimPlacement(pos=(0.5, 0.3, 0.75), target=(0.5, 0.3, 0.0)).parent_body is None True

CameraTopicKind

Bases: StrEnum

Per-camera stream suffix under CAMERA_TOPIC_PREFIX/<sensor name>/.

CapsuleShape

Bases: BaseModel

Capsule collision primitive — a segment swept by a radius.

The central segment runs along the local +Z axis from -length_m / 2 to +length_m / 2 (the MJCF / URDF capsule convention); it is placed and oriented by the owning frame (LinkCollisionGeometry.origin_xyz_rpy for a link, AttachedCollisionPrimitive.pose_in_object for a payload). Capsules bound most robot links tightly, so the safety check stays conservative.

Attributes:

Name	Type	Description
shape	Literal['capsule']	Discriminator (always "capsule").
radius_m	float	Capsule radius in metres.
length_m	float	Length of the central segment in metres (the cylinder portion; the total span is length_m + 2 * radius_m). 0.0 degenerates to a sphere.

Example

CapsuleShape(radius_m=0.04, length_m=0.3).length_m 0.3

ClockAuthority

Bases: BaseModel

Named origin for OpenRAL timestamps across sim, rSkills, ROS, and hardware.

The invariant is: every serialized stamp_ns is measured in the active deployment's OpenRAL timestamp domain. Simulation deployments may publish that domain onto ROS /clock; real deployments normally use host wall time or a PTP/hardware clock already synchronized into the graph. rSkills consume stamps from this domain; they do not create a separate clock.

Attributes:

Name	Type	Description
origin	ClockOrigin	Physical/logical source of the timestamp domain.
epoch	ClockEpoch	Epoch used by stamp_ns.
clock_id	str	Stable human-readable authority identifier.
publishes_ros_clock	bool	Whether this authority is projected onto ROS /clock for nodes using use_sim_time.
timestep_s	float | None	Optional logical simulation/control timestep in seconds. None means event-driven or externally determined.
notes	str | None	Optional operator-facing context.

Example

ClockAuthority.simulation("robocasa", timestep_s=0.05).epoch

hardware_synced(clock_id: str, *, epoch: ClockEpoch = ClockEpoch.UNIX) -> ClockAuthority classmethod

Return a hardware/PTP authority already synchronized to the graph.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def hardware_synced(
    cls,
    clock_id: str,
    *,
    epoch: ClockEpoch = ClockEpoch.UNIX,
) -> ClockAuthority:
    """Return a hardware/PTP authority already synchronized to the graph."""
    return cls(origin=ClockOrigin.HARDWARE_SYNCED, epoch=epoch, clock_id=clock_id)

host_wall(clock_id: str = 'host_wall') -> ClockAuthority classmethod

Return the default real-deployment authority: host wall/ROS system time.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def host_wall(cls, clock_id: str = "host_wall") -> ClockAuthority:
    """Return the default real-deployment authority: host wall/ROS system time."""
    return cls(origin=ClockOrigin.HOST_WALL, epoch=ClockEpoch.UNIX, clock_id=clock_id)

simulation(clock_id: str, *, timestep_s: float | None = None, publishes_ros_clock: bool = True) -> ClockAuthority classmethod

Return a simulator-owned elapsed-time authority.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def simulation(
    cls,
    clock_id: str,
    *,
    timestep_s: float | None = None,
    publishes_ros_clock: bool = True,
) -> ClockAuthority:
    """Return a simulator-owned elapsed-time authority."""
    return cls(
        origin=ClockOrigin.SIMULATION,
        epoch=ClockEpoch.SIMULATION_ELAPSED,
        clock_id=clock_id,
        publishes_ros_clock=publishes_ros_clock,
        timestep_s=timestep_s,
    )

ClockEpoch

Bases: str, Enum

Epoch that a stamp_ns value is measured from.

ClockOrigin

Bases: str, Enum

Authoritative source for OpenRAL stamp_ns values.

/clock is not an origin; it is the ROS 2 projection of one of these authorities into nodes that run with use_sim_time=true.

CollisionEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_COLLISION — a proposed motion would collide.

Attributes:

Name	Type	Description
kind	Literal['collision']	Discriminator (always "collision").
collision_kind	Literal['self', 'world']	"self" (link vs link) or "world" (link vs obstacle).
link_a	str	Robot link whose collision volume tripped the check.
link_b_or_object	str	The other robot link (self-collision) or the world object / occupancy region (world-collision).
horizon_step	int	Chunk step (horizon index) where the collision was first detected, or REACTIVE_HORIZON_STEP (-1) when the hit is a REACTIVE one — the robot's measured configuration already collides, so no prediction step is involved. The C++ safety kernel emits -1 for that check (check_config(q_meas, -1) in cpp/openral_safety_kernel/src/lifecycle_kernel.cpp); it is the common case for Cartesian control modes, whose reactive measured-state check is the guaranteed floor.
min_distance_m	float	Signed minimum distance at detection, in metres (negative means interpenetration).
joint_positions_rad	list[float]	The configuration the kernel ran forward kinematics on for horizon_step — the exact vector handed to forward_kinematics, in the envelope's dof order, with mobile-base dofs zeroed (the kernel checks the arm in base_link). This is what makes a stop adjudicatable: a predicted step's configuration exists in no other artifact, because it depends on the kernel's own damped-least-squares Jacobian, its lambda and its seed dt — so replaying such a stop against the measured joints reads geometry the kernel never checked. Empty only on a record emitted before the field existed: a kernel that reports a collision has a collision model by construction, and its FK buffer is always n_dof long.
world_grid_origin_m	tuple[float, float, float] | None	Base-frame position of voxel (0,0,0)'s minimum corner in the occupancy grid this check ran against, when the other party is a world voxel; None otherwise and on any producer that predates the field. The kernel names its cell only as voxel_<n>, an index into a grid it does not republish, so a consumer has to pair that index with a grid of its own — and the published window is snapped to the source lattice, so a base drift across one cell boundary shifts the whole window and the same index names a different cell. Matching on a stamp cannot separate two grids inside one clock tick (/clock quantises every stamp in the graph to the tick); matching on this origin is exact.

is_reactive: bool property

True when the hit came from the measured-state (reactive) check.

ComputeSpec

Bases: BaseModel

Compute profile for one deployment tier — runtime inference and GPU capabilities.

Used for all three deployment tiers:

• Edge — on-robot SoC (Jetson AGX Orin, NVIDIA Thor).
• Local — laptop / workstation tethered to the robot or running the simulation.
• Cloud — remote GPU node reachable via SSH or HTTP.

Attached to RobotDescription as compute_edge, compute_local, and compute_cloud. Consumed by rSkill.check_runtime and rSkill.check_quantization_dtype to match RSkillManifest.runtime / quantization.dtype against what the hardware can actually execute.

Attributes:

Name	Type	Description
compute_tops	float	Peak compute in INT8 TOPS (0 = unknown).
system_memory_gb	float	Total system / unified RAM in GB (0 = unknown).
num_gpus	int	Number of GPUs on this node (default 1; use > 1 for cloud multi-GPU pods — total VRAM budget = gpu_vram_gb * num_gpus).
gpu_vram_gb	float	VRAM of a single GPU in GB (0 when no discrete GPU).
cuda_compute_capability	tuple[int, int] | None	Highest CUDA compute capability (major, minor), e.g. (8, 9) for Ada Lovelace, (10, 0) for Blackwell. None on non-CUDA hosts.
cuda_toolkit_version	str | None	nvcc version string, e.g. "12.4". None when CUDA is absent or the version cannot be determined.
tensorrt_version	str | None	TensorRT runtime version string when importable.
gpu_supported_runtimes	list[RSkillRuntime]	Inference runtimes the hardware can execute, used by rSkill.check_capabilities to match RSkillManifest.runtime. Empty list = "unknown — skip check".
gpu_supported_dtypes	list[QuantizationDtype]	Quantization dtypes the accelerator supports (derived from CUDA compute capability or platform). Empty = "unknown — skip check".
nvmm_available	bool	Whether libnvbufsurface.so is present on this node (Tegra L4T multimedia stack). True enables the NVMM zero-copy sensor-ingest path; probed on all tiers and returns False gracefully when the library is absent.
endpoint	str | None	Remote access address for cloud / SSH nodes. Format: ssh://user@host[:port] or https://host:port. None for edge and local nodes.
network_latency_ms	float | None	Measured round-trip latency to this node in ms. Populated by the SSH probe; None for local/edge (no network hop). Used by the reasoner dispatcher to decide whether cloud dispatch fits within a skill's latency budget.

Example

spec = ComputeSpec( ... gpu_vram_gb=24.0, ... cuda_compute_capability=(8, 9), ... cuda_toolkit_version="13.2", ... ) spec.supports_cumotion() True

supports_cumotion() -> bool

Whether this compute spec meets the cuMotion (Isaac ROS) GPU floor.

cuMotion's CUDA motion planner requires an Ampere-or-newer GPU (compute capability >= 8.0), CUDA toolkit >= 13.0, and a nominal 8 GB card. The MoveIt planner gate uses this to choose the cuMotion planning pipeline; when False it falls back to OMPL.

Returns:

Type	Description
bool	True only when compute capability, CUDA toolkit version, and
bool	VRAM all clear the cuMotion floor. False on any non-CUDA host or
bool	when the CUDA toolkit version is unknown (the floor cannot be
bool	confirmed, so the gate stays closed).

Source code in python/core/src/openral_core/schemas.py

def supports_cumotion(self) -> bool:
    """Whether this compute spec meets the cuMotion (Isaac ROS) GPU floor.

    cuMotion's CUDA motion planner requires an Ampere-or-newer GPU (compute
    capability >= 8.0), CUDA toolkit >= 13.0, and a nominal 8 GB card. The
    MoveIt planner gate uses this to choose the cuMotion planning pipeline;
    when ``False`` it falls back to OMPL.

    Returns:
        ``True`` only when compute capability, CUDA toolkit version, and
        VRAM all clear the cuMotion floor. ``False`` on any non-CUDA host or
        when the CUDA toolkit version is unknown (the floor cannot be
        confirmed, so the gate stays closed).
    """
    if self.cuda_compute_capability is None:
        return False
    if self.cuda_compute_capability < _CUMOTION_MIN_COMPUTE_CAPABILITY:
        return False
    if self.cuda_toolkit_version is None:
        return False
    cuda_major = _cuda_major(self.cuda_toolkit_version)
    if cuda_major is None or cuda_major < _CUMOTION_MIN_CUDA_MAJOR:
        return False
    return self.gpu_vram_gb >= _CUMOTION_MIN_VRAM_GIB

ContactForceWitness

Bases: BaseModel

Bounded attestation of a MEASURED contact force on the declared target.

The second observable on the place path (ADR-0100, Path C): force is what distinguishes "set down" from "crush" once position clearance hits zero. The gate this feeds only ever adds a refusal — never widens an ADR-0097 allowance or moves a margin. magnitude_n is Newtons only when magnitude_calibrated (no published work validates MuJoCo contact force against real F/T measurements; FORGE arXiv:2408.04587 re-tunes on hardware); direction_in_object has no such caveat (arXiv:2602.14174). Absence of a witness is never evidence of no contact — contype/ conaffinity suppression can empty the solver's contact list under a payload demonstrably in contact.

Attributes:

Name	Type	Description
target_id	str	Surface the measured contact is against; the gate arms only when this names the live declaration's target or a surface of it.
direction_in_object	tuple[float, float, float]	Unit contact-force direction in the attached object's frame (not base frame — decorrelates as a mobile base drives).
magnitude_n	float	Contact force magnitude; Newtons only when magnitude_calibrated, else an ignored uncalibrated number.
magnitude_calibrated	bool	False (default, uncalibrated) means the force gate does not arm.
calibration_ref	str | None	Names the calibration magnitude_n was produced under; required when magnitude_calibrated.
confidence	float	Witness confidence in [0, 1].
evidence_kind	AttachmentEvidenceKind	Source that measured the force; the MuJoCo producer attests AttachmentEvidenceKind.SIM_CONTACT_FORCE.
evidence_ref	str | None	Optional trace / producer reference.
stamp_ns	int	Producer timestamp; with target_id keys the kernel's re-arm check.

Example

witness = ContactForceWitness( ... target_id="sim:cab_1_left_group_main", ... direction_in_object=(0.0, 0.0, 1.0), ... magnitude_n=4.5, ... magnitude_calibrated=False, ... confidence=1.0, ... evidence_kind=AttachmentEvidenceKind.SIM_CONTACT_FORCE, ... stamp_ns=1, ... ) witness.magnitude_calibrated False

fill_idl(msg: object) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS."""
    msg.target_id = self.target_id  # type: ignore[attr-defined]
    direction = msg.direction_in_object  # type: ignore[attr-defined]
    direction.x, direction.y, direction.z = self.direction_in_object
    msg.magnitude_n = float(self.magnitude_n)  # type: ignore[attr-defined]
    msg.magnitude_calibrated = bool(self.magnitude_calibrated)  # type: ignore[attr-defined]
    msg.calibration_ref = self.calibration_ref or ""  # type: ignore[attr-defined]
    msg.confidence = float(self.confidence)  # type: ignore[attr-defined]
    msg.evidence_kind = self.evidence_kind.value  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref or ""  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    direction = msg.direction_in_object  # type: ignore[attr-defined]
    return cls(
        target_id=str(msg.target_id),  # type: ignore[attr-defined]
        direction_in_object=(
            float(direction.x),
            float(direction.y),
            float(direction.z),
        ),
        magnitude_n=float(msg.magnitude_n),  # type: ignore[attr-defined]
        magnitude_calibrated=bool(msg.magnitude_calibrated),  # type: ignore[attr-defined]
        calibration_ref=str(msg.calibration_ref) or None,  # type: ignore[attr-defined]
        confidence=float(msg.confidence),  # type: ignore[attr-defined]
        evidence_kind=AttachmentEvidenceKind(str(msg.evidence_kind)),  # type: ignore[attr-defined]
        evidence_ref=str(msg.evidence_ref) or None,  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
    )

ControlMode

Bases: str, Enum

Action space / control interface exposed by a robot or skill.

ControlModeSemantics

Bases: BaseModel

Action-space semantics declared on each ActuatorRequirement.

Closes Gap 2 of the rSkill self-containment audit: kind alone ("joint_position") doesn't say whether the policy emits absolute targets or per-step deltas, in which joint order, in which reference frame, or how it encodes the gripper. Today these are assumed at the adapter level and silently broken when porting a skill between embodiments.

The block is REQUIRED on every actuator entry of a manifest. For the canonical embodiments the loader still auto-fills n_dof / vla_action_key from the robot YAML; semantics must be declared explicitly on the manifest because they are a property of the trained checkpoint, not the robot.

Attributes:

Name	Type	Description
mode	Literal['absolute', 'delta']	Whether the action vector represents absolute targets (joint positions, end-effector pose) or deltas from the current state. Required.
gripper_convention	GripperConvention | None	Encoding of the gripper slot in the action vector. Required when the parent ActuatorRequirement.kind is ControlMode.GRIPPER_BINARY or ControlMode.GRIPPER_POSITION; forbidden otherwise.
joint_order	list[str] | None	Ordered list of joint names matching the channels the policy emits. Optional for canonical embodiments (the robot YAML's ActionSpec.joint_names is the source of truth); REQUIRED for "custom" embodiments.
reference_frame	str | None	tf2 frame the action is expressed in. Required when the parent ActuatorRequirement.kind is one of ControlMode.CARTESIAN_POSE, ControlMode.CARTESIAN_DELTA, or ControlMode.CARTESIAN_TWIST; forbidden otherwise.

Example

sem = ControlModeSemantics(mode="absolute") sem.mode 'absolute' sem.gripper_convention is None and sem.reference_frame is None True

ControllerEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CONTROLLER — a ros2_control controller faulted.

Attributes:

Name	Type	Description
kind	Literal['controller']	Discriminator (always "controller").
controller_name	str	Failing controller (e.g. "joint_trajectory_controller").
state	str	Reported controller state (e.g. "inactive", "error").
detail	str	Free-form detail from the controller manager.

CriticEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CRITIC — a critic flagged an action below threshold.

Attributes:

Name	Type	Description
kind	Literal['critic']	Discriminator (always "critic").
critic_id	str	Identifier of the critic that fired (model id or hand-rolled).
score	float	Critic output in the critic's native range.
threshold	float	Configured pass threshold.

DeadlineOverrunPolicy

Bases: str, Enum

What the inference runner does when a tick exceeds the deadline.

warn logs + records an OTel attribute but still sends the action (mirrors lerobot's record loop). drop skips the action for this tick — used when stale actions are worse than no action (e.g. velocity control). raise is test-only: raises ROSDeadlineMissed.

DecomposeMissionTool

Bases: _ReasonerToolBase

Tool variant — write the reasoner's typed task queue.

The typed path for the decompose-mission playbook: the LLM emits an ordered list of finer subtasks and the node applies it to the deterministic MissionState, replacing the free-form-JSON gap with structured output (CLAUDE.md §3). Two modes, selected by target_task_id:

• populate (target_task_id empty) — build a fresh mission queue from subtasks, refining the single-task seed when the operator goal needs a finer decomposition than a single prompt.
• subdivide (target_task_id set) — flat-splice the named blocked task in place with subtasks (t2 → t2.1, t2.2), bounded by DEFAULT_MAX_SUBDIVIDE_DEPTH; past the bound the node hands off instead.

Each subtask is a GroundedSubtask (one specific object), not a free string — so a collective "first batch of objects" cannot be emitted. The node renders them to MissionState task text via rendered_subtasks.

Like every ReasonerToolCall variant it holds no authority over actuation — it only edits the S2 task ledger; a bad decomposition yields a worse plan the safety kernel still vetoes.

Attributes:

Name	Type	Description
tool	Literal['decompose_mission']	Discriminator (always "decompose_mission").
subtasks	list[GroundedSubtask]	Ordered, non-empty list of GroundedSubtask — each acts on exactly one specific object and becomes a TaskState.
target_task_id	str	TaskState.task_id of the blocked task to subdivide (e.g. "t2"); empty string populates/replaces the whole queue.

rendered_subtasks() -> list[str]

The ordered subtask instruction strings for MissionState.

Source code in python/core/src/openral_core/schemas.py

def rendered_subtasks(self) -> list[str]:
    """The ordered subtask instruction strings for ``MissionState``."""
    return [s.render() for s in self.subtasks]

DeployRuntime

Bases: BaseModel

Committed deploy-posture toggles for a workcell scene.

Every runtime leg openral deploy sim / deploy run can bring up is declared here so a committed DeployScene pins its posture explicitly instead of relying on per-invocation CLI flags. Precedence is CLI flag > scene runtime > auto — an explicit CLI flag always wins, and None means "auto" (the manifest-derived / built-in default the CLI documents for that flag: SLAM from capabilities.has_lidar/vision-SLAM, Nav2 tracks SLAM, octomap from a depth SensorSpec, detector on, kernel-check on, reward monitor / critic off).

Host-operational knobs (dashboard, foxglove, ports, dataset recording, --initial-task, --dry-run) stay CLI-only — they describe the invocation, not the workcell.

Relative *_manifest / object_detector_onnx paths that exist next to the scene YAML are resolved against the scene's directory; otherwise the value is passed through verbatim (repo-relative default / alias).

clock_origin: Literal['host_wall', 'simulation'] | None = None class-attribute instance-attribute

Pin the graph's clock authority instead of deriving it. None (auto) = host wall time for deploy run; for deploy sim, the simulator's clock whenever the backend exposes one (every bare MuJoCo twin does), unless octomap_cloud_topic names a real driver's cloud (outside /openral/cameras/), which selects host_wall.

A real ZED or RealSense driver stamps on wall-clock. Under the simulated clock octomap_server (use_sim_time) sits near t=0, treats every wall-stamped cloud as from the future and drops it, so the map and /openral/world_voxels stay empty while every node reports healthy.

simulation is refused for deploy run (a real robot has no sim clock), for a sim backend that exposes no clock, and together with a real driver's octomap_cloud_topic.

enable_scene_vlm: bool | None = None class-attribute instance-attribute

Bring up the scene-VLM query service so the reasoner is offered the read-only query_scene tool. None = off unless the CLI passes --enable-scene-vlm. Backward-compatible addition: a scene that omits it behaves exactly as before.

estimated_depth_camera: str | None = None class-attribute instance-attribute

An RGB camera (a manifest or scene sensor name) whose metric depth Depth Anything V3 estimates, so the world-collision map gets a cloud on a robot with no depth sensor (a top / context camera), in sim and on real hardware alike. The launch runs the depth provider on that camera with DA3METRIC-LARGE (its own sidecar, ZMQ port 5772): metric depth from the camera's own CameraInfo focal, published with that CameraInfo and back-projected to /openral/cameras/<name>/points, which becomes octomap_server's input (deploy_cloud_topic); the robot self filter always runs on it (DA3 sees the arm).

An ESTIMATE, not a measurement: it can misplace or miss surfaces (glass, low texture, far range), and it reaches the safety kernel's world voxels. The launch logs that at WARN. Any RGB camera with intrinsics qualifies (top, context or wrist) once it is tied to the robot by a declared mount: parent_frame + static_transform_xyz_rpy on it or on the sensor it shares_mount_with; in sim, an *_optical_frame camera the bridge places from the MJCF also does (openral_core.depth_extrinsic.resolve_estimated_depth_camera). A camera in world or on a bare robot link is refused. An estimate never replaces a measurement: the camera is REFUSED whenever the robot or the scene has any measured depth or point-cloud sensor (SensorSpec.is_cloud_source), in sim and on real hardware alike, with no override; the refusal names each such sensor and whether it is the robot's or the scene's. Map that sensor's cloud (octomap_cloud_topic) instead (Safety-WG, hazard HZ-0382-6). On deploy run that mount must be calibrated in units/<unit>.yaml like any cloud source's (depth_extrinsic_problems). Mutually exclusive with octomap_cloud_topic (which cloud is mapped must be one answer) and with slam_mono_camera naming the same camera (both would publish its depth topics). None = off. Backward-compatible addition.

grasp_allowance_enabled: bool = False class-attribute instance-attribute

Turn on the safety kernel's grasp-target exemption (its grasp_allowance_enabled): while a live GraspDeclaration carries a producer-measured region, world-voxel cells centred in that box do not trip the declared gripper contact links. The launch always passes the kernel its allowlist, grasp_contact_links = the manifest's role: gripper joints' child_links, so a declaration can never name a link the robot does not grip with. Default off, and off is the graph exactly as without it; turning it on is pending Safety-WG review (real pick-and-place design §2.1/§3, docs/reference/real-pick-place-adr-drafts.md). Backward-compatible addition. openral deploy run refuses it on unless vision_attachment.enabled and vision_attachment.grasp_target_enabled are on too: the kernel would arm with no producer measuring the region (deploy sim's MuJoCo evidence tracker measures it).

joint_states_topic: str | None = None class-attribute instance-attribute

Explicit override for the sensor_msgs/JointState topic the deploy runtime's Python nodes (in-process world state + the runner's joint-state cache) ingest. The C++ safety kernel always keeps the full-rate /joint_states.

None (auto) derives it: on deploy run of a robot whose real HAL is ros2_control-drivable (OpenArm, Franka, UR, Sawyer), the HAL node's own rate-limited /<hal_node>/joint_states republish; otherwise /joint_states. The broadcaster's full-rate stream (0.5-1 kHz) would wake the Python executor on every message and starve an in-process VLA of the GIL. See openral_hal.hal_joint_states_topic.

max_octree_age_s: float | None = Field(default=None, gt=0) class-attribute instance-attribute

How long the octomap bridge republishes the last octree it received. None = equal to world_voxel_deadline_s. Must not exceed it, so the kernel, not the bridge, fails closed on a silent camera: worst case from the last inserted cloud to the drop is this plus the deadline (hazard log Entry 033).

octomap_cloud_topic: str | None = None class-attribute instance-attribute

The PointCloud2 topic octomap_server consumes as cloud_in, i.e. what the world map is actually built from (deploy_cloud_topic). None under deploy sim = /openral/cameras/<name>/points of the manifest's one depth sensor with intrinsics, which the sim sensor bridge back-projects; a manifest with several must pin the one to map.

On real hardware nothing in-tree publishes a cloud, so deploy run refuses to start octomap (enabled explicitly or auto-enabled by a depth / point-cloud sensor) until this names what the driver already publishes: zed_wrapper emits /<name>/point_cloud/cloud_registered, RealSense /camera/depth/color/points. Set enable_octomap: false to run without the world map. A pin outside /openral/cameras/ on a deploy sim twin is a real driver, so the twin runs on host wall time (see clock_origin).

preload_prompt: str | None = None class-attribute instance-attribute

Prompt the preload warms the policy with. The runner keeps a policy resident per (rskill_id, revision) and passes each goal's own prompt to it per step, so a goal with a different prompt reuses the preloaded policy. For a single-instruction finetune use the training string, verbatim.

preload_rskill_id: str | None = None class-attribute instance-attribute

rSkill the skill_runner resolves and loads right after it activates, so the first execute_rskill goal finds it resident. The deadman watchdog opens its first-chunk window when a goal is accepted, so a multi-minute cold load of a large policy inside a goal is E-stopped — correctly. Loading before any goal exists keeps the watchdog exactly as strict. Goals are rejected while the preload is in flight (rskill_runner.preload_done marks the end). Any resolvable id: an installed Hub repo id or an in-tree name.

preload_rskill_revision: str | None = None class-attribute instance-attribute

Hub revision (branch, tag or commit) pinned for preload_rskill_id. Part of the resident key (rskill_id, revision, prompt): it must equal the revision later goals send, or they evict the preloaded skill. None = the unpinned default revision. Ignored without preload_rskill_id.

robot_self_filter_padding_m: float = Field(default=0.02, ge=0, json_schema_extra={'maximum': 0.1}) class-attribute instance-attribute

How far beyond the robot's collision primitives (and a held payload's) a real depth return still counts as the robot and is removed by openral_octomap_bridge's robot_self_filter before octomap inserts the cloud. Real camera path only; sim renders the robot transparent.

0.02 m is measured AT REST on the OpenArm Thor and Orin cells (2026-09-25, ZED-M, arms at q = 0): the robot's own depth returns end 5-7.5 mm outside the kernel's link hulls. The tail with the arms moving is unmeasured (scenes/deploy/openarm_real_world_voxels.yaml). Derive it per rig from the camera's depth noise at working range, the camera extrinsic error, how far a link moves between the cloud's capture and the joint state used to pose it, and half a voxel. It is also the width of the blind shell around the arm: an obstacle that close to the robot is removed with it, so a larger value hides more of the world from the kernel's check (hazard log Entry 035). Too small leaves robot surface in the map, which stops the robot against itself (fail closed).

Hard cap 0.10 m (2x the pre-2026-09-25 value), refused at load: a wider blind shell hides obstacles the arm can reach within one chunk. The self-filter node refuses it too (and a negative or non-finite value), forwarding nothing.

scene_vlm_manifest: str | None = None class-attribute instance-attribute

Path to the kind: "vlm" rSkill manifest backing query_scene. Scene-relative like the other manifest paths. None defaults to the in-tree rskills/qwen35-4b-nf4. Ignored unless enable_scene_vlm.

slam_depth_sidecar_autostart: bool = True class-attribute instance-attribute

Whether the deploy launch spawns the DA3 metric-depth sidecar (tools/da3_depth_sidecar.py, ZMQ port 5771) alongside the mono visual SLAM graph. Only meaningful when slam_mono_camera is set. False = the sidecar is operator-run / shared (e.g. one sidecar serving several deploys, or a dev venv via $OPENRAL_DA3_DEPTH_SIDECAR_VENV). First autostart provisions the sidecar venv, which can take minutes; the depth provider retries until it answers.

slam_mono_camera: str | None = None class-attribute instance-attribute

The single camera name the visual SLAM backend tracks in mono RGBD mode — one RGB camera fused with the DA3 metric-depth provider for scale, the lidar-less path for robots without a stereo rig. When set, the launch points PyCuVSLAM at /openral/cameras/<name>/image (+ /camera_info), auto-composes the depth provider (so cuVSLAM and nvblox get depth), and runs cuVSLAM's OdometryMode.RGBD. pycuvslam-only (the Isaac ROS impl has its own RGB-D wiring). Mutually exclusive with slam_stereo_cameras; None = stereo/multi-camera.

slam_stereo_cameras: tuple[str, str] | None = None class-attribute instance-attribute

The (left, right) camera names forming the rectified stereo rig the visual SLAM backend tracks, e.g. ["left", "right"] or ["front_left", "front_right"]. Each name maps by the OpenRAL convention to /openral/cameras/<name>/image (+ /camera_info); the launch forwards those topics to whichever visual impl is composed (Isaac ROS image_0/1_topic or PyCuVSLAM left/right_image_topic). None = the impl's built-in left/right default. The two names must differ.

slam_visual_impl: Literal['isaac_ros', 'pycuvslam'] | None = None class-attribute instance-attribute

Which cuVSLAM implementation the visual SLAM backend composes when slam_backend resolves to "visual" (capabilities.has_vision_slam, no lidar). Orthogonal to the backend choice: the backend is what sensing modality localises the robot (lidar vs camera), this is which engine binary the host has installed.

• "isaac_ros" — the composable isaac_ros_visual_slam C++ node (cuvslam.launch.py). NITROS zero-copy + raw-image undistortion + mono/IMU/RGB-D modes, but needs the operator's full Isaac ROS apt stack.
• "pycuvslam" — the same cuVSLAM engine run in-process from NVIDIA's PyCuVSLAM pip wheel (pycuvslam.launch.py); rectified stereo only, no apt stack. For sim hosts / GPU boxes without the Isaac ROS install.

None = "isaac_ros" (the established default — do not silently pick pycuvslam just because the wheel is present; §1.4). Ignored unless the resolved backend is "visual".

static_geometry_file: str | None = None class-attribute instance-attribute

Declared static geometry of the cell (StaticGeometry YAML; scene-relative, else repo-relative): walls, shelf frame, table, tote shell, in the robot's base_frame. The launch samples its surfaces and the octomap bridge marks every cell they pass through occupied in every published grid, whatever the camera has seen — so a fixture out of the head camera's view still stops an arm (docs/reference/ manipulation-workspace-contact.md §3.1). Only ADDS occupancy, the fail-closed direction, so it needs no measurement and is allowed on real hardware; the launch refuses it with the octomap leg off rather than drop declared walls. For a sim scene, tools/derive_scene_inputs.py exports it from the static colliders (an Isaac stage or a MuJoCo model). Backward-compatible addition.

vision_attachment: VisionAttachmentRuntime | None = None class-attribute instance-attribute

The vision attachment leg (segmenter + the HAL's attachment-evidence bridge). None or enabled: false = not launched, and the graph is exactly as without it. Enabled on deploy run, it always turns the kernel's attached check on too (see VisionAttachmentRuntime). Backward-compatible addition.

voxel_freshness_s: tuple[float, float] property

(world_voxel_deadline_s, max_octree_age_s) with the age defaulted.

Example

DeployRuntime().voxel_freshness_s (1.0, 1.0) DeployRuntime(world_voxel_deadline_s=2.0, max_octree_age_s=1.5).voxel_freshness_s (2.0, 1.5)

workspace_contact_enabled: bool = False class-attribute instance-attribute

Turn on the safety kernel's manipulation-workspace contact rule (issue #349, its workspace_contact_enabled; docs/reference/manipulation-workspace-contact.md): while a live GraspDeclaration carries a producer-confirmed workspace_region, world-voxel cells centred in that box do not trip the robot's hand links (the launch derives them from the manifest, RobotDescription.hand_links()) as long as the hand's speed is at most workspace_contact_max_speed_mps and its lead over the measured pose is at most workspace_contact_max_lead_m. Arm links, contact outside the workspace and a fast or far-leading hand stop as before. Requires grasp_allowance_enabled (the workspace rides the grasp declaration) and both bounds. Default off, and off is the graph exactly as without it. Sim-only until Safety-WG review: openral deploy run and deploy_e2e on a non-sim HAL refuse it on. Backward-compatible addition.

workspace_contact_max_lead_m: float | None = Field(default=None, gt=0.0, le=0.1) class-attribute instance-attribute

The hand lead (push) bound (m) of the workspace contact rule: the largest displacement of any point of a hand link's collision geometry between the MEASURED configuration and each checked row. On a joint-PD arm commanded-minus-measured is what the drives push with, so this bounds how hard a hand may press into what it touches — the speed bound alone would let a hand whose stationary target is buried deep keep pushing. Required when workspace_contact_enabled. Calibration point: see docs/reference/manipulation-workspace-contact.md §8 (the real-hardware value is a Safety-WG decision). Capped at 0.10 m (le=0.10), mirrored by the kernel's kMaxWorkspaceContactLeadM. Backward-compatible addition.

workspace_contact_max_speed_mps: float | None = Field(default=None, gt=0.0, le=1.0) class-attribute instance-attribute

The hand speed bound (m/s) of the workspace contact rule (impact energy): the largest displacement of any point of a hand link's collision geometry over one control period (action_spec.control_freq_hz) at the measured joint velocity (/joint_states) — impact energy is set by how fast the hand actually moves, and a commanded jump into an obstacle is the lead bound's job. Without a fresh measured velocity the kernel falls back to the speed between consecutive commanded rows (row 0 from the previously accepted command while fresh, else the measured state), which reads the hand faster. Required when workspace_contact_enabled — there is no default the kernel would silently use. Calibration point: see docs/reference/manipulation-workspace-contact.md §8. Capped at 1 m/s (le=1.0), beyond which the rule is not a contact-energy bound at all.

world_voxel_data_age_budget_s: float = Field(default=1.5, gt=0, json_schema_extra={'maximum': 3.0}) class-attribute instance-attribute

How old the sensor data behind a voxel grid may be when the safety kernel checks a chunk against it (its world_voxel_data_age_budget_ms), measured from the grid's source_stamp: the capture stamp of the newest cloud in the octree. Past it, or with no source stamp, the chunk drops as DROP_VOXEL_UNAVAILABLE (voxel_stale), not latched (hazard log Entry 034). The receipt deadline above cannot see pipeline latency; this can.

The default is the Thor ZED-M measurement (mock hardware, 2026-09-24): capture-to-kernel age p50 227 ms, p99 ~1.0 s, max 1.09 s, so 1.5 s clears the tail with headroom. Measure your own rig and declare its value: set below the rig's latency tail the robot stops (fail closed), never moves unsafely. It is independent of world_voxel_deadline_s / max_octree_age_s: those bound a silent source by receipt time, this bounds the world's age whatever the receipt time, so no ordering between them is required, and a smaller budget is only stricter.

Hard cap 3.0 s (2x the pre-2026-09-25 value), refused at load: a larger budget would certify chunks against a world seconds old. The kernel refuses it at configure too, and its own default is this 1.5 s: there is no "off" while the world check is enabled.

world_voxel_deadline_s: float = Field(default=1.0, gt=0, json_schema_extra={'maximum': 2.0}) class-attribute instance-attribute

How long the safety kernel trusts the last /openral/world_voxels grid (its world_voxel_deadline_ms); past it every chunk drops with DROP_VOXEL_UNAVAILABLE. It must exceed the octomap source's insert-to-publish gap on this rig (cloud period plus octomap's insert time), with margin: 1.0 s tolerates octomap down to about 1 Hz. A slower source (a 1 Hz scanning lidar, a CPU-only host inserting dense clouds) needs a longer deadline or the robot never moves; that is fail-closed, not unsafe.

Hard cap 2.0 s (2x the pre-2026-09-25 value), refused at load: past it the kernel would keep checking chunks against a map that old. A source that cannot keep octomap under it is not a world-voxel source. The kernel and the octomap bridge (max_octree_age_s) refuse it themselves too, so a node launched without this schema cannot run looser.

DeployScene

Bases: BaseModel

Unified deploy/workcell scene for openral deploy sim and deploy run.

Carries the physical/logical workcell: scene identity, optional robot mount, sim composition, and deploy-time safety/collision tightening. No task, no eval config — the reasoner/operator supplies goals at runtime.

composition lets a deploy scene declare the MJCF composer that builds its environment (e.g. the openarm tabletop arena: table + cubes + drawer + overview camera) instead of the robot manifest carrying it: the robot manifest describes the robot, the scene describes the scene. openral deploy sim threads it to the manifest-driven HAL node. openral deploy run ignores it because real-world geometry is physical, not MJCF.

safety tightens the robot manifest's SafetyEnvelope; loosening is rejected before launch. extra_allowed_collision_pairs is the only sanctioned collision loosening: additive per-pair ACM entries, never a global self-collision disable.

drivers: list[LaunchInclude] = Field(default_factory=list) class-attribute instance-attribute

Vendor sensor drivers this workcell needs on the bus.

A deploy_binding with a ros2_* backend subscribes to a topic somebody else publishes; these are the launches that publish them. deploy run includes each one on the real path (hal_mode:=real) before the sensor leg starts reading, so a camera reached over ROS comes up with the graph instead of needing a second terminal — the failure mode otherwise is a silently empty panel, not an error.

Empty for a workcell whose sensors OpenRAL opens directly (opencv_thread, gstreamer): those need no driver. Ignored on the sim path, where cameras are rendered rather than driven.

grasp_declaration: GraspDeclaration | None = None class-attribute instance-attribute

Committed grasp-phase declaration for direct dispatch (real pick-and-place design §2.1), the grasp mirror of place_declaration.

Names the grasp target and the gripper contact_links; openral deploy sim / deploy run inject it into the rSkill runner, which scopes it to each goal it dispatches (armed on start, retracted on end / cancel / E-stop) unless the goal carries its own. None means no declaration and no exemption. Same rule as the place one: a scene names a target, never a region — grasp_declaration.region is rejected by _reject_scene_supplied_place_region.

Direct-dispatch path only (reasoner off, attended): what to pick is task knowledge, so a committed cell scene carries none. The generic path is reasoner-grounded — the LLM names an ExecuteRskillTool.grasp_target, the reasoner grounds its search_box from perception, and the goal carries the declaration (design §2.2).

hal: HalParameters | None = None class-attribute instance-attribute

Deploy-time HAL binding for this workcell.

The scene's host-specific HAL construction defaults — serial port, lerobot calibration identity (id + calibration_dir), calibrate_on_connect — the same shape as the robot manifest's HalEntrypoints.parameters. This is the HAL analogue of the per-sensor SensorSpec.deploy_binding: the robot manifest stays authoritative for the HAL adapter + generic defaults, while the committed workcell scene carries the host-specific transport + calibration so openral deploy run --config <scene> is self-contained (no --hal overrides needed). deploy run resolves a relative calibration_dir against this scene file's directory (mirroring the --hal behaviour) and merges these defaults into the HAL params above the robot-manifest defaults but below any explicit --hal override, so the precedence is --hal > scene hal > robot.yaml hal.parameters.defaults. None = fall back to the manifest defaults + --hal overrides.

memory_dir: str | None = None class-attribute instance-attribute

Path to a per-robot deploy memory bundle directory holding any of MEMORY.md (self-maintained semantic memory), scene_graph.json (3D world-state graph → recall_object), and map.yaml (2D occupancy grid → nav2 map_server). openral deploy sim derives the three launch paths from it by convention (each artifact loaded by its correct consumer). --memory-dir on the CLI overrides this. None = no bundle (the reasoner starts with empty memory). Advisory only — never a safety-kernel input (§1.1).

place_declaration: PlaceDeclaration | None = None class-attribute instance-attribute

Committed place-phase declaration for direct dispatch (ADR-0097).

A reasoner grounds the place target per goal and puts the declaration on the ExecuteRskill goal, which wins over this. A direct dispatch has no reasoner in the loop, and the scene is the only place that knows the task's place target — so a counter→cabinet scene declares it here, and openral deploy sim / deploy run inject it into the rSkill runner, which scopes it to each goal it dispatches (armed on start, retracted on end / cancel / E-stop).

None means no declaration, so no place witness can ever arm and payload contact mid-carry stops the robot exactly as it does now. The declaration licenses nothing on its own: it only makes measured support contact on the declared target attestable.

A scene declares a target, never a region: a scene file cannot measure geometry, and PlaceRegion.frame_id is the robot base frame, which on a mobile base moves under any box a file could name. The producer that resolves the target measures it (in sim, from the declared body's MuJoCo subtree), so place_declaration.region is rejected here rather than carried to the kernel — see _reject_scene_supplied_place_region.

robot_unit: str | None = None class-attribute instance-attribute

The robots/<robot_id>/units/<unit>.yaml overlay this cell runs: per-host sensor bindings and per-unit calibration (SensorOverlay). $OPENRAL_ROBOT_UNIT overrides it, so one scene serves several cells. None = none; a real deploy of a robot that ships units/ then refuses (resolve_sensor_overlays).

runtime: DeployRuntime | None = None class-attribute instance-attribute

Committed deploy-posture toggles (see DeployRuntime). None = every leg on its CLI/auto default. CLI flags override field-by-field.

seed: int = 0 class-attribute instance-attribute

Simulator reset seed for deploy sim. Ignored by deploy run. Defaults to 0, preserving the pre-field deploy-sim episode.

sensors: list[SensorSpec] = Field(default_factory=list) class-attribute instance-attribute

Workcell-mounted cameras (overhead / front) — physically part of the cell, not the robot.

A deploy scene never touches a sensor the robot manifest defines: every entry must use a name the manifest does not (check_scene_sensor_overrides refuses a clash), and carries its own geometry and binding. A robot camera's real-hardware SensorSpec.deploy_binding lives in its unit overlay (robot_unit) or robots/<robot_id>/robot.yaml. merge_deploy_sensors appends these entries after the manifest's sensors.

Entries whose deploy_binding is set are opened by the real-deploy sensor leg and published on /openral/cameras/<name>/image.

from_yaml(path: str) -> Self classmethod

Load and validate a scene YAML from disk.

Inherited by SimScene and BenchmarkScene, which validate against their own (stricter) schemas via cls.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> Self:
    """Load and validate a scene YAML from disk.

    Inherited by ``SimScene`` and ``BenchmarkScene``, which
    validate against their own (stricter) schemas via ``cls``.
    """
    return _load_yaml_model(cls, path)

DetectedObject

Bases: BaseModel

A detected object in the scene.

Attributes:

Name	Type	Description
label	str	Semantic class label.
confidence	float	Detection confidence in [0, 1].
pose	Pose6D	6D pose in the world or camera frame.
bbox_3d	tuple[float, float, float, float, float, float] | None	3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
track_id	int | None	Persistent track ID across frames.

DetectionTracker2D(*, iou_threshold: float = 0.3, max_misses: int = 3)

Stateful 2D-IoU tracker assigning stable det_ids.

Parameters:

Name	Type	Description	Default
iou_threshold	float	Minimum 2D AABB IoU (same label) to associate a detection with an existing track.	0.3
max_misses	int	Retire a track after this many consecutive frames without a match (detectors flicker, so a few frames of grace).	3

Example

trk = DetectionTracker2D() a = [ObjectDetection2D(label="milk", confidence=0.9, bbox_xyxy=(10, 10, 50, 50))] trk.assign(a)[0].det_id 0 b = [ObjectDetection2D(label="milk", confidence=0.9, bbox_xyxy=(12, 11, 52, 51))] trk.assign(b)[0].det_id # same object next frame keeps its id 0

Validate the IoU threshold and miss budget.

Source code in python/core/src/openral_core/detection_tracker.py

def __init__(self, *, iou_threshold: float = 0.3, max_misses: int = 3) -> None:
    """Validate the IoU threshold and miss budget."""
    if not 0.0 <= iou_threshold <= 1.0:
        raise ValueError(f"iou_threshold must be in [0,1]; got {iou_threshold}")
    if max_misses < 1:
        raise ValueError(f"max_misses must be >= 1; got {max_misses}")
    self._iou = iou_threshold
    self._max_misses = max_misses
    self._tracks: list[_Track2D] = []
    self._next_id = 0

assign(detections: list[ObjectDetection2D]) -> list[ObjectDetection2D]

Stamp each detection with a stable det_id and age unmatched tracks.

Greedy highest-confidence-first same-label IoU association: a matched detection keeps its track's id (and refreshes the track's box); an unmatched detection opens a new track with a fresh monotonic id. Tracks not matched this frame accrue a miss and are retired past max_misses.

Parameters:

Name	Type	Description	Default
detections	list[ObjectDetection2D]	This frame's 2D detections (det_id ignored on input).	required

Returns:

Type	Description
list[ObjectDetection2D]	The same detections, in input order, each with det_id assigned.

Source code in python/core/src/openral_core/detection_tracker.py

def assign(self, detections: list[ObjectDetection2D]) -> list[ObjectDetection2D]:
    """Stamp each detection with a stable ``det_id`` and age unmatched tracks.

    Greedy highest-confidence-first same-label IoU association: a matched
    detection keeps its track's id (and refreshes the track's box); an
    unmatched detection opens a new track with a fresh monotonic id. Tracks
    not matched this frame accrue a miss and are retired past ``max_misses``.

    Args:
        detections: This frame's 2D detections (``det_id`` ignored on input).

    Returns:
        The same detections, in input order, each with ``det_id`` assigned.
    """
    matched_tracks: set[int] = set()
    # Greedy by confidence so the strongest detection claims a track first.
    order = sorted(range(len(detections)), key=lambda i: detections[i].confidence, reverse=True)
    assigned: dict[int, int] = {}  # detection index -> det_id
    for di in order:
        det = detections[di]
        best_t, best_iou = -1, self._iou
        for ti, tr in enumerate(self._tracks):
            if ti in matched_tracks or tr.label != det.label:
                continue
            iou = aabb_iou_2d(det.bbox_xyxy, tr.bbox_xyxy)
            if iou >= best_iou:
                best_iou, best_t = iou, ti
        if best_t >= 0:
            tr = self._tracks[best_t]
            tr.bbox_xyxy = det.bbox_xyxy
            tr.miss_count = 0
            matched_tracks.add(best_t)
            assigned[di] = tr.det_id
        else:
            self._tracks.append(
                _Track2D(
                    bbox_xyxy=det.bbox_xyxy,
                    label=det.label,
                    det_id=self._next_id,
                    miss_count=0,
                ),
            )
            matched_tracks.add(len(self._tracks) - 1)
            assigned[di] = self._next_id
            self._next_id += 1

    survivors: list[_Track2D] = []
    for ti, tr in enumerate(self._tracks):
        if ti in matched_tracks:
            survivors.append(tr)
            continue
        tr.miss_count += 1
        if tr.miss_count < self._max_misses:
            survivors.append(tr)
    self._tracks = survivors

    return [det.model_copy(update={"det_id": assigned[i]}) for i, det in enumerate(detections)]

DetectorContract

Bases: BaseModel

Manifest contract for kind: "detector" rSkills.

Carries the configuration the runtime ObjectsDetector needs to instantiate an exported detection model (RT-DETR / D-FINE ONNX) and match its output indices to semantic class labels.

Required when RSkillManifest.kind is "detector"; forbidden for all other kinds (enforced by RSkillManifest._check_kind_consistency).

Attributes:

Name	Type	Description
labels	list[str]	Ordered class-label list; the integer class-id the model emits is used as an index into this list. At least one label is required.
input_size	tuple[int, int]	Model input resolution as (width, height) in pixels. Both dimensions must be > 0. Default (640, 640) matches the RT-DETR / D-FINE default export resolution.
score_threshold	float	Detections with confidence below this value are discarded before publishing. Must be in [0.0, 1.0]. Default 0.5.
engine	DetectorEngine | None	Optional explicit backend selector (DetectorEngine). None (default) keeps the legacy runtime-keyed dispatch. Set it to disambiguate backends that share a runtime — e.g. zeroshot_hf for an in-process Transformers open-vocabulary detector, which is runtime: pytorch like the VLM sidecar.
mode	DetectorMode	Invocation mode (DetectorMode; default continuous). Declares whether the detector is an always-on background producer (output reaches the reasoner via world state) or an on-demand prompted locator (surfaces the locate_in_view tool).

Example

c = DetectorContract( ... labels=["person", "bicycle", "car"], ... input_size=(640, 640), ... score_threshold=0.4, ... ) c.labels[0] 'person' c.mode

DetectorEngine

Bases: str, Enum

Backend that executes a kind: "detector" rSkill.

Selects which runtime detector class build_manifest_detector constructs. None (the default on DetectorContract) preserves the legacy runtime-keyed dispatch: runtime: onnx/tensorrt → RT-DETR ONNX, runtime: pytorch → the LocateAnything VLM sidecar. Set it explicitly to opt into a backend that the runtime value alone cannot disambiguate (e.g. an in-process Transformers open-vocabulary detector, which is also runtime: pytorch).

Attributes:

Name	Type	Description
RTDETR_ONNX		Fixed-label RT-DETR / D-FINE ONNX export (CPU / NVMM tiers). Equivalent to leaving engine unset with an onnx/tensorrt runtime.
VLM_SIDECAR		Out-of-process open-vocabulary visual-grounding VLM (LocateAnything-3B). Equivalent to leaving engine unset with a pytorch runtime. Query-driven (prompted).
ZEROSHOT_HF		In-process Transformers open-vocabulary detector (AutoModelForZeroShotObjectDetection — e.g. OmDet-Turbo). Runs against a fixed class vocabulary (the manifest labels) every frame, so it behaves like a large closed-vocabulary detector that needs no prompting — an unprompted background producer that populates the world object list with far more than the 80 COCO classes.

Example

DetectorEngine.ZEROSHOT_HF.value 'zeroshot_hf'

DetectorMode

Bases: str, Enum

Invocation mode of a kind: "detector" rSkill.

The axis orthogonal to DetectorEngine: where engine says how the model runs, mode says when the reasoner invokes it and therefore how its output reaches the LLM.

Attributes:

Name	Type	Description
CONTINUOUS		An always-on background producer. Runs on the camera tee every frame and streams ObjectsMetadata into WorldState.detected_objects; the reasoner reads it passively (via world state / recall_object) and never prompts it. It is not an ExecuteRskill-dispatchable tool and carries no actuation authority. RT-DETR (closed vocab) and OmDet-Turbo (frozen open vocab) are continuous. The reasoner may still toggle it via LifecycleTransitionTool to free VRAM.
ON_DEMAND		A prompted locator the reasoner invokes only when it needs to find a specific object right now. Surfaces the read-only locate_in_view tool backed by an open-vocabulary detector; it is not run continuously. LocateAnything is on-demand.

The two modes cleanly separate "open-vocabulary" from "prompting": continuous detectors cover a fixed bank of classes the reasoner reads for free; the on-demand locator handles the long tail that bank does not cover.

Example

DetectorMode.CONTINUOUS.value 'continuous' DetectorMode.ON_DEMAND.value 'on_demand'

DeviceInfo

Bases: BaseModel

Snapshot of the host compute capabilities used for runtime selection.

Attributes:

Name	Type	Description
device_str	str	PyTorch-style device string ("cpu", "cuda:0", "mps").
gpu_memory_bytes	int	Total GPU VRAM in bytes (0 if CPU-only).
cuda_compute_capability	tuple[int, int] | None	CUDA compute capability major/minor pair.
cpu_count	int	Logical CPU count.
arch	str	CPU architecture string.

Example

info = DeviceInfo(device_str="cpu") info.gpu_memory_bytes 0

EmbodimentExtra

Bases: BaseModel

Sensor + actuator surface for a "custom" embodiment.

Required when "custom" appears in RSkillManifest.embodiment_tags; forbidden otherwise. This is the explicit "I know what I'm doing" hatch for skill manifests targeting embodiments that do not have a canonical robots/<id>/robot.yaml in tree.

Reuses SensorRequirement and ActuatorRequirement on the assumption that the natural shape of "what this embodiment offers" mirrors "what the skill needs" — the loader just runs the same compat check it would run against a real robot YAML.

Attributes:

Name	Type	Description
sensors	list[SensorRequirement]	Sensor slots this custom embodiment exposes (≥1).
actuators	list[ActuatorRequirement]	Actuator slots this custom embodiment exposes (≥1).

Example

from openral_core import SensorModality, ControlMode EmbodimentExtra( ... sensors=[SensorRequirement(modality=SensorModality.RGB)], ... actuators=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... n_dof=6, ... vla_action_key="action.joints.arm", ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) ... ], ... ) # doctest: +ELLIPSIS EmbodimentExtra(sensors=[...], actuators=[...])

EmbodimentKind

Bases: str, Enum

Top-level kinematic class of a robot body.

EmitPromptTool

Bases: _ReasonerToolBase

Tool variant — republish a PromptStamped onto another topic.

Dispatch: publish on target_topic (typically /openral/prompt for self-cascading, but any PromptStamped topic is valid). Lets the reasoner stage multi-step plans, talk to a peer reasoner, or feed a downstream prompt-aware skill without going through an ExecuteRskill goal.

Attributes:

Name	Type	Description
tool	Literal['emit_prompt']	Discriminator (always "emit_prompt").
target_topic	str	Absolute ROS topic name (must start with "/"). The reasoner's own subscription on /openral/prompt plus the prompt-router's FIFO queue handle the cascade.
text	str	Human-readable prompt body forwarded as PromptStamped.text.
metadata_json	str	Free-form JSON forwarded as PromptStamped.metadata_json. Empty string when no structured metadata is needed.

EndEffectorSpec

Bases: BaseModel

End-effector specification.

Attributes:

Name	Type	Description
name	str	End-effector name.
kind	Literal['parallel_gripper', 'suction', 'dexterous_hand', 'tool', 'none']	Type of end-effector.
hand	Hand	Laterality.
n_dof	int	Number of controllable degrees of freedom.
max_grip_force_n	float | None	Maximum grip force in Newtons.
max_payload_kg	float | None	Maximum payload in kg.
workspace_radius_m	float | None	Reach radius in meters.
tactile_sensors	list[str]	List of tactile SensorSpec names attached to this EE.
actuated	bool	Whether the end-effector is driven by an actuator. False for passive tools (inert flanges, magnetic plates without electromagnet, kinematic-only mounts). When False, the safety kernel rejects any chunk addressed at this EE — the chunk routes to a no-op rather than risking unintended motion. Default True: every actuated gripper / dexterous hand / suction cup we ship is driven.
command_convention	GripperConvention | None	Encoding of the value a GRIPPER_* action addressed at this end effector carries when it reaches the HAL (see GripperConvention). An rSkill whose gripper convention differs is refused at load time. Every actuated parallel_gripper in robots/ declares it.
command_range	tuple[float, float] | None	Inclusive (min, max) the safety kernel bounds every GRIPPER_* value addressed at this end effector by, in command_convention's unit. Defaults to the convention's own range for the normalized conventions; REQUIRED for the physical ones (raw_joint_rad, width_meters).

Example

ee = EndEffectorSpec( ... name="left_gripper", ... kind="parallel_gripper", ... command_convention="raw_joint_rad", ... command_range=(0.0, 0.7854), ... ) ee.resolved_command_range() (0.0, 0.7854) EndEffectorSpec( ... name="g", kind="parallel_gripper", command_convention="normalized_open_unit" ... ).resolved_command_range() (0.0, 1.0)

resolved_command_range() -> tuple[float, float] | None

The kernel's bound for this end effector, or None if undeclared.

Source code in python/core/src/openral_core/schemas.py

def resolved_command_range(self) -> tuple[float, float] | None:
    """The kernel's bound for this end effector, or ``None`` if undeclared."""
    if self.command_range is not None:
        return self.command_range
    if self.command_convention is None:
        return None
    return GRIPPER_CONVENTION_RANGES[self.command_convention]

ExecuteRskillTool

Bases: _ReasonerToolBase

Tool variant — invoke an installed, capability-matched rSkill.

Dispatch: action goal on /openral/execute_rskill (the openral_msgs/action/ExecuteRskill action server in F1's rskill_runner_node). The chunk path that follows is Skill → /openral/candidate_action → safety_node → /openral/safe_action → HAL.

Attributes:

Name	Type	Description
tool	Literal['execute_rskill']	Discriminator (always "execute_rskill").
rskill_id	str	RSkillManifest.name of an installed, capable skill. Validated against the local registry by the reasoner at palette-build time; an unknown id raises ROSReasonerInvalidPlan.
prompt	str	Natural-language prompt forwarded to the skill's ExecuteRskill goal alongside the rskill_id. For VLA skills this is the policy's task-conditioning signal (SmolVLA writes it into observation["task"]); for wrapped-ROS skills it's carried for trace / log context but the actual goal is built from goal_params_json merged over the manifest's default_goal_json.
goal_params_json	str	Serialised JSON object carrying per-skill typed parameters the LLM produces against the skill's RSkillManifest.goal_params_schema. Empty string disables the merge (today's behaviour). Wrapped-ROS skills (kind: ros_action / ros_service) deep-merge it over ros_integration.default_goal_json at configure-time; VLA skills accept the field and ignore it (their prompt is already the structured signal).
deadline_s	float	Hard deadline in seconds for the action server to complete the goal. 0 means "use the skill manifest's default latency budget".
patience_s	float | None	Task-adaptive execution ceiling override (short for a quick grab, long for a precise insertion). None → use the reward model's default_patience_s.
progress_tolerance	float | None	Override the reward model's plateau_tolerance when a task misbehaves (e.g. a noisy critic). None → use the model default.
grasp_target	GraspTargetRef | None	Optional pin on the object this goal grasps (GraspTargetRef); the reasoner grounds it to a GraspDeclaration search box at dispatch, or refuses the goal. None (the normal case: the policy picks the object) = no named target; with the HAL's approach-armed grasp target on, the runner's goal-scope declaration is measured where the gripper approaches.
place_target	PlaceTargetRef | None	Optional hint naming the surface this goal places onto (PlaceTargetRef); the reasoner grounds it to a PlaceDeclaration search box at dispatch, or refuses the goal. None (the normal case) = no dispatch place declaration; the real producer still measures the surface under the carried payload.

FixedAttachment

Bases: BaseModel

A rigid, zero-DoF parent→child link attachment in the kinematic tree.

RobotDescription.joints enumerates only the robot's movable joints — consumed unfiltered as the action-vector width, the sim qpos map, the reasoner's rSkill state-contract filter and the runner's joint permutation, so a zero-DoF entry there would corrupt all four. But a real robot's collision tree also contains rigid mounts (a Franka hand bolted to the flange, a bimanual rig's two arm pedestals), and without them the tree is a forest.

This list is that missing connectivity and nothing else: no limits, no sensors, no DoF, never a command channel, invisible to every joints consumer. The safety envelope loader unions it with joints to build the collision tree and refuses to load a robot whose collision links do not form one connected tree (openral_safety.envelope_loader.collision_params_from_description).

Every value must come from the robot's real URDF/MJCF at the zero configuration — the same source joints' own origin_xyz / origin_rpy come from. Never hand-estimate a mount transform.

Attributes:

Name	Type	Description
name	str	Attachment name, matching the source URDF <joint> name where one exists (a composed multi-hop mount is named for its endpoint).
parent_link	str	The link this attachment hangs from. Must be reachable from the tree's single root.
child_link	str	The rigidly attached link.
origin_xyz	tuple[float, float, float]	Translation (metres) of child_link's frame in parent_link's frame. When the source model inserts intermediate links between the two, this is the composed transform, exactly as openral_safety.urdf_lowering.lower_joint_fk computes it for movable joints.
origin_rpy	tuple[float, float, float]	Orientation (roll, pitch, yaw) in radians, same frame convention as origin_xyz.

Example

a = FixedAttachment( ... name="panda_hand_joint", ... parent_link="panda_link7", ... child_link="panda_hand", ... origin_xyz=(0.0, 0.0, 0.107), ... origin_rpy=(0.0, 0.0, -0.7853981633974483), ... ) a.child_link 'panda_hand'

ForceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_FORCE — measured force exceeded a safety limit.

Attributes:

Name	Type	Description
kind	Literal['force']	Discriminator (always "force").
joint_or_ee	str	Name of the joint or end-effector that tripped the limit.
measured_n	float	Measured force in newtons.
limit_n	float	Configured ceiling in newtons.

FrameEncoding

Bases: str, Enum

How the bytes inside a SensorFrame are interpreted.

The first four values are raw per-pixel layouts. JPEG / PNG are compressed forms; the runner decodes lazily. CUDA_NV12 / CUDA_RGBA and RAW mark frames whose payload is an opaque handle (NVMM pointer, DMA-BUF fd) — the data field is empty and the consumer must read via handle. CUDA_NV12 is the Tegra NVMM default; CUDA_RGBA is the x86 DeepStream NVMM layout.

GraspDeclaration

Bases: BaseModel

Dispatch's typed statement that a grasp phase is active for a target.

Field-for-field the grasp mirror of PlaceDeclaration (real pick-and-place design §2.1; ADR draft "Declaration-scoped grasp-target contact for gripper finger links" in docs/reference/real-pick-place-adr-drafts.md): dispatch names the target and the gripper contact links, the attachment evidence producer measures the target's oriented region, and the safety kernel bounds the exemption. On its own it exempts nothing — dispatch never supplies a region (the rSkill runner strips one) and region is None means no exemption. Scoped to one goal execution; dies on retraction, goal end/cancel, E-stop, or timeout_s after stamp_ns.

The caps MAX_TIMEOUT_S / MAX_HALF_EXTENT_M / MAX_VOLUME_M3 are Safety-WG placeholders from the ADR draft, not ratified bounds; they sit far below PlaceRegion's own caps because a grasp region names one graspable object, not a receptacle. region.geometry must be empty in v1.

Attributes:

Name	Type	Description
target_id	str	Identity of the declared grasp target, e.g. "cell:pick_box". Required while active.
object_id	str	Payload identity the grasp produces once attached (AttachedCollisionObject.object_id); empty = discovered at attach time.
contact_links	tuple[str, ...]	Gripper links the declaration names, e.g. ("openarm_left_finger_pair",). Non-empty while active. A consumer intersects these with its own launch-derived allowlist.
rskill_id	str	Dispatching skill, for attributability (HZ-0115-2).
trace_id	str	OTel trace id, for attributability (HZ-0115-2).
timeout_s	float	Backstop expiry window in seconds after stamp_ns; capped at MAX_TIMEOUT_S.
stamp_ns	int	Dispatcher timestamp; with active this is the whole liveness key.
active	bool	False retracts the declaration.
region	PlaceRegion | None	Producer-measured oriented box around the target; None = no exemption. Dies with the declaration.
search_box	PlaceRegion | None	Optional box (in the voxel grid's base frame, or a fixed tf2 frame such as odom / map that the producer re-expresses in the grid frame every tick, so a scene's box stays put while a mobile base moves) the target producer searches the occupancy map in for a seed: a search hint only, with no support semantics — the producer measures the support layer from the voxel map itself (the box is padded so that layer lies inside it). It only seeds perception and never arms anything: the exemption is always the measured region. The reasoner grounds it from a named GraspTargetRef (the generic path); unlike region a direct-dispatch scene may also supply it; dispatch passes it through unchanged. None = no search, so the producer never measures a region.
workspace_box	PlaceRegion | None	Optional manipulation workspace (issue #349, docs/reference/manipulation-workspace-contact.md) in the voxel grid's base frame, or a fixed tf2 frame like search_box (the confirmed workspace_region is always in the grid's frame) — the box the task works in (the source tote's interior for a pick). Like search_box a scene may declare it and dispatch passes it through unchanged; it arms nothing on its own. Capped by MAX_WORKSPACE_HALF_EXTENT_M / MAX_WORKSPACE_VOLUME_M3; geometry must be empty. None = no workspace.
workspace_region	PlaceRegion | None	The producer-confirmed workspace: workspace_box itself (never grown), stamped with the source_stamp of the fresh voxel grid that confirmed it holds occupied cells. Producer-supplied only, exactly like region — a scene can never supply it (DeployScene._reject_scene_supplied_place_region) and the rSkill runner strips it. With the kernel's workspace_contact_enabled the robot's hand links may touch occupied cells centred inside it while the hand is slow. None = no workspace contact.

Example

declaration = GraspDeclaration( ... target_id="cell:pick_box", ... contact_links=("openarm_left_finger_pair",), ... timeout_s=70.0, ... stamp_ns=1_000_000_000, ... ) declaration.is_live(now_ns=31_000_000_000) True declaration.is_live(now_ns=72_000_000_000) False

fill_idl(msg: object) -> None

Populate a duck-typed openral_msgs/GraspDeclaration without importing ROS.

A grasp region never carries geometry (v1), so no primitive factory is needed.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate a duck-typed ``openral_msgs/GraspDeclaration`` without importing ROS.

    A grasp region never carries geometry (v1), so no primitive factory is
    needed.
    """
    msg.target_id = self.target_id  # type: ignore[attr-defined]
    msg.object_id = self.object_id  # type: ignore[attr-defined]
    msg.contact_links = list(self.contact_links)  # type: ignore[attr-defined]
    msg.rskill_id = self.rskill_id  # type: ignore[attr-defined]
    msg.trace_id = self.trace_id  # type: ignore[attr-defined]
    msg.timeout_s = float(self.timeout_s)  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.active = bool(self.active)  # type: ignore[attr-defined]
    msg.region_valid = self.region is not None  # type: ignore[attr-defined]
    if self.region is not None:
        self.region.fill_idl(msg.region)  # type: ignore[attr-defined]
    msg.search_box_valid = self.search_box is not None  # type: ignore[attr-defined]
    if self.search_box is not None:
        self.search_box.fill_idl(msg.search_box)  # type: ignore[attr-defined]
    msg.workspace_box_valid = self.workspace_box is not None  # type: ignore[attr-defined]
    if self.workspace_box is not None:
        self.workspace_box.fill_idl(msg.workspace_box)  # type: ignore[attr-defined]
    msg.workspace_region_valid = self.workspace_region is not None  # type: ignore[attr-defined]
    if self.workspace_region is not None:
        self.workspace_region.fill_idl(msg.workspace_region)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    return cls(
        target_id=str(msg.target_id),  # type: ignore[attr-defined]
        object_id=str(msg.object_id),  # type: ignore[attr-defined]
        contact_links=tuple(str(link) for link in msg.contact_links),  # type: ignore[attr-defined]
        rskill_id=str(msg.rskill_id),  # type: ignore[attr-defined]
        trace_id=str(msg.trace_id),  # type: ignore[attr-defined]
        timeout_s=float(msg.timeout_s),  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        active=bool(msg.active),  # type: ignore[attr-defined]
        region=(
            PlaceRegion.from_idl(msg.region)  # type: ignore[attr-defined]
            if bool(getattr(msg, "region_valid", False))
            else None
        ),
        search_box=(
            PlaceRegion.from_idl(msg.search_box)  # type: ignore[attr-defined]
            if bool(getattr(msg, "search_box_valid", False))
            else None
        ),
        workspace_box=(
            PlaceRegion.from_idl(msg.workspace_box)  # type: ignore[attr-defined]
            if bool(getattr(msg, "workspace_box_valid", False))
            else None
        ),
        workspace_region=(
            PlaceRegion.from_idl(msg.workspace_region)  # type: ignore[attr-defined]
            if bool(getattr(msg, "workspace_region_valid", False))
            else None
        ),
    )

is_live(*, now_ns: int) -> bool

Is this declaration still in force at now_ns?

Same rule and same clock-domain contract as PlaceDeclaration.is_live: now_ns must be a reading of the publishing graph's ROS clock (simulator time under use_sim_time), or the newest stamp on the stream that carried the declaration — never time.time_ns. Retracted, expired and future-stamped declarations are all dead; a dead declaration exempts nothing.

Parameters:

Name	Type	Description	Default
now_ns	int	Consumer's current time, same clock as stamp_ns.	required

Returns:

Type	Description
bool	True only while the declaration is active and inside its
bool	backstop window.

Source code in python/core/src/openral_core/schemas.py

def is_live(self, *, now_ns: int) -> bool:
    """Is this declaration still in force at ``now_ns``?

    Same rule and same clock-domain contract as
    ``PlaceDeclaration.is_live``: ``now_ns`` must be a reading of the
    publishing graph's ROS clock (simulator time under ``use_sim_time``),
    or the newest stamp on the stream that carried the declaration — never
    ``time.time_ns``. Retracted, expired and future-stamped declarations
    are all dead; a dead declaration exempts nothing.

    Args:
        now_ns: Consumer's current time, same clock as ``stamp_ns``.

    Returns:
        ``True`` only while the declaration is active and inside its
        backstop window.
    """
    if not self.active:
        return False
    elapsed_ns = now_ns - self.stamp_ns
    return 0 <= elapsed_ns <= int(self.timeout_s * 1e9)

GraspTargetRef

Bases: BaseModel

The object an ExecuteRskillTool goal grasps, as the reasoner names it.

The reasoner names; perception grounds; the producer measures (real pick-and-place design §2.2). At dispatch the reasoner resolves this reference to a seed box in the robot base frame — a recalled spatial-memory node's 3D box when object_id is set, else the one live lifted detection whose label matches — and sends it as GraspDeclaration.search_box. A reference that grounds to nothing, or to more than one instance with no object_id, is refused and the goal is not sent. Naming arms nothing: the kernel only ever trusts the region the producer measured inside the seed.

Attributes:

Name	Type	Description
label	str	Open-vocabulary object label, e.g. "box"; matched case-insensitively against the live lifted detections' labels.
object_id	str | None	A spatial-memory node_id from recall_object when the reasoner already knows the instance; None = ground by label.
contact_links	list[str]	Gripper links the grasp declaration names, all child_links of role: gripper joints of ONE hand (the gripper joints hanging off one arm); empty = the hand of a single-hand robot. A robot with several hands refuses an empty list (the grounding would otherwise exempt every hand), and links of two hands, a non-gripper link, or only part of a hand's links (that would exempt one finger and the kernel would stop the grasp) are refused.

Example

GraspTargetRef(label="box").contact_links []

GripperClosureCalibration

Bases: BaseModel

How a role: gripper joint's position reads "closed on an object".

The position-stall grasp trigger (openral_hal._grasp_trigger) needs no effort channel: a jaw commanded toward closed that settles short of the command by more than its empty-close error is stalled on something. These are the numbers that turn that into thresholds, in the joint's own position units: robot-type constants of the gripper mechanism, declared in the robot manifest and derived from teleop data; not per cell (CLAUDE.md §1.2: cited measurements, never a library default).

Attributes:

Name	Type	Description
closed_position	float	The commanded position of a fully closed jaw. Must be an end of the joint's position_limits: "closer to closed" is read as the distance to it.
closed_rest_offset	float	How far from closed_position the jaw rests when it closes on nothing (mechanical stop, servo deadband). Measured from teleop data.
stall_gap	float	How much further from closed than the command (beyond closed_rest_offset) a settled jaw must sit to read as stalled on an object. Also the band a command must be within of closed_position to count as a close command, and (halved) the hysteresis for slip / release / re-seat.
settle_tolerance	float	Largest position span over the trigger's settle window for the jaw to count as stationary. A few position-encoder LSBs.

Example

GripperClosureCalibration( ... closed_position=0.0, ... closed_rest_offset=0.0086, ... stall_gap=0.08, ... settle_tolerance=0.001, ... ).stall_gap 0.08

GripperReadMode

Bases: str, Enum

How openral_hal.MujocoArmHAL reports the gripper qpos.

SUM_OVER_SCALE (default) — clip(sum(qpos[addrs]) / scale, 0, 1). Matches the Franka parallel gripper where two finger qpos are summed and divided by 2 * max_finger_extent (=0.08 m for Panda). Public surface is normalised [0, 1]. AFFINE_LOW_HIGH — (qpos[addrs[0]] - ctrl_range[0]) / (ctrl_range[1] - ctrl_range[0]). Used by the SO-100 menagerie Jaw joint, a 1-DoF revolute with a non-zero closed position (-0.174 rad). Public surface is normalised [0, 1]. PASSTHROUGH — qpos[addrs[0]] reported verbatim, in the same physical units as the MJCF (metres for Aloha prismatic fingers, radians for OpenArm revolute jaws). The public surface is not [0, 1] — Skills must accept the raw range.

GripperWriteMode

Bases: str, Enum

How openral_hal.MujocoArmHAL maps an Action's gripper value to ctrl.

NORMALISED (default) — input is [0, 1], mapped affinely to ctrl_range. low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). PASSTHROUGH — input is in the same physical units as ctrl_range (metres / radians), written directly to ctrl. MuJoCo's actuator ctrlrange does the clipping. Used by Aloha (positive-finger metres) and OpenArm (jaw radians).

GroundedSubtask

Bases: BaseModel

One subtask bound to exactly ONE specific object.

Makes the "smallest actionable unit" invariant a type: object_ref is the single concrete object (or place) this subtask acts on, and text is the instruction handed to the skill (the VLA prompt). The validator forbids a collective/quantified object_ref or text and requires text to name its object_ref — so an ungrounded "pick up the first batch of objects" is not a representable value and the provider's structured-output path re-prompts (the same mechanism that rejects a malformed ReasonerToolCall).

Attributes:

Name	Type	Description
object_ref	str	The single concrete object/place this subtask acts on ("milk", "the fridge"). Never a quantifier or generic plural.
text	str	The instruction the skill receives, naming object_ref ("pick up the milk and put it in the basket").

Example

GroundedSubtask(object_ref="milk", text="pick up the milk and bag it").render() 'pick up the milk and bag it' is_collective_target( ... GroundedSubtask(object_ref="milk", text="grab the milk").object_ref ... ) False

render() -> str

The instruction string handed to MissionState / the skill.

Source code in python/core/src/openral_core/schemas.py

def render(self) -> str:
    """The instruction string handed to ``MissionState`` / the skill."""
    return self.text.strip()

HalConfig

Bases: BaseModel

Configuration for a HAL adapter instantiation by the inference runner.

Picks which HAL adapter class to use (sim digital twin vs real hardware) and how to reach the robot (serial port, FCI URI, ROS 2 namespace). Adapter-specific fields are forwarded as params.

Attributes:

Name	Type	Description
adapter	str	HAL adapter id, e.g. "so100_follower", "so100_digital_twin", "franka_panda_real", "ur5e_real", "ros_control". Resolved against the HAL registry.
transport	dict[str, object]	Transport-layer parameters. Examples: {"port": "/dev/ttyACM0", "baud": 1_000_000} for SO-100; {"fci_uri": "172.16.0.2"} for Franka; {"namespace": "/ur5e"} for ros2_control.
params	dict[str, object]	Adapter-specific keyword arguments forwarded to the constructor (e.g. calibration overrides).

Example

HalConfig( ... adapter="so100_follower", ... transport={"port": "/dev/ttyACM0", "baud": 1_000_000}, ... ).adapter 'so100_follower'

HalEntrypoints

Bases: BaseModel

Per-robot simulation and real-hardware HAL import strings.

The two HALs a robot can expose, declared independently so the choice of HAL type lives in the manifest (never in environment config or runtime params). Each value is a "module:Attr" import string resolved by openral_hal.build_hal, or None when the robot has no HAL of that kind (sim-only / real-only / scene-only).

Attributes:

Name	Type	Description
sim	str | None	Import string for the simulation HAL. When None and RobotDescription.sim is populated, the resolver derives MujocoArmHAL.from_description — so every plain arm leaves this null. Set it explicitly only for a non-generic sim HAL (e.g. "openral_hal.panda_mobile:PandaMobileHAL", which has no sim: block to derive from).
real	str | None	Import string for the real-hardware HAL (e.g. "openral_hal.ur_real:UR5eRealHAL"). None for sim-only robots; the resolver raises ROSCapabilityMismatch if mode="real" is requested on a robot whose real is None.
parameters	HalParameters	Per-robot HAL construction defaults (serial port, robot_ip, …) merged into the constructor by openral_hal.build_hal. Empty by default.
real_bringup	str | None	Optional vendor ros2_control bringup that deploy run includes next to the real HAL, as "<ros_package>:<file>.launch.py" (resolved under that package's share/<pkg>/launch/). None means no bringup is included; there is no convention-based fallback.

Example

HalEntrypoints(real="openral_hal.ur_real:UR5eRealHAL").sim is None True HalEntrypoints(real_bringup="openral_hal_openarm:real_bringup.launch.py").real_bringup 'openral_hal_openarm:real_bringup.launch.py'

HalParameters

Bases: BaseModel

Per-robot HAL construction defaults declared in the manifest.

Carries the transport / constructor keyword arguments a robot's HAL needs — the SO-100's serial port + baud, a ros2_control arm's robot_ip / fci_ip — so the manifest is the single source of those defaults instead of a per-robot lifecycle-node subclass. This is the schema seam that lets the unified, robot.yaml-driven ManifestHALLifecycleNode serve a parameterised HAL without a bespoke _create_hal.

openral_hal.build_hal merges defaults underneath any explicit transport kwargs (so a deploy run override wins) and then drops every key the target HAL constructor does not accept — exactly the filtering it already applies to transport. Empty by default, so robots that need no construction kwargs (the derived MujocoArmHAL arms) are unaffected.

Attributes:

Name	Type	Description
defaults	dict[str, object]	HAL constructor / transport keyword defaults, e.g. {"port": "/dev/ttyACM0", "baud": 1_000_000} for the SO-100 or {"robot_ip": "192.168.1.10"} for a real UR arm.
can_bus_bindings	dict[str, str]	Maps a key of defaults to the role token that identifies which physical CAN bus fills it, e.g. {"left_can_interface": "left", "right_can_interface": "right"}. Declaring this opts the robot into having openral detect overwrite those defaults with the interface names actually found on the host, instead of shipping one lab's names to every user. A CAN interface name is a property of the host, not of the robot: the same arm is openarm_left on a machine with a udev rule and can1 on one without. The manifest therefore declares only which parameter each bus fills and how to recognise it; the value is discovered. Robots with no CAN bus leave this empty and are untouched. The token is matched case-insensitively as a substring of the interface name, so it works for any bus count — one entry for a single-bus arm, two for a bimanual station, N for an N-limb machine. Matching is never positional: alphabetical order is a coincidence, and binding the left arm's trajectory to the right arm's bus is exactly the failure this avoids. An entry that matches no interface, or more than one, is reported as a warning and leaves the manifest value untouched (CLAUDE.md §1.4).

Example

HalParameters(defaults={"port": "/dev/ttyACM0"}).defaults["port"] '/dev/ttyACM0' HalParameters(can_bus_bindings={"left_can_interface": "left"}).can_bus_bindings

Hand

Bases: str, Enum

Laterality of an end-effector.

HumanEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_HUMAN — a human triggered an intervention.

Attributes:

Name	Type	Description
kind	Literal['human']	Discriminator (always "human").
actor	str	Identifier of the operator (e.g. "slack:alice", "gui").
reason	str	Free-form reason given by the actor.

ImagePreprocessing

Bases: BaseModel

Per-rSkill image preprocessing contract.

Properties of how the checkpoint was trained against image frames, surfaced on the manifest so the sim adapter does not have to learn them from a YAML override. vla.extra overrides on the eval config still win — see openral_rskill._vla_core.resolve_image_preprocessing.

Attributes:

Name	Type	Description
flip_180	bool	Apply torch.flip(t, dims=[1, 2]) (H+W reversal) to every camera frame before the policy forward. SmolVLA / pi05 LIBERO checkpoints want this on; the RoboCasa checkpoints published by RoMALab / DAVIAN-Robotics want it off.
flip_vertical	bool	Apply img[::-1, :, :] (vertical-only, H-axis reversal) BEFORE any 180° rotation. Mirrors robocasa.wrappers.gym_wrapper.RoboCasaGymEnv.process_img, which the canonical openpi-robocasa eval applies before feeding frames to the policy. Required for robocasa/robocasa365_checkpoints/pi05_pretrain_human300 and its lerobot-converted siblings; the RoMALab MG_300 checkpoint does NOT want this flip.
input_template	str	Format string used to name image tensors in the policy input batch — for example "observation.images.{cam}" for SmolVLA / pi05 RoMALab vs "observation.image.{cam}" for ruiname/pi05-robocasa-10tasks-200k.
aliases	dict[str, str]	Per-checkpoint rename map from a VLA slot (the vla_feature_key suffix a sensors_required entry names, e.g. camera1) to the model's expected input feature name (e.g. image). Keys are slots — never sensor or scene camera names — so the map means the same thing on openral sim run, deploy sim and deploy run (see sensor_name_to_slot); unknown_alias_slots reports keys that break this rule. Empty means pass through unchanged. SmolVLA / pi05 LIBERO want {"camera1": "image", "camera2": "image2"}.
norm_tag	str | None	Normalization statistics tag for policies that use multiple checkpoints or training distributions (e.g. MolmoAct2 with norm_tag="so100_so101_molmoact2" for SO-100/101 finetuned weights). Optional; None means the adapter's default tag applies. Used by predict_action methods to select the right norm_stats entry at inference time.
image_max_crops	int | None	Cap on the number of image tiles a multi-crop image processor (Molmo / MolmoAct2 family) produces per camera frame. Each extra 378px crop adds ~182 pooled image tokens and attention cost is quadratic in the token count, so this is the primary activation-memory lever (the NF4 weights are fixed at ~3.5 GiB; what overflows an 8 GiB card is the crop activations). Optional; None keeps the checkpoint's own default (8 for MolmoAct2). A per-rollout vla.extra["image_max_crops"] (or the OPENRAL_MOLMOACT2_MAX_CROPS env) still overrides this per-checkpoint default. Must be >= 1 when set.
resize_width	int | None	Exact checkpoint input width. When set with resize_height, adapters resize the source frame before policy inference. This lets a high-resolution detector feed coexist with a policy trained at a lower native render size.
resize_height	int | None	Exact checkpoint input height; must be set together with resize_width.

unknown_alias_slots(sensors_required: list[SensorRequirement]) -> list[str]

Alias keys that are not a slot declared by sensors_required.

Example

req = SensorRequirement( ... modality=SensorModality.RGB, vla_feature_key="observation.images.camera1" ... ) ImagePreprocessing(aliases={"camera1": "image", "top": "x"}).unknown_alias_slots( ... [req] ... ) ['top']

Source code in python/core/src/openral_core/schemas.py

def unknown_alias_slots(self, sensors_required: list[SensorRequirement]) -> list[str]:
    """Alias keys that are not a slot declared by ``sensors_required``.

    Example:
        >>> req = SensorRequirement(
        ...     modality=SensorModality.RGB, vla_feature_key="observation.images.camera1"
        ... )
        >>> ImagePreprocessing(aliases={"camera1": "image", "top": "x"}).unknown_alias_slots(
        ...     [req]
        ... )
        ['top']
    """
    declared = _required_rgb_slots(sensors_required)
    return [key for key in self.aliases if key not in declared]

IntrinsicsPinhole

Bases: BaseModel

Pinhole camera intrinsics.

Attributes:

Name	Type	Description
width	int	Image width in pixels.
height	int	Image height in pixels.
fx	float	Focal length in x (pixels).
fy	float	Focal length in y (pixels).
cx	float	Principal point x (pixels).
cy	float	Principal point y (pixels).
distortion_model	Literal['plumb_bob', 'rational_polynomial', 'equidistant', 'none']	Distortion model name, in ROS CameraInfo naming: plumb_bob (Brown, [k1, k2, p1, p2, k3]), rational_polynomial (OpenCV rational, [k1, k2, p1, p2, k3, k4, k5, k6]: the radial term is (1 + k1 r^2 + k2 r^4 + k3 r^6) / (1 + k4 r^2 + k5 r^4 + k6 r^6)), equidistant (fisheye, [k1, k2, k3, k4]) or none.
distortion_coeffs	list[float]	Distortion coefficients, in the model's order.

Example

IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0) IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0, ...)

JointSpec

Bases: BaseModel

URDF-derived joint specification.

Attributes:

Name	Type	Description
name	str	Joint name matching the URDF.
joint_type	JointType	Kinematic joint type.
parent_link	str	Parent link name.
child_link	str	Child link name.
axis_xyz	tuple[float, float, float]	Rotation/translation axis unit vector.
position_limits	tuple[float, float] | None	(min, max) in the unit of the value that travels on this channel — radians or metres for arm joints, and (0.0, 1.0) for a gripper whose HAL contract is a normalised jaw fraction (i.e. one declared normalised in SimDescription.grippers). The safety kernel's envelope, the rSkill runner's pre-clamp and the supervisor's gripper_min / gripper_max all compare a commanded Action value against these bounds, so declaring a normalised gripper's mechanical radian range here stops the envelope from constraining that channel at all (issue #62). A mechanical range that differs from the channel unit belongs in SimGripperDescription.ctrl_range / the URDF.
velocity_limit	float | None	Maximum velocity.
effort_limit	float | None	Maximum effort (N or Nm).
has_position_sensor	bool	Whether position feedback is available.
has_velocity_sensor	bool	Whether velocity feedback is available.
has_torque_sensor	bool	Whether the joint has a dedicated joint torque sensor. A torque estimate derived from motor current does not count: declare False for current-derived estimates so force decisions do not over-trust them.
backlash_estimate	float | None	Estimated backlash in radians.
actuator_kind	Literal['dc', 'bldc', 'stepper', 'servo', 'tendon', 'hydraulic', 'pneumatic'] | None	Type of actuator.
sim_joint_name	str | None	Optional override carrying the MJCF/MuJoCo joint name as it appears in the simulator's compiled model when that name differs from name. The HAL uses name for the world-state contract (it's the URDF-shaped logical identifier the safety supervisor sees on every chunk); a separate sim_joint_name lets a sim-adapter look up mj_name2id without hardcoding robosuite / robocasa / MuJoCo naming conventions in backend modules. None means "the MJCF joint name matches name" — true for every fixed-base manipulator we ship; only mobile bases and humanoids whose MJCF auto-prefixes joints (robosuite's mobilebase0_… namespace, GR-1's robot0_…) need it.
role	JointRole	Structural classification. The downstream runner / safety / dataset-bridge code identifies grippers and base DoFs by this tag instead of substring-matching the joint name (which silently misclassifies any joint containing "gripper", e.g. "gripper_pose"). Default "unknown" keeps legacy manifests loadable; the fleet annotates incrementally as this rolls out.
closure_calibration	GripperClosureCalibration | None	Position-stall grasp-trigger calibration for a role: "gripper" joint (GripperClosureCalibration). None = uncalibrated: the HAL's vision attachment leg refuses to arm a trigger on that joint rather than guess thresholds.
origin_xyz	tuple[float, float, float]	Fixed translation (metres) of this joint's frame in its parent_link frame — the URDF <joint><origin xyz>. With origin_rpy and axis_xyz it gives the kernel everything it needs for forward kinematics. Default (0, 0, 0); populated by the offline lowering tool (from MJCF/URDF) only for robots that enable self-collision checking — legacy manifests are unaffected.
origin_rpy	tuple[float, float, float]	Fixed orientation (roll, pitch, yaw, radians) of this joint's frame in its parent_link frame — the URDF <joint><origin rpy>. Default (0, 0, 0).

JointState

Bases: BaseModel

Real-time joint state snapshot.

Attributes:

Name	Type	Description
name	list[str]	Ordered list of joint names.
position	list[float]	Joint positions (rad or m).
velocity	list[float]	Joint velocities (rad/s or m/s).
effort	list[float]	Joint efforts (Nm or N), all joints or none. Empty means not measured; a producer never zero-fills an unmeasured joint, since a torque observer reads 0.0 as an unloaded joint (#373).
stamp_ns	int	ROS 2 timestamp in nanoseconds.

JointType

Bases: str, Enum

Kinematic joint type following URDF conventions.

JointUnits

Bases: str, Enum

Angular units a joint-position checkpoint was trained in.

A joint-position VLA consumes state and emits actions in ONE angular convention (whatever its training dataset recorded). openral's JointState / Action contract is radians, so the skill_runner must convert deg<->rad at the policy boundary when the checkpoint is in degrees. Getting this wrong is not subtle: feeding a degrees-trained policy radians makes every observation ~57x too small (out-of-distribution -> garbage actions), and emitting its degree actions as radians makes the HAL command ~57x too large (the arm slams its limits). Declaring it on the manifest removes the fragile runtime guess.

LaunchInclude

Bases: BaseModel

A vendor ROS 2 launch file a deploy has to bring up alongside the graph.

Some sensors are not opened by OpenRAL at all — they are published by the vendor's own driver, and the scene's SensorDeployBinding merely subscribes to what that driver puts on the bus (backend: ros2_image). The binding names the topic but not who publishes it, so until this existed an operator had to start the driver by hand in a second terminal and every camera fed that way was silently empty when they forgot.

Same shape as the HAL's real_bringup.launch.py convention, but declared as data rather than by file name: a driver needs arguments (which camera model, which override YAML), and those are host-specific — which is exactly what a DeployScene is for.

Resolved through FindPackageShare, so the launch comes from the sourced overlay like any other vendor package, and a missing package fails loudly at launch-parse time rather than leaving a topic quietly unpublished.

Attributes:

Name	Type	Description
package	str	ROS 2 package providing the launch file.
launch_file	str	File name inside that package's launch/ directory.
args	dict[str, str]	Launch arguments, forwarded verbatim. Values are strings because that is what ros2 launch takes; "true"/"false" for booleans.

Example

LaunchInclude(package="zed_wrapper", launch_file="zed_camera.launch.py").args {}

LifecycleTransitionTool

Bases: _ReasonerToolBase

Tool variant — drive a ROS 2 lifecycle transition on a peer node.

Dispatch: service call on <node>/change_state with the matching Transition id (configure / activate / deactivate / cleanup). Used to bring a HAL back online, restart a faulted node, or stage a controlled shutdown.

Attributes:

Name	Type	Description
tool	Literal['lifecycle_transition']	Discriminator (always "lifecycle_transition").
node	str	Fully-qualified ROS node name (e.g. "/openral/hal/so100").
transition	Literal['configure', 'activate', 'deactivate', 'cleanup']	One of "configure", "activate", "deactivate", "cleanup". Other transitions ("shutdown", error-recovery) are deliberately omitted from the open-core palette per CLAUDE.md §6 Layer 6 — shutdown is owned by the safety supervisor, not the reasoner.

LinkCollisionGeometry

Bases: BaseModel

One convex collision volume rigidly attached to a robot link.

The lowered, kernel-facing form of a link's collision geometry. Authored by hand, or emitted by the offline lowering tool from a robot's MJCF or URDF + SRDF source; the kernel loads these into pre-sized buffers at on_configure and never parses the source geometry on the hot path. link_name is a JointSpec link (joints stays normative for the kinematic chain — this adds geometry only).

Attributes:

Name	Type	Description
link_name	str	The JointSpec.child_link (or JointSpec.parent_link) this volume is attached to.
shape	CollisionShape	The convex primitive (capsule, sphere, or box).
origin_xyz_rpy	tuple[float, float, float, float, float, float]	Pose of the primitive in the link frame — (x, y, z, roll, pitch, yaw) in metres and radians.
tight_geometry	TightCollisionGeometry | None	Optional tighter convex geometry the safety kernel uses in place of shape in two narrow phases: arm-link vs world voxel, and — when both links of a pair declare it — self-collision box vs box. shape remains the broad-phase bound, the containment proof, and the geometry every other check uses. Only valid on a BoxShape, and only when it fits inside it.

Example

g = LinkCollisionGeometry( ... link_name="link_1", ... shape=CapsuleShape(radius_m=0.04, length_m=0.3), ... ) g.origin_xyz_rpy (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)

LocateInViewTool

Bases: _ReasonerToolBase

Tool variant (read-only) — check if an object is in a live camera view.

The complement to RecallObjectTool: where recall_object recalls a remembered object from the scene-graph memory, locate_in_view asks a live camera-mounted VLM detector (e.g. LocateAnything) to look at the current frame right now and report whether the queried object is visible — and where. It runs the detector's open-vocabulary query on demand via the /openral/perception/locate_in_view ROS service and feeds the answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation — it only reads a frame; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['locate_in_view']	Discriminator (always "locate_in_view").
query	str	The object(s) to look for, as concrete object noun(s) — sent verbatim as the detector's open-vocabulary query. The fast default locator (omdet-turbo-locator) is a multi-label detector: it matches each comma-separated term as one object class, so query a single noun ("mug") or a comma-separated list for several ("cup, bowl, ketchup bottle"). A collective/quantified PHRASE ("the objects on the table", "everything", "all items") is treated as one literal class name and matches NOTHING — name the concrete objects instead. Referring expressions with relations ("the mug behind the bowl") need the locateanything-3b locator (set detector); omdet ignores the relation.
camera	str	Optional camera selector. Empty (default) uses the detector's primary camera; otherwise names one of the detector's configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name — the detector is camera-agnostic and maps the id to a topic.
detector	str	Optional on-demand locator selector. Empty (default) uses the deployment's default locator; otherwise an rSkill id / short alias of one of the on-demand locators in the graph (e.g. "omdet_turbo-any-locator-fp16" for fast simple "find X", "locateanything_3b-any-general-nf4" for complex referring expressions). The reasoner routes to /openral/perception/<detector>/locate_in_view. Still read-only — choosing a model does not grant actuation.

MemorySearchTool

Bases: _ReasonerToolBase

Tool variant (read-only) — search the archival memory log.

Pages in entries evicted from the bounded MEMORY.md core into the archival JSONL (MemGPT recall), so the LLM can recall an older fact — e.g. a stale object location used as a search prior — without loading the whole history. Read-only; no actuation.

Attributes:

Name	Type	Description
tool	Literal['memory_search']	Discriminator (always "memory_search").
query	str	Free-text query over archived memory.
section	MemorySection | None	Optional MemorySection filter.
limit	int	Maximum number of results to return.

MemoryWriteTool

Bases: _ReasonerToolBase

Tool variant (write) — edit the robot's self-maintained MEMORY.md.

The reasoner's first write-capable tool. It edits the persistent, human-readable semantic memory (preferences, corrections, lessons, durable home facts, object locations, open tasks) through an explicit operation — never a free-form rewrite (Mem0 ADD/UPDATE/DELETE + Zep temporal supersession). It writes ONLY to the advisory memory file and holds no authority over actuation: a wrong memory yields a bad plan the C++ kernel still vetoes (CLAUDE.md §1.1). The dispatch that applies the edit is wired in a later phase — this is the typed contract.

Attributes:

Name	Type	Description
tool	Literal['memory_write']	Discriminator (always "memory_write").
op	Literal['add', 'update', 'supersede', 'delete']	The edit operation — add / update / supersede / delete.
section	MemorySection	Which MemorySection to edit.
content	str	The fact, as a distilled NL rule/line. Required for every op except delete (validated).
importance	float	LLM-assigned salience in [0, 1] (Generative Agents), used by recency-importance-relevance retrieval when the section is capped.
target	str | None	The existing entry this op acts on. Required for update / supersede / delete; omitted for add (validated).

MimicJoint

Bases: BaseModel

A link posed by another joint's value: q = multiplier * q[mimics] + offset.

The URDF <mimic> / MuJoCo joint-equality relation, for a link that moves but has no command channel of its own — the OpenArm's second (follower) finger, driven through the first by the gripper's linkage. RobotDescription.joints is consumed unfiltered as the action width, the sim qpos map and the runner's permutation, so a follower cannot be a JointSpec; and a FixedAttachment cannot move. This is the third edge kind of the collision tree, and nothing else: no limits, no sensor, no DoF, invisible to every joints consumer. The safety kernel poses child_link rigidly at the mimicked value every check (JointKind::kMimicRevolute / kMimicPrismatic), so the follower finger is checked at the opening the gripper is commanded to, like every other link, instead of swept over its whole stroke (docs/reference/finger-mimic-geometry.md).

Every geometric value comes from the robot's real URDF/MJCF (openral collision lower re-injects origin_* / axis_xyz from the MJCF and refuses a multiplier / offset the MJCF equality disagrees with). Never hand-estimate one.

Attributes:

Name	Type	Description
name	str	Joint name, matching the source URDF <joint> where one exists.
parent_link	str	The link this joint hangs from.
child_link	str	The link it poses. Defined by no other edge.
mimics	str	The JointSpec (by name) whose value drives this one.
multiplier	float	q = multiplier * q[mimics] + offset; finite, non-zero.
offset	float	Same relation; finite, in the joint's own unit.
joint_type	JointType	revolute (or continuous) or prismatic.
axis_xyz	tuple[float, float, float]	This joint's own axis in its frame (the source model's, not the leader's: the OpenArm follower's axis is the leader's negated in the URDF, the same with multiplier = -1; the MJCF spells it as the same axis with multiplier = 1).
origin_xyz	tuple[float, float, float]	Translation (metres) of the joint frame in parent_link.
origin_rpy	tuple[float, float, float]	Orientation (roll, pitch, yaw), radians, same frame.
sim_joint_name	str | None	The MJCF joint this is, when the twin spells it differently (the OpenArm's openarm_left_finger_joint2).

Example

m = MimicJoint( ... name="openarm_left_finger_joint2", ... parent_link="openarm_left_link7", ... child_link="openarm_left_finger_follower", ... mimics="left_gripper", ... multiplier=1.0, ... axis_xyz=(1.0, 0.0, 0.0), ... origin_xyz=(-0.00143, 0.018, -0.068), ... ) m.value_at(0.5) 0.5

value_at(leader_value: float) -> float

This joint's value when the mimicked joint is at leader_value.

Source code in python/core/src/openral_core/schemas.py

def value_at(self, leader_value: float) -> float:
    """This joint's value when the mimicked joint is at ``leader_value``."""
    return self.multiplier * leader_value + self.offset

MotionMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/motion.

Emitted when a frame-difference detector measures sub-frame motion above a configured threshold. region_bbox is set when the detector localises the moving pixels (top-left origin in pixels); None means "motion magnitude only, no localisation."

Attributes:

Name	Type	Description
kind	Literal['motion']	Discriminator (always "motion").
magnitude	float	Mean absolute per-pixel difference, normalised to [0, 1] against the encoding's full-scale range.
threshold	float	Magnitude threshold that fired the event.
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the moving region in pixels.

ObjectDetection2D

Bases: BaseModel

A single 2D detection inside an ObjectsMetadata event.

The 3D DetectedObject lives in WorldState; the 2D form here is what a per-camera detector (nvinfer, tflite, cv2-CPU) emits before any pose lift / fusion.

Attributes:

Name	Type	Description
label	str	Semantic class label as produced by the model.
confidence	float	Detection confidence in [0, 1].
bbox_xyxy	tuple[int, int, int, int]	Axis-aligned bounding box in pixels (x_min, y_min, x_max, y_max); image origin top-left.
det_id	int	Stable per-detector, per-camera identity, assigned by a 2D-IoU tracker at detection time so an object can be referred to and de-duplicated even when the 3D lift cannot run (RGB-only / no depth). -1 means untracked (legacy detectors / single-shot). Propagated into DetectedObject.track_id by the lift so a physical object carries one id whether it appears in the camera-space in_view line (no depth) or the 3D scene_objects line (depth).

ObjectsMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/objects.

Emitted by the event leg's detector element (nvinfer on Jetson, tflite on CPU, etc.) when one or more objects are detected.

Attributes:

Name	Type	Description
kind	Literal['objects']	Discriminator (always "objects").
detections	list[ObjectDetection2D]	Per-object 2D detections, ordered by descending confidence.
model_id	str	Identifier of the detector that fired (e.g. "yolov8n", "nvinfer:resnet50").
frame_width	int	Pixel width of the frame the detector ran on; the bbox_xyxy of each detection is in this pixel space (the cross-frame lift scales it to the sensor's intrinsics resolution).
frame_height	int	Pixel height of that frame.
camera_frame_id	str | None	tf2 frame the source image's pixels live in (the camera's optical frame), from the driver's CameraInfo header. Set together with camera_intrinsics; None (no CameraInfo reached the detector) means a consumer falls back to the SensorSpec named by sensor_id, whose frame and intrinsics may be a sim stand-in on real hardware.
camera_intrinsics	IntrinsicsPinhole | None	The driver's live pinhole K for that frame, at the CameraInfo's own resolution (the lift rescales the box to it).

Example

md = ObjectsMetadata( ... sensor_id="top", ... detections=[], ... model_id="rtdetr", ... frame_width=1920, ... frame_height=1080, ... camera_frame_id="zed_left_camera_frame_optical", ... camera_intrinsics=IntrinsicsPinhole( ... width=1920, height=1080, fx=1498.18, fy=1498.18, cx=936.11, cy=541.81 ... ), ... ) md.camera_frame_id 'zed_left_camera_frame_optical'

ObservationSpec

Bases: BaseModel

VLA observation configuration for this robot.

Attributes:

Name	Type	Description
state_key	str	Observation dict key for the state vector.
state_shape	tuple[int, ...]	Shape of the state tensor, e.g. (6,) for 6-D EEF.
state_representation	StateRepresentation | None	How the state vector is encoded.
image_flip_180	bool	Deprecated; has no effect on image processing. The live knob is the per-checkpoint RSkillManifest.image_preprocessing.flip_180. Kept so existing manifests still load; setting it emits a FutureWarning.

OccupancyGridRef

Bases: BaseModel

Reference to a 2D occupancy grid for mobile-base world-collision.

Mirrors the nav_msgs/OccupancyGrid metadata that openral_runner.slam_bridge already decodes. The kernel consumes a bounded, fixed-capacity copy; a grid exceeding the configured capacity or older than the staleness deadline is treated as unavailable (fail-closed). The occupancy bytes are referenced by topic, not inlined.

Attributes:

Name	Type	Description
frame_id	str	tf2 frame the grid origin is expressed in (e.g. "map").
resolution_m	float	Edge length of one cell, in metres.
width	int	Grid width in cells.
height	int	Grid height in cells.
origin	Pose6D	Pose of cell (0, 0)'s lower-left corner.
data_topic	str	ROS 2 topic carrying the nav_msgs/OccupancyGrid.

OcrMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/ocr.

Attributes:

Name	Type	Description
kind	Literal['ocr']	Discriminator (always "ocr").
text	str	Recognised text (already stripped of leading / trailing whitespace by the detector).
confidence	float	Recogniser confidence in [0, 1].
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the recognised region in pixels.

PerceptionStaleEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_PERCEPTION — a sensor frame went stale.

Attributes:

Name	Type	Description
kind	Literal['perception']	Discriminator (always "perception").
sensor_id	str	SensorSpec.name of the stale sensor.
staleness_ms	float	Observed age of the last frame, in milliseconds.
threshold_ms	float	Staleness threshold that was crossed, in milliseconds.

PhysicsBackend

Bases: str, Enum

Physics / scene backend used to instantiate a SceneSpec.

Attributes:

Name	Type	Description
MUJOCO		Vanilla MuJoCo (CPU / single-env). Default for LIBERO, MetaWorld, and VLABench (native lerobot 0.6.0 VLABenchEnv, in-process).
MUJOCO_MJX		MuJoCo MJX (XLA, GPU-batched headless rollouts).
PYBULLET		PyBullet (legacy adapters, contact-rich tabletop).
SAPIEN		SAPIEN (Hillbot/UCSD physics + ray-traced rendering). The engine under ManiSkill3 and RoboTwin 2.0; the RoboTwin dual-arm benchmark backend runs it out-of-process via a py3.10 sidecar. ManiSkill3 scenes predate this slot and historically declared MUJOCO — new SAPIEN backends use this value.
ISAACSIM		NVIDIA Isaac Sim (Omniverse + PhysX + RTX, GPU) — the isaac_sim scene, driven out-of-process via a py3.11 sidecar.
COPPELIASIM		CoppeliaSim/PyRep — the RLBench benchmark backend, driven out-of-process via a py3.10 sidecar.
GENESIS		Genesis (physics-language unification). Future.
MOCK		In-process mock with no physics — used for wiring smoketests.

PlaceDeclaration

Bases: BaseModel

Dispatch's typed statement that a place phase is active for a payload.

The ADR-0097 distinction between "arrived at its declared destination" and "grazed a wall" mid-carry — never inferred from motion or simulator introspection. On its own it exempts nothing: it only lets a producer attest a place-phase SupportContactWitness on the declared target, which the kernel then exempts under the pick witness's own bounds/hysteresis (ADR-0092 D6). Scoped to one goal execution; dies on retraction, goal end/cancel, E-stop, or timeout_s after stamp_ns, whichever fires first (HZ-0097-3).

Drafted amendment (ADR-0097, default off): the real place producer (openral_hal._place_target_leg) measures the surface directly under the carried payload and attaches it as the region of dispatch's goal-scope declaration (target_id="surface", the runner's place_approach_enabled); it never declares on its own, and the region dies with the declaration, the payload and the region's age bound.

Attributes:

Name	Type	Description
target_id	str	Identity of the declared place target, e.g. "sim:cab_1_left_group_main"; an attested support_id must name it or a surface of it.
object_id	str	Payload the declaration scopes to (AttachedCollisionObject.object_id); empty = whichever payload is carried.
rskill_id	str	Dispatching skill, for attributability (HZ-0097-2).
trace_id	str	OTel trace id, for attributability (HZ-0097-2).
timeout_s	float	Backstop expiry window in seconds after stamp_ns; capped at MAX_TIMEOUT_S.
stamp_ns	int	Dispatcher timestamp; with active this is the whole liveness key.
active	bool	False retracts the declaration.
contact_force_threshold_n	float	Per-declaration contact-force bound (ADR-0100); 0.0 = no force gate. Never read from the robot-wide SafetyEnvelope.contact_force_threshold_n.
region	PlaceRegion | None	Producer-measured bounded region of the target (ADR-0097's 2026-08-14 amendment); None = no approach allowance (pre-amendment margins). Dies with the declaration.
search_box	PlaceRegion | None	Optional gravity-aligned box (in the voxel grid's base frame) naming the surface a place may use: a hint only. The real place producer measures the support directly under the carried payload with or without it; when set, it refuses a measured patch whose centre lies outside the box. The reasoner grounds it from an optional PlaceTargetRef (a lifted detection or a recalled memory node, padded); dispatch passes it through unchanged. It never arms anything — the allowance is always the producer-measured region. geometry must be empty.

Example

declaration = PlaceDeclaration( ... target_id="sim:cab_1_left_group_main", ... rskill_id="openral/pi05-robocasa", ... timeout_s=60.0, ... stamp_ns=1_000_000_000, ... ) declaration.is_live(now_ns=31_000_000_000) True declaration.is_live(now_ns=91_000_000_000) False

fill_idl(msg: object, *, primitive_factory: Callable[[], object] | None = None) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Parameters:

Name	Type	Description	Default
msg	object	A duck-typed openral_msgs/PlaceDeclaration to populate.	required
primitive_factory	Callable[[], object] | None	Forwarded to PlaceRegion.fill_idl; needed only when the region carries the declared target's own geometry (ADR-0098). Dispatch never measures geometry, so its publication path never needs one.	None

Source code in python/core/src/openral_core/schemas.py

def fill_idl(
    self, msg: object, *, primitive_factory: Callable[[], object] | None = None
) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

    Args:
        msg: A duck-typed ``openral_msgs/PlaceDeclaration`` to populate.
        primitive_factory: Forwarded to ``PlaceRegion.fill_idl``; needed
            only when the region carries the declared target's own geometry
            (ADR-0098). Dispatch never measures geometry, so its publication
            path never needs one.
    """
    msg.target_id = self.target_id  # type: ignore[attr-defined]
    msg.object_id = self.object_id  # type: ignore[attr-defined]
    msg.rskill_id = self.rskill_id  # type: ignore[attr-defined]
    msg.trace_id = self.trace_id  # type: ignore[attr-defined]
    msg.timeout_s = float(self.timeout_s)  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.active = bool(self.active)  # type: ignore[attr-defined]
    msg.contact_force_threshold_n = float(  # type: ignore[attr-defined]
        self.contact_force_threshold_n
    )
    msg.region_valid = self.region is not None  # type: ignore[attr-defined]
    if self.region is not None:
        self.region.fill_idl(
            msg.region,  # type: ignore[attr-defined]
            primitive_factory=primitive_factory,
        )
    # Never carries geometry (validated), so no primitive factory is needed.
    msg.search_box_valid = self.search_box is not None  # type: ignore[attr-defined]
    if self.search_box is not None:
        self.search_box.fill_idl(msg.search_box)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    return cls(
        target_id=str(msg.target_id),  # type: ignore[attr-defined]
        object_id=str(msg.object_id),  # type: ignore[attr-defined]
        rskill_id=str(msg.rskill_id),  # type: ignore[attr-defined]
        trace_id=str(msg.trace_id),  # type: ignore[attr-defined]
        timeout_s=float(msg.timeout_s),  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        active=bool(msg.active),  # type: ignore[attr-defined]
        contact_force_threshold_n=float(getattr(msg, "contact_force_threshold_n", 0.0)),
        region=(
            PlaceRegion.from_idl(msg.region)  # type: ignore[attr-defined]
            if bool(getattr(msg, "region_valid", False))
            else None
        ),
        search_box=(
            PlaceRegion.from_idl(msg.search_box)  # type: ignore[attr-defined]
            if bool(getattr(msg, "search_box_valid", False))
            else None
        ),
    )

is_live(*, now_ns: int) -> bool

Is this declaration still in force at now_ns?

Fails toward dead: a retracted declaration, one past its backstop, and one stamped in the future (a clock that jumped, which is not evidence of anything) are all dead. A dead declaration permits no place-witness attestation, which is the pre-ADR-0097 behaviour.

Clock-domain contract. now_ns and stamp_ns must be readings of the same clock, and that clock is the publishing graph's ROS clock: the dispatching rSkill runner stamps the declaration from node.get_clock().now(), the evidence producer re-stamps its publications from the same clock, and the safety kernel compares against its own. Under use_sim_time that domain is simulator time, whose readings are small (order 1e9 ns) where wall time is order 1.79e18 ns. A consumer that has no such reading of its own must use the newest stamp observed on the message stream carrying the declaration — never time.time_ns, which reports a wall epoch that puts every sim-stamped declaration ~57 years past its backstop and silently kills it.

Parameters:

Name	Type	Description	Default
now_ns	int	Consumer's current time, same clock as stamp_ns — see the clock-domain contract above.	required

Returns:

Type	Description
bool	True only while the declaration is active and inside its
bool	backstop window.

Source code in python/core/src/openral_core/schemas.py

def is_live(self, *, now_ns: int) -> bool:
    """Is this declaration still in force at ``now_ns``?

    Fails toward dead: a retracted declaration, one past its backstop, and
    one stamped in the future (a clock that jumped, which is not evidence
    of anything) are all dead. A dead declaration permits no place-witness
    attestation, which is the pre-ADR-0097 behaviour.

    **Clock-domain contract.** ``now_ns`` and ``stamp_ns`` must be
    readings of the *same* clock, and that clock is the publishing graph's
    ROS clock: the dispatching rSkill runner stamps the declaration from
    ``node.get_clock().now()``, the evidence producer re-stamps its
    publications from the same clock, and the safety kernel compares against
    its own. Under ``use_sim_time`` that domain is **simulator** time, whose
    readings are small (order 1e9 ns) where wall time is order 1.79e18 ns.
    A consumer that has no such reading of its own must use the newest stamp
    observed on the message stream carrying the declaration — never
    ``time.time_ns``, which reports a wall epoch that puts every sim-stamped
    declaration ~57 years past its backstop and silently kills it.

    Args:
        now_ns: Consumer's current time, same clock as ``stamp_ns`` — see
            the clock-domain contract above.

    Returns:
        ``True`` only while the declaration is active and inside its
        backstop window.
    """
    if not self.active:
        return False
    elapsed_ns = now_ns - self.stamp_ns
    return 0 <= elapsed_ns <= int(self.timeout_s * 1e9)

PlaceRegion

Bases: BaseModel

The bounded region of a declared place target (ADR-0097 amendment).

The volume inside which the declared payload's world-collision margin is reduced by min(1.5 x voxel, 40 mm) (ADR-0097's Second Amendment, 2026-08-15 — 37.5 mm at sim's 25 mm cells, 40 mm rather than 75 mm at a real 50 mm grid since the absolute ceiling still binds) so it can physically reach the support contact the place witness is earned by. Round-6 validation: without it the flagship counter->cabinet insertion cannot complete even with a correct declaration — passing through an enclosed target's opening means passing within margin distance of voxel-quantised walls, and at 25 mm cells that quantisation closes the opening's predicted clearance 22-30 mm before the payload arrives. A witness earned by touching can never arm if the payload is stopped before it can touch.

Producer-supplied, producer-specific. Sim computes it from the declared body's MuJoCo model subtree; real hardware measures the support slab directly under the carried payload from the live voxel map (openral_hal._place_target_leg, default off, drafted ADR amendments pending). The kernel is producer-agnostic: it consumes this box identically whoever measured it, and never derives one itself.

The box is oriented, not axis-aligned — the axis-aligned hull of a rotated receptacle is strictly larger, i.e. more permissive. Absent (PlaceDeclaration.region is None) means no allowance at all — the pre-amendment margins, unchanged. A degenerate or over-large region is rejected here and again in the kernel, both times toward "no allowance".

The same type also carries the producer-measured region of a GraspDeclaration (real pick-and-place design §2.1), under that declaration's tighter caps and with geometry always empty.

Attributes:

Name	Type	Description
frame_id	str	Frame pose is expressed in. Must be the robot base frame — the frame OccupancyVoxels is published in. The kernel refuses a region whose frame does not match the occupancy grid's, because a region measured in one frame and applied in another is a relaxation aimed at the wrong volume.
pose	Pose6D	Region box centre pose in frame_id.
half_extents	tuple[float, float, float]	Box half-extents along its own axes (m), each in (0, MAX_HALF_EXTENT_M].
evidence_ref	str	What the producer measured the box from, for attributability (HZ-0097-2/4). Free text, never parsed.
stamp_ns	int	Producer timestamp.
geometry	tuple[AttachedCollisionPrimitive, ...]	The declared target's own collision primitives, each posed in frame_id (ADR-0098, survey Path B). Empty — the default, and what every producer that cannot measure geometry publishes — leaves the region behaving exactly as it did before ADR-0098. Supplying it lets the kernel adjudicate a cell against the modelled surface instead of against the 25 mm cube that quantised it, bounded so that it can never be more permissive than the box alone: the kernel uses the modelled distance only while it is within the blanket allowance that cell would have been granted anyway.

Example

region = PlaceRegion( ... frame_id="base_link", ... pose=Pose6D( ... xyz=(0.6, 0.1, 1.1), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="base_link" ... ), ... half_extents=(0.20, 0.18, 0.12), ... evidence_ref="mujoco_body_subtree:cab_1_left_group_main", ... ) round(region.volume_m3(), 5) 0.03456

fill_idl(msg: object, *, primitive_factory: Callable[[], object] | None = None) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Parameters:

Name	Type	Description	Default
msg	object	A duck-typed openral_msgs/PlaceRegion to populate in place.	required
primitive_factory	Callable[[], object] | None	Builds one empty AttachedCollisionPrimitive message. Required only when geometry is non-empty — dispatch never measures geometry, so its publication path never needs one.	None

Raises:

Type	Description
ROSConfigError	geometry is non-empty and no primitive_factory was supplied. Dropping the geometry silently would leave the kernel adjudicating a declared target it was told nothing about, which reads on the wire exactly like a producer that measured nothing.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(
    self, msg: object, *, primitive_factory: Callable[[], object] | None = None
) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

    Args:
        msg: A duck-typed ``openral_msgs/PlaceRegion`` to populate in place.
        primitive_factory: Builds one empty ``AttachedCollisionPrimitive``
            message. Required only when ``geometry`` is non-empty —
            dispatch never measures geometry, so its publication path never
            needs one.

    Raises:
        ROSConfigError: ``geometry`` is non-empty and no
            ``primitive_factory`` was supplied. Dropping the geometry
            silently would leave the kernel adjudicating a declared target
            it was told nothing about, which reads on the wire exactly like
            a producer that measured nothing.
    """
    msg.frame_id = self.frame_id  # type: ignore[attr-defined]
    msg.pose.position.x = float(self.pose.xyz[0])  # type: ignore[attr-defined]
    msg.pose.position.y = float(self.pose.xyz[1])  # type: ignore[attr-defined]
    msg.pose.position.z = float(self.pose.xyz[2])  # type: ignore[attr-defined]
    msg.pose.orientation.x = float(self.pose.quat_xyzw[0])  # type: ignore[attr-defined]
    msg.pose.orientation.y = float(self.pose.quat_xyzw[1])  # type: ignore[attr-defined]
    msg.pose.orientation.z = float(self.pose.quat_xyzw[2])  # type: ignore[attr-defined]
    msg.pose.orientation.w = float(self.pose.quat_xyzw[3])  # type: ignore[attr-defined]
    msg.half_extents.x = float(self.half_extents[0])  # type: ignore[attr-defined]
    msg.half_extents.y = float(self.half_extents[1])  # type: ignore[attr-defined]
    msg.half_extents.z = float(self.half_extents[2])  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.geometry = []  # type: ignore[attr-defined]
    if self.geometry and primitive_factory is None:
        raise ROSConfigError(
            f"PlaceRegion for {self.frame_id!r} carries {len(self.geometry)} target "
            "primitives but fill_idl was given no primitive_factory to encode them. "
            "Publishing the region without its geometry is indistinguishable on the wire "
            "from a producer that measured none."
        )
    for primitive in self.geometry:
        item = primitive_factory()  # type: ignore[misc]  # reason: guarded above
        primitive.fill_idl(item)
        msg.geometry.append(item)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    pose = msg.pose  # type: ignore[attr-defined]
    half = msg.half_extents  # type: ignore[attr-defined]
    return cls(
        frame_id=str(msg.frame_id),  # type: ignore[attr-defined]
        pose=Pose6D(
            xyz=(float(pose.position.x), float(pose.position.y), float(pose.position.z)),
            quat_xyzw=(
                float(pose.orientation.x),
                float(pose.orientation.y),
                float(pose.orientation.z),
                float(pose.orientation.w),
            ),
            frame_id=str(msg.frame_id),  # type: ignore[attr-defined]
        ),
        half_extents=(float(half.x), float(half.y), float(half.z)),
        evidence_ref=str(msg.evidence_ref),  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        geometry=tuple(
            AttachedCollisionPrimitive.from_idl(item, object_id=str(msg.frame_id))  # type: ignore[attr-defined]
            for item in msg.geometry  # type: ignore[attr-defined]
        ),
    )

volume_m3() -> float

Volume of the region box in cubic metres.

Source code in python/core/src/openral_core/schemas.py

def volume_m3(self) -> float:
    """Volume of the region box in cubic metres."""
    hx, hy, hz = self.half_extents
    return 8.0 * hx * hy * hz

PlaceTargetRef

Bases: BaseModel

An OPTIONAL hint naming the surface an ExecuteRskillTool goal places onto.

Where to set the payload down is the policy's job, and the real place producer measures the support directly under the carried payload without being told (real pick-and-place design §2.3), so a goal never needs this. Set, it restricts the place to one labelled surface ("shelf", "table"): perception grounds it at dispatch to a padded, gravity-aligned box in the robot base frame — a recalled spatial-memory node's 3D box when object_id / place_node_id is set, else the one live lifted detection whose label matches — sent as PlaceDeclaration.search_box, and the producer refuses a patch measured outside it. Nothing about the cell is surveyed or predeclared. A reference that grounds to nothing, or to more than one instance with no node id, is refused and the goal is not sent. Naming arms nothing: the kernel only ever trusts the measured region.

Attributes:

Name	Type	Description
label	str	Open-vocabulary label of the surface, e.g. "shelf"; matched case-insensitively against the live lifted detections' labels.
object_id	str | None	A spatial-memory node_id from recall_object naming the surface's instance; None = ground by label (or place_node_id).
place_node_id	str | None	A spatial-memory place node id, for a remembered place that carries a 3D box; None = ground by label (or object_id). At most one of object_id / place_node_id.

Example

PlaceTargetRef(label="shelf").object_id is None True

PlaybookContract

Bases: BaseModel

Manifest contract for kind: "playbook" rSkills.

A playbook is a human-authored standard-operating-procedure — a structured Markdown document describing how the S2 Reasoner should approach a class of task: its trigger, the ordered decision steps it composes from existing reasoner tools, the bound on those steps, and its verifiable acceptance (done) predicate. It is content the reasoner reads, never code it executes, and carries no weights, actuators, ROS server, or Action contract — the symbolic counterpart to a vla policy.

Required when RSkillManifest.kind is "playbook"; forbidden for all other kinds (enforced by RSkillManifest._check_kind_consistency). The body_uri Markdown body is what the reasoner injects into its system prompt (or retrieves by trigger similarity at scale); the manifest description + trigger drive selection. Advisory only — a playbook changes what the LLM decides, never its actuation authority: every motion it triggers is still an ExecuteRskill → Action chunk → C++ safety kernel (CLAUDE.md §1.1).

Attributes:

Name	Type	Description
trigger	str	Natural-language description of when this playbook applies. Used as the retrieval key when more playbooks are installed than fit the system prompt (the deferred load_playbook tool).
body_uri	str	Repo-relative path to the Markdown SOP (e.g. "./PLAYBOOK.md").
composes_tools	list[str]	The ReasonerToolCall discriminators the SOP uses (execute_rskill, recall_object, locate_in_view, query_scene, query_task_progress, memory_write, memory_search …). Advisory — a manifest-level hint of the playbook's tool surface; at least one entry is required.
done_predicate	str	Natural-language acceptance test the playbook verifies before declaring success (e.g. "the target object is in the gripper").
max_steps	int	Hard bound on the number of tool calls before the playbook must terminate (no hidden default — CLAUDE.md §1.4). Must be > 0.

Example

p = PlaybookContract( ... trigger="the goal names a physical object whose location is not given", ... body_uri="./PLAYBOOK.md", ... composes_tools=["recall_object", "locate_in_view", "execute_rskill"], ... done_predicate="the target object is confirmed in view at a known pose", ... max_steps=12, ... ) p.max_steps 12

Pose6D

Bases: BaseModel

6D pose: position + quaternion.

Attributes:

Name	Type	Description
xyz	tuple[float, float, float]	Position (x, y, z) in meters.
quat_xyzw	tuple[float, float, float, float]	Quaternion (x, y, z, w).
frame_id	str	tf2 reference frame.

ProtocolSpec

Bases: BaseModel

Stand-alone eval-protocol descriptor.

Each BenchmarkScene now carries its own n_episodes / seed / task.success_key / task.max_steps; ProtocolSpec is retained as a public schema for describing a protocol independently of a suite (ADR drafts, benchmark-report tooling).

Pins the methodology so two rSkills evaluated under the same benchmark produce apples-to-apples numbers (e.g. LIBERO: 10 episodes per task, fixed seed range, is_success).

Attributes:

Name	Type	Description
n_episodes	int	Number of independent episodes per task. Honest success rates need >= 10 per task for the published LIBERO / MetaWorld protocols.
seeds	list[int]	Seed list applied per task (paired index-wise with the episode index). Length MUST be >= n_episodes; runners slice seeds[:n_episodes] so re-runs are reproducible.
success_key	str	info key on the gym step() return whose truthy value marks task success. Default "is_success" (LIBERO convention); MetaWorld uses "success".
max_steps	int	Per-task episode budget. Adapters may clip to scene-internal limits; the value here is the protocol ceiling.
min_reps	int | None	Minimum number of completed episodes per task before the result JSON is written. Guards against a crash mid-suite producing a misleadingly partial roll-up. None means "require every task x episode".

Example

p = ProtocolSpec(n_episodes=10, seeds=list(range(10)), max_steps=280) p.success_key 'is_success'

model_post_init(_context: object) -> None

Cross-field validation: seeds list must cover n_episodes.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: seeds list must cover n_episodes."""
    if len(self.seeds) < self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.seeds has {len(self.seeds)} entries but "
            f"n_episodes={self.n_episodes}; provide at least n_episodes "
            f"seeds so re-runs are reproducible."
        )
    if self.min_reps is not None and self.min_reps > self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.min_reps ({self.min_reps}) exceeds "
            f"n_episodes ({self.n_episodes}); a benchmark cannot require "
            f"more completed reps than it schedules."
        )

QuantizationBackend

Bases: str, Enum

Inference backend that will execute the quantized model.

Attributes:

Name	Type	Description
PYTORCH		torch.quantization / bitsandbytes / torchao.
ONNX		onnxruntime with quantization pre-applied at export.
TENSORRT		NVIDIA TensorRT engine (INT8 / FP8 calibrated).
GGUF		llama.cpp / ggml GGUF format (CPU and Metal).
MLX		Apple MLX framework (Apple Silicon only).

Example

QuantizationBackend.PYTORCH.value 'pytorch'

QuantizationConfig

Bases: BaseModel

Full specification of how to quantize a skill's model weights.

Attributes:

Name	Type	Description
dtype	QuantizationDtype	Target numeric format.
backend	QuantizationBackend	Inference backend that will execute the model.
per_channel	bool	Use per-channel (vs per-tensor) quantization.
calibration_dataset	str | None	HuggingFace dataset ID or local path used for post-training calibration (INT8 / TensorRT only).
extra	dict[str, object]	Backend-specific overrides (e.g. {"calibration_steps": 128}).

Example

cfg = QuantizationConfig(dtype=QuantizationDtype.INT8) cfg.backend

QuantizationDtype

Bases: str, Enum

Numeric format used to represent model weights and activations.

Attributes:

Name	Type	Description
FP32		32-bit float (full precision, reference).
FP16		16-bit float (GPU-native, common default).
BF16		Brain-float16 (Ampere+ GPUs, TPUs; better dynamic range than FP16).
INT8		8-bit integer (CPU/GPU; good accuracy/speed trade-off).
FP8		8-bit float, E4M3 (Ada/Hopper+; keeps floating-point dynamic range at INT8's memory cost, which matters for transformer activations that INT8's fixed scale clips). Carried in a TensorRT engine as explicit QuantizeLinear/DequantizeLinear node pairs in the ONNX graph, not as a builder precision flag.
INT4		4-bit integer (edge / memory-constrained; some accuracy loss).
FP4_NVFP4		NVIDIA FP4 format (Blackwell; highest throughput). Block size 16 with an FP32/FP16/BF16 scale — distinct from the MX formats, which scale per 32 elements with an E8M0 exponent.

Example

QuantizationDtype.INT8.value 'int8' QuantizationDtype.FP8.value 'fp8'

QuerySceneTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask a scene VLM a question about the live view.

Backed by a kind: "vlm" rSkill (e.g. Qwen3.5-4B NF4) running in an out-of-process ZMQ sidecar. Where LocateInViewTool answers where an object is (open-vocabulary localization via the detector), query_scene answers open-ended questions about the scene's state — task-progress and success/failure verification the reasoner needs for its replanning ladder: "has the robot grasped the mug?", "is the bowl on the shelf?", "did we drop the object?", "is the table clear?".

It captures the current frame of the requested camera, sends it plus the question to the VLM over the /openral/perception/query_scene ROS service, and feeds the free-text answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation — it only reads a frame; the dispatch never gates the safety kernel. It is not a localizer: use LocateInViewTool to find objects.

Attributes:

Name	Type	Description
tool	Literal['query_scene']	Discriminator (always "query_scene").
question	str	Natural-language question about the current scene (e.g. "Has the robot grasped the red mug?"). Sent verbatim to the VLM as the textual prompt alongside the frame.
camera	str	Optional camera selector. Empty (default) uses the perception node's primary camera; otherwise names one of its configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name.

QueryTaskProgressTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask the reward monitor how the task is going.

Backed by a kind: "reward" rSkill (Robometer-4B NF4) running in parallel with the active VLA in an out-of-process ZMQ sidecar. Where QuerySceneTool answers open-ended scene questions as free text, query_task_progress returns a quantitative windowed assessment of the current task: normalized progress and success over the last window_s seconds, plus their trends and a stalled flag.

It calls the /openral/perception/query_task_progress ROS service, which scores the monitor's buffered camera frames against the task instruction and returns progress_now / success_now / progress_trend / success_trend / stalled / succeeded. The reasoner uses it to decide whether to continue, escalate to QuerySceneTool, advance, or enter the replanning ladder. Like every variant it holds no authority over actuation — the reward signal is advisory; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['query_task_progress']	Discriminator (always "query_task_progress").
window_s	float	How many seconds of recent frames to assess. Must be > 0; clamped to the monitor's configured frame_window_s.
task	str	Optional task-instruction override. Empty (default) reuses the instruction the monitor was co-activated with (the active VLA's goal).

ROSCapabilityMismatch

Bases: ROSError

A skill requires a capability the target robot does not have.

ROSCollisionImminent

Bases: ROSSafetyViolation

A proposed motion would self-collide or strike a world obstacle.

Raised on the safety path when geometric checking finds a chunk whose forward-kinematic sweep brings a robot link within clearance of another link or a world primitive.

ROSConfigError

Bases: ROSError

Bad manifest, missing weights, or invalid YAML / URDF.

ROSDeadlineMissed

Bases: ROSFleetError

Cloud RTT exceeded the skill's deadline; fallback was not configured.

ROSDispatchUnavailable

Bases: ROSFleetError

No dispatcher (edge or cloud) is available for the requested skill.

ROSEStopRequested

Bases: ROSSafetyViolation

An emergency stop was requested; all actuation must cease immediately.

ROSError

Bases: Exception

Base class for all OpenRAL errors.

ROSFleetError

Bases: ROSError

A fleet-level or dispatch error.

ROSForceLimitExceeded

Bases: ROSSafetyViolation

Measured or predicted contact force exceeds the configured limit.

Not raised anywhere yet: no check currently compares a force against SafetyEnvelope.max_force_n. The class is exported so the force/current contact layer (issue #373) can raise it without an API change.

ROSGPUMemoryError

Bases: ROSRuntimeError

Out of GPU memory; use a smaller skill variant or quantize further.

ROSObjectNotInMemory

Bases: ROSPerceptionStale

A spatial-memory query matched no node, or only nodes that are stale.

Raised by the scene-graph query surface when a RecallObjectQuery / ResolvePlaceQuery cannot be satisfied. The caller degrades by treating the target as unknown (and may trigger active search) — it never fabricates a pose (CLAUDE.md §1.2, §1.4).

ROSPerceptionStale

Bases: ROSError

A sensor reading is older than the configured staleness deadline.

ROSPlanningError

Bases: ROSError

The reasoner failed to produce a valid plan.

ROSReasonerInvalidPlan

Bases: ROSPlanningError

The LLM returned a plan that failed schema or capability validation.

ROSRskillGoalSatisfied

Bases: ROSError

A wrapped-ROS rSkill has finished its goal successfully.

Used as a typed completion signal raised by openral_rskill.ros_action_rskill.ROSActionRskill._step_impl after the last waypoint of a one-shot planner (e.g. MoveIt) has been emitted, or after a result-only wrapped action (e.g. Nav2 NavigateToPose) reports success. The ExecuteRskill action server catches this specifically and closes the goal with success=True.

This is NOT an error — it inherits ROSError only to stay inside the OpenRAL exception surface so it is greppable and discoverable via the standard hierarchy. It must be caught ONLY at the rskill_runner_node execute-callback boundary; everywhere else it is a programming bug to raise it.

ROSRuntimeError

Bases: ROSError

General runtime failure during skill execution or HAL operation.

ROSSafetyViolation

Bases: ROSError

A safety constraint was violated.

NEVER catch this silently. Catch only at the safety supervisor boundary, trigger an E-stop, and emit a structured incident log entry.

ROSWorkspaceViolation

Bases: ROSSafetyViolation

An action would move the robot outside its allowed workspace.

RSkillAction

Bases: str, Enum

Closed vocabulary of high-level action verbs an rSkill can perform.

Declared on RSkillManifest.actions so the reasoner's LLM tool palette can present each skill with a structured "what does it do" label, in addition to the free-form description and the slug. The LLM scores tools primarily on natural-language description, but a closed verb vocabulary lets the palette pre-filter and lets the schema be unit-testable. New entries are additive.

Categories (descriptive only — not part of the wire format):

• Manipulation primitives: PICK, PLACE, PICK_AND_PLACE, TRANSFER, GRASP, RELEASE.
• Articulated / contact-rich: OPEN, CLOSE, PUSH, PULL, SLIDE, INSERT, POUR, WIPE, ROTATE.
• Motion: REACH; LOOK (aim a camera at a point).
• Mobile (for mobile-manipulator embodiments): NAVIGATE.
• Social / expressive: WAVE, SHAKE.
• Generalist marker: GENERALIST for foundation / multi-task checkpoints (e.g. RoboCasa-365, DROID, MetaWorld-MT50). The palette surfaces a generalist skill for goals that don't match a specific verb.
• Perception / reasoning (non-actuating S2 kinds): DETECT (detector), QUERY (scene VLM), MONITOR (reward monitor), PLAN (playbook decision procedure). These verbs are registry/discovery metadata only — their skills are reached via read-only reasoner tools or system-prompt injection, never an ExecuteRskill dispatch.

RSkillEvalBenchmark

Bases: BaseModel

The benchmark suite a RSkillEvalResult was measured against.

Attributes:

Name	Type	Description
name	str	Canonical suite name ("LIBERO", "MetaWorld", "ALOHA", "PushT", "RoboCasa" …). Used by openral benchmark report to group rows.
dataset	str | None	Optional HF dataset identifier.
protocol	str	Free-text description of the eval protocol.
robot	str	robot_id the benchmark was evaluated on.
simulator	str	Free-text simulator description.

RSkillEvalResult

Bases: BaseModel

Pydantic model of a rskills/<id>/eval/<benchmark>.json file.

The results and baselines blocks are intentionally benchmark-specific — LIBERO has four sub-suites with success rates, MetaWorld carries an MT50 average, ALOHA has a single cube-transfer rate. We require the metadata blocks (source, benchmark, eval_config) but leave results as a free-form dict so each benchmark can declare its own keys.

Attributes:

Name	Type	Description
schema_version	str	Semver-ish version string for this on-disk format.
source	RSkillEvalSource	Provenance — paper / reproduction state.
benchmark	RSkillEvalBenchmark	Suite identity (name, robot, simulator).
eval_config	dict[str, object]	Free-form configuration the benchmark was run with (chunk size, image size, denoising steps, …).
results	dict[str, object]	Free-form per-task / per-suite success rates.
baselines	dict[str, object]	Optional free-form comparison numbers from prior work.
trace_id	str | None	Hex OTel trace id (32 chars) for the rollout that produced this result. Set by openral benchmark run / benchmark scene from the cli.command root span so reviewers can deep-link from rskills/<id>/eval/<benchmark>.json straight to the trace tree in Jaeger / Tempo. None when there is no trace to point at: paper-cited numbers (reproduced_locally: false), and local runs with no OTLP endpoint configured (no --dashboard, no OTEL_EXPORTER_OTLP_ENDPOINT), where observability is in no-op mode.

Example

RSkillEvalResult.model_validate_json(

'{"schema_version": "0.1", "source": {...}, ...}')

from_json(path: str) -> RSkillEvalResult classmethod

Load and validate a rskills/<id>/eval/<benchmark>.json file.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the JSON file.	required

Returns:

Type	Description
RSkillEvalResult	A validated RSkillEvalResult.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the JSON fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_json(cls, path: str) -> RSkillEvalResult:
    """Load and validate a ``rskills/<id>/eval/<benchmark>.json`` file.

    Args:
        path: Filesystem path to the JSON file.

    Returns:
        A validated ``RSkillEvalResult``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the JSON fails schema validation.
    """
    import json as _json  # noqa: PLC0415  # reason: deferred import

    with open(path, encoding="utf-8") as fh:
        data = _json.load(fh)
    return cls.model_validate(data)

RSkillEvalSource

Bases: BaseModel

Provenance for a benchmark result block.

Used inside RSkillEvalResult so consumers know where the numbers came from (paper vs local reproduction) without having to read the _comment line.

Attributes:

Name	Type	Description
paper	str	Plain-text title of the source paper / report.
arxiv	str | None	Optional arxiv URL or id.
model_variant	str	Which checkpoint variant the numbers describe (e.g. "SmolVLA (0.45B)").
evaluated_by	str	Free-text — "upstream authors" or a contributor.
reproduced_locally	bool	True only when OpenRAL CI / contributor re-ran the benchmark and the listed numbers match.
reproduction_planned	str | None	Optional plan for closing the gap when reproduced_locally is False.
reproduction_cli	dict[str, object] | str | None	Either a single command string or a structured {description, single_suite_example, all_suites, suite_max_steps, notes} block (matches the LIBERO eval JSON shape).
table	str | None	Optional pointer at a specific table in the source paper.
status	str | None	Optional state marker — "in_progress", "deferred", "reproduced" — for benchmarks that are not yet final.

RSkillInfo

Bases: BaseModel

Snapshot of a Skill's runtime state — published on /skill/<name>/info.

Attributes:

Name	Type	Description
name	str	rSkill name (e.g. "noop_skill").
version	str	SemVer string.
state	RSkillState	Current primary lifecycle state.
weights_loaded	bool	Whether model weights have been loaded into memory.
quantized	bool	Whether weights have been quantized for the target runtime.
warmed_up	bool	Whether the model has been warmed up (first inference run).
embodiment_tags	list[str]	Embodiment tags from the skill manifest.
role	Literal['s0', 's1', 's2']	rSkill role: "s0" (cerebellar), "s1" (fast policy), "s2" (slow reasoning).
latency_budget_ms	float | None	Maximum allowed inference latency in milliseconds.
last_inference_ms	float | None	Actual latency of the most recent step() call.
error_msg	str | None	Human-readable error description when state == "error".
stamp_ns	int	Timestamp of this snapshot in nanoseconds.

Example

info = RSkillInfo(name="hello", version="0.1.0", state=RSkillState.UNCONFIGURED) info.weights_loaded False info.state

RSkillLatencyBudget

Bases: BaseModel

Per-stage latency budget declared in rskill.yaml.

Attributes:

Name	Type	Description
per_chunk_ms	float	End-to-end step() budget. CI fails if exceeded on the reference host (CLAUDE.md §2).
warmup_ms	float | None	Maximum allowed warm-up time during activate().
load_ms	float | None	Maximum allowed weight-load time during configure().
max_execution_s	float | None	Total execution budget for one execute_rskill goal, in graph-clock seconds under use_sim_time, else wall. A VLA policy never self-terminates (only wrapped-ROS skills raise ROSRskillGoalSatisfied), so a deploy dispatch with deadline_s=0 (the LLM's "use the manifest default" sentinel) would otherwise run forever. The skill_runner resolves deadline_s<=0 to this value (or a global default) and aborts the goal when it lapses so the reasoner re-evaluates the outcome — CLAUDE.md §3 "Deadline fallback mandatory". None = fall back to the runner's global default.

RSkillLicensePosture

Bases: str, Enum

License posture surfaced at install time (CLAUDE.md §3 VLA license matrix).

RSkillManifest

Bases: BaseModel

Pydantic model of the rskill.yaml package manifest (V1).

On-disk schema for an rSkill HF Hub repo; loaded by capability-checking against a RobotDescription, then constructing a runtime Skill. Commercial-use posture is derived from license via is_commercial_use_allowed.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk format version; "0.1" today (CLAUDE.md §1.6).
name	str	HF Hub identifier <owner>/<repo>, e.g. "openral/rskill-pick-cube-so100".
version	str	SemVer string of the rSkill package (not the wrapped weights).
license	RSkillLicensePosture	License posture surfaced at install time; drives is_commercial_use_allowed.
role	Literal['s0', 's1', 's2']	rSkill slot — "s1" fast policy (the only slot loaded today), "s0" cerebellar realtime (reserved), "s2" slow reasoning (reserved).
model_family	ModelFamily | None	Closed VLA / policy family; drives the runner's adapter dispatch.
embodiment_tags	list[EmbodimentTag]	Robot embodiments targeted; must intersect the robot's tags. "custom" requires embodiment_extra.
embodiment_extra	EmbodimentExtra | None	Sensor + actuator surface for "custom"; required iff "custom" is tagged, None otherwise.
capabilities_required	dict[str, bool | float | int | str]	Per-flag RobotCapabilities requirements the target robot must satisfy.
sensors_required	list[SensorRequirement]	Sensor inputs the policy expects. See SensorRequirement.
actuators_required	list[ActuatorRequirement]	At least one ActuatorRequirement; matched against the robot's action_spec.control_mode.
runtime	RSkillRuntime	Preferred inference runtime.
quantization	QuantizationConfig	Default QuantizationConfig for this rSkill.
weights_uri	str | None	HF Hub revision-pinned URI or local path to weights; production deployments pin a SHA (CLAUDE.md operating principle 8).
chunk_size	int	Action-chunk length emitted per Skill.step.
latency_budget	RSkillLatencyBudget	Latency contract enforced by CI on the reference host (CLAUDE.md §2).
min_vram_gb	dict[QuantizationDtype, float] | None	Optional minimum VRAM (GB) per quantization dtype; informational only.
fallback_skill_id	str | None	rSkill id to substitute on failure (RFC §8.3 replanning ladder).
benchmarks	dict[BenchmarkName, float]	Canonical suite id → success rate [0.0, 1.0]; full per-task breakdown lives in the matching eval/<key>.json.
evaluated_tasks	list[str]	Benchmark task ids / families trained or validated on; gates a scene's task.id against it (empty = permissive).
sim_env_control_mode	str | None	Optional simulator controller override ("relative"/"absolute"); None = backend default.
policy_extras	dict[str, object]	Adapter-owned runtime knobs copied into VLASpec.extra at CLI composition.
paper_url	str | None	Canonical paper URL for this skill / family.
dataset_uri	str | None	HF Hub URI for the training dataset.
source_repo	str | None	HF Hub URI for the upstream weights repo.
description	str	REQUIRED. Short (1-500 char) summary surfaced to operators and the reasoner's tool palette.
default_prompt	str | None	Optional verbatim task string the checkpoint was trained on; fallback when the goal's prompt is empty.
actions	list[RSkillAction]	REQUIRED. Closed-vocabulary action verbs (≥1); see RSkillAction.
objects	list[str]	Optional free-form object keywords (discriminative hints for the LLM).
scenes	list[str]	Optional free-form scene / environment keywords.
processors	RSkillProcessors | None	Per-file lerobot PolicyProcessorPipeline URIs; required for modern families (smolvla/pi05/xvla/diffusion/rldx).
image_preprocessing	ImagePreprocessing | None	Per-checkpoint image knobs (flip, camera renames); None = schema defaults.
state_contract	StateContract | None	Per-checkpoint proprioception layout.
action_contract	ActionContract | None	Per-checkpoint action vector contract (dim + optional representation).
n_action_steps	int | None	Replay cadence; omit when equal to chunk_size.

Example

m = RSkillManifest( ... name="openral/rskill-pick-cube-so100", ... version="0.1.0", ... license=RSkillLicensePosture.APACHE_2_0, ... role="s1", ... kind="vla", ... model_family="smolvla", ... embodiment_tags=["so100_follower"], ... runtime=RSkillRuntime.PYTORCH, ... weights_uri="hf://lerobot/smolvla_base", ... chunk_size=16, ... latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0), ... description="Pick a cube on the SO-100 follower arm.", ... actions=[RSkillAction.PICK], ... objects=["cube"], ... scenes=["tabletop"], ... actuators_required=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ), ... ], ... processors=RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json", ... ), ... ) m.is_commercial_use_allowed True m.schema_version '0.1'

is_commercial_use_allowed: bool property

Derive commercial-use posture from license.

apache-2.0 / mit / bsd → True. Every other posture (including unknown) → False, conservatively. The loader consults this in rSkill._check_license; users wanting to deploy a non-commercial skill in a research context set OPENRAL_ALLOW_NONCOMMERCIAL=1 per CLAUDE.md §1.9.

is_scaffold_placeholder: bool property

True when this is an unresolved rskills/template/ scaffold.

Checks the identity fields (name, weights_uri, source_repo) for the RSKILL_TEMPLATE_SENTINELS. The scaffold parses as a valid manifest (so tests can load it) but is not a real, loadable skill: the reasoner palette must not offer it as dispatchable, and the publish gate rejects it. One predicate so callers don't re-derive the sentinel check.

active_min_vram_gb() -> float | None

Declared VRAM (GB) for this skill at its active quantization dtype.

Reads min_vram_gb[quantization.dtype] — the footprint the skill will actually use at load, since quantization.dtype pins the runtime format . Returns None when min_vram_gb is unset or has no entry for the active dtype (the size is simply not declared — the caller decides whether that is an error). See assert_vla_reward_fits for the pair check that consumes this.

Source code in python/core/src/openral_core/schemas.py

def active_min_vram_gb(self) -> float | None:
    """Declared VRAM (GB) for this skill at its **active** quantization dtype.

    Reads ``min_vram_gb[quantization.dtype]`` — the footprint the skill will
    actually use at load, since ``quantization.dtype`` pins the runtime format
    . Returns ``None`` when ``min_vram_gb`` is unset or has no entry
    for the active dtype (the size is simply not declared — the caller decides
    whether that is an error). See ``assert_vla_reward_fits`` for the pair
    check that consumes this.
    """
    if self.min_vram_gb is None:
        return None
    return self.min_vram_gb.get(self.quantization.dtype)

from_yaml(path: str) -> RSkillManifest classmethod

Load and validate an rskill.yaml from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the manifest YAML.	required

Returns:

Type	Description
RSkillManifest	A validated RSkillManifest.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RSkillManifest:
    """Load and validate an ``rskill.yaml`` from disk.

    Args:
        path: Filesystem path to the manifest YAML.

    Returns:
        A validated ``RSkillManifest``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
    """
    return _load_yaml_model(cls, path)

RSkillProcessors

Bases: BaseModel

Explicit lerobot PolicyProcessorPipeline artefact pointers.

Closes Gap 1 + Gap 3 of the rSkill self-containment audit. Replaces the implicit snapshot_download(repo_id) → make_pre_post_processors(pretrained_path=...) fetch path used today by the SmolVLA / pi05 / xVLA / Diffusion / modern-ACT adapters. Two files are required because lerobot's pipeline ships them as a pair: a preprocessor that normalises the policy input batch and a postprocessor that un-normalises the action chunk.

Each URI must point at a specific file (the implicit-snapshot shape hf://owner/repo with no file tail is rejected). Two URI schemes accepted:

• hf://owner/repo[@rev]/path/to/file.json — file inside an HF Hub repo, revision-pinnable.

Attributes:

Name	Type	Description
preprocessor_uri	str	Per-file URI to the policy preprocessor JSON.
postprocessor_uri	str	Per-file URI to the policy postprocessor JSON.

Example

RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_libero/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_libero/policy_postprocessor.json", ... ) # doctest: +ELLIPSIS RSkillProcessors(preprocessor_uri='hf://...preprocessor.json', postprocessor_uri='hf://...postprocessor.json')

RSkillRuntime

Bases: str, Enum

Inference runtime hint declared in the manifest (RFC §7.3).

RSkillState

Bases: str, Enum

Primary lifecycle states for a Skill node.

Matches the ROS 2 Managed Node state machine with openral extensions.

States

unconfigured Initial state. Weights are not loaded. inactive Configured and weights loaded; not yet warmed up. Accepts no actions. active Ready to execute. step() may be called. finalized Terminal state after shutdown(). Cannot transition further. error Unrecoverable failure. Requires external intervention.

ReasonerEndpointPreset

Bases: NamedTuple

Everything a named endpoint implies beyond its URL.

dialect makes OPENRAL_REASONER_DIALECT unnecessary for a named endpoint — a preset knows its own wire format, and the escape hatch only has to ask when the operator supplies a bare URL nobody can classify.

Lives in openral_core next to REASONER_MODELS so the factory (openral_reasoner.tool_use) and openral doctor consume the SAME table — doctor used to hand-mirror it to avoid importing the optional reasoner package, and the mirror drifted twice.

ReasonerModel

Bases: BaseModel

A curated reasoner LLM — a member of REASONER_MODELS (ADR-0088).

Membership in REASONER_MODELS is the "tested for robotics tool calling" signal: every entry has been validated to reliably emit the discriminated ReasonerToolCall union (per-skill execute_rskill__* tools, nested GroundedSubtask, tool_choice honoured). There is deliberately no robotics_tool_calling flag — an untested model is simply not in the registry, and is reachable only through the uncurated escape hatch with a warning.

This model reorganises reasoner selection away from the location-leaking OPENRAL_REASONER_LLM_PROVIDER enum it replaced (ADR-0088; that spelling was removed in 0.3.0): the primary choice is the model; where it runs is the orthogonal, honest endpoint axis.

Attributes:

Name	Type	Description
id	str	Catalog key ("cosmos3-edge", "claude-opus-4-8"). The value of OPENRAL_REASONER_MODEL.
display_name	str	Human-facing label.
dialect	ReasonerDialect	Which wire client speaks to this model (anthropic uses the Messages API; openai uses Chat Completions — vLLM, OpenRouter, the Cosmos sidecar, etc. are all openai).
hosting	ReasonerHosting	cloud (vendor-run), managed_local (OpenRAL spawns the server), or byo_local (operator-run local server).
served_model_id	str	The id the endpoint expects, which may differ from id ("openai/gpt-5.5" on OpenRouter, "nvidia/Cosmos3-Edge" on the sidecar).
default_endpoint	str | None	Vendor URL, loopback URL, the REASONER_MANAGED_ENDPOINT sentinel, or None (the dialect's vendor default — e.g. the Anthropic API).
auth_required	bool	Whether the resolved endpoint needs an API key.
tool_choice	Literal['required', 'auto']	"required" forces one tool call per tick (the reasoner contract); "auto" for endpoints that reject "required".
max_tokens_default	int | None	Optional completion-token cap applied when the operator sets none — bounds a metered gateway's up-front reservation for reasoning models (GPT-5.x). None sends no cap.
min_gpu_vram_gb	float | None	BF16-resident VRAM floor for a comfortable local fit. None for cloud models (not a local-compute question).
required_dtype	QuantizationDtype | None	Dtype the local runtime must support, matched against ComputeSpec.gpu_supported_dtypes. None for cloud models.
weights_license	str | None	Informational weights-license posture for local models (§1.9), e.g. "OpenMDW-1.1". None for cloud API models (OpenRAL never holds their weights).

Example

m = REASONER_MODELS["cosmos3-edge"] m.dialect, m.hosting, m.auth_required ('openai', 'managed_local', False) REASONER_MODELS["claude-opus-4-8"].dialect 'anthropic'

is_local: bool property

True for managed_local / byo_local (needs local compute).

ReasonerTimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_REASONER_TIMEOUT — the LLM call missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['reasoner_timeout']	Discriminator (always "reasoner_timeout").
model	str	Model identifier (e.g. "claude-opus-4-7").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

RecallObjectMatch

Bases: BaseModel

One ranked match from a RecallObjectQuery.

Attributes:

Name	Type	Description
node_id	str	The matched node's id.
label	str	The matched node's label.
pose	Pose6D	The object's recalled pose (map frame).
score	float	Match score in [0, 1].
last_seen_ns	int	When the object was last observed, in nanoseconds.
approach	ApproachViewpoint | None	Optional computed camera-facing approach viewpoint.
inside_container_id	str | None	Set when the object is inside an occluding container that must be opened first; None otherwise.

RecallObjectQuery

Bases: BaseModel

Read-only query to recall a remembered object.

At least one of text / label must be non-empty. text is matched against node embeddings when an embedder is configured, otherwise matching falls back to label.

Attributes:

Name	Type	Description
text	str	Free-text query (open-vocabulary match).
label	str	Exact label match.
near	Pose6D | None	Optional pose to bias results toward (proximity).
max_age_ns	int | None	Optional recency filter — drop nodes whose last_seen_ns is older than this many nanoseconds before "now".
limit	int	Maximum number of matches to return.

RecallObjectResult

Bases: BaseModel

Result of a RecallObjectQuery.

Attributes:

Name	Type	Description
matches	list[RecallObjectMatch]	Ranked matches (possibly empty — an empty result is how the query reports "unknown"; callers raise / handle ROSObjectNotInMemory rather than fabricating a pose).

RecallObjectTool

Bases: _ReasonerToolBase

Tool variant (read-only) — recall a remembered object.

Queries the scene-graph spatial memory and returns the object's map-frame pose plus a camera-facing approach viewpoint and any occluding container to the reasoner's next reasoning step. Like every variant it holds no authority over actuation — it only reads memory. The dispatch that runs the query and feeds the result back to the LLM is wired separately (this is the typed contract).

Attributes:

Name	Type	Description
tool	Literal['recall_object']	Discriminator (always "recall_object").
query	str	Free-text or label naming the object to recall (e.g. "the red mug"). Mapped to a RecallObjectQuery at dispatch.
limit	int	Maximum number of ranked matches to return.

ReloadGstPipelineTool

Bases: _ReasonerToolBase

Tool variant — swap a sensor's GStreamer pipeline at runtime.

Dispatch: service call on /openral/sensors/<sensor_id>/reload_pipeline with the pipeline YAML payload. Lets the reasoner tune perception (switch resolution, enable an nvinfer leg, swap an RTSP source) without redeploying the runtime.

Attributes:

Name	Type	Description
tool	Literal['reload_gst_pipeline']	Discriminator (always "reload_gst_pipeline").
sensor_id	str	SensorSpec.name of the camera whose pipeline is being reloaded. Validated against the active runtime's sensor catalog at dispatch time.
pipeline_yaml	str	Full YAML body of the new SensorReaderConfig. The sensor node validates this against the Pydantic schema before accepting the swap.

ResolvePlaceQuery

Bases: BaseModel

Read-only query to resolve a place/room/agent reference to a goal.

Attributes:

Name	Type	Description
reference	str	Free-text, id, or label of the target (e.g. "kitchen", "where I was standing", a node id).
kind	SpatialNodeKind | None	Optional node-kind filter (room / place / agent).

ResolvePlaceResult

Bases: BaseModel

Result of a ResolvePlaceQuery.

Attributes:

Name	Type	Description
node_id	str	The resolved node's id.
goal	Pose6D	Navigation goal pose (map frame).
path_node_ids	list[str]	Ordered traversable_to path of node ids from the robot's current place to the goal (empty when no path is needed or known).

ResolvePlaceTool

Bases: _ReasonerToolBase

Tool variant (read-only) — resolve a place/room/agent to a goal.

Queries the scene-graph memory for a navigation goal pose plus a traversable_to path. Read-only; holds no authority over actuation. Dispatch + result-return are wired separately.

Attributes:

Name	Type	Description
tool	Literal['resolve_place']	Discriminator (always "resolve_place").
reference	str	Free-text, id, or label of the target (e.g. "the kitchen", "where I was standing"). Mapped to a ResolvePlaceQuery.

RewardContract

Bases: BaseModel

Manifest contract for kind: "reward" rSkills.

Carries the configuration a robotic reward / progress-monitor model (e.g. Robometer-4B, a Qwen3-VL-4B reward foundation model) needs to score a rollout. A reward skill runs in parallel with a kind: "vla" policy, continuously ingesting the VLA's camera frames into a rolling window, and emits per-frame progress ∈ progress_range and per-frame success probability. The Reasoner queries it on demand (QueryTaskProgressTool) to decide whether to continue, escalate to a scene VLM, advance, or replan. The signal is advisory only — it never actuates and never gates motors (CLAUDE.md §1.1).

Required when RSkillManifest.kind is "reward"; forbidden for all other kinds (enforced by RSkillManifest._check_kind_consistency). Like detector / vlm, a reward skill is a pure perception consumer: it emits no Action chunks and requires no actuators.

Attributes:

Name	Type	Description
progress_range	tuple[float, float]	(min, max) of the normalized per-frame progress scalar. Default (0.0, 1.0) — Robometer's discrete/binned mode (see num_bins) emits progress already in [0, 1].
success_threshold	float	Per-frame success probability at/above which the frame is considered a task success. In [0.0, 1.0]; default 0.5.
preference	bool	Whether the model also exposes a trajectory-preference head (Robometer does). Default False — the progress/success path is the Reasoner-facing contract; preference is future work.
backend	Literal['robometer', 'topreward']	Which reward runtime scores this skill. "robometer" (default) → the fine-tuned Robometer RewardModel in-process; "topreward" → the zero-shot TOPReward monitor (lerobot TOPRewardModel, per-frame progress from a prefix sweep of P("True" | video, instruction)), run in-process. Selects the backend in build_reward_monitor; existing manifests default to "robometer" so behavior is unchanged.
frame_window_s	float	Length of the rolling frame buffer in seconds. The node evicts frames older than this relative to the newest. Must be > 0.
target_fps	float	Frame-sampling rate fed to the model (Robometer's example uses 3 fps). Must be > 0. This is an S2-cadence monitor, not a per-control-step signal.
num_bins	int	Discrete-mode bin count for the progress head. Robometer's discrete mode yields per-frame normalized progress in [0, 1]; continuous mode yields raw regression values. Must be > 0; default 100.
instruction_required	bool	Whether a natural-language task instruction must accompany the frames (Robometer requires one). Default True.
check_floor	float	Below this progress value the attempt is clearly not done → no VLM adjudication, straight to the replanning ladder. Must be ≤ success_threshold; default 0.4 (calibrated for robometer's loose/general eval distribution that wanders ~0.55-0.78).
plateau_window_s	float	Trailing window (seconds) over which progress_trend ≈ 0 indicates the policy has stopped getting closer to the goal. Default 3.0.
plateau_tolerance	float	ε band absorbing critic noise when deciding whether the trend is flat. Default 0.05.
default_patience_s	float	Baseline execution ceiling (seconds) used as a backstop before the Reasoner escalates. Default 30.0.

Example

c = RewardContract(frame_window_s=8.0, target_fps=3.0) c.progress_range (0.0, 1.0) c.success_threshold 0.5 c.check_floor 0.4

RobotCapabilities

Bases: BaseModel

Physical capability flags for a robot body.

Describes what the robot can do (sensors, actuation, embodiment tags). Runtime compute / GPU properties live on ComputeSpec and are attached to the RobotDescription via its compute field.

Attributes:

Name	Type	Description
locomotion	list[LocomotionKind]	Locomotion types available.
can_lift_kg	float	Maximum payload in kg.
has_dexterous_hands	bool	Whether dexterous hands are present.
has_tactile	bool	Whether tactile sensing is available.
has_force_control	bool	Whether force/impedance control is supported.
has_vision	bool	Whether camera(s) are present.
has_lidar	bool	Whether LiDAR is present. Gates the 2D-lidar slam_toolbox backend.
has_vision_slam	bool	Whether the robot should run camera-based visual SLAM (cuVSLAM + nvblox) for localization/mapping, for robots that lack a lidar. Gates the visual SLAM backend. Independent of has_lidar; when both are set the lidar backend wins (it does not require an AI depth model).
has_audio	bool	Whether audio I/O is present.
bimanual	bool	Whether the robot has two arms.
supported_control_modes	list[ControlMode]	List of supported ControlMode values.
supported_vla_embodiments	list[str]	VLA embodiment IDs this robot can run.
embodiment_tags	list[str]	Short tags mapping to VLA heads / dataset splits.

RobotDescription

Bases: BaseModel

Top-level robot manifest — one per robot, published to HuggingFace Hub.

Attributes:

Name	Type	Description
name	str	Robot name, e.g. "so100_follower".
embodiment_kind	EmbodimentKind	Top-level kinematic class.
assets	AssetRefs	Unified URDF / MJCF / SRDF reference block (openral_core.assets.resolve_asset grammar). Empty by default.
base_frame	str	Base link tf2 frame name.
odom_frame	str	Odometry tf2 frame name.
map_frame	str	Map tf2 frame name.
joints	list[JointSpec]	List of joint specifications.
end_effectors	list[EndEffectorSpec]	List of end-effector specifications.
sensors	list[SensorSpec]	List of individual sensor specs.
sensor_bundles	list[SensorBundle]	List of multi-modal sensor bundles.
capabilities	RobotCapabilities	Capability flags for skill matching.
safety	SafetyEnvelope	Safety envelope constraints.
ros2_namespace	str	ROS 2 namespace prefix.
middleware	Literal['fastdds', 'cyclonedds', 'zenoh']	ROS 2 middleware selection.
onboard_compute	dict[str, object]	Onboard compute descriptors.
sdk_kind	Literal['open', 'closed_with_api', 'closed']	Whether the SDK is open or closed.
hal	HalEntrypoints	Simulation + real-hardware HAL import strings (deploy sim uses hal.sim, deploy run uses hal.real).
observation_spec	ObservationSpec | None	VLA observation configuration.
action_spec	ActionSpec | None	VLA action configuration.
sim	SimDescription | None	Optional MuJoCo wiring for openral_hal.MujocoArmHAL; lets the HAL build entirely from the manifest.
scene_defaults	SceneDefaults | None	Optional per-robot scene rendering defaults (TopCameraDefaults).
base_joints	list[str] | None	Optional ordered [forward, side, yaw] JointSpec.name refs for a planar mobile base; None for fixed-base manipulators.
collision_geometry	list[LinkCollisionGeometry]	Per-link convex collision primitives (capsules / spheres) for the safety kernel. Empty by default.
allowed_collision_pairs	list[tuple[str, str]]	Link-name pairs excluded from self-collision (from the SRDF disable_collisions block on real robots).
compute_edge	ComputeSpec | None	Compute spec for the on-robot accelerator; falls back to compute_local when None.
compute_local	ComputeSpec | None	Compute spec for the attached workstation / laptop.
compute_cloud	ComputeSpec | None	Optional remote compute endpoint (SSH or HTTPS).
schema_version	Literal['0.2']	On-disk schema version; only "0.2" loads. Any other version (including "0.1") is refused with a ROSConfigError.

Example

desc = RobotDescription( ... name="smoke_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["smoke"], ... ), ... safety=SafetyEnvelope(), ... ) assert desc.name == "smoke_robot"

control_rate_hz: float | None property

The robot's declared control rate, action_spec.control_freq_hz, or None.

The one place a robot's rate lives: the skill runner ticks at it, the HAL node publishes proprio at it, the real ros2_control HAL derives every trajectory deadline from it, and the recorder stamps it as fps.

Example

RobotDescription.from_yaml("robots/openarm/robot.yaml").control_rate_hz 30.0

lidar_sensor: SensorSpec | None property

The first declared 2-D LiDAR / scan sensor, or None.

The panda_mobile HAL synthesises a sensor_msgs/ LaserScan from MuJoCo ray-casts; its beam count (SensorSpec.n_channels), range (SensorSpec.range_min_m / SensorSpec.range_max_m) and rate (SensorSpec.rate_hz) live on this descriptor so openral deploy sim and the HAL lifecycle node read one source of truth from robot.yaml instead of each hardcoding scan params.

from_yaml(path: str) -> RobotDescription classmethod

Load and validate a RobotDescription YAML manifest from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the robot.yaml file.	required

Returns:

Type	Description
RobotDescription	A validated RobotDescription.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Example

RobotDescription.from_yaml("robots/franka_panda/robot.yaml")

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotDescription:
    """Load and validate a ``RobotDescription`` YAML manifest from disk.

    Args:
        path: Filesystem path to the ``robot.yaml`` file.

    Returns:
        A validated ``RobotDescription``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.

    Example:
        >>> # RobotDescription.from_yaml("robots/franka_panda/robot.yaml")
    """
    return _load_yaml_model(cls, path)

grasp_contact_links() -> tuple[str, ...]

The links a grasp may touch: every role: gripper jaw, follower fingers included.

The role: gripper joints' child_link first (the leader jaws the HAL names in a GraspDeclaration), then the child_link of every mimic_joints row following one of those joints — the other jaw of the same gripper, which the kernel must exempt in the grasp region too (ADR-0115). The launch passes this as the kernel's grasp_contact_links.

Example

d = RobotDescription.from_yaml("robots/openarm/robot.yaml") d.grasp_contact_links()[:2] ('openarm_left_finger_pair', 'openarm_right_finger_pair')

Source code in python/core/src/openral_core/schemas.py

def grasp_contact_links(self) -> tuple[str, ...]:
    """The links a grasp may touch: every ``role: gripper`` jaw, follower fingers included.

    The ``role: gripper`` joints' ``child_link`` first (the leader jaws the
    HAL names in a ``GraspDeclaration``), then the ``child_link`` of every
    ``mimic_joints`` row following one of those joints — the other jaw of
    the same gripper, which the kernel must exempt in the grasp region too
    (ADR-0115). The launch passes this as the kernel's ``grasp_contact_links``.

    Example:
        >>> d = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> d.grasp_contact_links()[:2]
        ('openarm_left_finger_pair', 'openarm_right_finger_pair')
    """
    grippers = [j for j in self.joints if j.role == "gripper"]
    names = {j.name for j in grippers}
    return tuple(j.child_link for j in grippers) + tuple(
        m.child_link for m in self.mimic_joints if m.mimics in names
    )

hand_links() -> tuple[str, ...]

The robot's hand links: the jaws, the link carrying them, and the wrist behind it.

Every role: gripper jaw and mimic follower (grasp_contact_links), then — once per hand — the mount the hand's leader jaws hang off (a chained finger climbed to its arm, as in gripper_hands): the link that carries the gripper; then the link that carrier hangs off, the wrist (never a root link: a gripper mounted on the robot's base adds nothing). The launch passes this as the safety kernel's workspace_contact_links (issue #349): the only links the manipulation-workspace contact rule ever relaxes. The wrist is in because a hand working at a tote's rim brings it within one voxel of the rim (8/8 deploy-sim stops, r23: PhysX 10 mm clear, the 15 mm map cells read -4 mm); the forearm and everything behind it never are. Derived from the manifest, never configured by name.

Example

d = RobotDescription.from_yaml("robots/openarm/robot.yaml") d.hand_links()[-2:] ('openarm_left_link6', 'openarm_right_link6')

Source code in python/core/src/openral_core/schemas.py

def hand_links(self) -> tuple[str, ...]:
    """The robot's hand links: the jaws, the link carrying them, and the wrist behind it.

    Every ``role: gripper`` jaw and mimic follower (``grasp_contact_links``), then —
    once per hand — the mount the hand's leader jaws hang off (a chained finger
    climbed to its arm, as in ``gripper_hands``): the link that carries the gripper;
    then the link that carrier hangs off, the wrist (never a root link: a gripper
    mounted on the robot's base adds nothing). The launch passes this as the safety
    kernel's ``workspace_contact_links`` (issue #349): the only links the
    manipulation-workspace contact rule ever relaxes. The wrist is in because a hand
    working at a tote's rim brings it within one voxel of the rim (8/8 deploy-sim
    stops, r23: PhysX 10 mm clear, the 15 mm map cells read -4 mm); the forearm and
    everything behind it never are. Derived from the manifest, never configured by
    name.

    Example:
        >>> d = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> d.hand_links()[-2:]
        ('openarm_left_link6', 'openarm_right_link6')
    """
    grippers = [j for j in self.joints if j.role == "gripper"]
    parent_of = {j.child_link: j.parent_link for j in grippers}
    joint_parent = {j.child_link: j.parent_link for j in self.joints}
    links = list(self.grasp_contact_links())
    carriers: list[str] = []
    for j in grippers:
        carrier = _gripper_mount(parent_of, j.parent_link)
        if carrier not in links:
            links.append(carrier)
            carriers.append(carrier)
    for carrier in carriers:
        wrist = joint_parent.get(carrier)
        # A root link (no joint of its own) is the robot's base, never a wrist.
        if wrist is not None and wrist in joint_parent and wrist not in links:
            links.append(wrist)
    return tuple(links)

nav2_footprint_param() -> str

This base's Nav2 footprint parameter string.

nav2_costmap_2d reads footprint as a string of [[x, y], ...] vertices and treats both "" and "[]" as "no polygon — use robot_radius". So a robot that declares footprint_polygon gets its measured outline, and one that declares only footprint_radius gets "[]", which keeps Nav2 on its cheaper circular path rather than silently inheriting the base param file's polygon (some other robot's shape).

A polygon matters beyond accuracy: only a polygon footprint can express a carried payload, which is what openral_nav2_bringup's footprint publisher pushes onto the costmaps' footprint topic while an object is attached.

Example

from openral_core import RobotDescription RobotDescription.from_yaml("robots/panda_mobile/robot.yaml").nav2_footprint_param() '[[0.35, 0.25], [-0.35, 0.25], [-0.35, -0.25], [0.35, -0.25]]'

Source code in python/core/src/openral_core/schemas.py

def nav2_footprint_param(self) -> str:
    """This base's Nav2 ``footprint`` parameter string.

    ``nav2_costmap_2d`` reads ``footprint`` as a string of
    ``[[x, y], ...]`` vertices and treats **both** ``""`` and ``"[]"``
    as "no polygon — use ``robot_radius``". So a robot that declares
    ``footprint_polygon`` gets its measured outline, and one that
    declares only ``footprint_radius`` gets ``"[]"``, which keeps
    Nav2 on its cheaper circular path rather than silently inheriting
    the base param file's polygon (some other robot's shape).

    A polygon matters beyond accuracy: only a polygon footprint can
    express a carried payload, which is what
    ``openral_nav2_bringup``'s footprint publisher pushes onto the
    costmaps' ``footprint`` topic while an object is attached.

    Example:
        >>> from openral_core import RobotDescription
        >>> RobotDescription.from_yaml("robots/panda_mobile/robot.yaml").nav2_footprint_param()
        '[[0.35, 0.25], [-0.35, 0.25], [-0.35, -0.25], [0.35, -0.25]]'
    """
    if not self.footprint_polygon:
        return "[]"
    inner = ", ".join(f"[{x:g}, {y:g}]" for x, y in self.footprint_polygon)
    return f"[{inner}]"

nav2_param_overrides() -> dict[str, str]

Nav2 param substitutions derived from this robot's base props.

Lets the Nav2 bringup stay generic: instead of hand-vendoring a per-robot nav2_<robot>.yaml, the launch rewrites a shared base param file with these key→value substitutions. Maps footprint_radius → robot_radius (collision envelope), the costmap inflation_radius (the circumscribed radius — footprint_radius or the farthest footprint_polygon vertex, whichever is larger — plus NAV2_INFLATION_CLEARANCE_M, so it stays ≥ the circumscribed radius Nav2 derives from the footprint it actually uses — otherwise Nav2 errors and falls back to slow full-footprint collision checks), footprint_polygon → the costmap footprint polygon (see nav2_footprint_param), and base_kinematics → the MPPI motion_model. Returns an empty dict for fixed-base arms (no mobile base → no Nav2). Velocity limits remain Nav2 tuning in the base param file; nav2.launch.py clamps them to safety.max_base_* separately (it needs the base file's values).

Source code in python/core/src/openral_core/schemas.py

def nav2_param_overrides(self) -> dict[str, str]:
    """Nav2 param substitutions derived from this robot's base props.

    Lets the Nav2 bringup stay generic: instead of
    hand-vendoring a per-robot ``nav2_<robot>.yaml``, the launch
    rewrites a shared base param file with these key→value
    substitutions. Maps ``footprint_radius`` → ``robot_radius``
    (collision envelope), the costmap ``inflation_radius`` (the
    circumscribed radius — ``footprint_radius`` or the farthest
    ``footprint_polygon`` vertex, whichever is larger — plus
    ``NAV2_INFLATION_CLEARANCE_M``, so it stays ≥ the circumscribed
    radius Nav2 derives from the footprint it actually uses —
    otherwise Nav2 errors and falls back to slow full-footprint
    collision checks), ``footprint_polygon`` →
    the costmap ``footprint`` polygon (see
    ``nav2_footprint_param``), and ``base_kinematics`` →
    the MPPI ``motion_model``. Returns an empty dict for fixed-base
    arms (no mobile base → no Nav2). Velocity limits remain Nav2
    tuning in the base param file; ``nav2.launch.py`` clamps them to
    ``safety.max_base_*`` separately (it needs the base file's values).
    """
    overrides: dict[str, str] = {}
    if self.footprint_radius is not None or self.footprint_polygon:
        # Emitted even for a radius-only robot, as Nav2's "[]" sentinel:
        # leaving the key unrewritten would hand that robot whatever
        # polygon the shared base file happens to ship.
        overrides["footprint"] = self.nav2_footprint_param()
        circumscribed = max(
            [self.footprint_radius or 0.0]
            + [math.hypot(x, y) for x, y in self.footprint_polygon or []]
        )
        overrides["inflation_radius"] = f"{circumscribed + NAV2_INFLATION_CLEARANCE_M:.3f}"
    if self.footprint_radius is not None:
        overrides["robot_radius"] = str(self.footprint_radius)
    if self.base_kinematics is not None:
        overrides["motion_model"] = {
            "omni": "Omni",
            "holonomic": "Omni",
            "differential": "DiffDrive",
            "ackermann": "Ackermann",
        }[self.base_kinematics]
    return overrides

validate_for_e2e_pipeline() -> None

Assert this manifest carries every field the e2e ROS graph needs.

The C++ safety kernel (cpp/openral_safety_kernel) reads per-joint position_limits / velocity_limit / effort_limit + the global safety: block. Per-joint limit fields are optional on JointSpec for sim-only robots, but they become mandatory the moment you ask openral deploy sim to bring up the kernel against this robot.

This method makes the e2e contract explicit and fails loud listing every missing field. Pair it with openral_safety.envelope_loader.compute_intersection(robot, skill=None) for the actual envelope synthesis — this method only validates.

Raises:

Type	Description
ROSConfigError	If any actuated joint is missing position_limits, velocity_limit, or effort_limit. The error lists every missing field at once so the operator does not have to fix one, re-run, fix the next.

Source code in python/core/src/openral_core/schemas.py

def validate_for_e2e_pipeline(self) -> None:
    """Assert this manifest carries every field the e2e ROS graph needs.

    The C++ safety kernel (``cpp/openral_safety_kernel``)
    reads per-joint ``position_limits`` / ``velocity_limit`` /
    ``effort_limit`` + the global ``safety:`` block. Per-joint
    limit fields are *optional* on ``JointSpec`` for sim-only
    robots, but they become mandatory the moment you ask
    ``openral deploy sim`` to bring up the kernel against this robot.

    This method makes the e2e contract explicit and fails loud
    listing every missing field. Pair it with
    ``openral_safety.envelope_loader.compute_intersection(robot,
    skill=None)`` for the actual envelope synthesis — this method
    only validates.

    Raises:
        ROSConfigError: If any actuated joint is missing
            ``position_limits``, ``velocity_limit``, or
            ``effort_limit``. The error lists every missing field
            at once so the operator does not have to fix one,
            re-run, fix the next.
    """
    missing: list[str] = []
    for joint in self.joints:
        if joint.joint_type not in (
            JointType.REVOLUTE,
            JointType.PRISMATIC,
            JointType.CONTINUOUS,
        ):
            continue
        if joint.position_limits is None:
            missing.append(f"joints[{joint.name}].position_limits")
        if joint.velocity_limit is None:
            missing.append(f"joints[{joint.name}].velocity_limit")
        if joint.effort_limit is None:
            missing.append(f"joints[{joint.name}].effort_limit")
    if missing:
        raise ROSConfigError(
            f"RobotDescription({self.name!r}) is missing fields required by "
            f"the e2e safety kernel: {missing}. Set them on the matching "
            "JointSpec(s) in robots/<robot_id>/robot.yaml."
        )

RobotUnit

Bases: BaseModel

One physical unit / host of a robot type: robots/<robot_id>/units/<unit>.yaml.

Selected by $OPENRAL_ROBOT_UNIT or DeployScene.robot_unit; see resolve_sensor_overlays.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk schema version.
robot_id	str	The robots/<robot_id> directory this unit belongs to.
unit	str	The unit name, equal to the file stem.
sensors	list[SensorOverlay]	Overlays for the robot manifest's sensors.

Example

RobotUnit(robot_id="so101_follower", unit="bench_laptop").sensors []

from_yaml(path: str) -> RobotUnit classmethod

Load and validate a unit overlay YAML file.

Example

RobotUnit.from_yaml("robots/openarm/units/orin.yaml").unit 'orin'

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotUnit:
    """Load and validate a unit overlay YAML file.

    Example:
        >>> RobotUnit.from_yaml("robots/openarm/units/orin.yaml").unit
        'orin'
    """
    return _load_yaml_model(cls, path)

RosIntegration

Bases: BaseModel

Wiring for an rSkill that wraps an existing ROS 2 action or service.

Populated when RSkillManifest.kind is "ros_action" or "ros_service". The ROSActionRskill adapter reads this block at configure time to build the right action / service client on the host LifecycleNode.

Attributes:

Name	Type	Description
package	str	IDL package name (e.g. "moveit_msgs", "nav2_msgs"). Imported lazily so manifests for un-installed ROS packages still parse — the loader raises ROSConfigError only when the skill is actually resolved.
interface_type	str	Action / service type name inside package (e.g. "MoveGroup", "NavigateToPose").
interface_name	str	Fully-qualified ROS path of the running server (e.g. "/move_action", "/navigate_to_pose").
result_trajectory_field	str | None	Dotted accessor pointing at a trajectory_msgs/JointTrajectory inside the action result (e.g. "planned_trajectory.joint_trajectory" for MoveIt's MoveGroup). When set, the adapter replays one waypoint per step call onto /openral/candidate_action (so the safety supervisor + HAL see every position). When None the action is treated as result-only: the adapter awaits the action result, raises ROSRskillGoalSatisfied on success, and never emits an Action chunk. This second mode covers wrapped ROS packages that drive actuators on their own (Nav2's behaviour tree publishes cmd_vel directly).
default_goal_json	str	JSON dict literal used to construct the goal message. v1 hard-codes the target here — the structured-prompt path that lowers LLM-emitted JSON into ExecuteRskill.Goal.prompt_metadata_json is a follow-up. REQUIRED so that a wrapped skill is always invocable end-to-end without per-call schema work.
ros_dependencies	list[str]	Apt / colcon packages the operator must have installed for the wrapped server to be reachable (e.g. "ros-${ROS_DISTRO}-moveit"). Surfaced by ral skill check and quoted in ROSConfigError messages when the action client fails to connect.

Example

ri = RosIntegration( ... package="moveit_msgs", ... interface_type="MoveGroup", ... interface_name="/move_action", ... result_trajectory_field="planned_trajectory.joint_trajectory", ... default_goal_json='{"target_joint_positions": [0, 0, 0, 0, 0, 0, 0]}', ... ros_dependencies=["ros-${ROS_DISTRO}-moveit"], ... ) ri.package 'moveit_msgs'

goal_builder: Literal['joint', 'pose', 'look_at'] | None = None class-attribute instance-attribute

Optional goal-lowering adapter over the shared ROSActionRskill MoveGroup engine. None (the default) sends default_goal_json + LLM overrides verbatim (the raw-IDL escape hatch). The named builders consume a typed block from the merged goal and lower it into MoveGroup constraints:

• "joint" — a joint block (positions, joint_names) → joint_constraints (JointGoalRskill).
• "pose" — a pose block (position, orientation as a quaternion array with quaternion_order, link_name) → MoveGroup position + orientation constraints for a Cartesian end-effector goal (PoseGoalRskill).
• "look_at" — a look_at block (target_xyz, camera, …) → the same pose-constraint lowering, with the gaze pose computed from the camera's live TF pose (LookAtRskill, a specialisation of the pose builder).

RunResult

Bases: BaseModel

Aggregated summary returned by InferenceRunner.run.

Attributes:

Name	Type	Description
n_ticks	int	Total ticks executed.
success	bool | None	Task success when the runner has a success signal (sim, scripted goal). None for open-ended hardware runs.
budget_violations	int	Count of ticks whose tick_ms exceeded the rSkill manifest's RSkillLatencyBudget.per_chunk_ms.
avg_inference_ms	float	Mean of TickResult.inference_ms.
p99_inference_ms	float	99th-percentile of TickResult.inference_ms.
avg_tick_ms	float	Mean of TickResult.tick_ms.
p99_tick_ms	float	99th-percentile of TickResult.tick_ms.
trace_id	str | None	OTel trace id (hex) of the run's root span. None when tracing is not configured.
save_dir	str | None	Directory where artefacts were written, when configured.
metadata	dict[str, object]	Free-form metadata (runner id, host, git SHA, …).

SafetyEnvelope

Bases: BaseModel

Safety constraints handed to the C++ safety kernel.

Not every field is enforced yet. Several are declared, not yet enforced by the kernel: max_force_n (issue #373), max_ee_accel_m_s2, contact_force_threshold_n and deadman_required are passed to the kernel as ROS parameters but no check consults them, and no_go_zones, e_stop_topic, e_stop_qos, cycle_time_violation_threshold_ms and human_in_loop_required are not read by any runtime component. Each attribute below says which case it is in.

Attributes:

Name	Type	Description
workspace_box_min_xyz	tuple[float, float, float] | None	Lower corner of allowed workspace (m). The kernel checks it only against CARTESIAN_POSE chunk positions.
workspace_box_max_xyz	tuple[float, float, float] | None	Upper corner of allowed workspace (m). Same scope as workspace_box_min_xyz.
no_go_zones	list[dict[str, object]]	List of polygon definitions (dicts with 'vertices'). Declared, not yet enforced: not passed to the kernel and not read by any runtime component.
max_ee_speed_m_s	float	Maximum end-effector linear speed in m/s. Also the C++ safety kernel's CARTESIAN_TWIST linear bound.
max_ee_accel_m_s2	float	Maximum end-effector acceleration in m/s². Declared, not yet enforced by the kernel: passed as a ROS parameter, but no check compares an acceleration against it.
max_joint_speed_factor	float	Fraction of joint velocity_limit allowed.
max_force_n	float	Declared maximum contact force in Newtons. Not enforced yet: it is validated here and passed to the C++ safety kernel as a parameter, but no check compares any measured or estimated force against it (issue #373).
max_torque_nm	float	Maximum joint torque in Nm. The kernel caps JOINT_TORQUE chunks at min(joint.torque_limit, this).
deadman_required	bool	Whether a deadman switch is required. Declared, not yet enforced by the kernel: passed as a ROS parameter but never consulted. The deploy launch starts the deadman watchdog unconditionally, whatever this says.
e_stop_topic	str	ROS 2 topic for E-stop commands. Declared, not read: the kernel latches on its hardcoded /openral/estop topic.
e_stop_qos	Literal['reliable']	QoS profile for E-stop topic. Declared, not read.
contact_force_threshold_n	float	Force threshold for contact detection. Declared, not yet enforced by the kernel: passed as a ROS parameter but never consulted. The kernel's place-contact force gate takes its threshold from each placement declaration instead.
cycle_time_violation_threshold_ms	float	Control cycle time violation threshold. Declared, not read by any runtime component.
human_in_loop_required	list[str]	rSkill names requiring human supervision. Declared, not read by any runtime component.
max_cartesian_step_m	float | None	Per-control-mode bound the C++ safety kernel enforces — per-step magnitude bound on CARTESIAN_DELTA's xyz triplet (Euclidean, after cartesian_delta_scale). None means "no per-mode check declared, skip". It is a norm, so a controller with a per-axis output limit a needs at least a·√3 here or in-distribution diagonal moves E-stop.
max_cartesian_step_rad	float | None	Per-control-mode kernel bound — per-step magnitude bound on CARTESIAN_DELTA's axis-angle triplet (Euclidean). None skips the check.
max_ee_angular_speed_rad_s	float | None	Per-control-mode kernel bound — angular component bound for CARTESIAN_TWIST (the linear bound reuses max_ee_speed_m_s). None skips the check.
max_base_linear_speed_m_s	float | None	Per-control-mode kernel bound — BODY_TWIST linear bound (Euclidean over vx,vy,vz). None skips the check; mobile manipulators / wheeled bases declare it. Must be finite and > 0 (inf would disable the kernel bound).
max_base_angular_speed_rad_s	float | None	Per-control-mode kernel bound — BODY_TWIST angular bound (Euclidean over wx,wy,wz; for planar bases only wz is non-zero). None skips the check.
self_collision_margin_m	float	Clearance margin (m) for the kernel's self/world/voxel geometric checks; a pair closer than this fires. Default 0.0 (collide on touch). A small negative value tolerates the grazing contact inherent to a compact arm's in-distribution operating envelope (e.g. the SO-101 pen VLA, whose links graze at true ≈ 0) while still catching a real jam (deeper penetration). Loosening this is a safety-WG decision and must be justified against the true (non-convex mesh) envelope clearance, not the conservative primitive distance.
starting_pose_max_joint_speed_rad_s	float | None	Approach speed (rad/s) of every non-prismatic joint on the skill runner's ramp from the live pose to a skill's starting_pose. Declared per robot, never derived: a rated velocity_limit is a ceiling, not an approach speed (issue 303 — deriving it put the OpenArm at 2 rad/s). The runner caps each joint at min(this, joint.velocity_limit). None = undeclared; a real robot with a non-prismatic joint must declare it.
starting_pose_tolerance_rad	float | None	Per-joint distance (rad) within which a non-prismatic joint counts as at its starting pose, both for the ramp and for verifying arrival before the policy starts. None = undeclared; a real robot with a non-prismatic joint must declare it.
starting_pose_max_joint_speed_m_s	float | None	The same approach speed for prismatic joints, in m/s (a gripper's 0.041 m stroke is not comparable to a radian). Also capped by joint.velocity_limit. A real robot with a prismatic joint must declare it.
starting_pose_tolerance_m	float | None	The per-joint arrival tolerance for prismatic joints, in m. A real robot with a prismatic joint must declare it.
joint_state_staleness_limit_s	float | None	Age (s) past which a joint-state reading is refused as stale — by the real HAL's read_state (ROSPerceptionStale) and by the skill runner's blocking waits. The one place a rig declares it: build_hal threads it into the HAL and deploy_e2e.launch.py into the runner in real mode. Must come from a measurement on the deploy host (tools/joint_state_staleness_probe.py). None = undeclared; a real robot must declare it.

SceneChangeMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/scene_change.

Emitted when a histogram / structural-similarity detector measures a frame-to-frame distance above a configured threshold — typically used to wake the reasoner when the scene has changed enough to warrant a re-plan.

Attributes:

Name	Type	Description
kind	Literal['scene_change']	Discriminator (always "scene_change").
distance	float	Frame-to-frame distance in the detector's native metric (e.g. cv2.HISTCMP_CHISQR_ALT distance, 1 - ssim).
threshold	float	Distance threshold that fired the event.
metric	str	Identifier of the distance metric (e.g. "chisqr_alt", "1-ssim", "hellinger").

SceneComposition

Bases: BaseModel

Declarative MJCF scene composition for a manifest-driven HAL.

Lets a robot whose sim HAL needs a composed MJCF (a bare arm spliced onto a tabletop + props) declare the composer in its manifest instead of a bespoke _create_hal lifecycle subclass (issue #191 Phase 3b). The manifest-driven node calls composer before constructing the HAL and threads the resulting MJCF path in as the HAL's mjcf_path.

The composer is a "module.path:function" import string. The function is called with **params and MUST return (xml: str, meshdir: Path) — the composed MJCF XML and the mesh directory it references (the node writes the XML next to meshdir so relative mesh paths resolve). Today's only composer is openral_sim.backends.openarm_robosuite._assets:compose_openarm_tabletop_mjcf.

Attributes:

Name	Type	Description
composer	str	"module.path:function" returning (xml, meshdir).
params	dict[str, object]	Keyword arguments for the composer (e.g. robot_lift_z, white_background).

Example

sc = SceneComposition( ... composer="pkg.scenes:compose_tabletop", ... params={"robot_lift_z": 0.36, "white_background": True}, ... ) sc.params["robot_lift_z"] 0.36

SceneDefaults

Bases: BaseModel

Per-robot scene rendering defaults.

These are values that scene composers may consult when the SimScene YAML does not override them. Today the only field is top_camera, which the openarm_tabletop_pnp backend consumes; future scenes can extend this submodel as new defaults are pulled out of backend hardcodes.

Attributes:

Name	Type	Description
top_camera	TopCameraDefaults | None	Default placement for the scene-overview camera.

Example

sd = SceneDefaults( ... top_camera=TopCameraDefaults( ... pos=(0.2, 0.0, 0.95), ... target=(0.65, 0.0, 0.05), ... fovy=65.0, ... ), ... ) sd.top_camera.pos (0.2, 0.0, 0.95)

SceneGraph

Bases: BaseModel

Persistent hierarchical scene-graph spatial memory.

The durable, queryable world model the S2 Reasoner consults to recall where objects/places/agents are and how to navigate to them. Distinct from the ephemeral collision grid and advisory only — never a safety input (CLAUDE.md §1.1).

Invariants (enforced): node ids are unique; every edge references existing nodes.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk schema version ("0.1"; no backward-incompatible change yet). Now the repo is published it is versioned for real (CLAUDE.md §1.6): an incompatible change bumps it and ships a migrator.
nodes	list[SpatialNode]	All scene-graph nodes.
edges	list[SpatialEdge]	All directed relations between nodes.

SceneSpec

Bases: BaseModel

A physics scene declaration — the WORLD the robot acts in.

A scene is a deterministic, reproducible MuJoCo / MJX / etc. world: assets, lighting, cameras, fixed objects. Tasks (TaskSpec) are evaluated INSIDE a scene; multiple tasks can share one scene (e.g. libero_spatial has 10 tasks per suite).

Attributes:

Name	Type	Description
id	str	Stable scene identifier used by the eval registry, e.g. "libero_spatial", "metaworld_mt50", "so100_tabletop".
backend	PhysicsBackend	Physics backend used to instantiate the scene.
assets_uri	str | None	Optional URI pointing at scene assets. When None, the registered adapter resolves assets internally (LIBERO / MetaWorld pull theirs from their own packages). Read by the isaac_sim scene as its environment USD (a local path, an http(s):// / omniverse:// URL, or isaac:<path> under the installed Isaac Sim asset root); other adapters ignore it.
observation_height	int	Default render height in pixels for camera obs.
observation_width	int	Default render width in pixels for camera obs.
cameras	list[str]	List of camera names the scene exposes. Adapters use this both to render and to map sensors to VLA feature keys.
backend_options	dict[str, object]	Backend-specific overrides (e.g. {"render_modes": ["rgb_array"]}). Opaque to the core schema: each sim backend owns the model that validates it, declared via SCENES.register(..., options_model=...) and run at load time by SCENES.validate_options.

Example

s = SceneSpec(id="libero_spatial", backend=PhysicsBackend.MUJOCO) s.observation_width 256

SceneTaskSpace

Bases: BaseModel

The action interface a scene-adapter family executes.

The scene leg of the cross-layer task-space contract. A scene picks a backend adapter (LIBERO OSC, RoboCasa composite, gym-aloha joints, the RLBench PyRep sidecar, …) and that adapter — not the coarse PhysicsBackend and not the individual scene instance — fixes which ControlMode s the policy's flat action vector is interpreted as. Many scenes share one adapter (the four LIBERO suites, RoboCasa's ~100 prebuilt PnP tasks), so the executed task space is declared once per family and keyed by the same vocabulary rSkills already use in RSkillManifest.evaluated_tasks (the leading token before any "/": "libero_spatial", "rlbench", "robocasa", …).

Attributes:

Name	Type	Description
modes	frozenset[ControlMode]	The ControlMode s the adapter executes. An rSkill is scene-compatible when its TaskSpace control modes are a subset of this set.
action_dim	int | None	The flat action-vector width the adapter steps, when fixed (LIBERO 7, MetaWorld 4, ALOHA 14). None when it varies by task or robot composition (RoboCasa 11-vs-12 base skew, ManiSkill per-task), in which case only modes is checked.
runs_via_default_packers	bool	True when every mode is in SIM_EXECUTABLE_CONTROL_MODES (the deploy-sim default packers could also drive it). False for adapters that drive a dedicated controller path outside the default packers — RLBench's CARTESIAN_POSE motion planner, RoboCasa-GR1's whole-body BASIC composite — mirroring KNOWN_SIM_GAPS in the sweep.

SegmenterContract

Bases: BaseModel

Manifest contract for kind: "segmenter" rSkills.

The sibling of DetectorContract for models that answer a geometric prompt rather than a semantic one: a detector is asked "where are the cups?" and replies with labelled, scored boxes; a segmenter is asked "what is the extent of the thing under this pixel?" and replies with a mask. It carries no labels and no score_threshold — it never classifies, and its consumer never gates on model confidence.

That is a safety property, not an oversight: the mask feeds the HAL's vision attachment-evidence producer, which turns masked depth into the collision payload a manipulator plans around. Only geometry (containment between the jaws, payload extent, depth validity) can reject a bad mask; a measured instance of a 59.8%-of-frame mask returned at the model's own top score of 0.977 is what removed score_threshold from this contract.

Required when RSkillManifest.kind is "segmenter"; forbidden for all other kinds (RSkillManifest._check_kind_consistency). Like detector / vlm, a pure perception producer: no Action chunks, no actuators.

Attributes:

Name	Type	Description
engine	SegmenterEngine	Backend that executes the model (SegmenterEngine). REQUIRED — there is no runtime-keyed fallback for this kind.
max_prompt_points	int	Upper bound on how many prompt points one request may carry (positive TCP point plus optional negative jaw-tip points). Bounds the request so a caller cannot make one segmentation unboundedly expensive. Default 8.
multimask	bool	Ask the backend for its multi-hypothesis head (SAM 2 returns three nested candidates per point prompt) instead of a single mask. The consumer picks among candidates on geometry, never on the model's per-candidate score. Default True.
min_mask_area_px	int	Masks smaller than this many set pixels are discarded as degenerate before they ever reach the depth intersection — a handful of pixels cannot bound a payload. Default 64.

Example

c = SegmenterContract(engine=SegmenterEngine.SAM2_HF) c.engine.value 'sam2_hf' (c.max_prompt_points, c.multimask, c.min_mask_area_px) (8, True, 64)

SegmenterEngine

Bases: str, Enum

Backend that executes a kind: "segmenter" rSkill.

The segmenter counterpart of DetectorEngine. There is no legacy runtime-keyed fallback to preserve here, so SegmenterContract.engine is REQUIRED — a segmenter manifest always names its backend outright.

Attributes:

Name	Type	Description
SAM2_HF		In-process Transformers Sam2Model promptable segmenter (SAM 2.1 Hiera). Consumes positive/negative point prompts in pixel coordinates and emits binary masks at the frame's own resolution.

Example

SegmenterEngine.SAM2_HF.value 'sam2_hf'

SelfVerifyEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SELFVERIFY — a self-check failed.

Attributes:

Name	Type	Description
kind	Literal['selfverify']	Discriminator (always "selfverify").
check	str	Short check identifier (e.g. "action_chunk.shape").
expected	str	Expected value as a string.
observed	str	Observed value as a string.

SensorBundle

Bases: BaseModel

Multi-modal sensor group, e.g. RealSense D455 = (RGB, depth, IMU).

Attributes:

Name	Type	Description
bundle_name	str	Unique name for this bundle.
sensors	list[SensorSpec]	List of constituent sensor specs.
sync	Literal['hardware', 'approximate', 'none']	Synchronization strategy.
sync_tolerance_ms	float	Tolerance for approximate synchronization.

SensorDeployBinding

Bases: BaseModel

Real-device binding that lets openral deploy run open a sensor.

The runtime counterpart to SensorSpec.sim_placement: sim_placement says how the sim renders a camera; deploy_binding says how a real deploy opens it — which SensorReaderBackend and its /dev/video* / pipeline parameters. It is host/site-specific (a /dev/video* index differs per machine).

A robot camera (wrist / head / the robot's own overview) takes it from its host's unit overlay, robots/<id>/units/<unit>.yaml (SensorOverlay), or from robots/<id>/robot.yaml for a robot with a single reference host; a deploy scene never redefines a robot sensor. A workcell-mounted camera (overhead / front) declares it on its own, distinctly named DeployScene.sensors entry. Either way the deploy sensor leg (openral_rskill_ros.sensor_leg) opens every SensorSpec that carries one and publishes it on /openral/cameras/<name>/image.

Attributes:

Name	Type	Description
backend	SensorReaderBackend	Which SensorReaderBackend opens the device.
backend_params	dict[str, object]	Backend-specific reader kwargs, e.g. {"device": "/dev/video0", "fps": 30} for opencv_thread or a {"pipeline": ...} string for gstreamer.
max_age_ms	int	How stale a frame may be before the reader raises.

Example

SensorDeployBinding( ... backend=SensorReaderBackend.OPENCV_THREAD, ... backend_params={"device": "/dev/video0", "fps": 30}, ... ).backend.value 'opencv_thread'

SensorFrame

Bases: BaseModel

A single sensor frame — metadata plus an optional inline payload.

A SensorFrame is the carrier passed from a SensorReader into WorldState.image_frames. Exactly one of data, topic, or handle is populated:

• data carries the pixel bytes for in-process delivery and for trace capture. JSON-serialized as base64 by Pydantic, so payloads round-trip through model_dump_json / model_validate_json.
• topic points at a ROS 2 topic — the bytes live on the ROS 2 bus and the consumer subscribes for them, which is what WorldState.images carries. Ros2ImageSensorReader does NOT return this shape: it subscribes on the caller's behalf and inlines the decoded pixels as data, because every read_latest consumer (the VLA observation builder, trace capture, the republishing sensor leg) needs the bytes, not a second subscription.
• handle is an opaque integer (e.g. CUDA NVMM pointer, DMA-BUF file descriptor) — in-process only, never serialized. Set by the GStreamerSensorReader when frames stay on the GPU.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; matches SensorSpec.name.
stamp_monotonic_ns	int	Capture-time monotonic timestamp in nanoseconds.
stamp_wall_ns	int	Capture-time wall-clock timestamp in nanoseconds.
encoding	FrameEncoding	How to interpret the bytes.
width	int	Image width in pixels.
height	int	Image height in pixels.
channels	int	Number of channels (3 for RGB/BGR, 1 for MONO/DEPTH).
data	bytes | None	Optional inline pixel payload (or compressed bytes for JPEG / PNG). Mutually exclusive with topic / handle.
topic	str | None	Optional ROS 2 topic reference. Mutually exclusive with data / handle.
handle	int | None	Optional opaque in-process handle (NVMM / DMA-BUF). Mutually exclusive with data / topic.
metadata	dict[str, object]	Free-form per-frame metadata (gain, exposure, …).

Example

SensorFrame( ... sensor_id="wrist_rgb", ... stamp_monotonic_ns=1, ... stamp_wall_ns=2, ... encoding=FrameEncoding.RGB8, ... width=640, ... height=480, ... topic=camera_topic("wrist_rgb"), ... ).channels 3

model_post_init(_context: object) -> None

Exactly one of (data, topic, handle) must be set.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Exactly one of (data, topic, handle) must be set."""
    populated = [bool(self.data), self.topic is not None, self.handle is not None]
    n = sum(populated)
    if n != 1:
        raise ValueError(
            f"SensorFrame({self.sensor_id!r}): exactly one of "
            f"(data, topic, handle) must be set; got {n}."
        )

SensorModality

Bases: str, Enum

Physical sensing modality.

SensorOverlay

Bases: BaseModel

Per-host / per-unit values for ONE sensor the robot manifest declares.

The robot manifest owns a sensor's identity and semantics (name, modality, frame_id, parent_frame); what differs between two physical units of the same robot type — the device path or serial in deploy_binding, the vendor driver topic, the calibrated mount pose and intrinsics — goes here. Any other field is refused (extra="forbid"). Applied by apply_sensor_overlays.

Attributes:

Name	Type	Description
name	str	The robot-manifest sensor this overlays.
deploy_binding	SensorDeployBinding | None	This host's real-device binding.
ros2_topic	str | None	This host's vendor-driver topic.
static_transform_xyz_rpy	tuple[float, float, float, float, float, float] | None	This unit's calibrated mount, in the manifest's parent_frame.
intrinsics	IntrinsicsPinhole | None	This unit's calibrated intrinsics.

Example

SensorOverlay(name="top", ros2_topic="/cam/image").ros2_topic '/cam/image'

SensorReaderBackend

Bases: str, Enum

Which SensorReader implementation to instantiate.

opencv_thread is the default and mirrors lerobot's per-camera background-thread pattern. galaxea_a1_camera_bridge consumes the versioned paired raw-frame service owned by an external A1 Runtime process without importing Runtime or opening either RealSense device. ros2_image subscribes to a ROS 2 image topic published by a vendor driver — the path for streams that exist only because an SDK computed them (ZED stereo depth, RealSense aligned depth), which no /dev/video* read can reach. gstreamer runs a GStreamer pipeline whose appsink delivers frames (NVMM / DMA-BUF on Jetson; CPU bytes on x86).

holoscan is reserved-but-unimplemented: a 2026-05-12 evaluation of NVIDIA Holoscan SDK as a parallel ingest backbone deferred adoption (lean GStreamer with custom NvBufSurface glue won the comparison). The enum value exists so a future PR can add the backend additively without bumping the schema again; configs that select it today raise ROSConfigError at factory time.

SensorReaderConfig

Bases: BaseModel

Per-sensor backend configuration for the inference runner.

Picks which SensorReader backend services a sensor and how the pipeline is parameterised. The optional publish_to_ros tee lets a GStreamer pipeline publish a downsampled stream to a ROS 2 topic for observability (rosbag2 / rqt_image_view) without putting the hot path through ROS.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; MUST match a SensorSpec.name in the target RobotDescription.
backend	SensorReaderBackend	Which SensorReaderBackend to instantiate.
backend_params	dict[str, object]	Backend-specific keyword arguments forwarded to the reader constructor. For gstreamer typically a single "pipeline" string. For opencv_thread typically {"device": "/dev/video0", "fps": 30}.
max_age_ms	int	How stale a frame may be before SensorReader.read_latest raises. Defaults to ~3 frames at 30 Hz.
publish_to_ros	bool	If True, the reader tees a downsampled stream to publish_topic at publish_rate_hz. gstreamer backend only; every other backend refuses it.
publish_topic	str | None	ROS 2 topic to publish to when publish_to_ros is True. Required iff publish_to_ros.
publish_rate_hz	float | None	Downsample rate for the ROS tee.
publish_frame_id	str | None	TF frame stamped on the tee's Image and CameraInfo headers (SensorSpec.frame_id). None stamps sensor_id. Only valid with publish_to_ros.
publish_camera_info	IntrinsicsPinhole | None	Manifest intrinsics (SensorSpec.intrinsics). When set, the tee also publishes sensor_msgs/CameraInfo on the image topic's sibling (.../<name>/image → .../<name>/camera_info), scaled to each published frame. None publishes images only. Only valid with publish_to_ros.

Example

SensorReaderConfig( ... sensor_id="wrist_rgb", ... backend=SensorReaderBackend.GSTREAMER, ... backend_params={ ... "pipeline": "v4l2src device=/dev/video0 ! " ... "nvv4l2decoder ! nvvideoconvert ! appsink" ... }, ... publish_to_ros=True, ... publish_topic=camera_topic("wrist_rgb"), ... publish_rate_hz=5.0, ... ).backend.value 'gstreamer'

model_post_init(_context: object) -> None

Cross-field validation for the ROS tee.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation for the ROS tee."""
    if not self.publish_to_ros and (
        self.publish_frame_id is not None or self.publish_camera_info is not None
    ):
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_frame_id / "
            f"publish_camera_info are set but publish_to_ros is False; they "
            f"only configure the ROS tee."
        )
    if self.publish_to_ros and self.backend != SensorReaderBackend.GSTREAMER:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_to_ros needs the "
            f"gstreamer backend (got {self.backend.value!r}); only its in-pipeline "
            f"tee publishes to ROS. Other backends are published by the deploy "
            f"sensor leg's SensorRosPublisher, not by this flag."
        )
    if self.publish_to_ros and self.publish_topic is None:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_to_ros is "
            f"True but publish_topic is unset; a ROS tee needs a topic."
        )
    if self.publish_topic is not None and not self.publish_to_ros:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_topic is "
            f"set but publish_to_ros is False; enable publish_to_ros "
            f"explicitly or drop the topic."
        )

SensorRequirement

Bases: BaseModel

One sensor an rSkill needs the robot to provide.

Used by RSkillManifest.sensors_required to declare the inputs the policy expects. The compatibility check resolves each entry against a RobotDescription's sensors list and rejects the pairing if no robot sensor satisfies the requirement.

Resolution rules (in order):

1. If vla_feature_key is set, the robot MUST expose exactly one sensor with that vla_feature_key. The check then verifies modality matches and (if specified) the sensor's intrinsics meet min_width / min_height.
2. Otherwise, the robot must expose at least count sensors of the requested modality (each meeting any specified resolution minimum).

Attributes:

Name	Type	Description
modality	SensorModality	Required physical modality (rgb, depth, imu…).
vla_feature_key	str | None	Optional exact key the VLA expects, e.g. "observation.images.camera1". When set, the robot's matching SensorSpec.vla_feature_key must be identical.
min_width	int | None	Minimum image width in pixels (RGB/depth/IR/stereo only).
min_height	int | None	Minimum image height in pixels.
count	int	Number of robot sensors of this modality required when vla_feature_key is unset. Ignored when vla_feature_key is provided (a key uniquely identifies one sensor).

Example

SensorRequirement( ... modality=SensorModality.RGB, ... vla_feature_key="observation.images.camera1", ... min_width=224, ... min_height=224, ... ) # doctest: +ELLIPSIS SensorRequirement(modality=, ...)

SensorSpec

Bases: BaseModel

Generalizable sensor descriptor — covers all modalities.

Attributes:

Name	Type	Description
name	str	Human-readable sensor name, e.g. "head_rgb", "left_wrist_depth".
modality	SensorModality	Physical sensing modality.
frame_id	str	tf2 frame name.
parent_frame	str | None	tf2 parent frame (for static transform).
static_transform_xyz_rpy	tuple[float, float, float, float, float, float] | None	Static transform from parent to this sensor.
rate_hz	float	Expected publishing rate in Hz.
intrinsics	IntrinsicsPinhole | None	Pinhole camera intrinsics (if applicable).
encoding	str | None	Image encoding, e.g. "rgb8", "16UC1".
vla_feature_key	str | None	VLA observation dict key this sensor maps to, e.g. 'observation.images.camera1'. Used by skill loaders to auto-wire sensors to VLA input_features.
shares_mount_with	str | None	Another sensor of this robot that is the same physical device (a stereo camera's colour stream and its depth): this one sits at THAT sensor's mount (parent_frame + static_transform_xyz_rpy, a unit's calibration included), so a sim renders both from one pose. None: the sensor declares its own mount, if any.
ros2_topic	str | None	The image topic an external ROS 2 driver publishes (/camera/color/image_raw, /zed/zed_node/...). None (every in-tree manifest) means OpenRAL itself produces the sensor (sim bridge or sensor leg) under the canonical camera_topic(name, kind) layout. Today it is read only by the RealSense calibration command. Nothing remaps it onto camera_topic(name) yet (ADR-0108 Decision 2, not implemented), so a camera served only by a vendor driver does not reach the canonical topics: bind it through a deploy_binding (e.g. the ros2_image backend) to publish it there.
ros2_msg_type	str | None	ROS 2 message type, e.g. "sensor_msgs/Image". None for non-ROS robots (USB, sim-only).
qos_profile	Literal['sensor_data', 'reliable', 'transient_local', 'parameters']	QoS profile key.
catalog_id	str | None	Optional sensor catalog id used as provenance for a catalog-backed physical device. The spec remains fully materialized; calibrated intrinsics, placement, feature keys, and serials stay on this instance.
vendor	str | None	Sensor vendor name.
model	str | None	Sensor model, e.g. "RealSense D455".
driver_pkg	str | None	ROS 2 driver package name.
metadata	dict[str, object]	Additional key-value metadata.

is_cloud_source: bool property

A depth camera or 3D lidar whose driver can feed octomap_server a cloud.

Wider than :attr:is_depth_camera: a real driver builds its own PointCloud2, so no pinhole intrinsics are needed (a 3D lidar has none). On a real deploy any such sensor auto-enables the octomap leg, which then needs the driver's topic pinned (:func:deploy_cloud_topic).

Example

SensorSpec( ... name="lidar", modality="point_cloud", frame_id="lidar", rate_hz=10.0 ... ).is_cloud_source True

is_depth_camera: bool property

A depth / point-cloud camera with intrinsics.

The one test for "can this be back-projected into a cloud": the sim bridge's depth synth, the sim cloud topic (deploy_cloud_topic), the Nav2-over-visual-SLAM guard and the launch's depth-camera pick all use it.

Example

desc = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml") ", ".join(s.name for s in desc.sensors if s.is_depth_camera) 'front_depth'

SimDescription

Bases: BaseModel

MuJoCo wiring for a single-arm robot, consumed by MujocoArmHAL.

The MJCF itself is named by RobotDescription.assets.mjcf; this block carries only the joint↔qpos/qvel/actuator plumbing. All fields are optional with defaults derived from RobotDescription.joints. The default mapping is "1:1 in joint order, offset by 7 (qpos) / 6 (qvel) if floating_base is True" — which is correct for every robot in the open core today bar minor gripper bookkeeping.

Attributes:

Name	Type	Description
floating_base	bool	If True, the robot's MJCF has a 6-DoF free joint before the actuated joints (humanoids). In that case the default joint_qpos_addr[joints[i].name] = 7 + i and joint_qvel_addr[joints[i].name] = 6 + i.
joint_qpos_addr	dict[str, int] | None	Optional override of the joint→qpos-index map. When omitted, the default 1:1 mapping is used.
joint_qvel_addr	dict[str, int] | None	Optional override of the joint→qvel-index map. When omitted, defaults to joint_qpos_addr (or its default).
actuator_index	dict[str, int] | None	Optional override of the joint→actuator-index map. When omitted, defaults to the 1:1 mapping joints[i].name → i.
grippers	list[SimGripperDescription]	Optional list of SimGripperDescription entries. Each must reference a joint by name that is also present in RobotDescription.joints. Single-arm robots have one entry (or none); bimanual robots have two (left + right).
settle_steps_default	int | None	Pinned number of mj_step calls executed per MujocoArmHAL.send_action. None (default) derives it at connect so one action is one control period of action_spec.control_freq_hz (issue #358); a pinned count is honoured and logged as an override that leaves the twin wall-paced.
keyframe_index	int | None	When set, MujocoArmHAL.connect calls mj_resetDataKeyframe(model, data, keyframe_index) before mj_forward. Required for MJCFs whose default MjData.qpos (zeros) sits outside the actuator ctrlrange — e.g. the gym-aloha parallel jaws, whose fingers have ctrlrange=[0.021, 0.057] and never recover from qpos = 0.
seed_ctrl_from_qpos	bool	When True, MujocoArmHAL.connect seeds data.ctrl[actuator] = data.qpos[joint_qpos_addr] for every controllable joint so position actuators hold the initial pose on the first mj_step. Required by the OpenArm v2 MJCF (its position actuators with per-class PD gains will drive qpos to ctrl == 0 otherwise).

Example

SimDescription(floating_base=True).floating_base True

SimEnvironment

Bases: BaseModel

A full sim configuration: the swappable (robot x scene x task x VLA) tuple.

The composed runtime form of a SimScene plus an RSkillManifest. openral_sim's CLI builds it; adapters consume it. Not loaded from YAML directly — from_yaml raises.

Attributes:

Name	Type	Description
robot_id	str	ID into the eval ROBOTS registry. Matches a robot's RobotDescription.name or an embodiment shortcut (e.g. "so100_follower", "franka_panda").
scene	SceneSpec	Scene the robot acts in.
task	TaskSpec	Task to evaluate. task.scene_id MUST equal scene.id.
vla	VLASpec	Policy that drives the robot.
base_pose	Pose6D | None	Optional per-episode mounting pose for the robot in the scene's world frame. None means "use the scene adapter's default" (e.g. an URDF's identity placement). Only honoured by free-axis scene adapters; setting it on a scene that registers with a fixed_robot= constraint (LIBERO, MetaWorld, RoboCasa, PushT, ALOHA) is rejected at CLI compose-time. The pose's frame_id is expected to be "world"; adapters anchor on the robot's RobotDescription.base_frame.
seed	int	Global random seed (env reset, action sampling, torch RNG).
n_episodes	int	Number of episodes to run. 1 for a smoketest; ≥ 50 for an honest single-task success rate.
record_video	bool	Whether to ask the runner to record video frames.
save_dir	str | None	Optional directory to write artefacts (video, traces, json summary). None means "discard outputs".
metadata	dict[str, object]	Free-form notes (commit SHA, run owner, etc.).

Example

env = SimEnvironment( ... robot_id="franka_panda", ... scene=SceneSpec(id="libero_spatial"), ... task=TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the cube", ... ), ... vla=VLASpec(id="smolvla", weights_uri="hf://lerobot/smolvla_libero"), ... ) env.task.scene_id == env.scene.id True

model_post_init(_context: object) -> None

Cross-field validation: task.scene_id must match scene.id.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: task.scene_id must match scene.id."""
    if self.task.scene_id != self.scene.id:
        raise ValueError(
            f"SimEnvironment.task.scene_id ({self.task.scene_id!r}) does not "
            f"match scene.id ({self.scene.id!r}); a task can only run in its "
            f"declared scene."
        )

SimGripperDescription

Bases: BaseModel

Gripper wiring inside a MuJoCo MJCF.

Attributes:

Name	Type	Description
joint	str	Name of the gripper joint as it appears in RobotDescription.joints (the public, lerobot-style name — not the menagerie MJCF name).
ctrl_range	tuple[float, float]	(low, high) raw control range for the gripper actuator. In NORMALISED write mode, low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). In PASSTHROUGH write mode this is informational — MuJoCo clips on its own.
qpos_addrs	tuple[int, ...]	qpos indices used to compute the reported gripper position. When the gripper has multiple finger joints (Franka), list every finger's qpos index.
qpos_scale	float	Span used to normalise the summed/raw gripper qpos to [0, 1]. For Franka with two fingers each in [0, 0.04] m, qpos_scale = 0.08. Ignored by AFFINE_LOW_HIGH and PASSTHROUGH read modes but kept for symmetry with the base-class invariant.
read_mode	GripperReadMode	How to report the qpos — see GripperReadMode.
write_mode	GripperWriteMode	How to map an Action's gripper value to ctrl — see GripperWriteMode.
actuator_index	int | None	Explicit MJCF actuator index that receives the (mapped) gripper command. When omitted, defaults to the position of joint in RobotDescription.joints (i.e. the 1:1 mapping derived by SimDescription for arm joints).
mirror_actuator_index	int | None	Optional second actuator that receives the negation of the (mapped) gripper command. Models the Aloha parallel jaws where one motor drives +x and the other -x to keep the fingers symmetric. None for single-actuator grippers.

SimScene

Bases: DeployScene

Scene + task for openral sim run.

All task fields are overridable at the CLI level. max_steps and success_key are optional — omit them for open-ended experiments. Accepts a BenchmarkScene YAML transparently (the eval-specific fields n_episodes, seed, and metadata simply fill the defaults).

SpatialEdge

Bases: BaseModel

A directed relation between two scene-graph nodes.

Attributes:

Name	Type	Description
src	str	node_id of the source node.
dst	str	node_id of the destination node.
kind	SpatialRelationKind	Relation kind.

SpatialNode

Bases: BaseModel

A persistent, typed node in the scene-graph spatial memory.

A superset of DetectedObject for kind == OBJECT; also used for places, rooms, and agents. The pose is anchored in a durable, drift-corrected frame (typically the tf2 map frame); consumers resolve it to the live base frame at query time via tf2 — the node never stores a raw transform.

This is advisory world-model state consumed by the S2 Reasoner. It is never a safety input (CLAUDE.md §1.1): the safety kernel gates only on the live, bounded geometric world.

Attributes:

Name	Type	Description
node_id	str	Stable identifier, unique within a SceneGraph.
kind	SpatialNodeKind	Node kind (object / place / room / agent).
pose	Pose6D	6D pose, anchored in the durable map frame.
label	str	Semantic label or name (e.g. "fridge", "kitchen").
confidence	float	Confidence in [0, 1] for a perceived node.
bbox_3d	tuple[float, float, float, float, float, float] | None	Optional 3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
embedding_ref	str | None	Optional handle into the vector store for open-vocabulary matching; None → label-only.
is_container	bool	Whether the node can hold other nodes (fridge, cabinet).
occludes_contents	bool	Whether contents are unobservable until the container is opened. Requires is_container.
first_seen_ns	int	Timestamp of first observation, in nanoseconds.
last_seen_ns	int	Timestamp of the most recent observation, in nanoseconds; must be >= first_seen_ns.
observation_count	int	Number of times this node has been observed.

Example

node = SpatialNode( ... node_id="fridge", ... kind=SpatialNodeKind.OBJECT, ... pose=Pose6D(xyz=(3.0, 1.0, 0.9), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"), ... label="fridge", ... is_container=True, ... occludes_contents=True, ... first_seen_ns=1, ... last_seen_ns=2, ... ) node.is_container True

SpatialNodeKind

Bases: str, Enum

Kind of node in the persistent scene-graph spatial memory.

OBJECT is the foundation (an accumulated DetectedObject); PLACE is a standable navigation waypoint; ROOM is a semantic area grouping places/objects; AGENT is a person or robot with a pose — the requester of a task is an AGENT so "bring it back to me" resolves to a concrete goal.

SpatialRelationKind

Bases: str, Enum

Kind of directed edge between scene-graph nodes.

CONTAINS links a room/container to what is inside it (a fridge CONTAINS a wine bottle); AT_PLACE links an object/agent to the waypoint to stand at to reach it; TRAVERSABLE_TO is the topological navigation graph between places/rooms; ON / NEAR are incidental object-to-object spatial relations.

SphereShape

Bases: BaseModel

Sphere collision primitive — the simplest convex link/obstacle volume.

Attributes:

Name	Type	Description
shape	Literal['sphere']	Discriminator (always "sphere").
radius_m	float	Sphere radius in metres.

Example

SphereShape(radius_m=0.05).shape 'sphere'

StateContract

Bases: BaseModel

Per-rSkill state-vector contract.

Surfaces the proprioception layout the checkpoint was trained against so the runtime adapter does not have to learn it from a YAML override.

Attributes:

Name	Type	Description
layout	StateLayout | None	Named proprioception layout — see StateLayout. LIBERO / MetaWorld / pusht / aloha leave this None and consume the raw joint-position vector directly.
dim	int | None	Explicit state dimension override. The runtime adapter clips or pads the env state vector to this width before handing it to the policy.
bindings	StateContractBindings | None	Per-robot source bindings — TF frame names + JointState entries the runtime adapter pulls values from. REQUIRED when layout is in WRAPPED_TASK_SPACE_LAYOUTS, FORBIDDEN otherwise (joint-space layouts have no source to parameterise).

StateContractBindings

Bases: BaseModel

Per-robot source bindings for an rSkill's state_contract.layout.

Symmetric to ControlModeSemantics on the action side (joint_order + reference_frame + gripper_convention): the rSkill manifest names the shape via StateContract.layout; these bindings name the sources on the deploying robot. The layout-adapter registry (openral_state_adapter) joins shape + bindings + live sensor_msgs/JointState + live /tf into the per-checkpoint state vector.

Bindings are required for layouts in WRAPPED_TASK_SPACE_LAYOUTS and forbidden otherwise (the joint-space layouts have no source to parameterise).

Attributes:

Name	Type	Description
eef_frame	str | None	tf2 link name of the end effector (e.g. "panda_hand"). Required for any layout reading EE position / orientation.
base_frame	str | None	tf2 link name of the mobile base (e.g. "base_link"). Required for any layout reading base position / orientation OR base-relative EE poses.
world_frame	str | None	tf2 root frame the base pose is expressed in (default "map" — slam_toolbox publishes map → odom → base_link). Set to "odom" for deployments without SLAM.
gripper_qpos_joints	list[str]	JointState.name entries whose positions populate the gripper-width slot(s) of the layout, in the order the policy expects (e.g. ["panda_finger_joint1", "panda_finger_joint2"]). Empty for grasper-less robots or layouts without a gripper slot.
quaternion_convention	Literal['xyzw', 'wxyz']	Component order of any quaternion in the assembled vector. ROS / TF2 default is "xyzw" (the geometry_msgs/Quaternion field order). Some upstream checkpoints expect "wxyz" — declare it here so the assembler permutes once, at the boundary.

StateRepresentation

Bases: str, Enum

State vector representation format.

StaticBox

Bases: BaseModel

One declared static fixture as an oriented box in the robot's base_frame.

Example

StaticBox( ... source="/World/Cell/LeftWall", ... xyz=(0.3, 0.55, 0.08), ... half_extents=(0.675, 0.018, 0.82), ... ).kind 'box'

StaticGeometry

Bases: BaseModel

A cell's declared static geometry (DeployRuntime.static_geometry_file).

Example

g = StaticGeometry.model_validate( ... { ... "static_geometry": [ ... {"kind": "box", "xyz": [0.9, 0.0, 0.1], "half_extents": [0.02, 0.55, 0.8]}, ... {"kind": "mesh", "file": "Rack.npz"}, ... ] ... } ... ) " ".join(e.kind for e in g.static_geometry) 'box mesh'

StaticMesh

Bases: BaseModel

One declared static fixture as a triangle mesh in the robot's base_frame.

file is an .npz with vertices (N, 3) metres and triangles (M, 3) indices, relative to the StaticGeometry YAML that lists it.

Example

StaticMesh(source="/World/OMAR/Rack", file="Rack.npz").kind 'mesh'

SupportContactWitness

Bases: BaseModel

Bounded attestation that a payload rests on a named support surface.

Attachment says what the robot is carrying; it does not say the payload is free of its environment (ADR-0092 D6). A grasped baguette still lies on the counter while the gripper closes, and the safety kernel correctly sees that as a payload-vs-world penetration. This witness is World State's statement that one specific contact is legitimate — the kernel exempts nothing else. Absent witness, absent exemption: the kernel stops.

Geometry is expressed in the attached object's own frame, deliberately not the base frame and not voxel indices: indices decorrelate as a mobile base drives and the occupancy lattice re-phases underneath the robot, while the physical contact persists — the object frame moves with the payload, so the attested plane keeps naming the same physical support face.

max_penetration_m is a physical depth. A 25 mm occupancy lattice reads a 1 mm physical contact as up to ~15 mm of penetration because the surface cell's cube overshoots the true surface; the kernel accounts for that discretisation separately and geometrically, so inflating this field to cover it would license real penetration.

Attributes:

Name	Type	Description
support_id	str	Identity of the supporting environment surface.
contact_point_in_object	tuple[float, float, float]	A point on the support plane, object frame.
contact_normal_in_object	tuple[float, float, float]	Unit outward support normal in the object frame, pointing from the supporting solid toward the payload.
patch_radius_m	float	Lateral radius of the supported patch about the contact point. Contact outside it is never exempt.
max_penetration_m	float	Attested bound on the physical penetration depth.
confidence	float	Witness confidence in [0, 1].
evidence_kind	AttachmentEvidenceKind	Source that measured the support contact. The MuJoCo producer attests SIM_GEOM_DISTANCE — a signed closest-distance probe, not the solver's contact list, which contype / conaffinity suppression can leave empty under a payload that is demonstrably resting on a surface.
evidence_ref	str | None	Optional trace/contact reference.
stamp_ns	int	Producer timestamp. With support_id this keys the safety kernel's re-arm check, so a witness that already died after separation stays dead until a genuinely new one is attested.

Example

witness = SupportContactWitness( ... support_id="sim:counter_main", ... contact_point_in_object=(0.0, 0.0, -0.025), ... contact_normal_in_object=(0.0, 0.0, 1.0), ... patch_radius_m=0.1, ... max_penetration_m=0.00118, ... confidence=1.0, ... evidence_kind=AttachmentEvidenceKind.SIM_GEOM_DISTANCE, ... stamp_ns=1, ... ) witness.patch_radius_m 0.1

fill_idl(msg: object) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS."""
    msg.support_id = self.support_id  # type: ignore[attr-defined]
    point = msg.contact_point_in_object  # type: ignore[attr-defined]
    point.x, point.y, point.z = self.contact_point_in_object
    normal = msg.contact_normal_in_object  # type: ignore[attr-defined]
    normal.x, normal.y, normal.z = self.contact_normal_in_object
    msg.patch_radius_m = float(self.patch_radius_m)  # type: ignore[attr-defined]
    msg.max_penetration_m = float(self.max_penetration_m)  # type: ignore[attr-defined]
    msg.confidence = float(self.confidence)  # type: ignore[attr-defined]
    msg.evidence_kind = self.evidence_kind.value  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref or ""  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    point = msg.contact_point_in_object  # type: ignore[attr-defined]
    normal = msg.contact_normal_in_object  # type: ignore[attr-defined]
    return cls(
        support_id=str(msg.support_id),  # type: ignore[attr-defined]
        contact_point_in_object=(float(point.x), float(point.y), float(point.z)),
        contact_normal_in_object=(float(normal.x), float(normal.y), float(normal.z)),
        patch_radius_m=float(msg.patch_radius_m),  # type: ignore[attr-defined]
        max_penetration_m=float(msg.max_penetration_m),  # type: ignore[attr-defined]
        confidence=float(msg.confidence),  # type: ignore[attr-defined]
        evidence_kind=AttachmentEvidenceKind(str(msg.evidence_kind)),  # type: ignore[attr-defined]
        evidence_ref=str(msg.evidence_ref) or None,  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
    )

SuppressedSummaryEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SUPPRESSED_SUMMARY — rolling rate-limit roll-up.

Emitted at ~1 Hz by openral_observability.FailureBusPublisher when one or more (kind, severity) buckets dropped events during the past window.

Attributes:

Name	Type	Description
kind	Literal['suppressed_summary']	Discriminator (always "suppressed_summary").
window_s	float	Length of the summarized window, in seconds.
kinds	list[int]	Parallel array of suppressed KIND_* values.
severities	list[int]	Parallel array of suppressed SEVERITY_* values.
counts	list[int]	Parallel array of dropped-event counts per bucket.

TaskSpace

Bases: BaseModel

Layer-neutral view of an action interface as ordered typed segments.

Produced from an rSkill's ActionContract (and, in a later phase, declared by a scene) so the three asset layers — robots, rSkills, scenes — share one comparable object instead of three implicit encodings (a flat supported_control_modes set, an ActionContract, and a runtime-probed action_dim). It is derived, never a hand-authored manifest field, so it cannot drift from the primitives it is built on.

Attributes:

Name	Type	Description
segments	list[TaskSpaceSegment]	Ordered, non-empty list of TaskSpaceSegment. Order matches the flat action-vector layout for skill-derived spaces.
representation	ActionRepresentation | None	The source ActionRepresentation when known (carried through from ActionContract), for diagnostics.

Example

ts = TaskSpace( ... segments=[ ... TaskSpaceSegment( ... family=TaskSpaceFamily.CARTESIAN, ... control_mode=ControlMode.CARTESIAN_DELTA, ... width=6, ... target="panda_hand", ... ), ... TaskSpaceSegment( ... family=TaskSpaceFamily.GRIPPER, ... control_mode=ControlMode.GRIPPER_POSITION, ... width=1, ... target="panda_hand", ... ), ... ], ... ) ts.total_dim 7 sorted(m.value for m in ts.control_modes) == ["cartesian_delta", "gripper_position"] True

control_modes: set[ControlMode] property

The distinct ControlMode s this space drives.

total_dim: int property

Sum of segment widths — the flat action-vector dimensionality.

from_action_contract(action: ActionContract, robot: RobotDescription) -> TaskSpace classmethod

Build the task space an rSkill emits, expanding slots / representation.

Resolution order:

1. action.slots set → one segment per non-discard slot.
2. else action.representation set → expand via canonical_slots_for_representation (returns None for joint representations, falling through to step 3).
3. else → a single whole-vector JOINT_POSITION segment (legacy path).

Parameters:

Name	Type	Description	Default
action	ActionContract	The rSkill's per-checkpoint action contract.	required
robot	RobotDescription	The robot the skill is being matched against — used by the representation expander to resolve the primary end-effector.	required

Returns:

Type	Description
TaskSpace	The normalized TaskSpace the checkpoint's action vector
TaskSpace	occupies.

Example

from openral_core.schemas import ( ... ActionSlot, ... EmbodimentKind, ... EndEffectorSpec, ... JointSpec, ... JointType, ... RobotCapabilities, ... SafetyEnvelope, ... ) robot = RobotDescription( ... name="franka_panda", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["franka_panda"], ... ), ... safety=SafetyEnvelope(), ... ) ac = ActionContract( ... dim=7, ... slots=[ ... ActionSlot( ... range=(0, 5), ... control_mode=ControlMode.CARTESIAN_DELTA, ... ee="panda_hand", ... frame="panda_hand", ... ), ... ActionSlot( ... range=(6, 6), control_mode=ControlMode.GRIPPER_POSITION, ee="panda_hand" ... ), ... ], ... ) ts = TaskSpace.from_action_contract(ac, robot) ts.total_dim 7 [seg.family.value for seg in ts.segments] == ["cartesian", "gripper"] True

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_action_contract(cls, action: ActionContract, robot: RobotDescription) -> TaskSpace:
    """Build the task space an rSkill emits, expanding slots / representation.

    Resolution order:

    1. ``action.slots`` set → one segment per non-discard slot.
    2. else ``action.representation`` set → expand via
       ``canonical_slots_for_representation`` (returns ``None`` for joint
       representations, falling through to step 3).
    3. else → a single whole-vector ``JOINT_POSITION`` segment (legacy path).

    Args:
        action: The rSkill's per-checkpoint action contract.
        robot: The robot the skill is being matched against — used by the
            representation expander to resolve the primary end-effector.

    Returns:
        The normalized ``TaskSpace`` the checkpoint's action vector
        occupies.

    Example:
        >>> from openral_core.schemas import (
        ...     ActionSlot,
        ...     EmbodimentKind,
        ...     EndEffectorSpec,
        ...     JointSpec,
        ...     JointType,
        ...     RobotCapabilities,
        ...     SafetyEnvelope,
        ... )
        >>> robot = RobotDescription(
        ...     name="franka_panda",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(
        ...         supported_control_modes=[ControlMode.JOINT_POSITION],
        ...         embodiment_tags=["franka_panda"],
        ...     ),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> ac = ActionContract(
        ...     dim=7,
        ...     slots=[
        ...         ActionSlot(
        ...             range=(0, 5),
        ...             control_mode=ControlMode.CARTESIAN_DELTA,
        ...             ee="panda_hand",
        ...             frame="panda_hand",
        ...         ),
        ...         ActionSlot(
        ...             range=(6, 6), control_mode=ControlMode.GRIPPER_POSITION, ee="panda_hand"
        ...         ),
        ...     ],
        ... )
        >>> ts = TaskSpace.from_action_contract(ac, robot)
        >>> ts.total_dim
        7
        >>> [seg.family.value for seg in ts.segments] == ["cartesian", "gripper"]
        True
    """
    slots = action.slots
    if slots is None and action.representation is not None:
        slots = canonical_slots_for_representation(
            action.representation, dim=action.dim, description=robot
        )
    if slots is None:
        # Legacy whole-vector path: an undeclared layout is joint-position.
        return cls(
            segments=[
                TaskSpaceSegment(
                    family=TaskSpaceFamily.JOINT,
                    control_mode=ControlMode.JOINT_POSITION,
                    width=action.dim,
                )
            ],
            representation=action.representation,
        )
    segments: list[TaskSpaceSegment] = []
    for slot in slots:
        mode = slot.control_mode
        if mode is None:  # discard slot — occupies range, emits no Action.
            continue
        lo, hi = slot.range
        segments.append(
            TaskSpaceSegment(
                family=_FAMILY_FOR_MODE[mode],
                control_mode=mode,
                width=hi - lo + 1,
                target=slot.ee,
            )
        )
    return cls(segments=segments, representation=action.representation)

TaskSpaceFamily

Bases: str, Enum

Coarse classification of a ControlMode for task-space views.

Redundant with ControlMode (derivable via _FAMILY_FOR_MODE) but stored on TaskSpaceSegment so the object reads cleanly in logs / dashboards and so family↔mode consistency is validated once at construction.

TaskSpaceMatch

Bases: BaseModel

Result of task_space_compatible.

Attributes:

Name	Type	Description
ok	bool	True when every skill segment is executable on the robot.
reasons	list[str]	Human-readable explanation of each incompatibility; empty when ok is True.

TaskSpaceSegment

Bases: BaseModel

One typed slice of an action vector, layer-neutral.

The normalized form of an ActionSlot (rSkill side) or a robot's advertised control mode (robot side), stripped of the slot's absolute [start, end] indices so robot capabilities and scene expectations — which are not policy vectors — can be expressed in the same vocabulary.

Attributes:

Name	Type	Description
family	TaskSpaceFamily	Coarse classification, derived from control_mode and validated to match _FAMILY_FOR_MODE.
control_mode	ControlMode	The ControlMode this slice routes to.
width	int	Number of scalar dimensions the slice occupies (> 0). For the gripper this is the audit's "is the gripper a dimension?" answered structurally — a 1-D GRIPPER_* segment.
target	str | None	End-effector name for CARTESIAN_* / GRIPPER_* / DEX_HAND_JOINT modes (the frame / actuator the slice drives); None for joint / base / composite modes. May be None even for EE modes when the producer could not resolve an end-effector (e.g. a representation-expanded skill on a robot with no declared EE) — the matcher reports that as a reason rather than failing construction.

TaskSpec

Bases: BaseModel

A task declaration — WHAT the robot must achieve inside a scene.

Tasks decouple scene assets (SceneSpec) from goal-conditioning. The same scene can host many tasks; the same task can occasionally be run in multiple compatible scenes (rare; usually 1:1 with scene_id).

Success is evaluated by the scene adapter — the reward / success signal comes from the underlying gym env (info['is_success'], info['success'], terminal reward, …). success_key lets manifest authors override which info field the runner reads.

Attributes:

Name	Type	Description
id	str	Stable task identifier, e.g. "libero_spatial/task_0", "metaworld/push-v3". Adapters split on / to resolve.
scene_id	str	ID of the SceneSpec this task runs in.
instruction	str	Natural-language goal handed to the VLA as the "task" text input. Some adapters override this with a description baked into the underlying suite (LIBERO, MetaWorld).
max_steps	int | None	Episode budget. Adapters may clip to scene-internal limits. None means unset; BenchmarkScene enforces a concrete value via its model validator.
success_key	str | None	Key inside info returned by env.step() whose truthy value marks task success. None means unset; BenchmarkScene enforces a concrete value via its model validator (required for paper-comparison benchmarks).
metadata	dict[str, object]	Free-form per-task notes (paper reference, dataset split, …).

Example

t = TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the black bowl", ... max_steps=200, ... success_key="is_success", ... ) t.max_steps 200 t.success_key 'is_success' TaskSpec(id="libero_spatial/task_0", scene_id="libero_spatial").max_steps is None True

TickResult

Bases: BaseModel

One tick's record returned by InferenceRunner.tick.

Carries the timing breakdown so the parent OTel span can attach exact sub-stage durations and the latency budget enforcement can flag violations without re-instrumenting.

The hardware fields (sensors_ms..hal_ms, safety_violations, action_applied) are the original (v1) surface used by DeployRunner. The sim-specific fields (step_idx..truncated) were added when SimRunner adopted per-step tick semantics; hardware leaves them at their defaults (None), so a hardware tick serialises identically to v1 under model_dump(exclude_none=True).

Attributes:

Name	Type	Description
stamp_ns	int	Tick wall-clock timestamp in nanoseconds (tick start).
tick_idx	int	0-indexed tick counter within a run.
sensors_ms	float	SensorReader.read_latest total wall-time.
world_state_ms	float	WorldStateAggregator.snapshot wall-time.
inference_ms	float	Skill.step wall-time (the chunk dispatch cost, not the full chunked inference — see ChunkedExecutor).
safety_ms	float	Safety check wall-time.
hal_ms	float	HAL.send_action wall-time.
tick_ms	float	End-to-end tick wall-time including the rate-limiter overhead.
chunk_index	int | None	Index of the action played out from a chunked executor. None for non-chunked skills.
safety_violations	list[str]	List of safety-violation reason strings emitted during this tick. Non-empty implies the action was either clamped or dropped per the safety policy.
action_applied	bool	False when the DeadlineOverrunPolicy was drop and the runner elected not to publish this tick's action, or when a sim tick is the reset-tick between episodes (no inference / env step happened).
step_idx	int | None	0-indexed step within the current episode. Set by SimRunner on step-ticks; None on hardware ticks and on sim reset-ticks.
episode_idx	int | None	0-indexed episode within the current run. Set by SimRunner on every tick (including reset-ticks); None on hardware ticks.
reward	float | None	Env step reward for this tick. Set by SimRunner on step-ticks; None on hardware ticks and reset-ticks.
terminated	bool | None	Whether the env signalled natural termination this tick. Set by SimRunner; None elsewhere.
truncated	bool | None	Whether the env hit its step budget this tick. Set by SimRunner; None elsewhere.
trace_context	str | None	Full W3C traceparent for this tick's rskill.tick span, in the form 00-<trace_id_hex>-<span_id_hex>-<flags_hex>. Optional — set by the runner when an OTel context is active so offline consumers (dataset writers, post-hoc analysers) can resume the trace without re-deriving it from the live span. Default None for byte-identical v1 JSON under model_dump(exclude_none=True).

TimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_TIMEOUT — an operation missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['timeout']	Discriminator (always "timeout").
operation	str	Short name of the operation that timed out (e.g. "skill.step", "hal.read_state", "reasoner.tick").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

TopCameraDefaults

Bases: BaseModel

Default placement for the scene-level "top" (a.k.a. "base") camera.

Per-robot scene defaults consumed by sim backends that render an overview camera (today: the openarm_tabletop_pnp scene). The values describe a look-at camera pointed from pos toward target with vertical field-of-view fovy in degrees.

Backend YAML overrides (scene.backend_options.top_camera_*) still win — this submodel is the default fed to the composer when no override is set, replacing the previous module-level constants in openral_sim.backends.openarm_robosuite._assets.

Attributes:

Name	Type	Description
pos	tuple[float, float, float]	(x, y, z) world-frame camera position in metres.
target	tuple[float, float, float]	(x, y, z) world-frame look-at point in metres.
fovy	float	Vertical field-of-view in degrees.

Example

TopCameraDefaults(pos=(0.2, 0.0, 0.95), target=(0.65, 0.0, 0.05), fovy=65.0).fovy 65.0

UrdfAsset

Bases: BaseModel

A URDF asset reference plus its robot_state_publisher wiring.

The ref is resolved by openral_core.assets.resolve_asset; root_frame and base_to_root_xyz_rpy carry the static transform that bridges a URDF whose root link differs from the robot's base_frame (e.g. Franka's panda_link0 mounted onto a base_link mobile platform).

Attributes:

Name	Type	Description
ref	str	Asset reference (rd:<module>, file:<relpath>, or ros2://robot_description for runtime topic-supplied URDFs).
root_frame	str | None	The URDF's root link name when it differs from RobotDescription.base_frame. None → the URDF root equals base_frame (no static transform needed).
base_to_root_xyz_rpy	tuple[float, float, float, float, float, float] | None	The 6-DoF transform [x, y, z, roll, pitch, yaw] (metres + radians) published via static_transform_publisher to bridge base_frame to root_frame. None when root_frame is None.

Example

UrdfAsset(ref="rd:panda_description", root_frame="panda_link0").root_frame 'panda_link0'

VLASpec

Bases: BaseModel

A VLA / policy declaration — the BRAIN driving the robot.

Lightweight pointer to a policy: either an installed rSkill (referenced by manifest name) or a raw HF Hub repo. The eval registry resolves this to a runtime adapter that returns Action objects.

Attributes:

Name	Type	Description
id	str	Adapter id in the eval registry, e.g. "smolvla", "pi05", "xvla", "random", "zero". Picks the loader.
weights_uri	str	Where to fetch weights from. Pass a bare rSkill reference: a name (smolvla-libero), a path (rskills/smolvla-libero), or a bare HF repo ID (OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16). The openral_sim.SimRunner requires a locally-resolvable reference — raw "hf://" URIs are rejected. Other URI shapes (e.g. "mock://") are still parsed by the schema so unit tests of the eval registries can run without an rSkill, but they will fail-fast if used with the runner.
device	str	Torch device override. "auto" picks cuda:0 if available, otherwise cpu.
quantization	QuantizationConfig | None	Optional quantization override for this run.
deterministic	bool	When True, set torch.use_deterministic_algorithms and disable cuDNN benchmarking.
extra	dict[str, object]	Adapter-specific options, e.g. {"chunk_size": 16}.

Example

v = VLASpec(id="smolvla", weights_uri="rskills/smolvla-libero") v.device 'auto'

ValidationGroundTruthAdjudication

Bases: BaseModel

What the simulator's own probe says about a kernel stop.

Derived from the sim.estop_ground_truth_snapshot line the sim HAL emits per stop. The probe measures against solid world geometry only and carries its own caveat that a zero contact count is not an emptiness test, so the adjudication is driven by the distance probes rather than the contact list.

Attributes:

Name	Type	Description
verdict	GroundTruthAdjudication	The adjudication.
stop_class	str | None	The probe's own class for the stop ("robot_world", "attached_payload").
sim_time_s	float | None	Simulator time at the stop.
grid_resolution_m	float | None	Occupancy-grid edge length; sets the quantization budget.
quantization_budget_m	float | None	Half the grid cell's body diagonal — the voxel-only term, not the whole admissible gap.
admissible_gap_m	float | None	Budget applied to discrepancy_m; prefers the HAL's own adjudication_budget.admissible_gap_m (88.2 mm vs. a 25 mm grid's 21.7 mm voxel-only term on the 2026-08-23 rounds), else quantization_budget_m for a pre-budget snapshot.
budget_source	str	Where admissible_gap_m came from — "hal-adjudication-budget", "grid-quantization" (a lower bound only: can prove within-quantization, never false-positive), or "" (no budget; rounds before #144 published none and two verdicts were over-convicted on that alone).
probe_collidability_filtered	bool | None	Whether both probe sides were solid-geom-restricted; False on pre-filter snapshots (a 0 m pair may be a visual mesh, so cannot support real-contact).
probe_distance_certified	bool | None	Whether every probe distance is proof-certified (openral_hal.convex_distance); False on mujoco.mj_geomDistance snapshots, which read confidently wrong 0.000 m for pairs whose certified value is +14.8/+82.2/ +107.9 mm — such a probe withdraws the verdict to unadjudicated rather than reversing it.
unadjudicated_reason	str	Why the verdict is unadjudicated (empty otherwise) — distinguishes a harness fault from a run fact.
nearest_any_m	float | None	Distance of the closest probed pair of any kind; None when nothing was within distmax_m.
nearest_tripping_party_m	float | None	Distance of the closest pair naming the kernel's tripping party; None when out of range.
nearest_pair	dict[str, object]	Bodies/geoms of the closest probed pair, verbatim.
discrepancy_m	float | None	nearest_tripping_party_m minus the kernel's min_distance_m — clearance the kernel did not see.
probed_pairs	int | None	Pairs the probe actually measured.
probe_truncated	bool | None	Whether the probe hit its call budget (a truncated probe can never support false-positive).
distmax_m	float | None	The probe's cutoff distance.
payload_contacts	int | None	MuJoCo contact count involving the payload.

Example

adj = ValidationGroundTruthAdjudication( ... verdict="real-contact", stop_class="attached_payload", sim_time_s=17.5 ... ) adj.verdict 'real-contact'

ValidationRoundDiff

Bases: BaseModel

Round-over-round comparison — the "what changed" artifact.

Attributes:

Name	Type	Description
schema_version	str	On-disk format version.
round_id	str	The newer round.
baseline_round_id	str	The round it is compared against.
executed_sha	str	SHA the newer round ran.
baseline_executed_sha	str	SHA the baseline ran.
seed	int | None	Scene seed the newer round pinned.
baseline_seed	int | None	Scene seed the baseline pinned. The SHA alone does not make a comparison a reproducibility one: the scene seed decides the initial configuration, so two rounds at one SHA on different seeds are a before/after of two different scenes. A seed-1-vs-seed-2 diff was labelled reproducibility for exactly that reason.
scenes	list[ValidationSceneDelta]	Per-scene deltas, in the newer round's order.

Example

d = ValidationRoundDiff( ... round_id="b", ... baseline_round_id="a", ... executed_sha="0" * 40, ... baseline_executed_sha="0" * 40, ... seed=1, ... baseline_seed=2, ... ) d.same_sha, d.is_reproducibility (True, False)

changed_scenes: list[str] property

Scene keys whose outcome moved between the two rounds.

is_reproducibility: bool property

Whether this compares one configuration with itself.

Same code and same seed. Anything else is a before/after: a seed change moves the scene's initial configuration, so the two rounds are not repeats of one another however equal their SHAs are.

same_seed: bool property

Whether both rounds pinned the same scene seed.

same_sha: bool property

Whether both rounds ran the same code.

ValidationRoundMetadata

Bases: BaseModel

What was executed, on what, from what — the reproducibility half.

Every field answers a "why couldn't we reproduce a past round" question: which code ran, whether the overlay was built from it, which dependency set, which entry point.

Attributes:

Name	Type	Description
round_id	str	Directory name of the round, e.g. "2026-08-22-master-1".
started_at	str	ISO-8601 UTC timestamp the round began.
host	str	Hostname of the validation runner.
executed_sha	str	Full git SHA of the checkout that ran.
worktree_clean	bool | None	Whether git status --porcelain was empty; None only for pre-harness imports (run refuses a dirty worktree).
overlay_built_at_ns	int | None	Modification time of the built ROS overlay, to prove it isn't older than the sources.
launcher_path	str	Absolute path of the openral entry point invoked.
repo_root	str	Absolute path exported as OPENRAL_REPO_ROOT.
robot_manifest_path	str | None	Repo-relative path of the resolved robot manifest.
robot_id	str | None	The robot id the scenes resolved to.
sync_groups	list[str]	Dependency groups the environment was synced with.
stack_argv	list[str]	The launch argv shared by every scene, verbatim.
safety_overrides_absent	bool	True only when the composed argv was checked and carries no safety-knob override; a round is refused rather than recorded False.
collision_scale	dict[str, float]	The #188 graded-velocity band the kernel ran with, when armed; empty = shipped default (disabled). Arrives via OPENRAL_COLLISION_SCALE_* env vars, invisible to argv inspection.
gpu_name	str | None	GPU the round ran on, when nvidia-smi was available.
notes_path	str | None	Round-relative path of the human-readable notes.
scene_dirs	dict[str, str]	Scene key → round-relative artifact directory; empty = same name as the scene key.
seed	int	The scene seed the round pinned; read with executed_sha for a true reproducibility pair (a seed mismatch is a before/after).
artifact_stem	str	Filename stem of every per-scene artifact; pre-harness rounds used "seed1".
imported_from	str | None	Absolute path of the pre-harness round this was imported from, if any.
scene_pins	dict[str, bool]	Scene runtime: keys the round pinned (no CLI flag exists for them); read with stack_argv for the whole executed stack.

Example

m = ValidationRoundMetadata( ... round_id="2026-08-22-master-1", ... started_at="2026-08-22T10:02:00Z", ... executed_sha="2edcf67" + "0" * 33, ... worktree_clean=True, ... ) m.safety_overrides_absent True

ros_automatic_discovery_range: str = '' class-attribute instance-attribute

ROS_AUTOMATIC_DISCOVERY_RANGE; "" means unset, i.e. SUBNET — discovery across the whole subnet, which is how a robot on another host reaches a simulation.

ros_domain_id: str = '' class-attribute instance-attribute

ROS_DOMAIN_ID the round ran on; "" means unset, i.e. domain 0.

Recorded from #227 onward. Before it, no round's DDS scope was captured, so it is not knowable after the fact whether one shared a ROS graph with another machine — which is exactly what happened on 2026-09-05, when a sim on the default scope consumed a live OpenArm's /joint_states.

ValidationRoundVerdicts

Bases: BaseModel

One validation round's machine-readable result — verdicts.json.

This is the on-disk contract tools/validation_matrix.py writes and docs/reference/collision-validation-evidence.md is fed from.

Attributes:

Name	Type	Description
schema_version	str	On-disk format version.
metadata	ValidationRoundMetadata	Reproducibility metadata for the round.
scenes	list[ValidationSceneVerdict]	One verdict per scene, in execution order.

Example

r = ValidationRoundVerdicts( ... metadata=ValidationRoundMetadata( ... round_id="r1", ... started_at="2026-08-22T10:02:00Z", ... executed_sha="0" * 40, ... worktree_clean=True, ... ), ... scenes=[], ... ) r.schema_version '0.1'

from_json(path: str) -> ValidationRoundVerdicts classmethod

Load and validate a round's verdicts.json.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the JSON file.	required

Returns:

Type	Description
ValidationRoundVerdicts	A validated ValidationRoundVerdicts.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the JSON fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_json(cls, path: str) -> ValidationRoundVerdicts:
    """Load and validate a round's ``verdicts.json``.

    Args:
        path: Filesystem path to the JSON file.

    Returns:
        A validated ``ValidationRoundVerdicts``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the JSON fails schema validation.
    """
    import json as _json  # noqa: PLC0415  # reason: deferred import

    with open(path, encoding="utf-8") as fh:
        data = _json.load(fh)
    return cls.model_validate(data)

scene(name: str) -> ValidationSceneVerdict | None

Return the verdict for name, or None when absent.

Parameters:

Name	Type	Description	Default
name	str	The scene key.	required

Returns:

Type	Description
ValidationSceneVerdict | None	The matching ValidationSceneVerdict, if the round ran it.

Source code in python/core/src/openral_core/schemas.py

def scene(self, name: str) -> ValidationSceneVerdict | None:
    """Return the verdict for ``name``, or ``None`` when absent.

    Args:
        name: The scene key.

    Returns:
        The matching ``ValidationSceneVerdict``, if the round ran it.
    """
    return next((s for s in self.scenes if s.scene == name), None)

ValidationSceneDelta

Bases: BaseModel

How one scene's verdict moved between two rounds.

Attributes:

Name	Type	Description
scene	str	The scene key.
baseline_outcome	ValidationOutcome | None	Outcome in the baseline round, None if the scene is new.
outcome	ValidationOutcome | None	Outcome in the current round, None if the scene was dropped.
changed	bool	Whether the outcome differs.
changed_fields	dict[str, dict[str, object]]	Field-by-field deltas, as {field: {"from": ..., "to": ...}}, for the comparable subset (outcome, task success, tripping pair, distances, allowance).

Example

d = ValidationSceneDelta( ... scene="baguette", ... baseline_outcome="estop-collision-real", ... outcome="estop-collision-false-positive", ... changed=True, ... ) d.changed True

ValidationSceneVerdict

Bases: BaseModel

One scene of one validation round, as a queryable record.

Attributes:

Name	Type	Description
scene	str	Short scene key ("baguette", "sink_cup", "fridge", "utensil").
config_path	str	Repo-relative path of the DeployScene YAML that ran.
config_sha256	str | None	Digest of that YAML's bytes as executed, so a round can be reproduced even if the scene later changes.
seed	int	The scene seed in force.
prompt	str	The instruction dispatched to the policy.
rskill_id	str	The rSkill the run dispatched.
outcome	ValidationOutcome	The scene's outcome.
task_success_final	bool | None	success from the terminal sim.task_success_final line. None when the run never emitted one.
task_success_ever	bool | None	ever_succeeded from the same line.
task_success_steps	int | None	Simulator steps the episode ran.
task_success_transitions	int | None	Success-predicate transitions observed.
stop	ValidationStopEvidence | None	The kernel's verdict, when the run was stopped.
ground_truth	ValidationGroundTruthAdjudication | None	Adjudication of that stop against the simulator.
witness	ValidationWitnessTimeline	The attach / witness / declaration lifecycle.
monitor_records	int | None	How many records the run's monitor wrote from a ROS subscription — every line except its own monitor_started / monitor_stopped pair. None when no monitor JSONL exists. Zero means the monitor received nothing, which is a harness fault (its DDS participant never saw the graph) and must not be read as "the run stopped before there was anything to see".
dispatch_failure_reason	str	failure_reason reported by the action client, verbatim — or, when the dispatcher died before writing a status line at all, why it did.
wall_s	float | None	Wall-clock seconds the dispatch took.
artifacts	dict[str, str]	Round-relative paths of the files this verdict was derived from, so every field is traceable to a file.
harness_error_reason	str	Why the run produced no usable artifact set — the graph never came up, or the deploy CLI rejected its own argv. Non-empty exactly when outcome is harness-error, so a failed launch can never be read as a clean deadline.

Example

v = ValidationSceneVerdict( ... scene="fridge", ... config_path="scenes/deploy/robocasa_fridge_drawer.yaml", ... seed=1, ... outcome="estop-initial-configuration", ... ) v.outcome 'estop-initial-configuration'

ValidationStopEvidence

Bases: BaseModel

The kernel's safety.collision verdict for one scene, transcribed.

Every field is read verbatim out of the safety.collision log line the C++ kernel emits; nothing is recomputed. min_distance_m is the depth of the cell the kernel reported, sweep_min_distance_m the depth of the deepest cell its sweep found — they differ only when a support-contact witness exempted a deeper cell, which makes their inequality the authoritative "was an exemption live at the trip" evidence.

Attributes:

Name	Type	Description
kind	str	The kernel's collision kind ("world", "self").
party_a	str	First party — a robot link, or attached:<object_id> when the declared payload is the subject.
party_b	str	Second party — usually voxel_<index> for a world stop.
horizon_step	int	Predicted-horizon index the verdict was evaluated at. -1 is the reactive (measured-state) check.
min_distance_m	float	Reported penetration depth, negative when inside.
sweep_min_distance_m	float | None	Deepest cell the sweep found, negative when inside. None when the kernel build did not report it.
place_allowance_active	bool	Whether the ADR-0097 place-approach allowance was applied on the trip itself.
place_target	str	Declared place target at the trip, empty when none.
depth_is_box_bound	bool	Whether the reported min_distance_m is the OBB's bound rather than a measurement of the geometry the evidence names (#213). Set by the kernel only for a self-collision pair whose two links both ship a stage-2 hull and whose hulls the GJK found overlapping — GJK proves an overlap but does not size one. Sound (the hull is inside its box, so the box can only report more penetration) but loose: -31.97 mm for a ~1.5 mm hull interpenetration on panda_link5/panda_link7. An adjudicator must charge the box budget for such a stop, never treat the number as a hull measurement. False on a snapshot recorded before 213.
exemption_active	bool | None	True when sweep_min_distance_m is strictly deeper than min_distance_m — a deeper cell was exempted. None when the sweep depth was not reported.

Example

ev = ValidationStopEvidence( ... kind="world", ... party_a="panda_link5", ... party_b="voxel_170781", ... horizon_step=0, ... min_distance_m=-0.0209178, ... sweep_min_distance_m=-0.0209178, ... ) ev.exemption_active False

exemption_active: bool | None property

Whether the sweep found a deeper cell than the one reported.

involves_payload: bool property

Whether the declared payload — not a bare robot link — tripped.

ValidationWitnessTimeline

Bases: BaseModel

The attach / witness / place-declaration lifecycle of one scene.

Reconstructed from the run's monitor JSONL (producer side) and the kernel's own safety.support_witness_* / safety.place_region_* log lines (consumer side). Both are recorded because a disagreement between them is itself a finding.

Attributes:

Name	Type	Description
attach_t_s	float | None	Monitor time of the first payload attach, None if the run never grasped.
detach_t_s	float | None	Monitor time of the payload detach, if any.
support_id	str | None	The support body the witness last attested against.
kernel_witness_armed	int	safety.support_witness_armed line count.
kernel_witness_separated	int	safety.support_witness_separated count.
place_declaration_seen	bool	Whether a place declaration with a producer-measured region reached the monitor.
place_region_armed	bool	Whether the kernel armed the declared place region.
place_allowance_active_lines	int	How many lines disclosed place_allowance_active=1. 0 means the approach allowance never applied anywhere in the run.

Example

ValidationWitnessTimeline(attach_t_s=176.32).place_allowance_active_lines 0

VisionAttachmentRuntime

Bases: BaseModel

The real-hardware vision attachment leg of a deploy (DeployRuntime.vision_attachment).

When enabled, openral deploy launches the SAM 2.1 segmenter lifecycle node (openral_perception_ros segmenter_node) and turns on the HAL's vision attachment-evidence bridge (the vision_attachment_* HAL parameters), which publishes the grasped payload on /openral/attachment_state.

Coupling rule: on real hardware the leg ALWAYS turns the safety kernel's attached-payload check on with it (attached_collision_enabled, 1000 ms deadline), never the leg alone. The octomap bridge clears a published payload from the map and the robot self-filter removes it from the cloud regardless of the kernel flag, so a leg without the kernel check would make the payload invisible to every collision check.

Default off. Turning it on for a real cell is a Safety-WG decision (hazard log): the grasp trigger is the gripper's position stalling short of a close command (the OpenArm gripper reports no effort), its thresholds the manifest's robot-type constants (JointSpec.closure_calibration) derived from teleop data, not per cell.

Attributes:

Name	Type	Description
enabled	bool	Bring the leg up. Requires all four topics below.
camera	str	Manifest sensor name the segmenter and the bridge use (e.g. head_zed).
rgb_topic	str | None	The camera driver's RGB Image topic the segmenter caches.
rgb_camera_info_topic	str | None	The driver's CameraInfo for that RGB stream.
depth_topic	str | None	The driver's metric depth Image the bridge back-projects.
depth_camera_info_topic	str | None	The driver's CameraInfo for the depth stream.
segmenter_manifest	str	kind: segmenter rSkill manifest (repo- or scene-relative).
device	str	Segmenter torch device (auto / cuda / cpu).
deadline_s	float	How long the bridge waits for one segmenter reply.
evidence_timeout_s	float	How old the newest joint-state sample carrying a jaw position for every gripper may be before the attachment heartbeat stops (HAL param vision_attachment_evidence_timeout_s).
tf_frames	dict[str, str]	{manifest link: TF frame} renames where the live TF tree's frame names differ from the manifest's links.
grasp_target_enabled	bool	Run the pre-grasp target producer on the bridge (HAL param vision_attachment_grasp_target_enabled): it measures the region the kernel's grasp-target exemption arms with, so deploy run refuses DeployRuntime.grasp_allowance_enabled without it. Default off.
grasp_target_approach_m	float | None	Approach-armed grasp target (HAL param vision_attachment_grasp_target_approach_m; needs grasp_target_enabled): with no named target, a hand whose TCP comes within this many metres of occupied voxels arms a one-hand declaration measured around its jaws, and the rSkill runner arms a goal-scope declaration for every goal so the policy, not the reasoner, picks what to grasp. None = off (default). Calibration point; at most GraspDeclaration.MAX_HALF_EXTENT_M.
grasp_target_premeasure	bool	Pre-measure every object instance near a hand (or in a named search box) from head-camera segmentation before any hand arms, each camera box confirmed by the voxel map, and arm the region from the tracked instance nearest the hand on the arming tick (HAL param vision_attachment_grasp_target_premeasure; needs grasp_target_enabled), so the region exists before contact without the policy pausing for an unoccluded view. Source order: camera instance first, the map confirms it and holds it while the declared hand occludes it (docs/reference/object-primitives-design.md, issue #349). Prototype; default off.
grasp_target_margin_m	float	How far the grasp target is bloated, metres (HAL param vision_attachment_grasp_target_margin_m; design note §2.1 "Grasp-target margin"). Before the handover the region the kernel exempts for the declared finger links is the measured target grown by this margin on every face, downward too — the finger links may press up to this far into the support under the target (and into anything else within it) without a stop. From the handover on, the attached payload — the target lowered onto the measured support top and grown by it on the four sides and the top, never down — is also the region the kernel latches and keeps exempting the finger links over until the payload leaves it, carried with the hand and checked against the environment. The producer's own gates keep the measured region. Default 0.0 (no bloat, the safer side): a scene that wants the wider exemption names it (the real OpenArm cell's 20 mm octomap needs >= 0.03 to cover the support layer, design note §2.1). In [0, MAX_GRASP_TARGET_MARGIN_M] (0.05 m); the margin, not the measurement, shrinks when a bloated box would exceed the declaration's caps (logged). Calibration point, task-dependent; a Safety-WG setting (hazard row HZ-0115-32).
place_target_enabled	bool	Run the real place producer (HAL param vision_attachment_place_target_enabled): while a payload is held it measures the support surface directly under it from the voxel map, arms a place region for it and attests the map-support proximity witness. No place target needs naming. Default off.
release_timeout_s	float	How long the bridge's release window waits for the payload to clear the jaws (HAL param vision_attachment_release_timeout_s). Calibration point. The window's grid age and clearance are not scene knobs: the deploy launch derives them from the kernel's voxel deadline, world margin and octree resolution.

Example

VisionAttachmentRuntime(camera="head_zed").enabled False

WaitTool

Bases: _ReasonerToolBase

Tool variant — deliberate no-op: observe and wait for the next tick.

The reasoner's tool choice is forced (tool_choice="any" / "required"), so without this variant the LLM MUST act every tick even when the correct decision is "nothing to do yet" — e.g. a skill is mid- execution and progress is nominal, or the mission is finished and no new goal has arrived. Forcing an action in those states produces spurious read-only queries or operator-prompt spam. wait makes "do nothing this tick" an explicit, logged, traceable decision instead of a side effect.

Dispatch: none. The node records the call (rationale included) on the reasoner span and returns — no ROS traffic, no actuation, no state change.

Attributes:

Name	Type	Description
tool	Literal['wait']	Discriminator (always "wait").

WamEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WAM — a world-action-model discrepancy.

Attributes:

Name	Type	Description
kind	Literal['wam']	Discriminator (always "wam").
horizon	int	Rollout horizon at which the discrepancy was measured.
discrepancy	float	Scalar discrepancy in the WAM's native units.
wam_id	str	Identifier of the WAM that fired.

WorkspaceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WORKSPACE — EE pose left the safety AABB.

Attributes:

Name	Type	Description
kind	Literal['workspace']	Discriminator (always "workspace").
ee_name	str	End-effector that violated the box.
measured_xyz	tuple[float, float, float]	Measured position in metres.
box_min	tuple[float, float, float]	Box minimum corner (x_min, y_min, z_min).
box_max	tuple[float, float, float]	Box maximum corner (x_max, y_max, z_max).

WorldState

Bases: BaseModel

Snapshot consumed by Reasoner and Skills.

Attributes:

Name	Type	Description
stamp_ns	int	Snapshot timestamp in nanoseconds.
joint_state	JointState	Current joint state.
base_pose	Pose6D | None	Base link pose (mobile robots).
base_twist	tuple[float, float, float, float, float, float] | None	Base link twist (vx, vy, vz, wx, wy, wz).
policy_state	list[float] | None	Optional simulator-native proprioception vector for a checkpoint whose state includes more than joint positions/velocities. Populated through a typed HAL -> WorldState topic; never inferred.
ee_poses	dict[str, Pose6D]	End-effector poses keyed by EE name.
contact_forces	dict[str, tuple[float, ...]]	Contact forces keyed by contact name.
images	dict[str, str]	Sensor name → ROS 2 topic reference (not the raw image).
image_frames	dict[str, SensorFrame] | None	Optional per-sensor SensorFrame snapshot for no-ROS / in-process deployments. When None the consumer reads frames via images topic refs as before.
point_clouds	dict[str, str]	Sensor name → ROS 2 topic reference.
tactile	dict[str, str]	Sensor name → ROS 2 topic reference.
detected_objects	list[DetectedObject]	List of detected objects.
battery_pct	float | None	Battery percentage in [0, 100].
diagnostics	dict[str, Literal['ok', 'warn', 'error', 'stale']]	Per-component diagnostic status.
attached_objects	list[AttachedCollisionObject]	Collision objects carried by robot links. These are absent from world occupancy and remain collision-active as payloads.
attachment_revision	int	Monotonic producer revision for atomic snapshots.
attachment_stamp_ns	int	Timestamp of the last accepted attachment update.
place_declaration	PlaceDeclaration | None	The place-phase declaration in force for the carried payload as the evidence producer resolved it (ADR-0097 and its 2026-08-14 amendment), including the producer-measured PlaceRegion. None means no place phase is declared, which is no place witness and no approach allowance. It rides here rather than on a channel of its own because the amendment extends the payload already crossing the dispatch → HAL → World State boundary, and because the kernel must apply the region and the attachment set it is scoped to from one and the same snapshot.
grasp_declaration	GraspDeclaration | None	The grasp-phase declaration in force as the evidence producer resolved it (real pick-and-place design §2.1), including the producer-measured target region. None means no grasp phase is declared, i.e. no exemption. Carried beside place_declaration for the same one-snapshot reason.
occupancy_grid	OccupancyGridRef | None	Optional 2D occupancy grid reference for mobile-base footprint checks. None until populated; an absent or stale grid is treated as unavailable (fail-closed).

aabb_iou_2d(a: tuple[int, int, int, int], b: tuple[int, int, int, int]) -> float

2D axis-aligned bbox IoU (pixel space).

Parameters:

Name	Type	Description	Default
a	tuple[int, int, int, int]	First box as (x_min, y_min, x_max, y_max).	required
b	tuple[int, int, int, int]	Second box as (x_min, y_min, x_max, y_max).	required

Returns:

Type	Description
float	Intersection-over-union in [0, 1]; 0.0 for disjoint or
float	degenerate (zero-area) boxes.

Example

aabb_iou_2d((0, 0, 10, 10), (0, 0, 10, 10)) 1.0 aabb_iou_2d((0, 0, 10, 10), (100, 100, 110, 110)) 0.0

Source code in python/core/src/openral_core/detection_tracker.py

def aabb_iou_2d(
    a: tuple[int, int, int, int],
    b: tuple[int, int, int, int],
) -> float:
    """2D axis-aligned bbox IoU (pixel space).

    Args:
        a: First box as ``(x_min, y_min, x_max, y_max)``.
        b: Second box as ``(x_min, y_min, x_max, y_max)``.

    Returns:
        Intersection-over-union in ``[0, 1]``; ``0.0`` for disjoint or
        degenerate (zero-area) boxes.

    Example:
        >>> aabb_iou_2d((0, 0, 10, 10), (0, 0, 10, 10))
        1.0
        >>> aabb_iou_2d((0, 0, 10, 10), (100, 100, 110, 110))
        0.0
    """
    ix = max(0, min(a[2], b[2]) - max(a[0], b[0]))
    iy = max(0, min(a[3], b[3]) - max(a[1], b[1]))
    inter = ix * iy
    if inter <= 0:
        return 0.0
    area_a = max(0, a[2] - a[0]) * max(0, a[3] - a[1])
    area_b = max(0, b[2] - b[0]) * max(0, b[3] - b[1])
    union = area_a + area_b - inter
    return inter / union if union > 0 else 0.0

apply_sensor_overlays(sensors: Iterable[SensorSpec], overlays: Iterable[SensorOverlay]) -> list[SensorSpec]

Robot-manifest sensors with each SensorOverlay's set fields laid over them.

An overlay only replaces the per-host / per-unit fields SensorOverlay allows (deploy_binding, ros2_topic, static_transform_xyz_rpy, intrinsics); name, modality and frames stay the manifest's. The result is re-validated as a SensorSpec.

Raises:

Type	Description
ROSConfigError	An overlay names no manifest sensor, or names one twice.

Example

desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml") fix = SensorOverlay(name="top", static_transform_xyz_rpy=(0, 0, 0.5, 0, 0, 0)) apply_sensor_overlays(desc.sensors, [fix])[0].static_transform_xyz_rpy (0.0, 0.0, 0.5, 0.0, 0.0, 0.0)

Source code in python/core/src/openral_core/schemas.py

def apply_sensor_overlays(
    sensors: Iterable[SensorSpec], overlays: Iterable[SensorOverlay]
) -> list[SensorSpec]:
    """Robot-manifest sensors with each ``SensorOverlay``'s set fields laid over them.

    An overlay only replaces the per-host / per-unit fields ``SensorOverlay`` allows
    (``deploy_binding``, ``ros2_topic``, ``static_transform_xyz_rpy``, ``intrinsics``);
    name, modality and frames stay the manifest's. The result is re-validated as a
    ``SensorSpec``.

    Raises:
        ROSConfigError: An overlay names no manifest sensor, or names one twice.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml")
        >>> fix = SensorOverlay(name="top", static_transform_xyz_rpy=(0, 0, 0.5, 0, 0, 0))
        >>> apply_sensor_overlays(desc.sensors, [fix])[0].static_transform_xyz_rpy
        (0.0, 0.0, 0.5, 0.0, 0.0, 0.0)
    """
    by_name: dict[str, SensorOverlay] = {}
    for overlay in overlays:
        if overlay.name in by_name:
            raise ROSConfigError(f"sensor overlay names {overlay.name!r} twice")
        by_name[overlay.name] = overlay
    out = list(sensors)
    unknown = sorted(set(by_name) - {s.name for s in out})
    if unknown:
        raise ROSConfigError(
            f"sensor overlay(s) {', '.join(repr(n) for n in unknown)} name no robot-manifest "
            f"sensor (manifest has: {', '.join(s.name for s in out)})"
        )
    return [
        SensorSpec.model_validate(
            {**s.model_dump(), **by_name[s.name].model_dump(exclude={"name"}, exclude_unset=True)}
        )
        if s.name in by_name
        else s
        for s in out
    ]

assert_vla_reward_fits(vla: RSkillManifest, reward: RSkillManifest, gpu_total_gb: float, *, margin_gb: float = 0.5) -> float

Verify a VLA + its paired reward model co-reside in GPU VRAM.

A VLA emits no success signal of its own, so the reasoner needs the reward model resident alongside the running policy. This is the pre-load gate: both sizes are declared in their manifests, so we check the pair fits before the VLA is ever loaded — failing fast and loud instead of a mid-run CUDA OOM. It checks the model pair footprint only (a necessary condition); the sim / ROS overhead is budgeted separately.

Parameters:

Name	Type	Description	Default
vla	RSkillManifest	The VLA manifest (kind == "vla") about to be loaded.	required
reward	RSkillManifest	Its paired reward manifest (kind == "reward").	required
gpu_total_gb	float	The GPU's total VRAM in GB.	required
margin_gb	float	Headroom reserved for fragmentation / framework overhead.	0.5

Returns:

Type	Description
float	The combined VRAM (GB) the pair requires (vla + reward).

Raises:

Type	Description
ROSConfigError	Either manifest does not declare min_vram_gb for its active dtype — co-residency we are about to require cannot be verified, so the operator must declare the size.
ROSGPUMemoryError	vla + reward + margin_gb > gpu_total_gb — the pair does not fit; the deploy must not run a VLA without its reward signal.

See tests/unit/test_vla_reward_pairing.py for fixture-backed coverage (smolvla-libero + robometer-4b-nf4 = 6.7 GB, fits 8 GB, OOMs a 4 GB card).

Source code in python/core/src/openral_core/schemas.py

def assert_vla_reward_fits(
    vla: RSkillManifest,
    reward: RSkillManifest,
    gpu_total_gb: float,
    *,
    margin_gb: float = 0.5,
) -> float:
    """Verify a VLA + its paired reward model co-reside in GPU VRAM.

    A VLA emits no success signal of its own, so the reasoner needs the reward
    model resident *alongside* the running policy. This is the
    pre-load gate: both sizes are declared in their manifests, so we check the
    pair fits before the VLA is ever loaded — failing fast and loud instead of a
    mid-run CUDA OOM. It checks the **model pair footprint** only (a necessary
    condition); the sim / ROS overhead is budgeted separately.

    Args:
        vla: The VLA manifest (``kind == "vla"``) about to be loaded.
        reward: Its paired reward manifest (``kind == "reward"``).
        gpu_total_gb: The GPU's total VRAM in GB.
        margin_gb: Headroom reserved for fragmentation / framework overhead.

    Returns:
        The combined VRAM (GB) the pair requires (``vla + reward``).

    Raises:
        ROSConfigError: Either manifest does not declare ``min_vram_gb`` for its
            active dtype — co-residency we are about to *require* cannot be
            verified, so the operator must declare the size.
        ROSGPUMemoryError: ``vla + reward + margin_gb > gpu_total_gb`` — the pair
            does not fit; the deploy must not run a VLA without its reward signal.

    See ``tests/unit/test_vla_reward_pairing.py`` for fixture-backed coverage
    (smolvla-libero + robometer-4b-nf4 = 6.7 GB, fits 8 GB, OOMs a 4 GB card).
    """
    vla_gb = vla.active_min_vram_gb()
    reward_gb = reward.active_min_vram_gb()
    missing = [m.name for m, gb in ((vla, vla_gb), (reward, reward_gb)) if gb is None]
    if missing:
        raise ROSConfigError(
            f"cannot verify VLA+reward VRAM co-residency: {missing!r} do not declare "
            f"min_vram_gb for their active dtype. Declare it so the pair can be "
            f"checked before load."
        )
    assert vla_gb is not None and reward_gb is not None  # narrowed by the guard above
    combined = vla_gb + reward_gb
    if combined + margin_gb > gpu_total_gb:
        raise ROSGPUMemoryError(
            f"VLA {vla.name!r} ({vla_gb:.2f} GB @ {vla.quantization.dtype.value}) + "
            f"reward {reward.name!r} ({reward_gb:.2f} GB @ {reward.quantization.dtype.value}) "
            f"= {combined:.2f} GB + {margin_gb:.2f} GB margin exceeds GPU VRAM "
            f"{gpu_total_gb:.2f} GB. A VLA must run with its reward model resident; "
            f"pick a smaller-footprint pair or a larger GPU."
        )
    return combined

camera_topic(name: str, kind: CameraTopicKind = CameraTopicKind.IMAGE, *, prefix: str = CAMERA_TOPIC_PREFIX) -> str

The canonical ROS topic of camera name's kind stream.

<prefix>/<name>/<kind> — the only place the camera layout is spelled. Producers (sim bridge, sensor leg) and consumers (world state, perception, reasoner, Foxglove, launch files) all build through it, so a camera's publisher and subscribers can never drift apart.

Parameters:

Name	Type	Description	Default
name	str	SensorSpec.name of the camera.	required
kind	CameraTopicKind	Which stream of that camera.	IMAGE
prefix	str	Topic root; override only for a namespaced graph.	CAMERA_TOPIC_PREFIX

Returns:

Type	Description
str	The absolute topic name.

Raises:

Type	Description
ROSConfigError	name is empty or contains /.

Example

camera_topic("wrist") '/openral/cameras/wrist/image' camera_topic("front_depth", CameraTopicKind.POINTS) '/openral/cameras/front_depth/points'

Source code in python/core/src/openral_core/schemas.py

def camera_topic(
    name: str,
    kind: CameraTopicKind = CameraTopicKind.IMAGE,
    *,
    prefix: str = CAMERA_TOPIC_PREFIX,
) -> str:
    """The canonical ROS topic of camera ``name``'s ``kind`` stream.

    ``<prefix>/<name>/<kind>`` — the only place the camera layout is spelled.
    Producers (sim bridge, sensor leg) and consumers (world state, perception,
    reasoner, Foxglove, launch files) all build through it, so a camera's
    publisher and subscribers can never drift apart.

    Args:
        name: ``SensorSpec.name`` of the camera.
        kind: Which stream of that camera.
        prefix: Topic root; override only for a namespaced graph.

    Returns:
        The absolute topic name.

    Raises:
        ROSConfigError: ``name`` is empty or contains ``/``.

    Example:
        >>> camera_topic("wrist")
        '/openral/cameras/wrist/image'
        >>> camera_topic("front_depth", CameraTopicKind.POINTS)
        '/openral/cameras/front_depth/points'
    """
    if not name or "/" in name:
        raise ROSConfigError(f"camera name must be a non-empty single path segment, got {name!r}")
    return f"{prefix}/{name}/{kind.value}"

canonical_slots_for_representation(rep: ActionRepresentation, *, dim: int, description: RobotDescription) -> list[ActionSlot] | None

Build the canonical ActionSlot layout for a representation.

The skill_runner calls this to expand a skill that declares only ActionContract.representation (no explicit slots) into a typed slot layout it can dispatch. Joint representations return None so the caller keeps the legacy whole-vector JOINT_POSITION path; cartesian / gripper representations get one slot per control mode, addressed at the robot's primary end-effector (description.end_effectors[0]).

EndEffectorSpec carries no explicit tf-frame field, so the end-effector's name is used as the slot frame (the EE name is the tf frame the cartesian pose/delta is expressed in for the canonical embodiments).

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required
dim	int	Dimensionality of the policy's flat action vector.	required
description	RobotDescription	The target robot — its primary end-effector names the cartesian/gripper slots.	required

Returns:

Type	Description
list[ActionSlot] | None	For joint representations: None (caller keeps the legacy
list[ActionSlot] | None	whole-vector JOINT_POSITION path). Otherwise a list of typed
list[ActionSlot] | None	ActionSlot s that satisfy the per-slot validators.

Raises:

Type	Description
ROSConfigError	If the representation needs an end-effector but description.end_effectors is empty, or if dim is too small for the layout (DELTA_EE_6D / CARTESIAN_POSE need dim >= 6; DELTA_EE_6D_PLUS_GRIPPER needs dim >= 7).

Example

desc = RobotDescription( ... name="ex_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")], ... capabilities=RobotCapabilities(), ... safety=SafetyEnvelope(), ... ) slots = canonical_slots_for_representation( ... ActionRepresentation.DELTA_EE_6D, dim=6, description=desc ... ) slots[0].control_mode is ControlMode.CARTESIAN_DELTA True

Source code in python/core/src/openral_core/schemas.py

def canonical_slots_for_representation(
    rep: ActionRepresentation,
    *,
    dim: int,
    description: RobotDescription,
) -> list[ActionSlot] | None:
    """Build the canonical ``ActionSlot`` layout for a representation.

    The skill_runner calls this to expand a skill that declares
    only ``ActionContract.representation`` (no explicit ``slots``) into a
    typed slot layout it can dispatch. Joint representations return
    ``None`` so the caller keeps the legacy whole-vector ``JOINT_POSITION``
    path; cartesian / gripper representations get one slot per control
    mode, addressed at the robot's primary end-effector
    (``description.end_effectors[0]``).

    ``EndEffectorSpec`` carries no explicit tf-frame field, so the
    end-effector's ``name`` is used as the slot ``frame`` (the EE name is
    the tf frame the cartesian pose/delta is expressed in for the
    canonical embodiments).

    Args:
        rep: The VLA's declared action-vector representation.
        dim: Dimensionality of the policy's flat action vector.
        description: The target robot — its primary end-effector names
            the cartesian/gripper slots.

    Returns:
        For joint representations: ``None`` (caller keeps the legacy
        whole-vector ``JOINT_POSITION`` path). Otherwise a list of typed
        ``ActionSlot`` s that satisfy the per-slot validators.

    Raises:
        ROSConfigError: If the representation needs an end-effector but
            ``description.end_effectors`` is empty, or if ``dim`` is too
            small for the layout (``DELTA_EE_6D`` / ``CARTESIAN_POSE``
            need ``dim >= 6``; ``DELTA_EE_6D_PLUS_GRIPPER`` needs
            ``dim >= 7``).

    Example:
        >>> desc = RobotDescription(
        ...     name="ex_robot",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> slots = canonical_slots_for_representation(
        ...     ActionRepresentation.DELTA_EE_6D, dim=6, description=desc
        ... )
        >>> slots[0].control_mode is ControlMode.CARTESIAN_DELTA
        True
    """
    if rep in (ActionRepresentation.JOINT_POSITIONS, ActionRepresentation.JOINT_VELOCITIES):
        # Legacy whole-vector JOINT path — the caller emits one implicit
        # JOINT_POSITION / JOINT_VELOCITY slot covering the whole vector.
        return None

    if not description.end_effectors:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} addresses "
            "an end-effector but the robot description declares no end_effectors; "
            "cannot build a cartesian/gripper slot layout."
        )
    primary = description.end_effectors[0]
    ee_name = primary.name
    # EndEffectorSpec has no explicit tf-frame field; the EE name is the
    # tf frame the cartesian pose/delta is expressed in.
    ee_frame = ee_name

    has_gripper = rep in (
        ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER,
        ActionRepresentation.DELTA_EE_3D_PLUS_GRIPPER,
    )
    # Translation-only representations carry a 3-wide cartesian slice; all
    # others (6-DoF delta / cartesian pose) carry the full 6-wide slice.
    cart_width = (
        _EE_3D_WIDTH if rep is ActionRepresentation.DELTA_EE_3D_PLUS_GRIPPER else _EE_6D_WIDTH
    )
    min_dim = cart_width + 1 if has_gripper else cart_width
    if dim < min_dim:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} requires "
            f"dim >= {min_dim} but action_contract.dim={dim} is too small for the layout."
        )

    cart_mode = (
        ControlMode.CARTESIAN_POSE
        if rep is ActionRepresentation.CARTESIAN_POSE
        else ControlMode.CARTESIAN_DELTA
    )
    slots: list[ActionSlot] = [
        ActionSlot(
            range=(0, cart_width - 1),
            control_mode=cart_mode,
            ee=ee_name,
            frame=ee_frame,
        )
    ]
    if has_gripper:
        slots.append(
            ActionSlot(
                range=(cart_width, dim - 1),
                control_mode=ControlMode.GRIPPER_POSITION,
                ee=ee_name,
            )
        )
    return slots

check_scene_sensor_overrides(manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec]) -> None

Refuse a scene sensor entry that names a sensor the robot manifest defines.

A sensor the robot manifest declares belongs to the robot: its name, modality and frames live in robot.yaml, so every scene on that robot sees the same camera. DeployScene.sensors only adds workcell cameras, under names the manifest does not use, and those carry their own geometry and binding. What differs per host or per physical unit (device path, driver topic, calibrated mount and intrinsics) goes in a RobotUnit overlay instead (resolve_sensor_overlays).

Parameters:

Name	Type	Description	Default
manifest_sensors	Iterable[SensorSpec]	The robot manifest's sensors.	required
scene_sensors	Iterable[SensorSpec]	The DeployScene.sensors entries.	required

Raises:

Type	Description
ROSConfigError	A scene entry reuses a manifest sensor's name.

Example

desc = RobotDescription.from_yaml("robots/openarm/robot.yaml") check_scene_sensor_overrides(desc.sensors, [])

Source code in python/core/src/openral_core/schemas.py

def check_scene_sensor_overrides(
    manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec]
) -> None:
    """Refuse a scene sensor entry that names a sensor the robot manifest defines.

    A sensor the robot manifest declares belongs to the robot: its name, modality and
    frames live in ``robot.yaml``, so every scene on that robot sees the same camera.
    ``DeployScene.sensors`` only adds workcell cameras, under names the manifest does
    not use, and those carry their own geometry and binding. What differs per host or
    per physical unit (device path, driver topic, calibrated mount and intrinsics) goes
    in a ``RobotUnit`` overlay instead (``resolve_sensor_overlays``).

    Args:
        manifest_sensors: The robot manifest's ``sensors``.
        scene_sensors: The ``DeployScene.sensors`` entries.

    Raises:
        ROSConfigError: A scene entry reuses a manifest sensor's name.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> check_scene_sensor_overrides(desc.sensors, [])
    """
    robot_names = {s.name for s in manifest_sensors}
    clashes = [entry.name for entry in scene_sensors if entry.name in robot_names]
    if clashes:
        raise ROSConfigError(
            f"scene sensor(s) {', '.join(repr(n) for n in clashes)} are defined by the robot "
            "manifest. A deploy scene never redefines a robot camera — its name, modality and "
            "frames live in robots/<robot_id>/robot.yaml. Put a per-host binding or per-unit "
            "calibration in a unit overlay (robots/<robot_id>/units/<unit>.yaml, selected by "
            "the scene's robot_unit or $OPENRAL_ROBOT_UNIT), or give a workcell camera its own "
            "name."
        )

check_time_base(sim_dt_per_tick_s: float, control_freq_hz: float | None, *, backend: str, robot: str) -> None

Refuse a sim backend whose step is not one control period of the robot.

Parameters:

Name	Type	Description	Default
sim_dt_per_tick_s	float	Sim seconds one step / action advances.	required
control_freq_hz	float | None	RobotDescription.control_rate_hz; None when the manifest declares no rate.	required
backend	str	Name for the message (the env / HAL class).	required
robot	str	RobotDescription.name for the message.	required

Raises:

Type	Description
ROSConfigError	the manifest declares no control rate, or the tick differs from 1 / control_freq_hz by more than TIME_BASE_TOL (a NaN tick is refused too).

Example

check_time_base(1 / 30, 30.0, backend="x", robot="r") check_time_base(1 / 60, 30.0, backend="x", robot="r") Traceback (most recent call last): ... openral_core.exceptions.ROSConfigError: ...

Source code in python/core/src/openral_core/time_base.py

def check_time_base(
    sim_dt_per_tick_s: float, control_freq_hz: float | None, *, backend: str, robot: str
) -> None:
    """Refuse a sim backend whose step is not one control period of the robot.

    Args:
        sim_dt_per_tick_s: Sim seconds one step / action advances.
        control_freq_hz: ``RobotDescription.control_rate_hz``; ``None`` when the
            manifest declares no rate.
        backend: Name for the message (the env / HAL class).
        robot: ``RobotDescription.name`` for the message.

    Raises:
        ROSConfigError: the manifest declares no control rate, or the tick
            differs from ``1 / control_freq_hz`` by more than ``TIME_BASE_TOL``
            (a NaN tick is refused too).

    Example:
        >>> check_time_base(1 / 30, 30.0, backend="x", robot="r")
        >>> check_time_base(1 / 60, 30.0, backend="x", robot="r")
        Traceback (most recent call last):
        ...
        openral_core.exceptions.ROSConfigError: ...
    """
    if control_freq_hz is None:
        raise ROSConfigError(
            f"{backend}: steps {sim_dt_per_tick_s:.6f} s of sim time per action, but robot "
            f"{robot!r} declares no action_spec.control_freq_hz to hold it to. Declare the "
            "control rate in the manifest."
        )
    # ``not <=`` so a NaN tick is refused too (NaN compares False both ways).
    if not abs(sim_dt_per_tick_s * control_freq_hz - 1.0) <= TIME_BASE_TOL:
        raise ROSConfigError(
            f"{backend}: steps {sim_dt_per_tick_s:.6f} s of sim time per action, but robot "
            f"{robot!r} ticks at {control_freq_hz:g} Hz ({1.0 / control_freq_hz:.6f} s). One "
            "step must be one control period, or the rehearsal runs at the wrong speed; fix "
            "the backend's physics steps per tick, not the robot's rate."
        )

contains_rskill_template_sentinel(text: str | None) -> bool

True when text carries an unresolved rSkill scaffold sentinel.

Example

contains_rskill_template_sentinel("TEMPLATE_ORG/rskill-TEMPLATE_ID") True contains_rskill_template_sentinel("OpenRAL/rskill-qwen35_4b-any-general-nf4") False contains_rskill_template_sentinel(None) False

Source code in python/core/src/openral_core/schemas.py

def contains_rskill_template_sentinel(text: str | None) -> bool:
    """True when ``text`` carries an unresolved rSkill scaffold sentinel.

    Example:
        >>> contains_rskill_template_sentinel("TEMPLATE_ORG/rskill-TEMPLATE_ID")
        True
        >>> contains_rskill_template_sentinel("OpenRAL/rskill-qwen35_4b-any-general-nf4")
        False
        >>> contains_rskill_template_sentinel(None)
        False
    """
    if not text:
        return False
    return any(s in text for s in RSKILL_TEMPLATE_SENTINELS)

control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]

Map an ActionRepresentation to the ControlMode s it drives.

Used by the reasoner's deploy-path palette gate: a skill is only offered when the target robot advertises every mode in the returned set.

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required

Returns:

Type	Description
set[ControlMode]	The set of ControlMode s the representation maps onto.

Example

control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == { ... ControlMode.CARTESIAN_DELTA, ... ControlMode.GRIPPER_POSITION, ... } True

Source code in python/core/src/openral_core/schemas.py

def control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]:
    """Map an ``ActionRepresentation`` to the ``ControlMode`` s it drives.

    Used by the reasoner's deploy-path palette gate: a skill is
    only offered when the target robot advertises *every* mode in the
    returned set.

    Args:
        rep: The VLA's declared action-vector representation.

    Returns:
        The set of ``ControlMode`` s the representation maps onto.

    Example:
        >>> control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == {
        ...     ControlMode.CARTESIAN_DELTA,
        ...     ControlMode.GRIPPER_POSITION,
        ... }
        True
    """
    if rep is ActionRepresentation.JOINT_POSITIONS:
        return {ControlMode.JOINT_POSITION}
    if rep is ActionRepresentation.JOINT_VELOCITIES:
        return {ControlMode.JOINT_VELOCITY}
    if rep is ActionRepresentation.DELTA_EE_6D:
        return {ControlMode.CARTESIAN_DELTA}
    if rep in (
        ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER,
        ActionRepresentation.DELTA_EE_3D_PLUS_GRIPPER,
    ):
        return {ControlMode.CARTESIAN_DELTA, ControlMode.GRIPPER_POSITION}
    # CARTESIAN_POSE — the only remaining enum member.
    return {ControlMode.CARTESIAN_POSE}

deploy_cloud_topic(manifest_sensors: Iterable[SensorSpec], *, pinned: str | None, hal_mode: str, estimated_camera: str | None = None) -> str

The PointCloud2 topic a deploy's cloud consumers read, or "" when it has none.

octomap_server (cloud_in) and the world-state object lift's depth fallback both read it. A pinned DeployRuntime.octomap_cloud_topic wins. Next, a DeployRuntime.estimated_depth_camera: the depth provider publishes that RGB camera's DA3 cloud on camera_topic(<name>, POINTS), sim and real alike. Otherwise, in sim, the sim sensor bridge back-projects each :attr:SensorSpec.is_depth_camera onto camera_topic(<name>, POINTS), so exactly one such camera names the cloud. On a real deploy nothing in-tree publishes a cloud (the sensor leg publishes depth images), so the answer is "" until the driver's own topic is pinned. The one rule openral deploy (pre-launch refusal) and deploy_e2e.launch.py share.

Raises:

Type	Description
ROSConfigError	sim, nothing pinned, and several depth cameras: which one is mapped must be declared (pin runtime.octomap_cloud_topic), not "first one wins".

Example

desc = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml") deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="sim") '/openral/cameras/front_depth/points' deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real") '' deploy_cloud_topic(desc.sensors, pinned="/camera/depth/color/points", hal_mode="real") '/camera/depth/color/points' deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real", estimated_camera="head") '/openral/cameras/head/points'

Source code in python/core/src/openral_core/schemas.py

def deploy_cloud_topic(
    manifest_sensors: Iterable[SensorSpec],
    *,
    pinned: str | None,
    hal_mode: str,
    estimated_camera: str | None = None,
) -> str:
    """The ``PointCloud2`` topic a deploy's cloud consumers read, or ``""`` when it has none.

    ``octomap_server`` (``cloud_in``) and the world-state object lift's depth fallback
    both read it. A pinned ``DeployRuntime.octomap_cloud_topic`` wins. Next, a
    ``DeployRuntime.estimated_depth_camera``: the depth provider publishes that RGB
    camera's DA3 cloud on ``camera_topic(<name>, POINTS)``, sim and real alike.
    Otherwise, in sim, the sim sensor bridge back-projects each :attr:`SensorSpec.is_depth_camera`
    onto ``camera_topic(<name>, POINTS)``, so exactly one such camera names the cloud.
    On a real deploy nothing in-tree publishes a cloud (the sensor leg publishes depth
    *images*), so the answer is ``""`` until the driver's own topic is pinned. The one
    rule ``openral deploy`` (pre-launch refusal) and ``deploy_e2e.launch.py`` share.

    Raises:
        ROSConfigError: sim, nothing pinned, and several depth cameras: which one is
            mapped must be declared (pin ``runtime.octomap_cloud_topic``), not "first
            one wins".

    Example:
        >>> desc = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml")
        >>> deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="sim")
        '/openral/cameras/front_depth/points'
        >>> deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real")
        ''
        >>> deploy_cloud_topic(desc.sensors, pinned="/camera/depth/color/points", hal_mode="real")
        '/camera/depth/color/points'
        >>> deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real", estimated_camera="head")
        '/openral/cameras/head/points'
    """
    if pinned:
        return pinned
    if estimated_camera:
        return camera_topic(estimated_camera, CameraTopicKind.POINTS)
    if hal_mode == "real":
        return ""
    depth = [s.name for s in manifest_sensors if s.is_depth_camera]
    if len(depth) > 1:
        raise ROSConfigError(
            f"several depth cameras ({', '.join(depth)}) could feed the world map; pin "
            "runtime.octomap_cloud_topic to the one to map, e.g. "
            f"{camera_topic(depth[0], CameraTopicKind.POINTS)!r}"
        )
    return camera_topic(depth[0], CameraTopicKind.POINTS) if depth else ""

detect_gpu_vram_gb(field: str) -> float

One nvidia-smi --query-gpu=<field> value for GPU 0 in GB, or 0.0.

Deliberately torch-free — callers must not pull in torch just to size the GPU. Any failure (no nvidia-smi, no GPU, parse error) returns 0.0; callers skip their check rather than blocking on a host where the value can't be read. field is an nvidia-smi --query-gpu column, e.g. memory.total or memory.free.

Example

detect_gpu_vram_gb("memory.total") >= 0.0 True

Source code in python/core/src/openral_core/gpu.py

def detect_gpu_vram_gb(field: str) -> float:
    """One ``nvidia-smi --query-gpu=<field>`` value for GPU 0 in GB, or ``0.0``.

    Deliberately torch-free — callers must not pull in torch just to size the
    GPU. Any failure (no ``nvidia-smi``, no GPU, parse error) returns ``0.0``;
    callers skip their check rather than blocking on a host where the value
    can't be read. ``field`` is an ``nvidia-smi --query-gpu`` column, e.g.
    ``memory.total`` or ``memory.free``.

    Example:
        >>> detect_gpu_vram_gb("memory.total") >= 0.0
        True
    """
    try:
        out = subprocess.run(
            ["nvidia-smi", f"--query-gpu={field}", "--format=csv,noheader,nounits"],
            capture_output=True,
            text=True,
            timeout=5.0,
            check=True,
        )
    except (OSError, subprocess.SubprocessError):
        return 0.0
    lines = out.stdout.strip().splitlines()
    if not lines:
        return 0.0
    try:
        return float(lines[0].strip()) / 1024.0  # MiB → GiB
    except ValueError:
        return 0.0

expected_repo_name(manifest: RSkillManifest) -> str

Suggest the canonical HF-repo name for an rSkill manifest.

Returns a name that satisfies repo_name_is_canonical for the manifest's kind — the value the publisher prints on a mismatch and --fix-name writes back. The <task> segment is a suggested default (see _task_slug_for_manifest); it is author-owned and the migration agent may refine it. The owner prefix is preserved from RSkillManifest.name (default _DEFAULT_RSKILL_OWNER).

Playbooks map to <owner>/rskill-playbook-<name>; ROS wrappers (ros_action / ros_service) to <owner>/rskill-<model>-<robot>-<task> (no <quant>); every other kind to <owner>/rskill-<model>-<robot>-<task>-<quant>.

Raises:

Type	Description
ValueError	If no canonical <model> token can be determined (see _model_token_for_manifest).

Example

m = RSkillManifest( ... name="OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16", ... version="0.1.0", ... license=RSkillLicensePosture.APACHE_2_0, ... role="s1", ... kind="vla", ... model_family="smolvla", ... embodiment_tags=["franka_panda"], ... runtime=RSkillRuntime.PYTORCH, ... weights_uri="hf://lerobot/smolvla_base", ... chunk_size=16, ... latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0), ... quantization=QuantizationConfig(dtype=QuantizationDtype.BF16), ... evaluated_tasks=["libero_spatial"], ... description="SmolVLA on LIBERO.", ... actions=[RSkillAction.GENERALIST], ... actuators_required=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ), ... ], ... processors=RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json", ... ), ... ) expected_repo_name(m) 'OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'

Source code in python/core/src/openral_core/schemas.py

def expected_repo_name(manifest: RSkillManifest) -> str:
    """Suggest the canonical HF-repo name for an rSkill manifest.

    Returns a name that satisfies ``repo_name_is_canonical`` for the
    manifest's ``kind`` — the value the publisher prints on a mismatch and
    ``--fix-name`` writes back. The ``<task>`` segment is a *suggested default*
    (see ``_task_slug_for_manifest``); it is author-owned and the migration
    agent may refine it. The owner prefix is preserved from
    ``RSkillManifest.name`` (default ``_DEFAULT_RSKILL_OWNER``).

    Playbooks map to ``<owner>/rskill-playbook-<name>``; ROS wrappers
    (``ros_action`` / ``ros_service``) to ``<owner>/rskill-<model>-<robot>-<task>``
    (no ``<quant>``); every other kind to
    ``<owner>/rskill-<model>-<robot>-<task>-<quant>``.

    Raises:
        ValueError: If no canonical ``<model>`` token can be determined (see
            ``_model_token_for_manifest``).

    Example:
        >>> m = RSkillManifest(
        ...     name="OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16",
        ...     version="0.1.0",
        ...     license=RSkillLicensePosture.APACHE_2_0,
        ...     role="s1",
        ...     kind="vla",
        ...     model_family="smolvla",
        ...     embodiment_tags=["franka_panda"],
        ...     runtime=RSkillRuntime.PYTORCH,
        ...     weights_uri="hf://lerobot/smolvla_base",
        ...     chunk_size=16,
        ...     latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0),
        ...     quantization=QuantizationConfig(dtype=QuantizationDtype.BF16),
        ...     evaluated_tasks=["libero_spatial"],
        ...     description="SmolVLA on LIBERO.",
        ...     actions=[RSkillAction.GENERALIST],
        ...     actuators_required=[
        ...         ActuatorRequirement(
        ...             kind=ControlMode.JOINT_POSITION,
        ...             control_mode_semantics=ControlModeSemantics(mode="absolute"),
        ...         ),
        ...     ],
        ...     processors=RSkillProcessors(
        ...         preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json",
        ...         postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json",
        ...     ),
        ... )
        >>> expected_repo_name(m)
        'OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'
    """
    owner = manifest.name.split("/", 1)[0] if "/" in manifest.name else _DEFAULT_RSKILL_OWNER
    if manifest.kind == "playbook":
        tail = _repo_tail_slug(manifest.name).removeprefix("playbook_").strip("_")
        name = tail if tail and _RSKILL_NAME_SEGMENT_RE.match(tail) else "main"
        return f"{owner}/rskill-playbook-{name}"
    model = _model_token_for_manifest(manifest)
    robot = _robot_token_for_manifest(manifest)
    task = _task_slug_for_manifest(manifest, model, robot)
    # Weightless wrappers (ros_action / ros_service) omit the <quant> segment.
    if manifest.kind in _WEIGHTLESS_KINDS:
        return f"{owner}/rskill-{model}-{robot}-{task}"
    quant = _quant_token_for_dtype(manifest.quantization.dtype)
    return f"{owner}/rskill-{model}-{robot}-{task}-{quant}"

extract_base_sim_joint_names(description: RobotDescription) -> tuple[str, str, str] | None

Return (forward, side, yaw) MJCF joint names from any mobile-base description.

Generic, robot-agnostic helper. Consumes RobotDescription.base_joints + each referenced joint's JointSpec.sim_joint_name override. Works for any robot whose robot.yaml declares both:

• base_joints: [<forward>, <side>, <yaw>] at the top level.
• Each of the three referenced joints carries a sim_joint_name: "..." mapping its URDF-shape name to the MJCF/MuJoCo joint name the simulator emits (typically auto-prefixed under a composed scene).

Returns None when the description has no base_joints block, fewer than three entries (the schema permits arbitrary list length for future non-planar bases — this helper specialises to the planar-base case), or any referenced joint lacks sim_joint_name. Callers should treat None as "fall back to module defaults" — the sim-side ray-cast helpers in openral_sim.backends.robocasa accept this.

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot's manifest, loaded via RobotDescription.from_yaml.	required

Returns:

Type	Description
tuple[str, str, str] | None	(forward, side, yaw) MJCF joint names, or None when
tuple[str, str, str] | None	the description doesn't carry a complete mobile-base block.

Example

See robots/panda_mobile/robot.yaml for a real fixture.

desc = RobotDescription( ... name="ex", ... embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR, ... joints=[ ... JointSpec( ... name="base_x", ... joint_type=JointType.PRISMATIC, ... parent_link="world", ... child_link="base_x_link", ... sim_joint_name="mobilebase0_joint_mobile_forward", ... ), ... JointSpec( ... name="base_y", ... joint_type=JointType.PRISMATIC, ... parent_link="base_x_link", ... child_link="base_y_link", ... sim_joint_name="mobilebase0_joint_mobile_side", ... ), ... JointSpec( ... name="base_yaw", ... joint_type=JointType.REVOLUTE, ... parent_link="base_y_link", ... child_link="base_link", ... sim_joint_name="mobilebase0_joint_mobile_yaw", ... ), ... ], ... capabilities=RobotCapabilities(embodiment_tags=["ex"]), ... safety=SafetyEnvelope(), ... base_joints=["base_x", "base_y", "base_yaw"], ... ) extract_base_sim_joint_names(desc)[0] 'mobilebase0_joint_mobile_forward'

Source code in python/core/src/openral_core/schemas.py

def extract_base_sim_joint_names(
    description: RobotDescription,
) -> tuple[str, str, str] | None:
    """Return ``(forward, side, yaw)`` MJCF joint names from any mobile-base description.

    Generic, robot-agnostic helper. Consumes
    ``RobotDescription.base_joints`` + each referenced joint's
    ``JointSpec.sim_joint_name`` override. Works for any robot whose
    ``robot.yaml`` declares both:

    * ``base_joints: [<forward>, <side>, <yaw>]`` at the top level.
    * Each of the three referenced joints carries a
      ``sim_joint_name: "..."`` mapping its URDF-shape ``name`` to the
      MJCF/MuJoCo joint name the simulator emits (typically auto-prefixed
      under a composed scene).

    Returns ``None`` when the description has no ``base_joints`` block,
    fewer than three entries (the schema permits arbitrary list length for
    future non-planar bases — this helper specialises to the planar-base
    case), or any referenced joint lacks ``sim_joint_name``. Callers should
    treat ``None`` as "fall back to module defaults" — the sim-side
    ray-cast helpers in ``openral_sim.backends.robocasa`` accept this.

    Args:
        description: The robot's manifest, loaded via
            ``RobotDescription.from_yaml``.

    Returns:
        ``(forward, side, yaw)`` MJCF joint names, or ``None`` when
        the description doesn't carry a complete mobile-base block.

    Example:
        >>> # See `robots/panda_mobile/robot.yaml` for a real fixture.
        >>> desc = RobotDescription(
        ...     name="ex",
        ...     embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="base_x",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="world",
        ...             child_link="base_x_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_forward",
        ...         ),
        ...         JointSpec(
        ...             name="base_y",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="base_x_link",
        ...             child_link="base_y_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_side",
        ...         ),
        ...         JointSpec(
        ...             name="base_yaw",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_y_link",
        ...             child_link="base_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_yaw",
        ...         ),
        ...     ],
        ...     capabilities=RobotCapabilities(embodiment_tags=["ex"]),
        ...     safety=SafetyEnvelope(),
        ...     base_joints=["base_x", "base_y", "base_yaw"],
        ... )
        >>> extract_base_sim_joint_names(desc)[0]
        'mobilebase0_joint_mobile_forward'
    """
    # Planar mobile bases carry exactly three holonomic axes (forward,
    # side, yaw); the schema doesn't constrain `base_joints` length so
    # future non-planar bases can declare richer surfaces, but this
    # helper specialises here.
    planar_base_dof = 3
    if description.base_joints is None or len(description.base_joints) < planar_base_dof:
        return None
    by_name = {j.name: j for j in description.joints}
    sim_names: list[str | None] = []
    for ref in description.base_joints[:planar_base_dof]:
        spec = by_name.get(ref)
        if spec is None:
            return None
        sim_names.append(spec.sim_joint_name)
    if any(n is None for n in sim_names):
        return None
    forward, side, yaw = sim_names
    # Narrow tuple[str | None, ...] → tuple[str, str, str] for type checkers.
    assert isinstance(forward, str)
    assert isinstance(side, str)
    assert isinstance(yaw, str)
    return (forward, side, yaw)

gripper_hands(description: RobotDescription) -> tuple[tuple[str, ...], ...]

The robot's hands: its role: gripper joints' child_links, grouped per hand.

A hand is the set of gripper joints that hang off one arm: joints sharing a parent_link, plus any gripper joint whose parent is another gripper joint's child (a chained finger). An SO-101's one jaw joint is one hand; an OpenArm's two finger-pair joints are two hands (one per arm); an R1 Pro's four finger joints are two hands of two fingers. The reasoner's grasp grounding makes a declaration name one hand; the HAL's approach-armed grasp target arms for one hand at a time.

Example

gripper_hands(RobotDescription.from_yaml("robots/openarm/robot.yaml")) (('openarm_left_finger_pair',), ('openarm_right_finger_pair',)) gripper_hands(RobotDescription.from_yaml("robots/r1pro/robot.yaml"))[0] ('left_gripper_finger_link1', 'left_gripper_finger_link2')

Source code in python/core/src/openral_core/schemas.py

def gripper_hands(description: RobotDescription) -> tuple[tuple[str, ...], ...]:
    """The robot's hands: its ``role: gripper`` joints' ``child_link``s, grouped per hand.

    A hand is the set of gripper joints that hang off one arm: joints sharing a
    ``parent_link``, plus any gripper joint whose parent is another gripper joint's
    child (a chained finger). An SO-101's one jaw joint is one hand; an OpenArm's two
    finger-pair joints are two hands (one per arm); an R1 Pro's four finger joints are
    two hands of two fingers. The reasoner's grasp grounding makes a declaration name
    one hand; the HAL's approach-armed grasp target arms for one hand at a time.

    Example:
        >>> gripper_hands(RobotDescription.from_yaml("robots/openarm/robot.yaml"))
        (('openarm_left_finger_pair',), ('openarm_right_finger_pair',))
        >>> gripper_hands(RobotDescription.from_yaml("robots/r1pro/robot.yaml"))[0]
        ('left_gripper_finger_link1', 'left_gripper_finger_link2')
    """
    grippers = [j for j in description.joints if j.role == "gripper"]
    parent_of = {j.child_link: j.parent_link for j in grippers}
    hands: dict[str, list[str]] = {}
    for j in grippers:
        hands.setdefault(_gripper_mount(parent_of, j.parent_link), []).append(j.child_link)
    return tuple(tuple(links) for links in hands.values())

is_collective_target(text: str) -> bool

True when text targets a set rather than one specific object.

A quantifier (all/every/each/both/everything) or a bare generic plural (objects/items/things). Narrow by design: a specific goal never trips it.

Example

is_collective_target("put all the objects in the basket") True is_collective_target("pick up the alphabet soup and put it in the basket") False

Source code in python/core/src/openral_core/schemas.py

def is_collective_target(text: str) -> bool:
    """True when ``text`` targets a *set* rather than one specific object.

    A quantifier (``all``/``every``/``each``/``both``/``everything``) or a bare
    generic plural (``objects``/``items``/``things``). Narrow by design: a
    specific goal never trips it.

    Example:
        >>> is_collective_target("put all the objects in the basket")
        True
        >>> is_collective_target("pick up the alphabet soup and put it in the basket")
        False
    """
    return _COLLECTIVE_TARGET_RE.search(text) is not None

is_tegra_host(release_path: Path | None = None) -> bool

True on an NVIDIA Jetson / L4T host (release_path exists).

The one Tegra probe. release_path defaults to TEGRA_RELEASE_PATH, read at call time, so a test can select the Jetson path for every caller by pointing that constant at a captured nv_tegra_release fixture. It answers "is this an L4T image", not "is the GPU integrated / unified memory": DGX Spark (GB10) is unified-memory but runs DGX OS.

Example

is_tegra_host(Path("/nonexistent/nv_tegra_release")) False

Source code in python/core/src/openral_core/gpu.py

def is_tegra_host(release_path: Path | None = None) -> bool:
    """True on an NVIDIA Jetson / L4T host (``release_path`` exists).

    The one Tegra probe. ``release_path`` defaults to ``TEGRA_RELEASE_PATH``,
    read at call time, so a test can select the Jetson path for every caller by
    pointing that constant at a captured ``nv_tegra_release`` fixture. It
    answers "is this an L4T image", not "is the GPU integrated / unified
    memory": DGX Spark (GB10) is unified-memory but runs DGX OS.

    Example:
        >>> is_tegra_host(Path("/nonexistent/nv_tegra_release"))
        False
    """
    return (TEGRA_RELEASE_PATH if release_path is None else release_path).exists()

load_benchmark_suite(path: str) -> list[BenchmarkScene]

Load a bare list of BenchmarkScenes from benchmarks/<id>.yaml.

A benchmark suite YAML is a bare YAML list at the root; the suite id is derived from the filename stem (e.g. benchmarks/libero_spatial.yaml → "libero_spatial"). The pre-June-2026 {id, tasks, metadata} wrapper mapping (BenchmarkSpec, since deleted) is rejected with an explicit redirect message.

Per-scene Pydantic validation runs here. Suite-level invariants (uniformity, uniqueness, non-empty) are NOT enforced here; call raise_on_invalid_suite separately (typically with suite_id=Path(path).stem) so tests can construct invalid in-memory suites without touching the filesystem.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to a benchmark YAML file.	required

Returns:

Type	Description
list[BenchmarkScene]	Validated list of BenchmarkScenes in YAML order.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ROSConfigError	If the YAML root is not a list (legacy dict-shape gets an explicit redirect), or any entry fails BenchmarkScene validation.

Example

from pathlib import Path from openral_core import load_benchmark_suite, raise_on_invalid_suite scenes = load_benchmark_suite("benchmarks/libero_spatial.yaml") suite_id = Path("benchmarks/libero_spatial.yaml").stem raise_on_invalid_suite(scenes, suite_id=suite_id) len(scenes) 10

Source code in python/core/src/openral_core/loaders.py

def load_benchmark_suite(path: str) -> list[BenchmarkScene]:
    """Load a bare list of ``BenchmarkScene``s from ``benchmarks/<id>.yaml``.

    A benchmark suite YAML is a bare YAML list at the root; the suite id is
    derived from the filename stem (e.g. ``benchmarks/libero_spatial.yaml`` →
    ``"libero_spatial"``). The pre-June-2026 ``{id, tasks, metadata}``
    wrapper mapping (``BenchmarkSpec``, since deleted) is rejected with an
    explicit redirect message.

    Per-scene Pydantic validation runs here. Suite-level invariants
    (uniformity, uniqueness, non-empty) are NOT enforced here; call
    ``raise_on_invalid_suite`` separately (typically with
    ``suite_id=Path(path).stem``) so tests can construct invalid in-memory
    suites without touching the filesystem.

    Args:
        path: Filesystem path to a benchmark YAML file.

    Returns:
        Validated list of ``BenchmarkScene``s in YAML order.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        ROSConfigError: If the YAML root is not a list (legacy dict-shape
            gets an explicit redirect), or any entry fails
            ``BenchmarkScene`` validation.

    Example:
        >>> from pathlib import Path
        >>> from openral_core import load_benchmark_suite, raise_on_invalid_suite
        >>> scenes = load_benchmark_suite("benchmarks/libero_spatial.yaml")
        >>> suite_id = Path("benchmarks/libero_spatial.yaml").stem
        >>> raise_on_invalid_suite(scenes, suite_id=suite_id)
        >>> len(scenes)
        10
    """
    raw_obj = _yaml.safe_load(Path(path).read_text(encoding="utf-8"))

    if isinstance(raw_obj, dict):
        raise ROSConfigError(
            f"{path}: YAML root is a mapping, but the June 2026 schema change "
            "deleted the BenchmarkSpec wrapper. A benchmark suite YAML is "
            "now a bare list of BenchmarkScene mappings at the root; the "
            "suite id is derived from the filename stem. Remove the "
            "top-level `id:` / `metadata:` block and inline the scenes "
            "directly, promoting `suite` / `simulator` from the suite-level "
            "metadata to per-scene `metadata.display_name` / "
            "`metadata.simulator`."
        )

    if not isinstance(raw_obj, list):
        raise ROSConfigError(
            f"{path}: YAML root must be a list of BenchmarkScene mappings, "
            f"got {type(raw_obj).__name__}."
        )

    scenes: list[BenchmarkScene] = []
    for i, raw_scene in enumerate(raw_obj):
        if not isinstance(raw_scene, dict):
            raise ROSConfigError(
                f"{path}: entry [{i}] must be a mapping, got {type(raw_scene).__name__}."
            )
        try:
            scenes.append(BenchmarkScene.model_validate(raw_scene))
        except ValidationError as exc:
            raise ROSConfigError(
                f"{path}: entry [{i}] is not a valid BenchmarkScene: {exc}\n"
                "BenchmarkScene requires `n_episodes`, `seed`, "
                "`metadata.paper`, `metadata.honest_scope`, `task.max_steps`, "
                "and `task.success_key`."
            ) from exc
    return scenes

load_robot_unit(robot_yaml: str | Path, unit: str) -> RobotUnit

Load <robot_yaml dir>/units/<unit>.yaml and check it belongs to that robot.

Raises:

Type	Description
ROSConfigError	The file is missing, or its robot_id / unit disagree with the manifest directory name / the requested unit.

Example

load_robot_unit("robots/openarm/robot.yaml", "thor").robot_id 'openarm'

Source code in python/core/src/openral_core/schemas.py

def load_robot_unit(robot_yaml: str | Path, unit: str) -> RobotUnit:
    """Load ``<robot_yaml dir>/units/<unit>.yaml`` and check it belongs to that robot.

    Raises:
        ROSConfigError: The file is missing, or its ``robot_id`` / ``unit`` disagree with
            the manifest directory name / the requested unit.

    Example:
        >>> load_robot_unit("robots/openarm/robot.yaml", "thor").robot_id
        'openarm'
    """
    manifest_dir = Path(robot_yaml).parent
    path = manifest_dir / "units" / f"{unit}.yaml"
    if not path.is_file():
        have = sorted(p.stem for p in (manifest_dir / "units").glob("*.yaml"))
        raise ROSConfigError(
            f"robot unit {unit!r} not found at {path} (available: {', '.join(have) or 'none'})"
        )
    loaded = RobotUnit.from_yaml(str(path))
    if loaded.robot_id != manifest_dir.name or loaded.unit != unit:
        raise ROSConfigError(
            f"{path} declares robot_id={loaded.robot_id!r} unit={loaded.unit!r}; "
            f"expected robot_id={manifest_dir.name!r} unit={unit!r}"
        )
    return loaded

load_scene_strict(path: str, expected: type[DeployScene] | type[SimScene] | type[BenchmarkScene]) -> DeployScene | SimScene | BenchmarkScene

load_scene_strict(
    path: str, expected: type[BenchmarkScene]
) -> BenchmarkScene

load_scene_strict(
    path: str, expected: type[SimScene]
) -> SimScene

load_scene_strict(
    path: str, expected: type[DeployScene]
) -> DeployScene

Load path as exactly expected; reject other tiers with a clear error.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to a YAML file containing a single scene mapping.	required
expected	type[DeployScene] | type[SimScene] | type[BenchmarkScene]	One of DeployScene, SimScene, BenchmarkScene — the exact tier the calling CLI accepts.	required

Returns:

Type	Description
DeployScene | SimScene | BenchmarkScene	A validated instance of expected.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ROSConfigError	If the YAML root is not a mapping, or if it loads as a tier other than expected (with a redirect message naming the right CLI command).

Example

from openral_core import SimScene, load_scene_strict scene = load_scene_strict("scenes/sim/tabletop_cube_push.yaml", SimScene) scene.scene.id # scene_id is the @SCENES.register("…") key, not the filename 'tabletop_push'

Source code in python/core/src/openral_core/loaders.py

def load_scene_strict(
    path: str,
    expected: type[DeployScene] | type[SimScene] | type[BenchmarkScene],
) -> DeployScene | SimScene | BenchmarkScene:
    """Load ``path`` as exactly ``expected``; reject other tiers with a clear error.

    Args:
        path: Filesystem path to a YAML file containing a single scene mapping.
        expected: One of ``DeployScene``, ``SimScene``, ``BenchmarkScene`` —
            the exact tier the calling CLI accepts.

    Returns:
        A validated instance of ``expected``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        ROSConfigError: If the YAML root is not a mapping, or if it loads as
            a tier other than ``expected`` (with a redirect message naming the
            right CLI command).

    Example:
        >>> from openral_core import SimScene, load_scene_strict
        >>> scene = load_scene_strict("scenes/sim/tabletop_cube_push.yaml", SimScene)
        >>> scene.scene.id  # scene_id is the @SCENES.register("…") key, not the filename
        'tabletop_push'
    """
    raw_obj = _yaml.safe_load(Path(path).read_text(encoding="utf-8"))
    if not isinstance(raw_obj, dict):
        raise ROSConfigError(f"{path}: YAML root must be a mapping, got {type(raw_obj).__name__}")
    raw: dict[str, object] = raw_obj

    if expected is DeployScene:
        return _load_as_deploy(path, raw)
    if expected is SimScene:
        return _load_as_sim(path, raw)
    if expected is BenchmarkScene:
        return _load_as_benchmark(path, raw)
    raise ROSConfigError(
        f"load_scene_strict: unsupported expected type {expected!r} — "
        "must be DeployScene, SimScene, or BenchmarkScene."
    )

merge_deploy_sensors(manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec]) -> list[SensorSpec]

The deploy's sensor set: the robot manifest's sensors, then the scene's workcell ones.

A scene only adds sensors (check_scene_sensor_overrides refuses one that reuses a manifest sensor's name), so this is a checked concatenation, in declaration order.

Raises:

Type	Description
ROSConfigError	A scene entry names a sensor the robot manifest defines.

Example

desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml") ", ".join(s.name for s in merge_deploy_sensors(desc.sensors, [])) 'top, wrist'

Source code in python/core/src/openral_core/schemas.py

def merge_deploy_sensors(
    manifest_sensors: Iterable[SensorSpec],
    scene_sensors: Iterable[SensorSpec],
) -> list[SensorSpec]:
    """The deploy's sensor set: the robot manifest's sensors, then the scene's workcell ones.

    A scene only adds sensors (``check_scene_sensor_overrides`` refuses one that reuses a
    manifest sensor's name), so this is a checked concatenation, in declaration order.

    Raises:
        ROSConfigError: A scene entry names a sensor the robot manifest defines.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml")
        >>> ", ".join(s.name for s in merge_deploy_sensors(desc.sensors, []))
        'top, wrist'
    """
    manifest = list(manifest_sensors)
    scene = list(scene_sensors)
    check_scene_sensor_overrides(manifest, scene)
    return [*manifest, *scene]

parse_reasoner_tool_choice(raw: str | None) -> str | None

Validate an OPENRAL_REASONER_TOOL_CHOICE value; None when unset.

Shared by the reasoner's client factory and openral doctor so the two cannot drift. Case-insensitive, like OPENRAL_REASONER_DIALECT.

Raises:

Type	Description
ROSConfigError	When set to anything but auto / required.

Example

parse_reasoner_tool_choice(" AUTO ") 'auto' parse_reasoner_tool_choice("") is None True

Source code in python/core/src/openral_core/schemas.py

def parse_reasoner_tool_choice(raw: str | None) -> str | None:
    """Validate an ``OPENRAL_REASONER_TOOL_CHOICE`` value; ``None`` when unset.

    Shared by the reasoner's client factory and ``openral doctor`` so the two
    cannot drift. Case-insensitive, like ``OPENRAL_REASONER_DIALECT``.

    Raises:
        ROSConfigError: When set to anything but ``auto`` / ``required``.

    Example:
        >>> parse_reasoner_tool_choice(" AUTO ")
        'auto'
        >>> parse_reasoner_tool_choice("") is None
        True
    """
    value = (raw or "").strip().lower()
    if not value:
        return None
    if value not in REASONER_TOOL_CHOICES:
        raise ROSConfigError(
            f"OPENRAL_REASONER_TOOL_CHOICE={raw!r}; expected one of {list(REASONER_TOOL_CHOICES)}."
        )
    return value

physics_steps_per_tick(control_freq_hz: float, physics_dt_s: float) -> tuple[int, float]

Physics steps per control tick, and the physics dt that makes them exact.

The control period must be a whole number of physics steps. When the engine's dt already divides it the pair is (period / dt, dt); otherwise the dt is shrunk (never grown) to the largest value at or below physics_dt_s that divides the period, so the integration is at least as fine as the model's author chose. Shrinking by less than one step is what MuJoCo's default 0.002 s needs at 30 Hz (1 / 30 s is 16.67 steps).

Parameters:

Name	Type	Description	Default
control_freq_hz	float	The robot's action_spec.control_freq_hz.	required
physics_dt_s	float	The engine's physics time step (MjModel.opt.timestep, robosuite's SIMULATION_TIMESTEP).	required

Returns:

Type	Description
tuple[int, float]	(steps, dt_s) with steps * dt_s == 1 / control_freq_hz.

Raises:

Type	Description
ROSConfigError	a rate or dt that is not finite and positive.

Example

physics_steps_per_tick(20.0, 0.002) (25, 0.002) steps, dt = physics_steps_per_tick(30.0, 0.002) steps, round(dt, 6) (17, 0.001961)

Source code in python/core/src/openral_core/time_base.py

def physics_steps_per_tick(control_freq_hz: float, physics_dt_s: float) -> tuple[int, float]:
    """Physics steps per control tick, and the physics dt that makes them exact.

    The control period must be a whole number of physics steps. When the
    engine's dt already divides it the pair is ``(period / dt, dt)``; otherwise
    the dt is **shrunk** (never grown) to the largest value at or below
    ``physics_dt_s`` that divides the period, so the integration is at least as
    fine as the model's author chose. Shrinking by less than one step is what
    MuJoCo's default 0.002 s needs at 30 Hz (1 / 30 s is 16.67 steps).

    Args:
        control_freq_hz: The robot's ``action_spec.control_freq_hz``.
        physics_dt_s: The engine's physics time step (``MjModel.opt.timestep``,
            robosuite's ``SIMULATION_TIMESTEP``).

    Returns:
        ``(steps, dt_s)`` with ``steps * dt_s == 1 / control_freq_hz``.

    Raises:
        ROSConfigError: a rate or dt that is not finite and positive.

    Example:
        >>> physics_steps_per_tick(20.0, 0.002)
        (25, 0.002)
        >>> steps, dt = physics_steps_per_tick(30.0, 0.002)
        >>> steps, round(dt, 6)
        (17, 0.001961)
    """
    if not (math.isfinite(control_freq_hz) and control_freq_hz > 0):
        raise ROSConfigError(f"control_freq_hz must be finite and > 0, got {control_freq_hz!r}")
    if not (math.isfinite(physics_dt_s) and physics_dt_s > 0):
        raise ROSConfigError(f"physics_dt_s must be finite and > 0, got {physics_dt_s!r}")
    period = 1.0 / control_freq_hz
    exact = period / physics_dt_s
    steps = round(exact) if math.isclose(exact, round(exact), rel_tol=1e-9) else math.ceil(exact)
    return max(1, steps), period / max(1, steps)

publishing_sensors(manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec], hal_mode: str) -> list[SensorSpec]

The sensors that actually publish a camera topic on a deploy.

Sim: the manifest's sensors, which the sim sensor bridge renders (bound or not; a scene-only hardware camera has no sim publisher). Real: merge_deploy_sensors, keeping only sensors with a deploy_binding — an unbound sensor gets no reader and so no topic. The one rule both openral deploy (pre-launch decisions) and deploy_e2e.launch.py (which camera each consumer subscribes to) use, so they cannot disagree.

Example

arm = RobotDescription.from_yaml("robots/openarm/robot.yaml") sim = publishing_sensors(arm.sensors, [], "sim") ", ".join(s.name for s in sim if s.modality == "rgb") 'top, wrist_left, wrist_right' thor = resolve_sensor_overlays("robots/openarm/robot.yaml", "thor", required=True) bound = publishing_sensors(apply_sensor_overlays(arm.sensors, thor), [], "real") ", ".join(s.name for s in bound) 'head_zed, top, wrist_left, wrist_right'

Source code in python/core/src/openral_core/schemas.py

def publishing_sensors(
    manifest_sensors: Iterable[SensorSpec],
    scene_sensors: Iterable[SensorSpec],
    hal_mode: str,
) -> list[SensorSpec]:
    """The sensors that actually publish a camera topic on a deploy.

    Sim: the manifest's sensors, which the sim sensor bridge renders (bound or not; a
    scene-only hardware camera has no sim publisher). Real: ``merge_deploy_sensors``, keeping
    only sensors with a ``deploy_binding`` — an unbound sensor gets no reader and so no topic.
    The one rule both ``openral deploy`` (pre-launch decisions) and ``deploy_e2e.launch.py``
    (which camera each consumer subscribes to) use, so they cannot disagree.

    Example:
        >>> arm = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> sim = publishing_sensors(arm.sensors, [], "sim")
        >>> ", ".join(s.name for s in sim if s.modality == "rgb")
        'top, wrist_left, wrist_right'
        >>> thor = resolve_sensor_overlays("robots/openarm/robot.yaml", "thor", required=True)
        >>> bound = publishing_sensors(apply_sensor_overlays(arm.sensors, thor), [], "real")
        >>> ", ".join(s.name for s in bound)
        'head_zed, top, wrist_left, wrist_right'
    """
    if hal_mode != "real":
        return list(manifest_sensors)
    return [s for s in merge_deploy_sensors(manifest_sensors, scene_sensors) if s.deploy_binding]

raise_on_invalid_suite(scenes: list[BenchmarkScene], *, suite_id: str) -> None

Raise ROSConfigError if scenes violates suite-level invariants.

Originally enforced inside BenchmarkSpec.model_post_init (deleted in June 2026). Extracted as a free function so callers can validate freshly loaded suites independently and so tests can exercise the rules without touching the filesystem.

Suite-level invariants

• scenes MUST be non-empty.
• Every scenes[i].task.id MUST be unique within the suite.
• Every scenes[i].robot_id MUST be non-None — benchmark suites always pin an embodiment.
• All scenes[i].robot_id / n_episodes / seed / metadata MUST be equal across the list — a suite is a single fixed protocol applied across (potentially varying) scenes / tasks.

Per-scene task.success_key / task.max_steps MAY differ (e.g. ManiSkill3 pick + stack share one suite but each task has its own step budget).

Parameters:

Name	Type	Description	Default
scenes	list[BenchmarkScene]	The list of BenchmarkScenes to validate.	required
suite_id	str	The suite identifier — embedded in every error message so failures point back to the right benchmarks/<id>.yaml file.	required

Raises:

Type	Description
ROSConfigError	If any invariant is violated. The first violation wins (no batched reporting).

Source code in python/core/src/openral_core/loaders.py

def raise_on_invalid_suite(
    scenes: list[BenchmarkScene],
    *,
    suite_id: str,
) -> None:
    """Raise ``ROSConfigError`` if ``scenes`` violates suite-level invariants.

    Originally enforced inside ``BenchmarkSpec.model_post_init`` (deleted in
    June 2026). Extracted as a free function so callers can validate freshly
    loaded suites independently and so tests can exercise the rules without
    touching the filesystem.

    Suite-level invariants:
        * ``scenes`` MUST be non-empty.
        * Every ``scenes[i].task.id`` MUST be unique within the suite.
        * Every ``scenes[i].robot_id`` MUST be non-``None`` — benchmark
          suites always pin an embodiment.
        * All ``scenes[i].robot_id`` / ``n_episodes`` / ``seed`` /
          ``metadata`` MUST be equal across the list — a suite is a single
          fixed protocol applied across (potentially varying) scenes / tasks.

    Per-scene ``task.success_key`` / ``task.max_steps`` MAY differ (e.g.
    ManiSkill3 pick + stack share one suite but each task has its own step
    budget).

    Args:
        scenes: The list of ``BenchmarkScene``s to validate.
        suite_id: The suite identifier — embedded in every error message so
            failures point back to the right ``benchmarks/<id>.yaml`` file.

    Raises:
        ROSConfigError: If any invariant is violated. The first violation
            wins (no batched reporting).
    """
    if not scenes:
        raise ROSConfigError(
            f"benchmark suite {suite_id!r}: scene list is empty; a benchmark "
            f"must declare at least one BenchmarkScene."
        )

    first = scenes[0]
    if first.robot_id is None:
        raise ROSConfigError(
            f"benchmark suite {suite_id!r}: scene {first.task.id!r} has "
            f"robot_id=None; benchmark suites must pin an embodiment "
            f"(add `robot_id: <id>` to the BenchmarkScene YAML)."
        )

    seen: set[str] = set()
    for scene in scenes:
        task_id = scene.task.id
        if task_id in seen:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: duplicate task id "
                f"{task_id!r}; task ids must be unique within a suite."
            )
        seen.add(task_id)

        if scene.robot_id is None:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"robot_id=None; benchmark suites must pin an embodiment "
                f"(add `robot_id: <id>` to the BenchmarkScene YAML)."
            )
        if scene.robot_id != first.robot_id:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"robot_id={scene.robot_id!r} but the suite uses "
                f"{first.robot_id!r}; every scene in a suite must run on "
                f"the same robot."
            )
        if scene.n_episodes != first.n_episodes:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"n_episodes={scene.n_episodes} but the suite uses "
                f"{first.n_episodes}; every scene in a suite must share "
                f"the same episode count."
            )
        if scene.seed != first.seed:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"seed={scene.seed} but the suite uses {first.seed}; "
                f"every scene in a suite must share the same seed offset."
            )
        if scene.metadata != first.metadata:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"metadata={scene.metadata!r} but the suite uses "
                f"{first.metadata!r}; every scene in a suite shares the "
                f"same provenance block (paper, honest_scope, optional "
                f"display_name + simulator)."
            )

repo_name_is_canonical(name: str, *, kind: RSkillKind, model_family: str | None = None) -> bool

Return True when name obeys the repo-naming grammar for kind.

Three shapes, chosen by kind:

• playbook → rskill-playbook-<name> (3 parts).
• ros_action / ros_service (_WEIGHTLESS_KINDS) → rskill-<model>-<robot>-<task> (4 parts) — no <quant> segment, since a ROS wrapper carries no weights.
• every other kind → rskill-<model>-<robot>-<task>-<quant> (5 parts) with <quant> ∈ CANONICAL_QUANT_TOKENS.

In all cases <robot> and <task> match the segment shape (robot ids are an open registry, vocabulary-checked in CI). The owner prefix (<owner>/) is ignored.

When model_family is given (a VLA skill), the <model> segment must be family-consistent — one of _MODEL_FAMILY_ALLOWED_TOKENS for that family, or the family id itself for a family with no entry — so a checkpoint can't be mislabelled with another family's token. When model_family is None (tool / ROS skills, or a pure name check) <model> ∈ CANONICAL_MODEL_TOKENS is required.

Example

repo_name_is_canonical( ... "OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16", kind="vla" ... ) True repo_name_is_canonical( ... "OpenRAL/rskill-moveit-multi-eef_pose", kind="ros_action" ... ) # ROS wrapper: no quant segment True repo_name_is_canonical( ... "OpenRAL/rskill-pi05-franka_panda-libero-bf16", ... kind="vla", ... model_family="smolvla", ... ) # wrong family token False repo_name_is_canonical("OpenRAL/rskill-playbook-find_object", kind="playbook") True repo_name_is_canonical("OpenRAL/rskill-smolvla-libero", kind="vla") # old 3-part name False

Source code in python/core/src/openral_core/schemas.py

def repo_name_is_canonical(name: str, *, kind: RSkillKind, model_family: str | None = None) -> bool:
    """Return ``True`` when ``name`` obeys the repo-naming grammar for ``kind``.

    Three shapes, chosen by ``kind``:

    * ``playbook`` → ``rskill-playbook-<name>`` (3 parts).
    * ``ros_action`` / ``ros_service`` (``_WEIGHTLESS_KINDS``) →
      ``rskill-<model>-<robot>-<task>`` (4 parts) — no ``<quant>`` segment, since
      a ROS wrapper carries no weights.
    * every other kind → ``rskill-<model>-<robot>-<task>-<quant>`` (5 parts) with
      ``<quant>`` ∈ ``CANONICAL_QUANT_TOKENS``.

    In all cases ``<robot>`` and ``<task>`` match the segment shape (robot ids
    are an open registry, vocabulary-checked in CI). The owner prefix
    (``<owner>/``) is ignored.

    When ``model_family`` is given (a VLA skill), the ``<model>`` segment must
    be family-consistent — one of ``_MODEL_FAMILY_ALLOWED_TOKENS`` for that
    family, or the family id itself for a family with no entry — so a
    checkpoint can't be mislabelled with another family's token. When
    ``model_family`` is ``None`` (tool / ROS skills, or a pure name check)
    ``<model>`` ∈ ``CANONICAL_MODEL_TOKENS`` is required.

    Example:
        >>> repo_name_is_canonical(
        ...     "OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16", kind="vla"
        ... )
        True
        >>> repo_name_is_canonical(
        ...     "OpenRAL/rskill-moveit-multi-eef_pose", kind="ros_action"
        ... )  # ROS wrapper: no quant segment
        True
        >>> repo_name_is_canonical(
        ...     "OpenRAL/rskill-pi05-franka_panda-libero-bf16",
        ...     kind="vla",
        ...     model_family="smolvla",
        ... )  # wrong family token
        False
        >>> repo_name_is_canonical("OpenRAL/rskill-playbook-find_object", kind="playbook")
        True
        >>> repo_name_is_canonical("OpenRAL/rskill-smolvla-libero", kind="vla")  # old 3-part name
        False
    """
    repo = name.split("/", 1)[1] if "/" in name else name
    parts = repo.split("-")
    if parts[0] != "rskill":
        return False
    if kind == "playbook":
        return (
            len(parts) == _RSKILL_PLAYBOOK_NAME_PARTS
            and parts[1] == "playbook"
            and bool(_RSKILL_NAME_SEGMENT_RE.match(parts[2]))
        )
    if kind in _WEIGHTLESS_KINDS:
        if len(parts) != _RSKILL_ROS_NAME_PARTS:
            return False
        _, model, robot, task = parts
        quant_ok = True
    else:
        if len(parts) != _RSKILL_NAME_PARTS:
            return False
        _, model, robot, task, quant = parts
        quant_ok = quant in CANONICAL_QUANT_TOKENS
    model_ok = (
        model in CANONICAL_MODEL_TOKENS
        if model_family is None
        else model in _allowed_model_tokens(model_family)
    )
    return (
        model_ok
        and bool(_RSKILL_NAME_SEGMENT_RE.match(robot))
        and bool(_RSKILL_NAME_SEGMENT_RE.match(task))
        and quant_ok
    )

required_vla_camera_slots(manifest: RSkillManifest, description: RobotDescription | None) -> tuple[str, ...]

RGB VLA slots an rSkill consumes on this robot, in robot manifest order.

The robot's slots (sensor_name_to_slot) trimmed to the ones the manifest's sensors_required names by vla_feature_key — a Franka exposes camera3 for VLABench while a LIBERO skill wants only camera1/camera2. All robot slots when the manifest names none.

Example

desc = RobotDescription.from_yaml("robots/franka_panda/robot.yaml") skill = RSkillManifest.from_yaml("rskills/smolvla-libero/rskill.yaml") required_vla_camera_slots(skill, desc) ('camera1', 'camera2')

Source code in python/core/src/openral_core/schemas.py

def required_vla_camera_slots(
    manifest: RSkillManifest, description: RobotDescription | None
) -> tuple[str, ...]:
    """RGB VLA slots an rSkill consumes on this robot, in robot manifest order.

    The robot's slots (``sensor_name_to_slot``) trimmed to the ones the
    manifest's ``sensors_required`` names by ``vla_feature_key`` — a Franka
    exposes ``camera3`` for VLABench while a LIBERO skill wants only
    ``camera1``/``camera2``. All robot slots when the manifest names none.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/franka_panda/robot.yaml")
        >>> skill = RSkillManifest.from_yaml("rskills/smolvla-libero/rskill.yaml")
        >>> required_vla_camera_slots(skill, desc)
        ('camera1', 'camera2')
    """
    slots = tuple(sensor_name_to_slot(description).values())
    required = _required_rgb_slots(manifest.sensors_required)
    return tuple(slot for slot in slots if slot in required) if required else slots

resolve_sensor_overlays(robot_yaml: str | Path, scene_unit: str | None, *, required: bool) -> list[SensorOverlay]

The sensor overlays of the robot unit this host runs.

The unit is $OPENRAL_ROBOT_UNIT if set, else the scene's robot_unit. With neither, there is nothing to overlay; but when required (a real deploy) and the robot ships a units/ directory, that is refused: its bindings and calibration are per unit, so running on the manifest's nominal values would be a silent guess.

Raises:

Type	Description
ROSConfigError	A required unit is not selected, or the selected one is invalid.

Example

resolve_sensor_overlays("robots/franka_panda/robot.yaml", None, required=True) []

Source code in python/core/src/openral_core/schemas.py

def resolve_sensor_overlays(
    robot_yaml: str | Path, scene_unit: str | None, *, required: bool
) -> list[SensorOverlay]:
    """The sensor overlays of the robot unit this host runs.

    The unit is ``$OPENRAL_ROBOT_UNIT`` if set, else the scene's ``robot_unit``. With
    neither, there is nothing to overlay; but when ``required`` (a real deploy) and the
    robot ships a ``units/`` directory, that is refused: its bindings and calibration
    are per unit, so running on the manifest's nominal values would be a silent guess.

    Raises:
        ROSConfigError: A required unit is not selected, or the selected one is invalid.

    Example:
        >>> resolve_sensor_overlays("robots/franka_panda/robot.yaml", None, required=True)
        []
    """
    unit = os.environ.get(ROBOT_UNIT_ENV) or scene_unit
    if not unit:
        units_dir = Path(robot_yaml).parent / "units"
        if required and units_dir.is_dir():
            have = sorted(p.stem for p in units_dir.glob("*.yaml"))
            raise ROSConfigError(
                f"{Path(robot_yaml).parent.name} has per-unit sensor overlays "
                f"({', '.join(have)}) but none is selected: set ${ROBOT_UNIT_ENV} or the "
                "scene's robot_unit to the unit this host drives"
            )
        return []
    return list(load_robot_unit(robot_yaml, unit).sensors)

scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole

Linearly rescale pinhole intrinsics to a new render resolution.

For a camera with a fixed field of view, fx/fy/cx/cy scale linearly with the image dimensions: a frame rendered at twice the width has twice the focal length (px) and twice the principal-point x. This is the consistency rule deploy-sim needs — the canonical manifest pins one nominal resolution, but a scene may render the same MuJoCo camera at another (scene.observation_width/height). Publishing the manifest's nominal intrinsics on a different-resolution render would back-project depth pixels and project occupancy voxels with the wrong focal length, corrupting the OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera model matched to whatever was actually rendered.

The distortion model and coefficients are preserved unchanged (coefficients are resolution-independent for the normalised plumb-bob model). When the target already equals base's resolution the input is returned as-is.

Parameters:

Name	Type	Description	Default
base	IntrinsicsPinhole	Nominal pinhole intrinsics (e.g. from a SensorSpec).	required
width	int	Target render width in pixels (> 0).	required
height	int	Target render height in pixels (> 0).	required

Returns:

Type	Description
IntrinsicsPinhole	Intrinsics scaled so (fx, fy, cx, cy) correspond to ``(width,
IntrinsicsPinhole	height)at the same field of view asbase``.

Raises:

Type	Description
ValueError	If width or height is not strictly positive.

Example

base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0) hi = scale_intrinsics_to(base, 640, 640) (hi.width, hi.fx, hi.cx) (640, 640.0, 320.0)

Source code in python/core/src/openral_core/schemas.py

def scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole:
    """Linearly rescale pinhole intrinsics to a new render resolution.

    For a camera with a *fixed* field of view, ``fx``/``fy``/``cx``/``cy`` scale
    linearly with the image dimensions: a frame rendered at twice the width has
    twice the focal length (px) and twice the principal-point x. This is the
    consistency rule deploy-sim needs — the canonical manifest pins one nominal
    resolution, but a scene may render the same MuJoCo camera at another
    (``scene.observation_width``/``height``). Publishing the manifest's nominal
    intrinsics on a different-resolution render would back-project depth pixels
    and project occupancy voxels with the wrong focal length, corrupting the
    OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera
    model matched to whatever was actually rendered.

    The distortion model and coefficients are preserved unchanged (coefficients
    are resolution-independent for the normalised plumb-bob model). When the
    target already equals ``base``'s resolution the input is returned as-is.

    Args:
        base: Nominal pinhole intrinsics (e.g. from a ``SensorSpec``).
        width: Target render width in pixels (``> 0``).
        height: Target render height in pixels (``> 0``).

    Returns:
        Intrinsics scaled so ``(fx, fy, cx, cy)`` correspond to ``(width,
        height)`` at the same field of view as ``base``.

    Raises:
        ValueError: If ``width`` or ``height`` is not strictly positive.

    Example:
        >>> base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0)
        >>> hi = scale_intrinsics_to(base, 640, 640)
        >>> (hi.width, hi.fx, hi.cx)
        (640, 640.0, 320.0)
    """
    if width <= 0 or height <= 0:
        raise ValueError(f"target resolution must be positive; got ({width}, {height})")
    if width == base.width and height == base.height:
        return base
    sx = width / base.width
    sy = height / base.height
    return IntrinsicsPinhole(
        width=width,
        height=height,
        fx=base.fx * sx,
        fy=base.fy * sy,
        cx=base.cx * sx,
        cy=base.cy * sy,
        distortion_model=base.distortion_model,
        distortion_coeffs=list(base.distortion_coeffs),
    )

scene_family(task_id: str) -> str

Reduce an evaluated_tasks entry to its scene-family key.

The family is the leading token before any "/" — "rlbench/open_drawer" → "rlbench", "libero_spatial" → "libero_spatial".

Example

scene_family("rlbench/open_drawer") 'rlbench' scene_family("metaworld") 'metaworld'

Source code in python/core/src/openral_core/schemas.py

def scene_family(task_id: str) -> str:
    """Reduce an ``evaluated_tasks`` entry to its scene-family key.

    The family is the leading token before any ``"/"`` — ``"rlbench/open_drawer"``
    → ``"rlbench"``, ``"libero_spatial"`` → ``"libero_spatial"``.

    Example:
        >>> scene_family("rlbench/open_drawer")
        'rlbench'
        >>> scene_family("metaworld")
        'metaworld'
    """
    return task_id.split("/", 1)[0]

scene_task_space_compatible(family: str, skill_space: TaskSpace) -> TaskSpaceMatch

Check an rSkill's TaskSpace is executable by a scene family.

The third leg of the cross-layer gate: a skill fits a scene when every ControlMode it emits is in the scene family's executed set, and — when the family fixes a width — its total dimensionality matches. Pairs with task_space_compatible (rSkill x robot); together they close the rSkill x robot x scene triangle the audit found unconnected.

Parameters:

Name	Type	Description	Default
family	str	The scene-family key (see scene_family).	required
skill_space	TaskSpace	The task space the rSkill emits (TaskSpace.from_action_contract).	required

Returns:

Type	Description
TaskSpaceMatch	A TaskSpaceMatch — ok plus a reason per incompatibility.
TaskSpaceMatch	An unknown family is reported as a single reason (not an exception)
TaskSpaceMatch	so a sweep can collect every gap in one pass.

Example

space = TaskSpace( ... segments=[ ... TaskSpaceSegment( ... family=TaskSpaceFamily.JOINT, ... control_mode=ControlMode.JOINT_POSITION, ... width=14, ... ) ... ] ... ) scene_task_space_compatible("robotwin", space).ok True

Source code in python/core/src/openral_core/schemas.py

def scene_task_space_compatible(family: str, skill_space: TaskSpace) -> TaskSpaceMatch:
    """Check an rSkill's ``TaskSpace`` is executable by a scene family.

    The third leg of the cross-layer gate: a skill fits a
    scene when every ``ControlMode`` it emits is in the scene family's
    executed set, and — when the family fixes a width — its total dimensionality
    matches. Pairs with ``task_space_compatible`` (rSkill x robot); together
    they close the rSkill x robot x scene triangle the audit found unconnected.

    Args:
        family: The scene-family key (see ``scene_family``).
        skill_space: The task space the rSkill emits
            (``TaskSpace.from_action_contract``).

    Returns:
        A ``TaskSpaceMatch`` — ``ok`` plus a reason per incompatibility.
        An unknown ``family`` is reported as a single reason (not an exception)
        so a sweep can collect every gap in one pass.

    Example:
        >>> space = TaskSpace(
        ...     segments=[
        ...         TaskSpaceSegment(
        ...             family=TaskSpaceFamily.JOINT,
        ...             control_mode=ControlMode.JOINT_POSITION,
        ...             width=14,
        ...         )
        ...     ]
        ... )
        >>> scene_task_space_compatible("robotwin", space).ok
        True
    """
    spec = SCENE_FAMILY_TASK_SPACE.get(family)
    if spec is None:
        return TaskSpaceMatch(
            ok=False,
            reasons=[
                f"scene family {family!r} has no declared task space in SCENE_FAMILY_TASK_SPACE"
            ],
        )
    reasons: list[str] = []
    extra = skill_space.control_modes - spec.modes
    if extra:
        reasons.append(
            f"control mode(s) {sorted(m.value for m in extra)} not executable by "
            f"scene family {family!r} (executes {sorted(m.value for m in spec.modes)})"
        )
    if spec.action_dim is not None and skill_space.total_dim != spec.action_dim:
        reasons.append(
            f"action dim {skill_space.total_dim} != scene family {family!r} "
            f"expected dim {spec.action_dim}"
        )
    return TaskSpaceMatch(ok=not reasons, reasons=reasons)

sensor_name_to_slot(description: RobotDescription | None) -> dict[str, str]

Map each RGB sensor NAME to its VLA slot, in manifest order.

The slot is the vla_feature_key suffix (observation.images.camera1 -> camera1), falling back to the sensor name when the sensor declares no key. Slots are the one camera namespace shared by openral sim run, deploy sim and deploy run: image_preprocessing.aliases keys and adapter camera_keys are slots, while topics and sim renders stay keyed by sensor name.

Example

desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml") sensor_name_to_slot(desc)

Source code in python/core/src/openral_core/schemas.py

def sensor_name_to_slot(description: RobotDescription | None) -> dict[str, str]:
    """Map each RGB sensor NAME to its VLA slot, in manifest order.

    The slot is the ``vla_feature_key`` suffix (``observation.images.camera1``
    -> ``camera1``), falling back to the sensor name when the sensor declares
    no key. Slots are the one camera namespace shared by ``openral sim run``,
    ``deploy sim`` and ``deploy run``: ``image_preprocessing.aliases`` keys
    and adapter ``camera_keys`` are slots, while topics and sim renders stay
    keyed by sensor name.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml")
        >>> sensor_name_to_slot(desc)
        {'top': 'camera1', 'wrist': 'camera2'}
    """
    if description is None:
        return {}
    return {
        s.name: s.vla_feature_key.rsplit(".", 1)[-1] if s.vla_feature_key else s.name
        for s in description.sensors
        if s.modality == "rgb"
    }

task_space_compatible(skill_space: TaskSpace, robot: RobotDescription, *, hal_mode: Literal['sim', 'real'] = 'real') -> TaskSpaceMatch

Check an rSkill's TaskSpace is executable on a robot.

The single cross-layer gate replacing implicit embodiment_tags/dim matching. Compatible when every segment's control_mode is executable for hal_mode ("sim": SIM_EXECUTABLE_CONTROL_MODES; "real": the robot's capabilities.supported_control_modes), every CARTESIAN_*/GRIPPER_*/DEX_HAND_JOINT segment names a real end-effector, and the total joint-segment width fits the robot's joint count.

Parameters:

Name	Type	Description	Default
skill_space	TaskSpace	The task space the rSkill emits (TaskSpace.from_action_contract).	required
robot	RobotDescription	The target robot description.	required
hal_mode	Literal['sim', 'real']	"sim" (default-sim OSC packers) or "real" (hardware controllers).	'real'

Returns:

Type	Description
TaskSpaceMatch	A TaskSpaceMatch — ok plus a reason per incompatibility.

Example

from openral_core.schemas import ( ... EmbodimentKind, ... EndEffectorSpec, ... JointSpec, ... JointType, ... RobotCapabilities, ... SafetyEnvelope, ... ) robot = RobotDescription( ... name="franka_panda", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["franka_panda"], ... ), ... safety=SafetyEnvelope(), ... ) space = TaskSpace( ... segments=[ ... TaskSpaceSegment( ... family=TaskSpaceFamily.CARTESIAN, ... control_mode=ControlMode.CARTESIAN_DELTA, ... width=6, ... target="panda_hand", ... ) ... ] ... )

Real hardware needs a cartesian controller the Franka does not declare:

task_space_compatible(space, robot, hal_mode="real").ok False

In sim the robosuite OSC packer synthesises it from joint commands:

task_space_compatible(space, robot, hal_mode="sim").ok True

Source code in python/core/src/openral_core/schemas.py

def task_space_compatible(
    skill_space: TaskSpace,
    robot: RobotDescription,
    *,
    hal_mode: Literal["sim", "real"] = "real",
) -> TaskSpaceMatch:
    """Check an rSkill's ``TaskSpace`` is executable on a robot.

    The single cross-layer gate replacing implicit ``embodiment_tags``/dim
    matching. Compatible when every segment's ``control_mode`` is executable
    for ``hal_mode`` (``"sim"``: ``SIM_EXECUTABLE_CONTROL_MODES``;
    ``"real"``: the robot's ``capabilities.supported_control_modes``), every
    ``CARTESIAN_*``/``GRIPPER_*``/``DEX_HAND_JOINT`` segment names a real
    end-effector, and the total joint-segment width fits the robot's joint count.

    Args:
        skill_space: The task space the rSkill emits (``TaskSpace.from_action_contract``).
        robot: The target robot description.
        hal_mode: ``"sim"`` (default-sim OSC packers) or ``"real"`` (hardware controllers).

    Returns:
        A ``TaskSpaceMatch`` — ``ok`` plus a reason per incompatibility.

    Example:
        >>> from openral_core.schemas import (
        ...     EmbodimentKind,
        ...     EndEffectorSpec,
        ...     JointSpec,
        ...     JointType,
        ...     RobotCapabilities,
        ...     SafetyEnvelope,
        ... )
        >>> robot = RobotDescription(
        ...     name="franka_panda",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(
        ...         supported_control_modes=[ControlMode.JOINT_POSITION],
        ...         embodiment_tags=["franka_panda"],
        ...     ),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> space = TaskSpace(
        ...     segments=[
        ...         TaskSpaceSegment(
        ...             family=TaskSpaceFamily.CARTESIAN,
        ...             control_mode=ControlMode.CARTESIAN_DELTA,
        ...             width=6,
        ...             target="panda_hand",
        ...         )
        ...     ]
        ... )
        >>> # Real hardware needs a cartesian controller the Franka does not declare:
        >>> task_space_compatible(space, robot, hal_mode="real").ok
        False
        >>> # In sim the robosuite OSC packer synthesises it from joint commands:
        >>> task_space_compatible(space, robot, hal_mode="sim").ok
        True
    """
    reasons: list[str] = []
    if hal_mode == "sim":
        executable: set[ControlMode] = set(SIM_EXECUTABLE_CONTROL_MODES)
    else:
        executable = {ControlMode(m) for m in robot.capabilities.supported_control_modes}
    ee_names = {ee.name for ee in robot.end_effectors}
    n_joints = len(robot.joints)

    joint_width = 0
    for seg in skill_space.segments:
        if seg.control_mode not in executable:
            reasons.append(
                f"control_mode {seg.control_mode.value!r} not executable in "
                f"hal_mode={hal_mode!r} (segment family={seg.family.value!r}, "
                f"width={seg.width})"
            )
        if seg.family in (
            TaskSpaceFamily.CARTESIAN,
            TaskSpaceFamily.GRIPPER,
            TaskSpaceFamily.DEX_HAND,
        ):
            if seg.target is None:
                reasons.append(
                    f"{seg.family.value} segment ({seg.control_mode.value}) has no "
                    "target end-effector to address"
                )
            elif seg.target not in ee_names:
                reasons.append(
                    f"{seg.family.value} segment targets end-effector "
                    f"{seg.target!r}, not declared on robot {robot.name!r} "
                    f"(have: {sorted(ee_names)})"
                )
        if seg.family is TaskSpaceFamily.JOINT:
            joint_width += seg.width

    if joint_width > n_joints:
        reasons.append(
            f"joint-segment width {joint_width} exceeds robot joint count "
            f"{n_joints} on {robot.name!r}"
        )

    return TaskSpaceMatch(ok=not reasons, reasons=reasons)

openral schema v0 — normative Pydantic v2 contracts for all layers.

This is the single source of truth for the data contracts between layers. Anything imported from openral_core.init is public API. Breaking changes require a SemVer MAJOR bump (pre-1.0: MINOR) and a migration entry.

BenchmarkName: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

Benchmark suite id (open; _REGISTRY_ID_PATTERN).

One per benchmarks/<id>.yaml suite in tree.

Membership is checked against the checkout (benchmarks/*.yaml plus scenes/benchmark/*.yaml stems, openral_rskill.loader.known_benchmark_ids) by the benchmark writeback (update_rskill_benchmarks), the openral benchmark scene guard and CI; rSkill.from_pretrained only WARNS on an unknown suite, since a Hub install may have no checkout to compare against.

Used as keys in RSkillManifest.benchmarks. Each value is the headline success rate [0.0, 1.0] the skill achieves on that suite. The full breakdown lives in the matching rskills/<id>/eval/<key>.json.

aloha_insertion / aloha_transfer_cube are retained as task-level ids (the act-aloha* manifests cite their per-task paper numbers) even though the two single-task suites were unified into aloha.yaml — the unified suite auto-filters per rSkill so a single run scores one ACT checkpoint's one task.

CANONICAL_MODEL_TOKENS: frozenset[str] = frozenset({'smolvla', 'pi05', 'xvla', 'xr1', 'act', 'diffusion', 'molmoact2', 'openvla', 'openvla_oft', 'gr00t_n17', 'rldx1_ft', '3d_diffuser_actor', 'lingbot_vla', 'lingbot_vla2', 'lingbot_va_a1', 'internvla_n1', 'sam2_1', 'omdet_turbo', 'rtdetr_coco_r18', 'rtdetr_v2_r50vd', 'robometer_4b', 'topreward_qwen3vl_4b', 'locateanything_3b', 'qwen35_4b', 'moveit', 'nav2'}) module-attribute

Canonical <model> name tokens — the versioned checkpoint vocabulary the repo-naming convention allows. Distinct from ModelFamily (a runner dispatch key): several tokens can share one family (openvla / openvla_oft) and non-VLA tool models (omdet_turbo, robometer_4b, moveit, …) have no family at all. Adding a checkpoint token here lets a new rSkill publish under it.

CANONICAL_QUANT_TOKENS: frozenset[str] = frozenset({'fp32', 'fp16', 'bf16', 'int8', 'nf4'}) module-attribute

Canonical <quant> name tokens for weight-bearing kinds. int4 → nf4 (bitsandbytes NF4). The ROS wrappers (_WEIGHTLESS_KINDS) carry no weights and OMIT the <quant> segment entirely (a 4-part name).

CANONICAL_ROBOT_NAME_TOKENS: frozenset[str] = frozenset({'any', 'multi'}) module-attribute

Reserved robot-agnostic <robot> name tokens: the "any" wildcard and the "multi" aggregate (a skill declaring >1 concrete robot). Concrete robot tokens are an open registry (robots/*/robot.yaml): repo_name_is_canonical checks them by shape only, and CI checks in-tree names against the robot tags (with the so10x_follower tags shortened to so100/so101 via _EMBODIMENT_TO_ROBOT_TOKEN) plus these reserved tokens.

CollisionShape: TypeAlias = Annotated[CapsuleShape | SphereShape | BoxShape, Field(discriminator='shape')] module-attribute

Discriminated union of convex collision primitives.

Discriminator field is shape. Used by LinkCollisionGeometry (robot links) and AttachedCollisionPrimitive (carried payloads). Mesh primitives are excluded — the allocation-free safety kernel checks only convex analytic shapes; mesh-accurate collision is a planning-layer concern.

Notes:

• Validating a mapping requires the shape key; an untagged dict (e.g. {"radius_m": ...}) is rejected with union_tag_not_found rather than structurally guessed. Every robots/ manifest already writes the tag.
• Constructing a member directly is unaffected: BoxShape(...) still fills shape="box" from its default.
• Never dump a shape with exclude_defaults=True — it drops the tag and the result no longer re-validates.

DOP_AXES: tuple[tuple[float, float, float], ...] = ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0), (0.5773502691896258, 0.5773502691896258, 0.5773502691896258), (0.5773502691896258, 0.5773502691896258, -0.5773502691896258), (0.5773502691896258, -0.5773502691896258, 0.5773502691896258), (0.5773502691896258, -0.5773502691896258, -0.5773502691896258), (0.7071067811865476, 0.7071067811865476, 0.0), (0.7071067811865476, -0.7071067811865476, 0.0), (0.7071067811865476, 0.0, 0.7071067811865476), (0.7071067811865476, 0.0, -0.7071067811865476), (0.0, 0.7071067811865476, 0.7071067811865476), (0.0, 0.7071067811865476, -0.7071067811865476)) module-attribute

The 26-DOP's 13 unit axis directions, in the owning box's own local frame.

Deliberate mirror of kDopAxis in cpp/openral_safety_kernel/include/openral_safety_kernel/collision.hpp: the manifest and the kernel must index the same slab with the same direction, or the containment proof recorded in one does not transfer to the other. See docs/methods/14-duplication-watch.md.

The first three entries are the box's own axes, which is load-bearing rather than cosmetic: a 26-DOP lies inside its own first three slabs, so checking those three against the box's half-extents proves containment for the whole polytope without enumerating a single vertex.

EmbodimentTag: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

Embodiment tag (open; _REGISTRY_ID_PATTERN).

One per robots/<id>/robot.yaml shipped in tree, plus "custom" as the explicit "I know what I'm doing" escape hatch and "any" as the explicit embodiment-agnostic wildcard.

"multi" is a repo-name aggregate token (CLAUDE.md §3 rSkill packaging): the <robot> segment a skill declaring >1 concrete robot uses (e.g. the moveit-* ROS wrappers). It is a member of the Literal so the name validator can accept it; a manifest normally lists the specific robots in embodiment_tags rather than the aggregate.

"any" is the declared way to say "this rSkill runs on every embodiment": perception kinds (detector / vlm / reward) and playbook decision procedures use embodiment_tags: ["any"]. An empty embodiment_tags is rejected by RSkillManifest._check_embodiment_tags_present — agnosticism must be declared, never derived from an empty list (CLAUDE.md §1.4). The rSkill↔robot gate (openral_rskill.loader.rSkill.check_embodiment_tags) treats "any" in a skill's tags as match-any; a robot never declares "any".

"mobile_base" is a CLASS tag (not a specific robot): any robot with a planar base + body_twist actuator declares it so base-only rSkills (Nav2 NavigateToPose, etc.) can target the whole class without naming each specific mobile platform. Robot-specific tags (e.g. "panda_mobile") coexist on the same RobotDescription for skills that DO depend on the specific composition.

The in-tree CI guard (tests/unit/test_manifest_registry_ids.py) restricts every rskills/*/rskill.yaml tag to {any, custom, multi} plus the tags some robots/*/robot.yaml declares, so a typo cannot land in a manifest where the loader's compat check would silently never match; openral_rskill.loader.intree_embodiment_tags is that set. When "custom" is used, the manifest MUST also populate embodiment_extra (see EmbodimentExtra); the cross-validator on RSkillManifest enforces this.

FailureEvidence: TypeAlias = TimeoutEvidence | ForceEvidence | WorkspaceEvidence | PerceptionStaleEvidence | CriticEvidence | ControllerEvidence | SelfVerifyEvidence | HumanEvidence | WamEvidence | ReasonerTimeoutEvidence | CollisionEvidence | SuppressedSummaryEvidence module-attribute

Discriminated union for FailureTrigger.evidence_json payloads.

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming evidence_json string with::

from pydantic import TypeAdapter
from openral_core import FailureEvidence

evidence = TypeAdapter(FailureEvidence).validate_json(msg.evidence_json)

Producers serialize via evidence.model_dump_json().

The union mirrors the KIND_* constants on openral_msgs/msg/FailureTrigger. Adding a new variant requires adding a new KIND_* constant to the IDL.

GripperConvention: TypeAlias = Literal['normalized_open_unit', 'normalized_open_symmetric', 'normalized_close_symmetric', 'binary_close_one', 'raw_joint_rad', 'width_meters'] module-attribute

How a gripper command value is encoded — on both sides of the wire.

An rSkill declares the encoding it emits on its GRIPPER_* actuator (ControlModeSemantics.gripper_convention); a robot declares the encoding each gripper end effector consumes (EndEffectorSpec.command_convention). rSkill.check_gripper_conventions refuses to load a skill whose encoding differs from the end effector it drives — silently mis-encoding the gripper between scenes / robots is a top observed mis-actuation failure — and the safety kernel bounds every GRIPPER_* chunk by that end effector's command_range.

Conventions:

• normalized_open_unit — 0.0 = fully closed, 1.0 = fully open. Most lerobot / SmolVLA / pi0.5 LIBERO checkpoints.
• normalized_open_symmetric — -1.0 = fully closed, +1.0 = fully open. Some MetaWorld checkpoints.
• normalized_close_symmetric — -1.0 = fully open, +1.0 = fully closed. robosuite's gripper action (RoboCasa).
• binary_close_one — 0.0 = open, 1.0 = close (single bit).
• raw_joint_rad — raw per-finger joint angle in radians (Fourier dexhands, some humanoid skills).
• width_meters — physical gripper width in metres (Franka FCI native).

GroundTruthAdjudication: TypeAlias = Literal['real-contact', 'false-positive', 'within-quantization', 'unadjudicated'] module-attribute

Verdict of the simulator ground-truth probe on a kernel stop.

real-contact — some probed pair of solid geoms is at or below 0 m, so geometry really is touching. false-positive — the nearest true geometry is further from the tripping party than the admissible kernel-vs-probe gap can explain; this requires a published gap, since the voxel term alone is a lower bound on it and exceeding a lower bound establishes nothing. within-quantization — the kernel was conservative by an amount that gap accounts for, which is correct behaviour. unadjudicated — the probe did not cover the stop (truncated, no snapshot recorded, no budget known, or the recorded probe does not attest that both of its sides were collidability filtered, in which case a 0 m pair may be a purely visual mesh and cannot support real-contact).

JointRole: TypeAlias = Literal['arm', 'base', 'gripper', 'torso', 'leg', 'head', 'neck', 'wheel', 'unknown'] module-attribute

Structural classification of a JointSpec.

Carries the joint's purpose in the embodiment's morphology — what the runner, safety kernel, and dataset bridge need to identify a channel without relying on name-substring heuristics (e.g. "gripper" in name.lower() in rskill_runner_node._build_joint_permutation which silently misclassifies any joint with "gripper" in the name).

"unknown" is the default so legacy manifests load unchanged; the fleet annotates incrementally as this rolls out.

MAX_TIGHT_HULL_VERTICES = 320 module-attribute

Ceiling on a stage-2 hull's vertex count, mirroring kMaxTightHullVertices.

A cost bound set by measurement, not a geometry choice: the kernel's stage-2 support function is an exhaustive scan (the only form whose result is provably the true support, which is what keeps its distance bound sound), so its cost is linear in the vertex count and the staged path stops paying for itself past roughly this many vertices. See docs/reference/collision-hull-narrow-phase.md.

MemorySection: TypeAlias = Literal['home_map', 'preferences', 'lessons', 'object_locations', 'open_tasks'] module-attribute

The fixed sections of the self-maintained MEMORY.md core.

• home_map — stable places / region-connectivity (EDIT in place).
• preferences — distilled user-preference rules (TidyBot; EDIT in place).
• lessons — distilled human corrections / hard-won lessons (DROC; APPEND).
• object_locations — timestamped object→place log (SUPERSEDE on re-observation).
• open_tasks — recurring commitments / open items (EDIT; remove on completion).

ModelFamily: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

VLA / policy family the skill belongs to (open; _REGISTRY_ID_PATTERN).

Membership is checked against the policy registry openral_sim.POLICIES (the runner refuses an unregistered family at dispatch; CI checks every in-tree manifest). In-tree families: smolvla, pi05, xvla, xr1, act, diffusion, rldx, molmoact2, gr00t, diffuser_actor, openvla, lingbot_vla2, lingbot_vla, lingbot_va_a1, internvla_n1.

Used by the eval / runner adapters to dispatch to the right openral_sim.backends.<family> policy adapter without string-matching the skill name. Adding a family means registering the matching adapter in openral_sim.POLICIES — not editing this module.

xr1 (Xiaomi Robotics XR-1 / MiBoTForActionGeneration) runs out-of-process because its released checkpoints pin transformers 4.57.1 and FlashAttention 2.8.3 (the sidecar installs torch 2.9.1 rather than upstream's 2.8.0, which publishes no linux-aarch64 wheel). The adapter consumes the three public benchmark checkpoints; the bare 5B model_states.pt is a post-training seed, not an executable policy.

gr00t (NVIDIA Isaac GR00T N1.x / N2) runs out-of-process via a ZMQ sidecar in an isolated Python 3.10 venv, sharing the architecture of the rldx adapter (RLDX-1 is itself a GR00T-N1.5 finetune).

lingbot_vla2 (Robbyant LingBot-VLA 2.0, Qwen3-VL-4B backbone + sparse-MoE flow-matching action expert, Apache-2.0 code + weights) runs out-of-process via a ZMQ sidecar in its own Python 3.12 + torch-2.9.1 venv — the upstream lingbotvla package pins torch==2.8.0 / transformers==4.57.3 + custom Triton MoE kernels, incompatible with the workspace transformers>=5 (the torch half of that pin set is overridden to 2.9.1 because 2.8.0 has no linux-aarch64 wheel; adapter: openral_sim.policies.lingbot_vla2).

lingbot_vla (Robbyant LingBot-VLA 1.0, 4B Qwen2.5-VL-3B backbone + a dense Qwen2 flow-matching action expert; paper "A Pragmatic VLA Foundation Model", arXiv 2601.18692) is the RoboTwin post-train checkpoint. It shares the v2 sidecar transport + adapter but loads from the separate V1 upstream repo (github.com/robbyant/lingbot-vla) in its own transformers==4.51.3 / lerobot==0.4.2 venv, driven by tools/_lingbot_vla2_server.py --variant v1 (adapter: openral_sim.policies.lingbot_vla2 id lingbot_vla).

lingbot_va_a1 is the Galaxea A1 EEF deployment adapter for the LingBot-VA checkpoint. It connects to versioned local Camera Bridge and policy-gateway services owned by the separately maintained A1 Runtime, without importing its checkout. Runtime owns the exact model configuration, EEF transforms, cache semantics, and IK; OpenRAL validates the gateway identity and returns its joint/gripper proposals through the normal safety and HAL path.

openvla (OpenVLA / OpenVLA-OFT) is a transformers custom-code model loaded in-process (trust_remote_code, gated by OPENRAL_ALLOW_REMOTE_CODE=1); the adapter de-normalizes the policy's discrete action tokens with the checkpoint's embedded unnorm_key stats and replays the action chunk closed-loop.

internvla_n1 (InternRobotics InternVLA-N1 / DualVLN, arXiv:2512.08186) is a dual-system vision-language NAVIGATION policy (Qwen2.5-VL-7B System-2 + NavDP DiT System-1). Upstream InternNav pins transformers==4.51.0 — incompatible with the workspace — so it runs out-of-process via the same ZMQ sidecar architecture as rldx. The adapter consumes RGB-D + instruction and emits a 6-D BODY_TWIST velocity command for a mobile base.

PerceptionEventMetadata: TypeAlias = MotionMetadata | ObjectsMetadata | OcrMetadata | SceneChangeMetadata module-attribute

Discriminated union for PromptStamped.metadata_json payloads on the /openral/perception/<kind> topics.

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming metadata_json string with::

from pydantic import TypeAdapter
from openral_core import PerceptionEventMetadata

metadata = TypeAdapter(PerceptionEventMetadata).validate_json(msg.metadata_json)

Producers serialise via metadata.model_dump_json().

The four variants map 1:1 onto the four per-kind topics fixed in the perception-event contract (motion, objects, ocr, scene_change). Adding a new variant requires adding a new /openral/perception/<kind> topic to the contract — by design, new kinds get new topics, not a schema bump, so subscribers can subscribe to exactly the kinds they care about.

REASONER_TOOL_CHOICES: tuple[str, ...] = ('auto', 'required') module-attribute

Values OPENRAL_REASONER_TOOL_CHOICE accepts (OpenAI tool_choice modes).

ROBOT_UNIT_ENV = 'OPENRAL_ROBOT_UNIT' module-attribute

Env var naming the robots/<robot_id>/units/<unit>.yaml overlay for this host.

Wins over DeployScene.robot_unit so one committed scene runs on several cells.

RSkillKind: TypeAlias = Literal['vla', 'wam', 'ros_action', 'ros_service', 'detector', 'segmenter', 'vlm', 'reward', 'playbook'] module-attribute

Discriminator selecting how an rSkill is instantiated at the loader.

• "vla" — learnable Vision-Language-Action policy. Requires RSkillManifest.model_family and RSkillManifest.weights_uri; resolved by the policy adapter dispatch in openral_rskill (the pre-existing path used by every in-tree rSkill prior to this discriminator landing).
• "wam" — World Action Model (planning-layer mental-simulation / failure-anticipation component per CLAUDE.md §3). Reserved so the discriminator is forward-compatible; the loader / runner branch is not implemented yet and kind: wam manifests are rejected at resolve time with ROSConfigError.
• "ros_action" — wraps an existing ROS 2 action server. Requires a RosIntegration block. model_family and weights_uri are forbidden. chunk_size is pinned to 1 so the safety supervisor's per-row check sees every commanded position.
• "ros_service" — wraps an existing ROS 2 service. Same constraints as "ros_action".
• "detector" — perception producer that runs an exported detection model (RT-DETR / D-FINE ONNX) on the camera tee and publishes ObjectsMetadata; emits no Action. Requires a DetectorContract block and RSkillManifest.weights_uri (the exported ONNX / TensorRT engine). model_family and action_contract / state_contract are forbidden.
• "segmenter" — perception producer that answers a geometric prompt (a point, or a small set of positive/negative points, in pixel coordinates) with a binary mask over the current camera frame. Unlike "detector" it carries no label vocabulary and no score threshold: it does not say what the pixels are, only which pixels belong to the prompted thing. Its consumer is the HAL's vision attachment-evidence producer, which intersects the mask with the wrist depth frame to bound a grasped payload. Requires a SegmenterContract block and RSkillManifest.weights_uri; model_family, detector, reward, ros_integration, action_contract / state_contract, processors, image_preprocessing, n_action_steps and starting_pose are FORBIDDEN, and actuators_required MUST be empty — a segmenter actuates nothing.
• "vlm" — vision/video-language model used as a scene-understanding perception component (e.g. Qwen3.5-4B NF4). Accepts RGB image or video frames and a natural-language query; returns a text answer. Emits no actions or bounding boxes. Runs at S2 rate (role: "s2"), so it is surfaced to the reasoner as a read-only scene-query tool, never as an ExecuteRskill policy. weights_uri REQUIRED; actuators_required MUST be empty; action_contract, state_contract, detector, ros_integration, processors, image_preprocessing, n_action_steps, and starting_pose are FORBIDDEN. model_family is OPTIONAL metadata.
• "playbook" — a symbolic, human-authored decision procedure (a Markdown standard-operating-procedure) the S2 Reasoner reads, not code it executes. Carries no weights, no actuators, no ROS server, no Action contract — it is the symbolic counterpart to a vla policy. Requires a PlaybookContract block and role: "s2"; chunk_size MUST be 1 (required field, no Action rows); actuators_required MUST be empty; actions MUST include RSkillAction.PLAN. weights_uri, model_family, min_vram_gb, detector, reward, ros_integration, processors, image_preprocessing, action_contract, state_contract, n_action_steps, starting_pose are FORBIDDEN. Surfaced to the reasoner by injecting its PLAYBOOK.md body into the system prompt (or via a retrieval tool at scale), never as an ExecuteRskill policy.

ReasonerDialect = Literal['anthropic', 'openai'] module-attribute

Wire client an endpoint speaks — the only two the reasoner ships (ADR-0088).

ReasonerHosting = Literal['cloud', 'managed_local', 'byo_local'] module-attribute

Where a reasoner model runs: a vendor's cloud, a server OpenRAL spawns and manages locally, or a local server the operator brings up themselves.

ReasonerToolCall: TypeAlias = Annotated[ExecuteRskillTool | ReloadGstPipelineTool | LifecycleTransitionTool | EmitPromptTool | WaitTool | RecallObjectTool | ResolvePlaceTool | LocateInViewTool | QuerySceneTool | QueryTaskProgressTool | MemoryWriteTool | MemorySearchTool | DecomposeMissionTool, Field(discriminator='tool')] module-attribute

Discriminated union over the reasoner tool variants.

The discriminator field is tool (a string Literal on each variant), and it is declared as one: a malformed payload reports only the errors of the variant it tagged itself as, instead of every variant's. The caller always sets tool from the provider's function name before validating (openral_reasoner.tool_use), so an untagged payload is a bug in the caller, not a shape to guess at.

Consumers decode an LLM tool-use payload with::

from pydantic import TypeAdapter
from openral_core import ReasonerToolCall

call = TypeAdapter(ReasonerToolCall).validate_json(payload)

Producers (LLM clients) serialise via call.model_dump_json().

Actuation/effect palette: ExecuteRskillTool, ReloadGstPipelineTool, LifecycleTransitionTool, EmitPromptTool, WaitTool. Read-only query variants (no actuation authority): RecallObjectTool, ResolvePlaceTool, LocateInViewTool, QuerySceneTool, QueryTaskProgressTool, MemorySearchTool. Write-to-memory: MemoryWriteTool. S2-ledger-only: DecomposeMissionTool (populates/flat-splices MissionState's task queue, never actuation).

Extending the palette requires (a) a new variant here, (b) the corresponding ROS-side dispatch in openral_reasoner_ros.reasoner_node, (c) a CLAUDE.md §3 amendment if the new tool shifts the reasoner's authority surface. The query variants' dispatch + result-return path is wired separately; until then they are a typed contract not yet exposed in the live provider palette.

StateLayout: TypeAlias = Annotated[str, StringConstraints(pattern=_REGISTRY_ID_PATTERN)] module-attribute

Per-checkpoint proprioception layout id (open; _REGISTRY_ID_PATTERN).

Membership is NOT checked here: a task-space layout without a registered assembler is dropped by the reasoner palette and refused by openral_state_adapter.assemble_state (ROSConfigError); CI checks every in-tree manifest layout against registered_layouts() plus the sim adapters' layout maps.

In-tree layouts:

• smolvla_9d, human300_16d, gr1, rc365.
• simpler_widowx matches RLWRLD/RLDX-1-FT-SIMPLER-WIDOWX's bridge_orig modality config (8 scalar state keys, single video.image_0 camera, Bridge-data orientation rotation). simpler_google matches RLWRLD/RLDX-1-FT-SIMPLER-GOOGLE's fractal20220817_data modality config (8 scalar state keys including a 4-D quaternion in state.r{x,y,z,w}, single video.image camera, sticky-gripper postprocessing).
• libero_eef8d — LIBERO 8-D task-space proprio eef_pos(3) ‖ eef_axisangle(3) ‖ gripper_qpos(2) in the world frame, what the lerobot/smolvla_libero, pi05-libero and xvla-libero checkpoints were trained on. The benchmark (openral sim run) supplies it directly; deploy assembles it from live TF (EE pose) + JointState (gripper) via the libero_eef8d assembler. Without it the runner would feed raw joint-space state to a task-space policy.

A layout names the SHAPE the checkpoint was trained on — field order, frame convention, gripper encoding, quaternion handedness. The per-robot SOURCE bindings (which TF frame is "the EE", which joint names are "the gripper") live on StateContractBindings. The openral_state_adapter registry maps each layout id to an assembler function that joins shape + bindings + live JointState + live TF.

TIGHT_CONTAINMENT_EPSILON_M = 1e-09 module-attribute

Floating-point slack when checking a hull vertex against its own DOP slab.

Mirrors kTightContainmentEpsilonM. The slab bound is the maximum of u·x over the same vertex set, so the relation is an equality on at least one vertex per axis and the only open question is evaluation order. One nanometre buys numerical agreement, not geometric room.

ValidationOutcome: TypeAlias = Literal['completed', 'estop-collision-real', 'estop-collision-false-positive', 'estop-collision-within-quantization', 'estop-collision-unadjudicated', 'estop-initial-configuration', 'deadline-after-grasp', 'deadline-no-grasp', 'harness-error'] module-attribute

How one scene of a validation round ended.

completed is the only success. The four estop-collision-* values all mean the C++ safety kernel stopped the run and differ only in what the simulator's ground truth says about that stop; estop-initial-configuration means the stop landed before any action reached the HAL, so it is a scene-initialisation defect rather than a stack one (see openral_hal.sim_sensor_bridge.initial_configuration_stop_record). harness-error means the run never produced a usable artifact set.

WRAPPED_TASK_SPACE_LAYOUTS: frozenset[StateLayout] = frozenset({'rc365', 'human300_16d', 'libero_eef8d'}) module-attribute

Layouts that are TASK-space composites (Cartesian poses + gripper widths), NOT one-scalar-per-joint. These layouts REQUIRE StateContract.bindings to name the source TF frames + JointState entries — the cross-validator on StateContract enforces this at manifest load. The remaining layouts (smolvla_9d, gr1, simpler_*) are joint-space slices (potentially across multiple controller groups, as in GR1's 29-D waist+arms+hands composite) that the runner serves verbatim from observation.joint_state.position; a robot.yaml with the matching joint count dispatches them without an assembler.

Action

Bases: BaseModel

A single action step or chunk produced by a Skill.

Attributes:

Name	Type	Description
control_mode	ControlMode	Target action space.
horizon	int	Number of steps (1 = single step, H = chunk).
joint_targets	list[list[float]] | None	Joint position targets, shape (H, N).
joint_velocities	list[list[float]] | None	Joint velocity targets, shape (H, N).
joint_torques	list[list[float]] | None	Joint torque targets, shape (H, N).
cartesian_pose	list[Pose6D] | None	EE pose targets.
cartesian_delta	list[tuple[float, ...]] | None	EE pose deltas.
cartesian_twist	list[tuple[float, ...]] | None	EE velocity targets.
body_twist	list[tuple[float, float, float, float, float, float]] | None	Base twist targets.
foot_placements	list[dict[str, object]] | None	Discrete footstep targets.
gripper	list[float] | None	Gripper commands in [0, 1].
dex_hand_joints	list[list[float]] | None	Dexterous hand joint targets.
confidence	float	rSkill confidence in [0, 1].
stamp_ns	int	Action timestamp in nanoseconds.
ee_name	str | None	Target end-effector name.
frame_id	str | None	Reference frame for Cartesian actions.
cartesian_delta_scale	tuple[float, ...] | None	Per-axis physical range for a native normalized Cartesian-delta controller. Predictive safety applies clip(raw, -1, 1) * scale; the raw delta remains unchanged for the HAL. None means the policy already emits physical units.
joint_names	list[str] | None	Robot joint names this action's rows target, in the emitting slot's order (ADR-0102). None = a whole-vector action in RobotDescription.joints order. Set for JOINT_* slots, which are zero-padded to full dof and so cannot otherwise say which joints they own; authoritative over slot arrival order when present.
tick_index	int	Shared 1-based inference tick for every slot emitted by one policy step. Preserved across the ROS safety wire so a simulation HAL can commit a multi-surface action atomically after every slot passes.
tick_group_size	int	Number of non-discard slots emitted for this inference tick. 1 keeps single-surface actions unchanged.
runner_session_id	int	Random nonzero id of the rSkill runner process that emitted this action (ActionChunk.runner_session_id). A HAL keys its replay watermark on (runner_session_id, tick_index) because tick_index restarts with every runner process. 0 = unknown (session-less producer), which keeps the tick-only replay heuristic.
safety_overrides	dict[str, object]	Operator-approved safety override tokens.

ActionContract

Bases: BaseModel

Per-rSkill action-vector contract.

Mirrors StateContract for the action side. Carries the output dimensionality the checkpoint emits so the dataset bridge (and any downstream consumer) can bind the LeRobot v3 action feature shape without consulting the sim or hardware adapter.

The sim-specific action contract belongs on the per-checkpoint rSkill manifest, not on the physical RobotDescription (the same Franka emits 7-D delta-EEF on LIBERO vs 8-D joint pos on a hardware deploy).

Attributes:

Name	Type	Description
dim	int	Dimensionality of the action vector emitted by Skill.step() / PolicyAdapter.step(). Required when this contract is set.
representation	ActionRepresentation | None	Optional named representation (mirrors ActionSpec.representation). When set, downstream consumers can map between equivalent representations (e.g. joint_positions → delta_ee_6d_plus_gripper).
slots	list[ActionSlot] | None	Declarative slot layout. When set, every index in [0, dim) must be covered by exactly one ActionSlot (no gaps, no overlaps). The skill_runner reads this to dispatch slices of the policy vector onto typed Action objects. When None, the runner falls back to the legacy single-Action path (one implicit JOINT_POSITION slot covering the whole vector). Manifests carrying slots are exempt from the dim <= len(robot.joints) invariant because the slot decoder gives a typed contract per slice instead.
cartesian_delta_scale	tuple[float, ...] | None	Per-axis physical range for a normalized Cartesian controller. Predictive safety applies clip(raw, -1, 1) * scale. None means identity (the policy already emits physical units).
joint_names	list[str] | None	The checkpoint's joint order on the whole-vector joint path, spelled as RobotDescription.joints names (matched BY NAME by openral_rskill._policy_io.PolicyIOCodec). None means the codec derives the order from the checkpoint's own feature names, or identity. Forbidden together with slots (each JOINT_* slot carries its own joint_names).
gripper_scale	float	Policy-unit gripper value per robot-unit value. The HAL gripper contract is a normalized [0, 1] jaw fraction; a LeRobot SO-ARM checkpoint's gripper channel is [0, 100] → 100.0. Applied by the codec on BOTH the joint and the slot path.
control_freq_hz	float | None	The rate (Hz) the checkpoint was trained to be executed at — one action-chunk row per control period. None = undeclared (the runner uses the robot's action_spec.control_freq_hz unchecked). When declared, the skill runner refuses a goal whose tick rate differs: there is no temporal resampler, and executing a 15 Hz policy at 30 Hz doubles its commanded speed.

ActionRepresentation

Bases: str, Enum

Action vector representation format.

ActionSlot

Bases: BaseModel

One contiguous slice of an rSkill's action vector.

The skill_runner reads ActionContract.slots and emits one typed Action per non-discard slot per step. All actions inherit the parent step's trace_id so the safety supervisor and downstream telemetry can join them post-hoc.

Attributes:

Name	Type	Description
range	tuple[int, int]	Inclusive [start, end] indices into the flat policy action vector. range[0] must be ≤ range[1]; both must fall within [0, ActionContract.dim).
control_mode	ControlMode | None	The ControlMode the slice is routed to. The HAL whitelist on the target robot must include this mode (the palette filter rejects the rSkill at install time otherwise). None only when discard is True.
discard	bool	When True the slice is dropped silently — used for dataset artefacts like RoboCasa365's torso placeholder dim or paired gripper channels. The slot still occupies its range so coverage validation works; no Action is emitted.
ee	str | None	End-effector name from the robot's RobotDescription.end_effectors / RobotDescription.joints. REQUIRED for CARTESIAN_* modes (the pose is computed in the named EE's frame) and for GRIPPER_* modes (names the actuator). FORBIDDEN for BODY_TWIST and JOINT_POSITION.
frame	str | None	tf2 frame name. REQUIRED for cartesian + body-twist modes (the slice's bytes are expressed in this frame). FORBIDDEN for joint-position and gripper modes.
joint_names	list[str]	Robot joint names this slice targets, in slot order. REQUIRED for JOINT_POSITION / JOINT_VELOCITY / JOINT_TORQUE when the slot covers fewer than all robot joints; FORBIDDEN for non-joint modes. Length must equal range[1] - range[0] + 1.
input_bounds	tuple[float, float] | None	Optional inclusive controller-input bounds. The skill runner clips this slot before safety and HAL dispatch, matching native controllers that clip normalized policy outputs before applying physical scaling.

ActionSpec

Bases: BaseModel

VLA action configuration for this robot.

Attributes:

Name	Type	Description
dim	int | None	Dimensionality of the action vector. None = not declared: the dataset recorder then takes the width from the action itself. A robot whose policy contract is not committed declares only the control rate rather than a guessed width (issue #303).
representation	ActionRepresentation | None	How the action vector is encoded.
control_freq_hz	float | None	Control frequency the actions are executed at — the runner's tick, the HAL node's proprio publish rate, the real ros2_control HAL's trajectory deadline, and the recorder's fps.
chunk_size	int | None	Number of action steps per inference call (chunk size H).

ActuatorRequirement

Bases: BaseModel

One actuator slot an rSkill emits actions for.

Symmetric with SensorRequirement on the action side. The compatibility check resolves each entry against a RobotDescription's action_spec and rejects pairings whose declared ControlMode is not advertised by the robot.

For predefined embodiments (one of the 9 canonical robots/<id>/robot.yaml slugs), n_dof and vla_action_key are optional — the loader auto-fills them from the robot YAML at compatibility-check time. For the "custom" embodiment escape hatch they MUST be set on the manifest; the RSkillManifest cross-validator enforces this.

Attributes:

Name	Type	Description
kind	ControlMode	The action interface the skill emits (e.g. ControlMode.JOINT_POSITION, ControlMode.CARTESIAN_DELTA). Reuses the canonical ControlMode enum so the manifest, the safety kernel, Action.control_mode, and RobotDescription.action_spec all share one source of truth.
n_dof	int | None	Optional degrees of freedom the skill emits. Auto-filled from the robot YAML for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
vla_action_key	str | None	Optional slot name the policy emits, e.g. "action.joints.arm_left". Auto-filled for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
control_mode_semantics	ControlModeSemantics	Action-space semantics for this slot. REQUIRED — closes Gap 2 of the rSkill self-containment audit. Cross-validators enforce: gripper kinds require gripper_convention; cartesian kinds require reference_frame; other kinds forbid both.

Example

a = ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) a.kind is ControlMode.JOINT_POSITION True a.n_dof is None and a.vla_action_key is None True

ApproachViewpoint

Bases: BaseModel

A camera-facing standoff pose for viewing/manipulating an object.

Attributes:

Name	Type	Description
pose	Pose6D	Base/EE goal pose (map frame) at a standoff from the object, oriented so the gripper-mounted camera faces it.
standoff_m	float	Standoff distance from the object, in metres.
camera_frame_id	str	tf2 frame of the camera the viewpoint orients toward.

AssetRefs

Bases: BaseModel

The unified description-asset block on RobotDescription.

Replaces the scattered urdf_path / mjcf_uri / srdf_path (+ the earlier URDF-root fields) with one block whose refs share the openral_core.assets.resolve_asset grammar.

Attributes:

Name	Type	Description
urdf	UrdfAsset | None	URDF asset (with optional robot_state_publisher wiring), or None when the robot ships no URDF.
mjcf	str | None	MJCF asset ref (MuJoCo wiring), or None.
srdf	str | None	SRDF asset ref whose disable_collisions block seeds RobotDescription.allowed_collision_pairs, or None.

Example

AssetRefs(urdf=UrdfAsset(ref="rd:panda_description")).urdf.ref 'rd:panda_description'

AttachedCollisionObject

Bases: BaseModel

Collision geometry rigidly attached to a robot link.

The object is removed from world occupancy only when the same geometry is present here, so collision checking never loses the carried payload. The removal itself is the occupancy bridge's (openral_octomap_bridge clears the payload's own cells out of /openral/world_voxels while it appears here); the kernel keeps checking the payload as robot geometry.

Attributes:

Name	Type	Description
object_id	str	Stable perception/world identity.
attach_link	str	Robot link that owns the link-relative pose.
touch_links	list[str]	Explicit robot links allowed to contact the object.
primitives	list[AttachedCollisionPrimitive]	Bounded collision geometry in the object frame.
pose_in_link	Pose6D	Object pose whose frame_id equals attach_link.
mass_kg	float | None	Optional payload mass for real controller integration.
center_of_mass_m	tuple[float, float, float] | None	Optional payload CoG in the attached-object frame.
inertia_kg_m2	tuple[float, ...] | None	Optional row-major 3x3 inertia matrix.
confidence	float	Attachment confidence in [0, 1].
evidence_kind	AttachmentEvidenceKind	Source that confirmed the attachment.
evidence_ref	str | None	Optional trace/contact/track reference.
stamp_ns	int	Confirmation timestamp.
contact_force	ContactForceWitness | None	Optional attestation of a measured contact force on the declared place target (ADR-0100). None — the default, and what every producer that cannot measure force publishes — means the declaration-scoped force gate never arms and geometry decides alone. The gate it feeds only ever adds a refusal; it never licenses a contact the geometry refuses.
support_contact	SupportContactWitness | None	Optional attestation that this payload is in bounded support contact with a named environment surface (ADR-0092 D6). None — the honest value for any producer that cannot measure support contact, including the vision attachment producer — means the safety kernel exempts no payload-vs-world contact at all.

fill_idl(msg: object, *, primitive_factory: Callable[[], object]) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object, *, primitive_factory: Callable[[], object]) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS."""
    msg.object_id = self.object_id  # type: ignore[attr-defined]
    msg.attach_link = self.attach_link  # type: ignore[attr-defined]
    msg.touch_links = list(self.touch_links)  # type: ignore[attr-defined]
    msg.primitives = []  # type: ignore[attr-defined]
    for primitive in self.primitives:
        item = primitive_factory()
        primitive.fill_idl(item)
        msg.primitives.append(item)  # type: ignore[attr-defined]
    pose = msg.pose_in_link  # type: ignore[attr-defined]
    pose.position.x, pose.position.y, pose.position.z = self.pose_in_link.xyz
    (
        pose.orientation.x,
        pose.orientation.y,
        pose.orientation.z,
        pose.orientation.w,
    ) = self.pose_in_link.quat_xyzw
    msg.mass_valid = self.mass_kg is not None  # type: ignore[attr-defined]
    msg.mass_kg = float(self.mass_kg or 0.0)  # type: ignore[attr-defined]
    msg.center_of_mass_valid = self.center_of_mass_m is not None  # type: ignore[attr-defined]
    if self.center_of_mass_m is not None:
        center = msg.center_of_mass_m  # type: ignore[attr-defined]
        center.x, center.y, center.z = self.center_of_mass_m
    msg.inertia_valid = self.inertia_kg_m2 is not None  # type: ignore[attr-defined]
    msg.inertia_kg_m2 = (  # type: ignore[attr-defined]
        [float(value) for value in self.inertia_kg_m2]
        if self.inertia_kg_m2 is not None
        else [0.0] * 9
    )
    msg.confidence = float(self.confidence)  # type: ignore[attr-defined]
    msg.evidence_kind = self.evidence_kind.value  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref or ""  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.support_contact_valid = self.support_contact is not None  # type: ignore[attr-defined]
    if self.support_contact is not None:
        self.support_contact.fill_idl(msg.support_contact)  # type: ignore[attr-defined]
    msg.contact_force_valid = self.contact_force is not None  # type: ignore[attr-defined]
    if self.contact_force is not None:
        self.contact_force.fill_idl(msg.contact_force)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    center_of_mass = None
    if bool(msg.center_of_mass_valid):  # type: ignore[attr-defined]
        center = msg.center_of_mass_m  # type: ignore[attr-defined]
        center_of_mass = (float(center.x), float(center.y), float(center.z))
    inertia = (
        tuple(float(value) for value in msg.inertia_kg_m2)  # type: ignore[attr-defined]
        if bool(msg.inertia_valid)  # type: ignore[attr-defined]
        else None
    )
    pose = msg.pose_in_link  # type: ignore[attr-defined]
    attach_link = str(msg.attach_link)  # type: ignore[attr-defined]
    return cls(
        object_id=str(msg.object_id),  # type: ignore[attr-defined]
        attach_link=attach_link,
        touch_links=list(msg.touch_links),  # type: ignore[attr-defined]
        primitives=[
            AttachedCollisionPrimitive.from_idl(item, object_id=str(msg.object_id))  # type: ignore[attr-defined]
            for item in msg.primitives  # type: ignore[attr-defined]
        ],
        pose_in_link=Pose6D(
            xyz=(
                float(pose.position.x),
                float(pose.position.y),
                float(pose.position.z),
            ),
            quat_xyzw=(
                float(pose.orientation.x),
                float(pose.orientation.y),
                float(pose.orientation.z),
                float(pose.orientation.w),
            ),
            frame_id=attach_link,
        ),
        mass_kg=(
            float(msg.mass_kg)  # type: ignore[attr-defined]
            if bool(msg.mass_valid)  # type: ignore[attr-defined]
            else None
        ),
        center_of_mass_m=center_of_mass,
        inertia_kg_m2=inertia,
        confidence=float(msg.confidence),  # type: ignore[attr-defined]
        evidence_kind=AttachmentEvidenceKind(str(msg.evidence_kind)),  # type: ignore[attr-defined]
        evidence_ref=str(msg.evidence_ref) or None,  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        contact_force=(
            ContactForceWitness.from_idl(msg.contact_force)  # type: ignore[attr-defined]
            if bool(getattr(msg, "contact_force_valid", False))
            else None
        ),
        support_contact=(
            SupportContactWitness.from_idl(msg.support_contact)  # type: ignore[attr-defined]
            if bool(msg.support_contact_valid)  # type: ignore[attr-defined]
            else None
        ),
    )

AttachedCollisionPrimitive

Bases: BaseModel

One bounded payload primitive positioned in its object frame.

Attributes:

Name	Type	Description
shape	CollisionShape	The convex primitive (capsule, sphere, or box).
pose_in_object	Pose6D	Pose of the primitive in the owning object's frame.
tight_geometry	TightCollisionGeometry | None	Optional tighter convex geometry the safety kernel uses in place of shape in the payload-vs-world-voxel check — the same staged 26-DOP → exact-hull narrow phase a robot link gets from LinkCollisionGeometry.tight_geometry, and the same containment rule: only valid on a BoxShape, and only when it fits inside it. shape stays the broad-phase bound and the geometry every other check uses. None is not a defect — a sphere/box/capsule sim geom lowers exactly and has nothing to refine. It is only a mesh geom, whose lowering is its local AABB, that leaves a real gap: measured across the 2026-09-10 resolution A/B (424 samples, 4 scenes) a carried payload's box corners stand a median 50.78 mm proud of the mesh, 2.3-3.9x the entire world-voxel quantisation term, and 97 % of 15 mm-arm stops were payload-vs-world (issue #266).

Example

p = AttachedCollisionPrimitive( ... shape=BoxShape(half_extents_m=(0.03, 0.03, 0.09)), ... pose_in_object=Pose6D( ... xyz=(0.0, 0.0, 0.0), ... quat_xyzw=(0.0, 0.0, 0.0, 1.0), ... frame_id="sim:obj_main", ... ), ... ) p.tight_geometry is None True

fill_idl(msg: object) -> None

Populate one duck-typed attached primitive without importing ROS.

Parameters:

Name	Type	Description	Default
msg	object	A duck-typed openral_msgs/AttachedCollisionPrimitive to populate in place.	required

Raises:

Type	Description
ROSConfigError	The primitive is none of the three shapes the IDL can carry.

Fail-closed: an unrepresentable shape raises here rather than publishing shape_type=0 (the IDL default, indistinguishable from "no shape"). Mirrors the fail-closed pattern in openral_cli.collision.collision_primitive_envelope and this class's own from_idl, which already refuses an unknown shape_type.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate one duck-typed attached primitive without importing ROS.

    Args:
        msg: A duck-typed ``openral_msgs/AttachedCollisionPrimitive`` to
            populate in place.

    Raises:
        ROSConfigError: The primitive is none of the three shapes the IDL
            can carry.

    Fail-closed: an unrepresentable shape raises here rather than
    publishing ``shape_type=0`` (the IDL default, indistinguishable from
    "no shape"). Mirrors the fail-closed pattern in
    ``openral_cli.collision.collision_primitive_envelope`` and this
    class's own ``from_idl``, which already refuses an unknown
    ``shape_type``.
    """
    if isinstance(self.shape, SphereShape):
        msg.shape_type = msg.SHAPE_SPHERE  # type: ignore[attr-defined]
        msg.shape_dimensions = [float(self.shape.radius_m)]  # type: ignore[attr-defined]
    elif isinstance(self.shape, CapsuleShape):
        msg.shape_type = msg.SHAPE_CAPSULE  # type: ignore[attr-defined]
        msg.shape_dimensions = [  # type: ignore[attr-defined]
            float(self.shape.radius_m),
            float(self.shape.length_m),
        ]
    elif isinstance(self.shape, BoxShape):
        msg.shape_type = msg.SHAPE_BOX  # type: ignore[attr-defined]
        msg.shape_dimensions = [  # type: ignore[attr-defined]
            float(value) for value in self.shape.half_extents_m
        ]
    else:
        raise ROSConfigError(
            f"attached collision primitive {self.shape.shape!r} has no "
            "openral_msgs/AttachedCollisionPrimitive encoding. Add a SHAPE_* "
            "constant and a branch here (and in from_idl) before carrying it "
            "on a payload — it must never be published as an unset shape."
        )
    pose = msg.pose_in_object  # type: ignore[attr-defined]
    pose.position.x, pose.position.y, pose.position.z = self.pose_in_object.xyz
    (
        pose.orientation.x,
        pose.orientation.y,
        pose.orientation.z,
        pose.orientation.w,
    ) = self.pose_in_object.quat_xyzw
    tight = self.tight_geometry
    msg.tight_dop_lo = [] if tight is None else list(tight.dop_lo_m)  # type: ignore[attr-defined]
    msg.tight_dop_hi = [] if tight is None else list(tight.dop_hi_m)  # type: ignore[attr-defined]
    msg.tight_hull_vertices = (  # type: ignore[attr-defined]
        []
        if tight is None
        else [coordinate for vertex in tight.hull_vertices_m for coordinate in vertex]
    )

from_idl(msg: object, *, object_id: str) -> Self classmethod

Decode one duck-typed attached primitive without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object, *, object_id: str) -> Self:
    """Decode one duck-typed attached primitive without importing ROS."""
    dimensions = [float(value) for value in msg.shape_dimensions]  # type: ignore[attr-defined]
    shape_type = int(msg.shape_type)  # type: ignore[attr-defined]
    if shape_type == int(msg.SHAPE_SPHERE):  # type: ignore[attr-defined]
        if len(dimensions) != 1:
            raise ValueError("Attached sphere requires one shape dimension.")
        shape: CollisionShape = SphereShape(radius_m=dimensions[0])
    elif shape_type == int(msg.SHAPE_CAPSULE):  # type: ignore[attr-defined]
        if len(dimensions) != 2:  # noqa: PLR2004
            raise ValueError("Attached capsule requires two shape dimensions.")
        shape = CapsuleShape(radius_m=dimensions[0], length_m=dimensions[1])
    elif shape_type == int(msg.SHAPE_BOX):  # type: ignore[attr-defined]
        if len(dimensions) != 3:  # noqa: PLR2004
            raise ValueError("Attached box requires three shape dimensions.")
        shape = BoxShape(half_extents_m=(dimensions[0], dimensions[1], dimensions[2]))
    else:
        raise ValueError(f"Unknown attached collision shape type: {shape_type}")
    pose = msg.pose_in_object  # type: ignore[attr-defined]
    return cls(
        shape=shape,
        pose_in_object=Pose6D(
            xyz=(
                float(pose.position.x),
                float(pose.position.y),
                float(pose.position.z),
            ),
            quat_xyzw=(
                float(pose.orientation.x),
                float(pose.orientation.y),
                float(pose.orientation.z),
                float(pose.orientation.w),
            ),
            frame_id=object_id,
        ),
        tight_geometry=_tight_geometry_from_idl(msg),
    )

AttachmentEvidenceKind

Bases: str, Enum

Evidence source that confirmed a robot/object attachment or its support.

SIM_CONTACT and SIM_GEOM_DISTANCE are both simulator ground truth, and the distinction is load-bearing rather than cosmetic. A simulator's contact list is not a proximity oracle: MuJoCo's contype / conaffinity bitmasks suppress whole geom pairs, so an object flush on a counter can generate no contact record at all. SIM_GEOM_DISTANCE names the signed closest-distance probe (mj_geomDistance), which sees the pairs the contact list hides. A consumer reading a witness must not infer from SIM_CONTACT that a solver contact existed, nor from SIM_GEOM_DISTANCE that one did not.

BenchmarkMetadata

Bases: BaseModel

Provenance block required on every BenchmarkScene.

paper / honest_scope are the published-protocol citation + a one-sentence "what this eval actually measured" statement.

display_name / simulator are optional paper-comparison labels that surface into RSkillEvalResult.benchmark when present — display_name becomes benchmark.name and simulator becomes benchmark.simulator. Before the scene-hierarchy convergence these lived in a free-form dict on the deleted BenchmarkSpec; moving them per-scene keeps them with their provenance and lets the aggregator emit identical JSON whether driven by run_benchmark (suite) or run_benchmark_scene (single scene).

BenchmarkScene

Bases: SimScene

Full benchmark eval spec for openral benchmark.

n_episodes, seed, and metadata are required with no defaults — they must match the published evaluation protocol. The task must supply success_key and max_steps.

BoxShape

Bases: BaseModel

Oriented box (OBB) collision primitive — a rectangular convex block.

An axis-aligned box in the primitive's local frame (placed and oriented by the owning frame's origin_xyz_rpy / pose), spanning [-half_extents_m[k], +half_extents_m[k]] along each local axis k.

A box fits a blocky link (a near-cubic housing, e.g. the SO-ARM100/101 base) far tighter than a capsule: a capsule's circular cross-section must bulge past the block's flat faces, so it over-reports clearance at poses where the flat faces are what actually face a neighbour. That bulge is what makes the manifest capsule model false-E-stop the SO-101 at its home pose (base↔lower_arm / base↔wrist reported penetrating while the true mesh clearance is +0.16 m / +0.24 m). See issue #84.

Attributes:

Name	Type	Description
shape	Literal['box']	Discriminator (always "box").
half_extents_m	tuple[PositiveFloat, PositiveFloat, PositiveFloat]	Half-sizes (hx, hy, hz) along the local x/y/z axes in metres (the full box is 2*hx x 2*hy x 2*hz).

Example

BoxShape(half_extents_m=(0.055, 0.048, 0.036)).shape 'box'

CameraSimPlacement

Bases: BaseModel

Where a camera sensor's (RGB or depth) camera sits in the sim MJCF.

Lets the generic HAL camera rig (openral_hal._camera_rig) splice a manifest camera into a bare-arm MJCF that ships no <camera> elements, so a deploy sim digital twin renders the robot's declared cameras without a per-robot scene composer. A camera is a property of the robot's sensor suite (the wrist camera is bolted to the gripper; the overhead camera is a declared sensor), so its sim pose lives on the SensorSpec, not in a scene.

The rig only splices a camera that is absent from the loaded MJCF, so this is a no-op for scene-attached or already-composed models (those carry their own cameras).

Attributes:

Name	Type	Description
parent_body	str | None	MJCF body the camera is rigidly mounted to (tracks that body's pose — e.g. the gripper/Fixed_Jaw for a wrist camera). None mounts it world-fixed in <worldbody> (an overhead / third-person camera).
pos	tuple[float, float, float]	Camera position (x, y, z) in metres, in parent_body's frame (or world when parent_body is None).
target	tuple[float, float, float]	Look-at point (x, y, z) in the same frame; the camera's MuJoCo -z view axis is oriented from pos toward it.
up	tuple[float, float, float]	World-up hint used to level the camera roll. Defaults to (0, 0, 1). A camera physically mounted upside-down (e.g. a wrist phone bolted inverted on the gripper) uses (0, 0, -1) to roll the image 180°.
fovy_deg	float | None	Vertical field of view in degrees. None derives it from the sensor's pinhole intrinsics (2·atan(height / (2·fy))) so the rendered FoV matches the declared camera model.

Example

CameraSimPlacement(pos=(0.5, 0.3, 0.75), target=(0.5, 0.3, 0.0)).parent_body is None True

CameraTopicKind

Bases: StrEnum

Per-camera stream suffix under CAMERA_TOPIC_PREFIX/<sensor name>/.

CapsuleShape

Bases: BaseModel

Capsule collision primitive — a segment swept by a radius.

The central segment runs along the local +Z axis from -length_m / 2 to +length_m / 2 (the MJCF / URDF capsule convention); it is placed and oriented by the owning frame (LinkCollisionGeometry.origin_xyz_rpy for a link, AttachedCollisionPrimitive.pose_in_object for a payload). Capsules bound most robot links tightly, so the safety check stays conservative.

Attributes:

Name	Type	Description
shape	Literal['capsule']	Discriminator (always "capsule").
radius_m	float	Capsule radius in metres.
length_m	float	Length of the central segment in metres (the cylinder portion; the total span is length_m + 2 * radius_m). 0.0 degenerates to a sphere.

Example

CapsuleShape(radius_m=0.04, length_m=0.3).length_m 0.3

ClockAuthority

Bases: BaseModel

Named origin for OpenRAL timestamps across sim, rSkills, ROS, and hardware.

The invariant is: every serialized stamp_ns is measured in the active deployment's OpenRAL timestamp domain. Simulation deployments may publish that domain onto ROS /clock; real deployments normally use host wall time or a PTP/hardware clock already synchronized into the graph. rSkills consume stamps from this domain; they do not create a separate clock.

Attributes:

Name	Type	Description
origin	ClockOrigin	Physical/logical source of the timestamp domain.
epoch	ClockEpoch	Epoch used by stamp_ns.
clock_id	str	Stable human-readable authority identifier.
publishes_ros_clock	bool	Whether this authority is projected onto ROS /clock for nodes using use_sim_time.
timestep_s	float | None	Optional logical simulation/control timestep in seconds. None means event-driven or externally determined.
notes	str | None	Optional operator-facing context.

Example

ClockAuthority.simulation("robocasa", timestep_s=0.05).epoch

hardware_synced(clock_id: str, *, epoch: ClockEpoch = ClockEpoch.UNIX) -> ClockAuthority classmethod

Return a hardware/PTP authority already synchronized to the graph.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def hardware_synced(
    cls,
    clock_id: str,
    *,
    epoch: ClockEpoch = ClockEpoch.UNIX,
) -> ClockAuthority:
    """Return a hardware/PTP authority already synchronized to the graph."""
    return cls(origin=ClockOrigin.HARDWARE_SYNCED, epoch=epoch, clock_id=clock_id)

host_wall(clock_id: str = 'host_wall') -> ClockAuthority classmethod

Return the default real-deployment authority: host wall/ROS system time.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def host_wall(cls, clock_id: str = "host_wall") -> ClockAuthority:
    """Return the default real-deployment authority: host wall/ROS system time."""
    return cls(origin=ClockOrigin.HOST_WALL, epoch=ClockEpoch.UNIX, clock_id=clock_id)

simulation(clock_id: str, *, timestep_s: float | None = None, publishes_ros_clock: bool = True) -> ClockAuthority classmethod

Return a simulator-owned elapsed-time authority.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def simulation(
    cls,
    clock_id: str,
    *,
    timestep_s: float | None = None,
    publishes_ros_clock: bool = True,
) -> ClockAuthority:
    """Return a simulator-owned elapsed-time authority."""
    return cls(
        origin=ClockOrigin.SIMULATION,
        epoch=ClockEpoch.SIMULATION_ELAPSED,
        clock_id=clock_id,
        publishes_ros_clock=publishes_ros_clock,
        timestep_s=timestep_s,
    )

ClockEpoch

Bases: str, Enum

Epoch that a stamp_ns value is measured from.

ClockOrigin

Bases: str, Enum

Authoritative source for OpenRAL stamp_ns values.

/clock is not an origin; it is the ROS 2 projection of one of these authorities into nodes that run with use_sim_time=true.

CollisionEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_COLLISION — a proposed motion would collide.

Attributes:

Name	Type	Description
kind	Literal['collision']	Discriminator (always "collision").
collision_kind	Literal['self', 'world']	"self" (link vs link) or "world" (link vs obstacle).
link_a	str	Robot link whose collision volume tripped the check.
link_b_or_object	str	The other robot link (self-collision) or the world object / occupancy region (world-collision).
horizon_step	int	Chunk step (horizon index) where the collision was first detected, or REACTIVE_HORIZON_STEP (-1) when the hit is a REACTIVE one — the robot's measured configuration already collides, so no prediction step is involved. The C++ safety kernel emits -1 for that check (check_config(q_meas, -1) in cpp/openral_safety_kernel/src/lifecycle_kernel.cpp); it is the common case for Cartesian control modes, whose reactive measured-state check is the guaranteed floor.
min_distance_m	float	Signed minimum distance at detection, in metres (negative means interpenetration).
joint_positions_rad	list[float]	The configuration the kernel ran forward kinematics on for horizon_step — the exact vector handed to forward_kinematics, in the envelope's dof order, with mobile-base dofs zeroed (the kernel checks the arm in base_link). This is what makes a stop adjudicatable: a predicted step's configuration exists in no other artifact, because it depends on the kernel's own damped-least-squares Jacobian, its lambda and its seed dt — so replaying such a stop against the measured joints reads geometry the kernel never checked. Empty only on a record emitted before the field existed: a kernel that reports a collision has a collision model by construction, and its FK buffer is always n_dof long.
world_grid_origin_m	tuple[float, float, float] | None	Base-frame position of voxel (0,0,0)'s minimum corner in the occupancy grid this check ran against, when the other party is a world voxel; None otherwise and on any producer that predates the field. The kernel names its cell only as voxel_<n>, an index into a grid it does not republish, so a consumer has to pair that index with a grid of its own — and the published window is snapped to the source lattice, so a base drift across one cell boundary shifts the whole window and the same index names a different cell. Matching on a stamp cannot separate two grids inside one clock tick (/clock quantises every stamp in the graph to the tick); matching on this origin is exact.

is_reactive: bool property

True when the hit came from the measured-state (reactive) check.

ComputeSpec

Bases: BaseModel

Compute profile for one deployment tier — runtime inference and GPU capabilities.

Used for all three deployment tiers:

• Edge — on-robot SoC (Jetson AGX Orin, NVIDIA Thor).
• Local — laptop / workstation tethered to the robot or running the simulation.
• Cloud — remote GPU node reachable via SSH or HTTP.

Attached to RobotDescription as compute_edge, compute_local, and compute_cloud. Consumed by rSkill.check_runtime and rSkill.check_quantization_dtype to match RSkillManifest.runtime / quantization.dtype against what the hardware can actually execute.

Attributes:

Name	Type	Description
compute_tops	float	Peak compute in INT8 TOPS (0 = unknown).
system_memory_gb	float	Total system / unified RAM in GB (0 = unknown).
num_gpus	int	Number of GPUs on this node (default 1; use > 1 for cloud multi-GPU pods — total VRAM budget = gpu_vram_gb * num_gpus).
gpu_vram_gb	float	VRAM of a single GPU in GB (0 when no discrete GPU).
cuda_compute_capability	tuple[int, int] | None	Highest CUDA compute capability (major, minor), e.g. (8, 9) for Ada Lovelace, (10, 0) for Blackwell. None on non-CUDA hosts.
cuda_toolkit_version	str | None	nvcc version string, e.g. "12.4". None when CUDA is absent or the version cannot be determined.
tensorrt_version	str | None	TensorRT runtime version string when importable.
gpu_supported_runtimes	list[RSkillRuntime]	Inference runtimes the hardware can execute, used by rSkill.check_capabilities to match RSkillManifest.runtime. Empty list = "unknown — skip check".
gpu_supported_dtypes	list[QuantizationDtype]	Quantization dtypes the accelerator supports (derived from CUDA compute capability or platform). Empty = "unknown — skip check".
nvmm_available	bool	Whether libnvbufsurface.so is present on this node (Tegra L4T multimedia stack). True enables the NVMM zero-copy sensor-ingest path; probed on all tiers and returns False gracefully when the library is absent.
endpoint	str | None	Remote access address for cloud / SSH nodes. Format: ssh://user@host[:port] or https://host:port. None for edge and local nodes.
network_latency_ms	float | None	Measured round-trip latency to this node in ms. Populated by the SSH probe; None for local/edge (no network hop). Used by the reasoner dispatcher to decide whether cloud dispatch fits within a skill's latency budget.

Example

spec = ComputeSpec( ... gpu_vram_gb=24.0, ... cuda_compute_capability=(8, 9), ... cuda_toolkit_version="13.2", ... ) spec.supports_cumotion() True

supports_cumotion() -> bool

Whether this compute spec meets the cuMotion (Isaac ROS) GPU floor.

cuMotion's CUDA motion planner requires an Ampere-or-newer GPU (compute capability >= 8.0), CUDA toolkit >= 13.0, and a nominal 8 GB card. The MoveIt planner gate uses this to choose the cuMotion planning pipeline; when False it falls back to OMPL.

Returns:

Type	Description
bool	True only when compute capability, CUDA toolkit version, and
bool	VRAM all clear the cuMotion floor. False on any non-CUDA host or
bool	when the CUDA toolkit version is unknown (the floor cannot be
bool	confirmed, so the gate stays closed).

Source code in python/core/src/openral_core/schemas.py

def supports_cumotion(self) -> bool:
    """Whether this compute spec meets the cuMotion (Isaac ROS) GPU floor.

    cuMotion's CUDA motion planner requires an Ampere-or-newer GPU (compute
    capability >= 8.0), CUDA toolkit >= 13.0, and a nominal 8 GB card. The
    MoveIt planner gate uses this to choose the cuMotion planning pipeline;
    when ``False`` it falls back to OMPL.

    Returns:
        ``True`` only when compute capability, CUDA toolkit version, and
        VRAM all clear the cuMotion floor. ``False`` on any non-CUDA host or
        when the CUDA toolkit version is unknown (the floor cannot be
        confirmed, so the gate stays closed).
    """
    if self.cuda_compute_capability is None:
        return False
    if self.cuda_compute_capability < _CUMOTION_MIN_COMPUTE_CAPABILITY:
        return False
    if self.cuda_toolkit_version is None:
        return False
    cuda_major = _cuda_major(self.cuda_toolkit_version)
    if cuda_major is None or cuda_major < _CUMOTION_MIN_CUDA_MAJOR:
        return False
    return self.gpu_vram_gb >= _CUMOTION_MIN_VRAM_GIB

ContactForceWitness

Bases: BaseModel

Bounded attestation of a MEASURED contact force on the declared target.

The second observable on the place path (ADR-0100, Path C): force is what distinguishes "set down" from "crush" once position clearance hits zero. The gate this feeds only ever adds a refusal — never widens an ADR-0097 allowance or moves a margin. magnitude_n is Newtons only when magnitude_calibrated (no published work validates MuJoCo contact force against real F/T measurements; FORGE arXiv:2408.04587 re-tunes on hardware); direction_in_object has no such caveat (arXiv:2602.14174). Absence of a witness is never evidence of no contact — contype/ conaffinity suppression can empty the solver's contact list under a payload demonstrably in contact.

Attributes:

Name	Type	Description
target_id	str	Surface the measured contact is against; the gate arms only when this names the live declaration's target or a surface of it.
direction_in_object	tuple[float, float, float]	Unit contact-force direction in the attached object's frame (not base frame — decorrelates as a mobile base drives).
magnitude_n	float	Contact force magnitude; Newtons only when magnitude_calibrated, else an ignored uncalibrated number.
magnitude_calibrated	bool	False (default, uncalibrated) means the force gate does not arm.
calibration_ref	str | None	Names the calibration magnitude_n was produced under; required when magnitude_calibrated.
confidence	float	Witness confidence in [0, 1].
evidence_kind	AttachmentEvidenceKind	Source that measured the force; the MuJoCo producer attests AttachmentEvidenceKind.SIM_CONTACT_FORCE.
evidence_ref	str | None	Optional trace / producer reference.
stamp_ns	int	Producer timestamp; with target_id keys the kernel's re-arm check.

Example

witness = ContactForceWitness( ... target_id="sim:cab_1_left_group_main", ... direction_in_object=(0.0, 0.0, 1.0), ... magnitude_n=4.5, ... magnitude_calibrated=False, ... confidence=1.0, ... evidence_kind=AttachmentEvidenceKind.SIM_CONTACT_FORCE, ... stamp_ns=1, ... ) witness.magnitude_calibrated False

fill_idl(msg: object) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS."""
    msg.target_id = self.target_id  # type: ignore[attr-defined]
    direction = msg.direction_in_object  # type: ignore[attr-defined]
    direction.x, direction.y, direction.z = self.direction_in_object
    msg.magnitude_n = float(self.magnitude_n)  # type: ignore[attr-defined]
    msg.magnitude_calibrated = bool(self.magnitude_calibrated)  # type: ignore[attr-defined]
    msg.calibration_ref = self.calibration_ref or ""  # type: ignore[attr-defined]
    msg.confidence = float(self.confidence)  # type: ignore[attr-defined]
    msg.evidence_kind = self.evidence_kind.value  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref or ""  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    direction = msg.direction_in_object  # type: ignore[attr-defined]
    return cls(
        target_id=str(msg.target_id),  # type: ignore[attr-defined]
        direction_in_object=(
            float(direction.x),
            float(direction.y),
            float(direction.z),
        ),
        magnitude_n=float(msg.magnitude_n),  # type: ignore[attr-defined]
        magnitude_calibrated=bool(msg.magnitude_calibrated),  # type: ignore[attr-defined]
        calibration_ref=str(msg.calibration_ref) or None,  # type: ignore[attr-defined]
        confidence=float(msg.confidence),  # type: ignore[attr-defined]
        evidence_kind=AttachmentEvidenceKind(str(msg.evidence_kind)),  # type: ignore[attr-defined]
        evidence_ref=str(msg.evidence_ref) or None,  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
    )

ControlMode

Bases: str, Enum

Action space / control interface exposed by a robot or skill.

ControlModeSemantics

Bases: BaseModel

Action-space semantics declared on each ActuatorRequirement.

Closes Gap 2 of the rSkill self-containment audit: kind alone ("joint_position") doesn't say whether the policy emits absolute targets or per-step deltas, in which joint order, in which reference frame, or how it encodes the gripper. Today these are assumed at the adapter level and silently broken when porting a skill between embodiments.

The block is REQUIRED on every actuator entry of a manifest. For the canonical embodiments the loader still auto-fills n_dof / vla_action_key from the robot YAML; semantics must be declared explicitly on the manifest because they are a property of the trained checkpoint, not the robot.

Attributes:

Name	Type	Description
mode	Literal['absolute', 'delta']	Whether the action vector represents absolute targets (joint positions, end-effector pose) or deltas from the current state. Required.
gripper_convention	GripperConvention | None	Encoding of the gripper slot in the action vector. Required when the parent ActuatorRequirement.kind is ControlMode.GRIPPER_BINARY or ControlMode.GRIPPER_POSITION; forbidden otherwise.
joint_order	list[str] | None	Ordered list of joint names matching the channels the policy emits. Optional for canonical embodiments (the robot YAML's ActionSpec.joint_names is the source of truth); REQUIRED for "custom" embodiments.
reference_frame	str | None	tf2 frame the action is expressed in. Required when the parent ActuatorRequirement.kind is one of ControlMode.CARTESIAN_POSE, ControlMode.CARTESIAN_DELTA, or ControlMode.CARTESIAN_TWIST; forbidden otherwise.

Example

sem = ControlModeSemantics(mode="absolute") sem.mode 'absolute' sem.gripper_convention is None and sem.reference_frame is None True

ControllerEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CONTROLLER — a ros2_control controller faulted.

Attributes:

Name	Type	Description
kind	Literal['controller']	Discriminator (always "controller").
controller_name	str	Failing controller (e.g. "joint_trajectory_controller").
state	str	Reported controller state (e.g. "inactive", "error").
detail	str	Free-form detail from the controller manager.

CriticEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CRITIC — a critic flagged an action below threshold.

Attributes:

Name	Type	Description
kind	Literal['critic']	Discriminator (always "critic").
critic_id	str	Identifier of the critic that fired (model id or hand-rolled).
score	float	Critic output in the critic's native range.
threshold	float	Configured pass threshold.

DeadlineOverrunPolicy

Bases: str, Enum

What the inference runner does when a tick exceeds the deadline.

warn logs + records an OTel attribute but still sends the action (mirrors lerobot's record loop). drop skips the action for this tick — used when stale actions are worse than no action (e.g. velocity control). raise is test-only: raises ROSDeadlineMissed.

DecomposeMissionTool

Bases: _ReasonerToolBase

Tool variant — write the reasoner's typed task queue.

The typed path for the decompose-mission playbook: the LLM emits an ordered list of finer subtasks and the node applies it to the deterministic MissionState, replacing the free-form-JSON gap with structured output (CLAUDE.md §3). Two modes, selected by target_task_id:

• populate (target_task_id empty) — build a fresh mission queue from subtasks, refining the single-task seed when the operator goal needs a finer decomposition than a single prompt.
• subdivide (target_task_id set) — flat-splice the named blocked task in place with subtasks (t2 → t2.1, t2.2), bounded by DEFAULT_MAX_SUBDIVIDE_DEPTH; past the bound the node hands off instead.

Each subtask is a GroundedSubtask (one specific object), not a free string — so a collective "first batch of objects" cannot be emitted. The node renders them to MissionState task text via rendered_subtasks.

Like every ReasonerToolCall variant it holds no authority over actuation — it only edits the S2 task ledger; a bad decomposition yields a worse plan the safety kernel still vetoes.

Attributes:

Name	Type	Description
tool	Literal['decompose_mission']	Discriminator (always "decompose_mission").
subtasks	list[GroundedSubtask]	Ordered, non-empty list of GroundedSubtask — each acts on exactly one specific object and becomes a TaskState.
target_task_id	str	TaskState.task_id of the blocked task to subdivide (e.g. "t2"); empty string populates/replaces the whole queue.

rendered_subtasks() -> list[str]

The ordered subtask instruction strings for MissionState.

Source code in python/core/src/openral_core/schemas.py

def rendered_subtasks(self) -> list[str]:
    """The ordered subtask instruction strings for ``MissionState``."""
    return [s.render() for s in self.subtasks]

DeployRuntime

Bases: BaseModel

Committed deploy-posture toggles for a workcell scene.

Every runtime leg openral deploy sim / deploy run can bring up is declared here so a committed DeployScene pins its posture explicitly instead of relying on per-invocation CLI flags. Precedence is CLI flag > scene runtime > auto — an explicit CLI flag always wins, and None means "auto" (the manifest-derived / built-in default the CLI documents for that flag: SLAM from capabilities.has_lidar/vision-SLAM, Nav2 tracks SLAM, octomap from a depth SensorSpec, detector on, kernel-check on, reward monitor / critic off).

Host-operational knobs (dashboard, foxglove, ports, dataset recording, --initial-task, --dry-run) stay CLI-only — they describe the invocation, not the workcell.

Relative *_manifest / object_detector_onnx paths that exist next to the scene YAML are resolved against the scene's directory; otherwise the value is passed through verbatim (repo-relative default / alias).

clock_origin: Literal['host_wall', 'simulation'] | None = None class-attribute instance-attribute

Pin the graph's clock authority instead of deriving it. None (auto) = host wall time for deploy run; for deploy sim, the simulator's clock whenever the backend exposes one (every bare MuJoCo twin does), unless octomap_cloud_topic names a real driver's cloud (outside /openral/cameras/), which selects host_wall.

A real ZED or RealSense driver stamps on wall-clock. Under the simulated clock octomap_server (use_sim_time) sits near t=0, treats every wall-stamped cloud as from the future and drops it, so the map and /openral/world_voxels stay empty while every node reports healthy.

simulation is refused for deploy run (a real robot has no sim clock), for a sim backend that exposes no clock, and together with a real driver's octomap_cloud_topic.

enable_scene_vlm: bool | None = None class-attribute instance-attribute

Bring up the scene-VLM query service so the reasoner is offered the read-only query_scene tool. None = off unless the CLI passes --enable-scene-vlm. Backward-compatible addition: a scene that omits it behaves exactly as before.

estimated_depth_camera: str | None = None class-attribute instance-attribute

An RGB camera (a manifest or scene sensor name) whose metric depth Depth Anything V3 estimates, so the world-collision map gets a cloud on a robot with no depth sensor (a top / context camera), in sim and on real hardware alike. The launch runs the depth provider on that camera with DA3METRIC-LARGE (its own sidecar, ZMQ port 5772): metric depth from the camera's own CameraInfo focal, published with that CameraInfo and back-projected to /openral/cameras/<name>/points, which becomes octomap_server's input (deploy_cloud_topic); the robot self filter always runs on it (DA3 sees the arm).

An ESTIMATE, not a measurement: it can misplace or miss surfaces (glass, low texture, far range), and it reaches the safety kernel's world voxels. The launch logs that at WARN. Any RGB camera with intrinsics qualifies (top, context or wrist) once it is tied to the robot by a declared mount: parent_frame + static_transform_xyz_rpy on it or on the sensor it shares_mount_with; in sim, an *_optical_frame camera the bridge places from the MJCF also does (openral_core.depth_extrinsic.resolve_estimated_depth_camera). A camera in world or on a bare robot link is refused. An estimate never replaces a measurement: the camera is REFUSED whenever the robot or the scene has any measured depth or point-cloud sensor (SensorSpec.is_cloud_source), in sim and on real hardware alike, with no override; the refusal names each such sensor and whether it is the robot's or the scene's. Map that sensor's cloud (octomap_cloud_topic) instead (Safety-WG, hazard HZ-0382-6). On deploy run that mount must be calibrated in units/<unit>.yaml like any cloud source's (depth_extrinsic_problems). Mutually exclusive with octomap_cloud_topic (which cloud is mapped must be one answer) and with slam_mono_camera naming the same camera (both would publish its depth topics). None = off. Backward-compatible addition.

grasp_allowance_enabled: bool = False class-attribute instance-attribute

Turn on the safety kernel's grasp-target exemption (its grasp_allowance_enabled): while a live GraspDeclaration carries a producer-measured region, world-voxel cells centred in that box do not trip the declared gripper contact links. The launch always passes the kernel its allowlist, grasp_contact_links = the manifest's role: gripper joints' child_links, so a declaration can never name a link the robot does not grip with. Default off, and off is the graph exactly as without it; turning it on is pending Safety-WG review (real pick-and-place design §2.1/§3, docs/reference/real-pick-place-adr-drafts.md). Backward-compatible addition. openral deploy run refuses it on unless vision_attachment.enabled and vision_attachment.grasp_target_enabled are on too: the kernel would arm with no producer measuring the region (deploy sim's MuJoCo evidence tracker measures it).

joint_states_topic: str | None = None class-attribute instance-attribute

Explicit override for the sensor_msgs/JointState topic the deploy runtime's Python nodes (in-process world state + the runner's joint-state cache) ingest. The C++ safety kernel always keeps the full-rate /joint_states.

None (auto) derives it: on deploy run of a robot whose real HAL is ros2_control-drivable (OpenArm, Franka, UR, Sawyer), the HAL node's own rate-limited /<hal_node>/joint_states republish; otherwise /joint_states. The broadcaster's full-rate stream (0.5-1 kHz) would wake the Python executor on every message and starve an in-process VLA of the GIL. See openral_hal.hal_joint_states_topic.

max_octree_age_s: float | None = Field(default=None, gt=0) class-attribute instance-attribute

How long the octomap bridge republishes the last octree it received. None = equal to world_voxel_deadline_s. Must not exceed it, so the kernel, not the bridge, fails closed on a silent camera: worst case from the last inserted cloud to the drop is this plus the deadline (hazard log Entry 033).

octomap_cloud_topic: str | None = None class-attribute instance-attribute

The PointCloud2 topic octomap_server consumes as cloud_in, i.e. what the world map is actually built from (deploy_cloud_topic). None under deploy sim = /openral/cameras/<name>/points of the manifest's one depth sensor with intrinsics, which the sim sensor bridge back-projects; a manifest with several must pin the one to map.

On real hardware nothing in-tree publishes a cloud, so deploy run refuses to start octomap (enabled explicitly or auto-enabled by a depth / point-cloud sensor) until this names what the driver already publishes: zed_wrapper emits /<name>/point_cloud/cloud_registered, RealSense /camera/depth/color/points. Set enable_octomap: false to run without the world map. A pin outside /openral/cameras/ on a deploy sim twin is a real driver, so the twin runs on host wall time (see clock_origin).

preload_prompt: str | None = None class-attribute instance-attribute

Prompt the preload warms the policy with. The runner keeps a policy resident per (rskill_id, revision) and passes each goal's own prompt to it per step, so a goal with a different prompt reuses the preloaded policy. For a single-instruction finetune use the training string, verbatim.

preload_rskill_id: str | None = None class-attribute instance-attribute

rSkill the skill_runner resolves and loads right after it activates, so the first execute_rskill goal finds it resident. The deadman watchdog opens its first-chunk window when a goal is accepted, so a multi-minute cold load of a large policy inside a goal is E-stopped — correctly. Loading before any goal exists keeps the watchdog exactly as strict. Goals are rejected while the preload is in flight (rskill_runner.preload_done marks the end). Any resolvable id: an installed Hub repo id or an in-tree name.

preload_rskill_revision: str | None = None class-attribute instance-attribute

Hub revision (branch, tag or commit) pinned for preload_rskill_id. Part of the resident key (rskill_id, revision, prompt): it must equal the revision later goals send, or they evict the preloaded skill. None = the unpinned default revision. Ignored without preload_rskill_id.

robot_self_filter_padding_m: float = Field(default=0.02, ge=0, json_schema_extra={'maximum': 0.1}) class-attribute instance-attribute

How far beyond the robot's collision primitives (and a held payload's) a real depth return still counts as the robot and is removed by openral_octomap_bridge's robot_self_filter before octomap inserts the cloud. Real camera path only; sim renders the robot transparent.

0.02 m is measured AT REST on the OpenArm Thor and Orin cells (2026-09-25, ZED-M, arms at q = 0): the robot's own depth returns end 5-7.5 mm outside the kernel's link hulls. The tail with the arms moving is unmeasured (scenes/deploy/openarm_real_world_voxels.yaml). Derive it per rig from the camera's depth noise at working range, the camera extrinsic error, how far a link moves between the cloud's capture and the joint state used to pose it, and half a voxel. It is also the width of the blind shell around the arm: an obstacle that close to the robot is removed with it, so a larger value hides more of the world from the kernel's check (hazard log Entry 035). Too small leaves robot surface in the map, which stops the robot against itself (fail closed).

Hard cap 0.10 m (2x the pre-2026-09-25 value), refused at load: a wider blind shell hides obstacles the arm can reach within one chunk. The self-filter node refuses it too (and a negative or non-finite value), forwarding nothing.

scene_vlm_manifest: str | None = None class-attribute instance-attribute

Path to the kind: "vlm" rSkill manifest backing query_scene. Scene-relative like the other manifest paths. None defaults to the in-tree rskills/qwen35-4b-nf4. Ignored unless enable_scene_vlm.

slam_depth_sidecar_autostart: bool = True class-attribute instance-attribute

Whether the deploy launch spawns the DA3 metric-depth sidecar (tools/da3_depth_sidecar.py, ZMQ port 5771) alongside the mono visual SLAM graph. Only meaningful when slam_mono_camera is set. False = the sidecar is operator-run / shared (e.g. one sidecar serving several deploys, or a dev venv via $OPENRAL_DA3_DEPTH_SIDECAR_VENV). First autostart provisions the sidecar venv, which can take minutes; the depth provider retries until it answers.

slam_mono_camera: str | None = None class-attribute instance-attribute

The single camera name the visual SLAM backend tracks in mono RGBD mode — one RGB camera fused with the DA3 metric-depth provider for scale, the lidar-less path for robots without a stereo rig. When set, the launch points PyCuVSLAM at /openral/cameras/<name>/image (+ /camera_info), auto-composes the depth provider (so cuVSLAM and nvblox get depth), and runs cuVSLAM's OdometryMode.RGBD. pycuvslam-only (the Isaac ROS impl has its own RGB-D wiring). Mutually exclusive with slam_stereo_cameras; None = stereo/multi-camera.

slam_stereo_cameras: tuple[str, str] | None = None class-attribute instance-attribute

The (left, right) camera names forming the rectified stereo rig the visual SLAM backend tracks, e.g. ["left", "right"] or ["front_left", "front_right"]. Each name maps by the OpenRAL convention to /openral/cameras/<name>/image (+ /camera_info); the launch forwards those topics to whichever visual impl is composed (Isaac ROS image_0/1_topic or PyCuVSLAM left/right_image_topic). None = the impl's built-in left/right default. The two names must differ.

slam_visual_impl: Literal['isaac_ros', 'pycuvslam'] | None = None class-attribute instance-attribute

Which cuVSLAM implementation the visual SLAM backend composes when slam_backend resolves to "visual" (capabilities.has_vision_slam, no lidar). Orthogonal to the backend choice: the backend is what sensing modality localises the robot (lidar vs camera), this is which engine binary the host has installed.

• "isaac_ros" — the composable isaac_ros_visual_slam C++ node (cuvslam.launch.py). NITROS zero-copy + raw-image undistortion + mono/IMU/RGB-D modes, but needs the operator's full Isaac ROS apt stack.
• "pycuvslam" — the same cuVSLAM engine run in-process from NVIDIA's PyCuVSLAM pip wheel (pycuvslam.launch.py); rectified stereo only, no apt stack. For sim hosts / GPU boxes without the Isaac ROS install.

None = "isaac_ros" (the established default — do not silently pick pycuvslam just because the wheel is present; §1.4). Ignored unless the resolved backend is "visual".

static_geometry_file: str | None = None class-attribute instance-attribute

Declared static geometry of the cell (StaticGeometry YAML; scene-relative, else repo-relative): walls, shelf frame, table, tote shell, in the robot's base_frame. The launch samples its surfaces and the octomap bridge marks every cell they pass through occupied in every published grid, whatever the camera has seen — so a fixture out of the head camera's view still stops an arm (docs/reference/ manipulation-workspace-contact.md §3.1). Only ADDS occupancy, the fail-closed direction, so it needs no measurement and is allowed on real hardware; the launch refuses it with the octomap leg off rather than drop declared walls. For a sim scene, tools/derive_scene_inputs.py exports it from the static colliders (an Isaac stage or a MuJoCo model). Backward-compatible addition.

vision_attachment: VisionAttachmentRuntime | None = None class-attribute instance-attribute

The vision attachment leg (segmenter + the HAL's attachment-evidence bridge). None or enabled: false = not launched, and the graph is exactly as without it. Enabled on deploy run, it always turns the kernel's attached check on too (see VisionAttachmentRuntime). Backward-compatible addition.

voxel_freshness_s: tuple[float, float] property

(world_voxel_deadline_s, max_octree_age_s) with the age defaulted.

Example

DeployRuntime().voxel_freshness_s (1.0, 1.0) DeployRuntime(world_voxel_deadline_s=2.0, max_octree_age_s=1.5).voxel_freshness_s (2.0, 1.5)

workspace_contact_enabled: bool = False class-attribute instance-attribute

Turn on the safety kernel's manipulation-workspace contact rule (issue #349, its workspace_contact_enabled; docs/reference/manipulation-workspace-contact.md): while a live GraspDeclaration carries a producer-confirmed workspace_region, world-voxel cells centred in that box do not trip the robot's hand links (the launch derives them from the manifest, RobotDescription.hand_links()) as long as the hand's speed is at most workspace_contact_max_speed_mps and its lead over the measured pose is at most workspace_contact_max_lead_m. Arm links, contact outside the workspace and a fast or far-leading hand stop as before. Requires grasp_allowance_enabled (the workspace rides the grasp declaration) and both bounds. Default off, and off is the graph exactly as without it. Sim-only until Safety-WG review: openral deploy run and deploy_e2e on a non-sim HAL refuse it on. Backward-compatible addition.

workspace_contact_max_lead_m: float | None = Field(default=None, gt=0.0, le=0.1) class-attribute instance-attribute

The hand lead (push) bound (m) of the workspace contact rule: the largest displacement of any point of a hand link's collision geometry between the MEASURED configuration and each checked row. On a joint-PD arm commanded-minus-measured is what the drives push with, so this bounds how hard a hand may press into what it touches — the speed bound alone would let a hand whose stationary target is buried deep keep pushing. Required when workspace_contact_enabled. Calibration point: see docs/reference/manipulation-workspace-contact.md §8 (the real-hardware value is a Safety-WG decision). Capped at 0.10 m (le=0.10), mirrored by the kernel's kMaxWorkspaceContactLeadM. Backward-compatible addition.

workspace_contact_max_speed_mps: float | None = Field(default=None, gt=0.0, le=1.0) class-attribute instance-attribute

The hand speed bound (m/s) of the workspace contact rule (impact energy): the largest displacement of any point of a hand link's collision geometry over one control period (action_spec.control_freq_hz) at the measured joint velocity (/joint_states) — impact energy is set by how fast the hand actually moves, and a commanded jump into an obstacle is the lead bound's job. Without a fresh measured velocity the kernel falls back to the speed between consecutive commanded rows (row 0 from the previously accepted command while fresh, else the measured state), which reads the hand faster. Required when workspace_contact_enabled — there is no default the kernel would silently use. Calibration point: see docs/reference/manipulation-workspace-contact.md §8. Capped at 1 m/s (le=1.0), beyond which the rule is not a contact-energy bound at all.

world_voxel_data_age_budget_s: float = Field(default=1.5, gt=0, json_schema_extra={'maximum': 3.0}) class-attribute instance-attribute

How old the sensor data behind a voxel grid may be when the safety kernel checks a chunk against it (its world_voxel_data_age_budget_ms), measured from the grid's source_stamp: the capture stamp of the newest cloud in the octree. Past it, or with no source stamp, the chunk drops as DROP_VOXEL_UNAVAILABLE (voxel_stale), not latched (hazard log Entry 034). The receipt deadline above cannot see pipeline latency; this can.

The default is the Thor ZED-M measurement (mock hardware, 2026-09-24): capture-to-kernel age p50 227 ms, p99 ~1.0 s, max 1.09 s, so 1.5 s clears the tail with headroom. Measure your own rig and declare its value: set below the rig's latency tail the robot stops (fail closed), never moves unsafely. It is independent of world_voxel_deadline_s / max_octree_age_s: those bound a silent source by receipt time, this bounds the world's age whatever the receipt time, so no ordering between them is required, and a smaller budget is only stricter.

Hard cap 3.0 s (2x the pre-2026-09-25 value), refused at load: a larger budget would certify chunks against a world seconds old. The kernel refuses it at configure too, and its own default is this 1.5 s: there is no "off" while the world check is enabled.

world_voxel_deadline_s: float = Field(default=1.0, gt=0, json_schema_extra={'maximum': 2.0}) class-attribute instance-attribute

How long the safety kernel trusts the last /openral/world_voxels grid (its world_voxel_deadline_ms); past it every chunk drops with DROP_VOXEL_UNAVAILABLE. It must exceed the octomap source's insert-to-publish gap on this rig (cloud period plus octomap's insert time), with margin: 1.0 s tolerates octomap down to about 1 Hz. A slower source (a 1 Hz scanning lidar, a CPU-only host inserting dense clouds) needs a longer deadline or the robot never moves; that is fail-closed, not unsafe.

Hard cap 2.0 s (2x the pre-2026-09-25 value), refused at load: past it the kernel would keep checking chunks against a map that old. A source that cannot keep octomap under it is not a world-voxel source. The kernel and the octomap bridge (max_octree_age_s) refuse it themselves too, so a node launched without this schema cannot run looser.

DeployScene

Bases: BaseModel

Unified deploy/workcell scene for openral deploy sim and deploy run.

Carries the physical/logical workcell: scene identity, optional robot mount, sim composition, and deploy-time safety/collision tightening. No task, no eval config — the reasoner/operator supplies goals at runtime.

composition lets a deploy scene declare the MJCF composer that builds its environment (e.g. the openarm tabletop arena: table + cubes + drawer + overview camera) instead of the robot manifest carrying it: the robot manifest describes the robot, the scene describes the scene. openral deploy sim threads it to the manifest-driven HAL node. openral deploy run ignores it because real-world geometry is physical, not MJCF.

safety tightens the robot manifest's SafetyEnvelope; loosening is rejected before launch. extra_allowed_collision_pairs is the only sanctioned collision loosening: additive per-pair ACM entries, never a global self-collision disable.

drivers: list[LaunchInclude] = Field(default_factory=list) class-attribute instance-attribute

Vendor sensor drivers this workcell needs on the bus.

A deploy_binding with a ros2_* backend subscribes to a topic somebody else publishes; these are the launches that publish them. deploy run includes each one on the real path (hal_mode:=real) before the sensor leg starts reading, so a camera reached over ROS comes up with the graph instead of needing a second terminal — the failure mode otherwise is a silently empty panel, not an error.

Empty for a workcell whose sensors OpenRAL opens directly (opencv_thread, gstreamer): those need no driver. Ignored on the sim path, where cameras are rendered rather than driven.

grasp_declaration: GraspDeclaration | None = None class-attribute instance-attribute

Committed grasp-phase declaration for direct dispatch (real pick-and-place design §2.1), the grasp mirror of place_declaration.

Names the grasp target and the gripper contact_links; openral deploy sim / deploy run inject it into the rSkill runner, which scopes it to each goal it dispatches (armed on start, retracted on end / cancel / E-stop) unless the goal carries its own. None means no declaration and no exemption. Same rule as the place one: a scene names a target, never a region — grasp_declaration.region is rejected by _reject_scene_supplied_place_region.

Direct-dispatch path only (reasoner off, attended): what to pick is task knowledge, so a committed cell scene carries none. The generic path is reasoner-grounded — the LLM names an ExecuteRskillTool.grasp_target, the reasoner grounds its search_box from perception, and the goal carries the declaration (design §2.2).

hal: HalParameters | None = None class-attribute instance-attribute

Deploy-time HAL binding for this workcell.

The scene's host-specific HAL construction defaults — serial port, lerobot calibration identity (id + calibration_dir), calibrate_on_connect — the same shape as the robot manifest's HalEntrypoints.parameters. This is the HAL analogue of the per-sensor SensorSpec.deploy_binding: the robot manifest stays authoritative for the HAL adapter + generic defaults, while the committed workcell scene carries the host-specific transport + calibration so openral deploy run --config <scene> is self-contained (no --hal overrides needed). deploy run resolves a relative calibration_dir against this scene file's directory (mirroring the --hal behaviour) and merges these defaults into the HAL params above the robot-manifest defaults but below any explicit --hal override, so the precedence is --hal > scene hal > robot.yaml hal.parameters.defaults. None = fall back to the manifest defaults + --hal overrides.

memory_dir: str | None = None class-attribute instance-attribute

Path to a per-robot deploy memory bundle directory holding any of MEMORY.md (self-maintained semantic memory), scene_graph.json (3D world-state graph → recall_object), and map.yaml (2D occupancy grid → nav2 map_server). openral deploy sim derives the three launch paths from it by convention (each artifact loaded by its correct consumer). --memory-dir on the CLI overrides this. None = no bundle (the reasoner starts with empty memory). Advisory only — never a safety-kernel input (§1.1).

place_declaration: PlaceDeclaration | None = None class-attribute instance-attribute

Committed place-phase declaration for direct dispatch (ADR-0097).

A reasoner grounds the place target per goal and puts the declaration on the ExecuteRskill goal, which wins over this. A direct dispatch has no reasoner in the loop, and the scene is the only place that knows the task's place target — so a counter→cabinet scene declares it here, and openral deploy sim / deploy run inject it into the rSkill runner, which scopes it to each goal it dispatches (armed on start, retracted on end / cancel / E-stop).

None means no declaration, so no place witness can ever arm and payload contact mid-carry stops the robot exactly as it does now. The declaration licenses nothing on its own: it only makes measured support contact on the declared target attestable.

A scene declares a target, never a region: a scene file cannot measure geometry, and PlaceRegion.frame_id is the robot base frame, which on a mobile base moves under any box a file could name. The producer that resolves the target measures it (in sim, from the declared body's MuJoCo subtree), so place_declaration.region is rejected here rather than carried to the kernel — see _reject_scene_supplied_place_region.

robot_unit: str | None = None class-attribute instance-attribute

The robots/<robot_id>/units/<unit>.yaml overlay this cell runs: per-host sensor bindings and per-unit calibration (SensorOverlay). $OPENRAL_ROBOT_UNIT overrides it, so one scene serves several cells. None = none; a real deploy of a robot that ships units/ then refuses (resolve_sensor_overlays).

runtime: DeployRuntime | None = None class-attribute instance-attribute

Committed deploy-posture toggles (see DeployRuntime). None = every leg on its CLI/auto default. CLI flags override field-by-field.

seed: int = 0 class-attribute instance-attribute

Simulator reset seed for deploy sim. Ignored by deploy run. Defaults to 0, preserving the pre-field deploy-sim episode.

sensors: list[SensorSpec] = Field(default_factory=list) class-attribute instance-attribute

Workcell-mounted cameras (overhead / front) — physically part of the cell, not the robot.

A deploy scene never touches a sensor the robot manifest defines: every entry must use a name the manifest does not (check_scene_sensor_overrides refuses a clash), and carries its own geometry and binding. A robot camera's real-hardware SensorSpec.deploy_binding lives in its unit overlay (robot_unit) or robots/<robot_id>/robot.yaml. merge_deploy_sensors appends these entries after the manifest's sensors.

Entries whose deploy_binding is set are opened by the real-deploy sensor leg and published on /openral/cameras/<name>/image.

from_yaml(path: str) -> Self classmethod

Load and validate a scene YAML from disk.

Inherited by SimScene and BenchmarkScene, which validate against their own (stricter) schemas via cls.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> Self:
    """Load and validate a scene YAML from disk.

    Inherited by ``SimScene`` and ``BenchmarkScene``, which
    validate against their own (stricter) schemas via ``cls``.
    """
    return _load_yaml_model(cls, path)

DetectedObject

Bases: BaseModel

A detected object in the scene.

Attributes:

Name	Type	Description
label	str	Semantic class label.
confidence	float	Detection confidence in [0, 1].
pose	Pose6D	6D pose in the world or camera frame.
bbox_3d	tuple[float, float, float, float, float, float] | None	3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
track_id	int | None	Persistent track ID across frames.

DetectorContract

Bases: BaseModel

Manifest contract for kind: "detector" rSkills.

Carries the configuration the runtime ObjectsDetector needs to instantiate an exported detection model (RT-DETR / D-FINE ONNX) and match its output indices to semantic class labels.

Required when RSkillManifest.kind is "detector"; forbidden for all other kinds (enforced by RSkillManifest._check_kind_consistency).

Attributes:

Name	Type	Description
labels	list[str]	Ordered class-label list; the integer class-id the model emits is used as an index into this list. At least one label is required.
input_size	tuple[int, int]	Model input resolution as (width, height) in pixels. Both dimensions must be > 0. Default (640, 640) matches the RT-DETR / D-FINE default export resolution.
score_threshold	float	Detections with confidence below this value are discarded before publishing. Must be in [0.0, 1.0]. Default 0.5.
engine	DetectorEngine | None	Optional explicit backend selector (DetectorEngine). None (default) keeps the legacy runtime-keyed dispatch. Set it to disambiguate backends that share a runtime — e.g. zeroshot_hf for an in-process Transformers open-vocabulary detector, which is runtime: pytorch like the VLM sidecar.
mode	DetectorMode	Invocation mode (DetectorMode; default continuous). Declares whether the detector is an always-on background producer (output reaches the reasoner via world state) or an on-demand prompted locator (surfaces the locate_in_view tool).

Example

c = DetectorContract( ... labels=["person", "bicycle", "car"], ... input_size=(640, 640), ... score_threshold=0.4, ... ) c.labels[0] 'person' c.mode

DetectorEngine

Bases: str, Enum

Backend that executes a kind: "detector" rSkill.

Selects which runtime detector class build_manifest_detector constructs. None (the default on DetectorContract) preserves the legacy runtime-keyed dispatch: runtime: onnx/tensorrt → RT-DETR ONNX, runtime: pytorch → the LocateAnything VLM sidecar. Set it explicitly to opt into a backend that the runtime value alone cannot disambiguate (e.g. an in-process Transformers open-vocabulary detector, which is also runtime: pytorch).

Attributes:

Name	Type	Description
RTDETR_ONNX		Fixed-label RT-DETR / D-FINE ONNX export (CPU / NVMM tiers). Equivalent to leaving engine unset with an onnx/tensorrt runtime.
VLM_SIDECAR		Out-of-process open-vocabulary visual-grounding VLM (LocateAnything-3B). Equivalent to leaving engine unset with a pytorch runtime. Query-driven (prompted).
ZEROSHOT_HF		In-process Transformers open-vocabulary detector (AutoModelForZeroShotObjectDetection — e.g. OmDet-Turbo). Runs against a fixed class vocabulary (the manifest labels) every frame, so it behaves like a large closed-vocabulary detector that needs no prompting — an unprompted background producer that populates the world object list with far more than the 80 COCO classes.

Example

DetectorEngine.ZEROSHOT_HF.value 'zeroshot_hf'

DetectorMode

Bases: str, Enum

Invocation mode of a kind: "detector" rSkill.

The axis orthogonal to DetectorEngine: where engine says how the model runs, mode says when the reasoner invokes it and therefore how its output reaches the LLM.

Attributes:

Name	Type	Description
CONTINUOUS		An always-on background producer. Runs on the camera tee every frame and streams ObjectsMetadata into WorldState.detected_objects; the reasoner reads it passively (via world state / recall_object) and never prompts it. It is not an ExecuteRskill-dispatchable tool and carries no actuation authority. RT-DETR (closed vocab) and OmDet-Turbo (frozen open vocab) are continuous. The reasoner may still toggle it via LifecycleTransitionTool to free VRAM.
ON_DEMAND		A prompted locator the reasoner invokes only when it needs to find a specific object right now. Surfaces the read-only locate_in_view tool backed by an open-vocabulary detector; it is not run continuously. LocateAnything is on-demand.

The two modes cleanly separate "open-vocabulary" from "prompting": continuous detectors cover a fixed bank of classes the reasoner reads for free; the on-demand locator handles the long tail that bank does not cover.

Example

DetectorMode.CONTINUOUS.value 'continuous' DetectorMode.ON_DEMAND.value 'on_demand'

DeviceInfo

Bases: BaseModel

Snapshot of the host compute capabilities used for runtime selection.

Attributes:

Name	Type	Description
device_str	str	PyTorch-style device string ("cpu", "cuda:0", "mps").
gpu_memory_bytes	int	Total GPU VRAM in bytes (0 if CPU-only).
cuda_compute_capability	tuple[int, int] | None	CUDA compute capability major/minor pair.
cpu_count	int	Logical CPU count.
arch	str	CPU architecture string.

Example

info = DeviceInfo(device_str="cpu") info.gpu_memory_bytes 0

EmbodimentExtra

Bases: BaseModel

Sensor + actuator surface for a "custom" embodiment.

Required when "custom" appears in RSkillManifest.embodiment_tags; forbidden otherwise. This is the explicit "I know what I'm doing" hatch for skill manifests targeting embodiments that do not have a canonical robots/<id>/robot.yaml in tree.

Reuses SensorRequirement and ActuatorRequirement on the assumption that the natural shape of "what this embodiment offers" mirrors "what the skill needs" — the loader just runs the same compat check it would run against a real robot YAML.

Attributes:

Name	Type	Description
sensors	list[SensorRequirement]	Sensor slots this custom embodiment exposes (≥1).
actuators	list[ActuatorRequirement]	Actuator slots this custom embodiment exposes (≥1).

Example

from openral_core import SensorModality, ControlMode EmbodimentExtra( ... sensors=[SensorRequirement(modality=SensorModality.RGB)], ... actuators=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... n_dof=6, ... vla_action_key="action.joints.arm", ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) ... ], ... ) # doctest: +ELLIPSIS EmbodimentExtra(sensors=[...], actuators=[...])

EmbodimentKind

Bases: str, Enum

Top-level kinematic class of a robot body.

EmitPromptTool

Bases: _ReasonerToolBase

Tool variant — republish a PromptStamped onto another topic.

Dispatch: publish on target_topic (typically /openral/prompt for self-cascading, but any PromptStamped topic is valid). Lets the reasoner stage multi-step plans, talk to a peer reasoner, or feed a downstream prompt-aware skill without going through an ExecuteRskill goal.

Attributes:

Name	Type	Description
tool	Literal['emit_prompt']	Discriminator (always "emit_prompt").
target_topic	str	Absolute ROS topic name (must start with "/"). The reasoner's own subscription on /openral/prompt plus the prompt-router's FIFO queue handle the cascade.
text	str	Human-readable prompt body forwarded as PromptStamped.text.
metadata_json	str	Free-form JSON forwarded as PromptStamped.metadata_json. Empty string when no structured metadata is needed.

EndEffectorSpec

Bases: BaseModel

End-effector specification.

Attributes:

Name	Type	Description
name	str	End-effector name.
kind	Literal['parallel_gripper', 'suction', 'dexterous_hand', 'tool', 'none']	Type of end-effector.
hand	Hand	Laterality.
n_dof	int	Number of controllable degrees of freedom.
max_grip_force_n	float | None	Maximum grip force in Newtons.
max_payload_kg	float | None	Maximum payload in kg.
workspace_radius_m	float | None	Reach radius in meters.
tactile_sensors	list[str]	List of tactile SensorSpec names attached to this EE.
actuated	bool	Whether the end-effector is driven by an actuator. False for passive tools (inert flanges, magnetic plates without electromagnet, kinematic-only mounts). When False, the safety kernel rejects any chunk addressed at this EE — the chunk routes to a no-op rather than risking unintended motion. Default True: every actuated gripper / dexterous hand / suction cup we ship is driven.
command_convention	GripperConvention | None	Encoding of the value a GRIPPER_* action addressed at this end effector carries when it reaches the HAL (see GripperConvention). An rSkill whose gripper convention differs is refused at load time. Every actuated parallel_gripper in robots/ declares it.
command_range	tuple[float, float] | None	Inclusive (min, max) the safety kernel bounds every GRIPPER_* value addressed at this end effector by, in command_convention's unit. Defaults to the convention's own range for the normalized conventions; REQUIRED for the physical ones (raw_joint_rad, width_meters).

Example

ee = EndEffectorSpec( ... name="left_gripper", ... kind="parallel_gripper", ... command_convention="raw_joint_rad", ... command_range=(0.0, 0.7854), ... ) ee.resolved_command_range() (0.0, 0.7854) EndEffectorSpec( ... name="g", kind="parallel_gripper", command_convention="normalized_open_unit" ... ).resolved_command_range() (0.0, 1.0)

resolved_command_range() -> tuple[float, float] | None

The kernel's bound for this end effector, or None if undeclared.

Source code in python/core/src/openral_core/schemas.py

def resolved_command_range(self) -> tuple[float, float] | None:
    """The kernel's bound for this end effector, or ``None`` if undeclared."""
    if self.command_range is not None:
        return self.command_range
    if self.command_convention is None:
        return None
    return GRIPPER_CONVENTION_RANGES[self.command_convention]

ExecuteRskillTool

Bases: _ReasonerToolBase

Tool variant — invoke an installed, capability-matched rSkill.

Dispatch: action goal on /openral/execute_rskill (the openral_msgs/action/ExecuteRskill action server in F1's rskill_runner_node). The chunk path that follows is Skill → /openral/candidate_action → safety_node → /openral/safe_action → HAL.

Attributes:

Name	Type	Description
tool	Literal['execute_rskill']	Discriminator (always "execute_rskill").
rskill_id	str	RSkillManifest.name of an installed, capable skill. Validated against the local registry by the reasoner at palette-build time; an unknown id raises ROSReasonerInvalidPlan.
prompt	str	Natural-language prompt forwarded to the skill's ExecuteRskill goal alongside the rskill_id. For VLA skills this is the policy's task-conditioning signal (SmolVLA writes it into observation["task"]); for wrapped-ROS skills it's carried for trace / log context but the actual goal is built from goal_params_json merged over the manifest's default_goal_json.
goal_params_json	str	Serialised JSON object carrying per-skill typed parameters the LLM produces against the skill's RSkillManifest.goal_params_schema. Empty string disables the merge (today's behaviour). Wrapped-ROS skills (kind: ros_action / ros_service) deep-merge it over ros_integration.default_goal_json at configure-time; VLA skills accept the field and ignore it (their prompt is already the structured signal).
deadline_s	float	Hard deadline in seconds for the action server to complete the goal. 0 means "use the skill manifest's default latency budget".
patience_s	float | None	Task-adaptive execution ceiling override (short for a quick grab, long for a precise insertion). None → use the reward model's default_patience_s.
progress_tolerance	float | None	Override the reward model's plateau_tolerance when a task misbehaves (e.g. a noisy critic). None → use the model default.
grasp_target	GraspTargetRef | None	Optional pin on the object this goal grasps (GraspTargetRef); the reasoner grounds it to a GraspDeclaration search box at dispatch, or refuses the goal. None (the normal case: the policy picks the object) = no named target; with the HAL's approach-armed grasp target on, the runner's goal-scope declaration is measured where the gripper approaches.
place_target	PlaceTargetRef | None	Optional hint naming the surface this goal places onto (PlaceTargetRef); the reasoner grounds it to a PlaceDeclaration search box at dispatch, or refuses the goal. None (the normal case) = no dispatch place declaration; the real producer still measures the surface under the carried payload.

FixedAttachment

Bases: BaseModel

A rigid, zero-DoF parent→child link attachment in the kinematic tree.

RobotDescription.joints enumerates only the robot's movable joints — consumed unfiltered as the action-vector width, the sim qpos map, the reasoner's rSkill state-contract filter and the runner's joint permutation, so a zero-DoF entry there would corrupt all four. But a real robot's collision tree also contains rigid mounts (a Franka hand bolted to the flange, a bimanual rig's two arm pedestals), and without them the tree is a forest.

This list is that missing connectivity and nothing else: no limits, no sensors, no DoF, never a command channel, invisible to every joints consumer. The safety envelope loader unions it with joints to build the collision tree and refuses to load a robot whose collision links do not form one connected tree (openral_safety.envelope_loader.collision_params_from_description).

Every value must come from the robot's real URDF/MJCF at the zero configuration — the same source joints' own origin_xyz / origin_rpy come from. Never hand-estimate a mount transform.

Attributes:

Name	Type	Description
name	str	Attachment name, matching the source URDF <joint> name where one exists (a composed multi-hop mount is named for its endpoint).
parent_link	str	The link this attachment hangs from. Must be reachable from the tree's single root.
child_link	str	The rigidly attached link.
origin_xyz	tuple[float, float, float]	Translation (metres) of child_link's frame in parent_link's frame. When the source model inserts intermediate links between the two, this is the composed transform, exactly as openral_safety.urdf_lowering.lower_joint_fk computes it for movable joints.
origin_rpy	tuple[float, float, float]	Orientation (roll, pitch, yaw) in radians, same frame convention as origin_xyz.

Example

a = FixedAttachment( ... name="panda_hand_joint", ... parent_link="panda_link7", ... child_link="panda_hand", ... origin_xyz=(0.0, 0.0, 0.107), ... origin_rpy=(0.0, 0.0, -0.7853981633974483), ... ) a.child_link 'panda_hand'

ForceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_FORCE — measured force exceeded a safety limit.

Attributes:

Name	Type	Description
kind	Literal['force']	Discriminator (always "force").
joint_or_ee	str	Name of the joint or end-effector that tripped the limit.
measured_n	float	Measured force in newtons.
limit_n	float	Configured ceiling in newtons.

FrameEncoding

Bases: str, Enum

How the bytes inside a SensorFrame are interpreted.

The first four values are raw per-pixel layouts. JPEG / PNG are compressed forms; the runner decodes lazily. CUDA_NV12 / CUDA_RGBA and RAW mark frames whose payload is an opaque handle (NVMM pointer, DMA-BUF fd) — the data field is empty and the consumer must read via handle. CUDA_NV12 is the Tegra NVMM default; CUDA_RGBA is the x86 DeepStream NVMM layout.

GraspDeclaration

Bases: BaseModel

Dispatch's typed statement that a grasp phase is active for a target.

Field-for-field the grasp mirror of PlaceDeclaration (real pick-and-place design §2.1; ADR draft "Declaration-scoped grasp-target contact for gripper finger links" in docs/reference/real-pick-place-adr-drafts.md): dispatch names the target and the gripper contact links, the attachment evidence producer measures the target's oriented region, and the safety kernel bounds the exemption. On its own it exempts nothing — dispatch never supplies a region (the rSkill runner strips one) and region is None means no exemption. Scoped to one goal execution; dies on retraction, goal end/cancel, E-stop, or timeout_s after stamp_ns.

The caps MAX_TIMEOUT_S / MAX_HALF_EXTENT_M / MAX_VOLUME_M3 are Safety-WG placeholders from the ADR draft, not ratified bounds; they sit far below PlaceRegion's own caps because a grasp region names one graspable object, not a receptacle. region.geometry must be empty in v1.

Attributes:

Name	Type	Description
target_id	str	Identity of the declared grasp target, e.g. "cell:pick_box". Required while active.
object_id	str	Payload identity the grasp produces once attached (AttachedCollisionObject.object_id); empty = discovered at attach time.
contact_links	tuple[str, ...]	Gripper links the declaration names, e.g. ("openarm_left_finger_pair",). Non-empty while active. A consumer intersects these with its own launch-derived allowlist.
rskill_id	str	Dispatching skill, for attributability (HZ-0115-2).
trace_id	str	OTel trace id, for attributability (HZ-0115-2).
timeout_s	float	Backstop expiry window in seconds after stamp_ns; capped at MAX_TIMEOUT_S.
stamp_ns	int	Dispatcher timestamp; with active this is the whole liveness key.
active	bool	False retracts the declaration.
region	PlaceRegion | None	Producer-measured oriented box around the target; None = no exemption. Dies with the declaration.
search_box	PlaceRegion | None	Optional box (in the voxel grid's base frame, or a fixed tf2 frame such as odom / map that the producer re-expresses in the grid frame every tick, so a scene's box stays put while a mobile base moves) the target producer searches the occupancy map in for a seed: a search hint only, with no support semantics — the producer measures the support layer from the voxel map itself (the box is padded so that layer lies inside it). It only seeds perception and never arms anything: the exemption is always the measured region. The reasoner grounds it from a named GraspTargetRef (the generic path); unlike region a direct-dispatch scene may also supply it; dispatch passes it through unchanged. None = no search, so the producer never measures a region.
workspace_box	PlaceRegion | None	Optional manipulation workspace (issue #349, docs/reference/manipulation-workspace-contact.md) in the voxel grid's base frame, or a fixed tf2 frame like search_box (the confirmed workspace_region is always in the grid's frame) — the box the task works in (the source tote's interior for a pick). Like search_box a scene may declare it and dispatch passes it through unchanged; it arms nothing on its own. Capped by MAX_WORKSPACE_HALF_EXTENT_M / MAX_WORKSPACE_VOLUME_M3; geometry must be empty. None = no workspace.
workspace_region	PlaceRegion | None	The producer-confirmed workspace: workspace_box itself (never grown), stamped with the source_stamp of the fresh voxel grid that confirmed it holds occupied cells. Producer-supplied only, exactly like region — a scene can never supply it (DeployScene._reject_scene_supplied_place_region) and the rSkill runner strips it. With the kernel's workspace_contact_enabled the robot's hand links may touch occupied cells centred inside it while the hand is slow. None = no workspace contact.

Example

declaration = GraspDeclaration( ... target_id="cell:pick_box", ... contact_links=("openarm_left_finger_pair",), ... timeout_s=70.0, ... stamp_ns=1_000_000_000, ... ) declaration.is_live(now_ns=31_000_000_000) True declaration.is_live(now_ns=72_000_000_000) False

fill_idl(msg: object) -> None

Populate a duck-typed openral_msgs/GraspDeclaration without importing ROS.

A grasp region never carries geometry (v1), so no primitive factory is needed.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate a duck-typed ``openral_msgs/GraspDeclaration`` without importing ROS.

    A grasp region never carries geometry (v1), so no primitive factory is
    needed.
    """
    msg.target_id = self.target_id  # type: ignore[attr-defined]
    msg.object_id = self.object_id  # type: ignore[attr-defined]
    msg.contact_links = list(self.contact_links)  # type: ignore[attr-defined]
    msg.rskill_id = self.rskill_id  # type: ignore[attr-defined]
    msg.trace_id = self.trace_id  # type: ignore[attr-defined]
    msg.timeout_s = float(self.timeout_s)  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.active = bool(self.active)  # type: ignore[attr-defined]
    msg.region_valid = self.region is not None  # type: ignore[attr-defined]
    if self.region is not None:
        self.region.fill_idl(msg.region)  # type: ignore[attr-defined]
    msg.search_box_valid = self.search_box is not None  # type: ignore[attr-defined]
    if self.search_box is not None:
        self.search_box.fill_idl(msg.search_box)  # type: ignore[attr-defined]
    msg.workspace_box_valid = self.workspace_box is not None  # type: ignore[attr-defined]
    if self.workspace_box is not None:
        self.workspace_box.fill_idl(msg.workspace_box)  # type: ignore[attr-defined]
    msg.workspace_region_valid = self.workspace_region is not None  # type: ignore[attr-defined]
    if self.workspace_region is not None:
        self.workspace_region.fill_idl(msg.workspace_region)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    return cls(
        target_id=str(msg.target_id),  # type: ignore[attr-defined]
        object_id=str(msg.object_id),  # type: ignore[attr-defined]
        contact_links=tuple(str(link) for link in msg.contact_links),  # type: ignore[attr-defined]
        rskill_id=str(msg.rskill_id),  # type: ignore[attr-defined]
        trace_id=str(msg.trace_id),  # type: ignore[attr-defined]
        timeout_s=float(msg.timeout_s),  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        active=bool(msg.active),  # type: ignore[attr-defined]
        region=(
            PlaceRegion.from_idl(msg.region)  # type: ignore[attr-defined]
            if bool(getattr(msg, "region_valid", False))
            else None
        ),
        search_box=(
            PlaceRegion.from_idl(msg.search_box)  # type: ignore[attr-defined]
            if bool(getattr(msg, "search_box_valid", False))
            else None
        ),
        workspace_box=(
            PlaceRegion.from_idl(msg.workspace_box)  # type: ignore[attr-defined]
            if bool(getattr(msg, "workspace_box_valid", False))
            else None
        ),
        workspace_region=(
            PlaceRegion.from_idl(msg.workspace_region)  # type: ignore[attr-defined]
            if bool(getattr(msg, "workspace_region_valid", False))
            else None
        ),
    )

is_live(*, now_ns: int) -> bool

Is this declaration still in force at now_ns?

Same rule and same clock-domain contract as PlaceDeclaration.is_live: now_ns must be a reading of the publishing graph's ROS clock (simulator time under use_sim_time), or the newest stamp on the stream that carried the declaration — never time.time_ns. Retracted, expired and future-stamped declarations are all dead; a dead declaration exempts nothing.

Parameters:

Name	Type	Description	Default
now_ns	int	Consumer's current time, same clock as stamp_ns.	required

Returns:

Type	Description
bool	True only while the declaration is active and inside its
bool	backstop window.

Source code in python/core/src/openral_core/schemas.py

def is_live(self, *, now_ns: int) -> bool:
    """Is this declaration still in force at ``now_ns``?

    Same rule and same clock-domain contract as
    ``PlaceDeclaration.is_live``: ``now_ns`` must be a reading of the
    publishing graph's ROS clock (simulator time under ``use_sim_time``),
    or the newest stamp on the stream that carried the declaration — never
    ``time.time_ns``. Retracted, expired and future-stamped declarations
    are all dead; a dead declaration exempts nothing.

    Args:
        now_ns: Consumer's current time, same clock as ``stamp_ns``.

    Returns:
        ``True`` only while the declaration is active and inside its
        backstop window.
    """
    if not self.active:
        return False
    elapsed_ns = now_ns - self.stamp_ns
    return 0 <= elapsed_ns <= int(self.timeout_s * 1e9)

GraspTargetRef

Bases: BaseModel

The object an ExecuteRskillTool goal grasps, as the reasoner names it.

The reasoner names; perception grounds; the producer measures (real pick-and-place design §2.2). At dispatch the reasoner resolves this reference to a seed box in the robot base frame — a recalled spatial-memory node's 3D box when object_id is set, else the one live lifted detection whose label matches — and sends it as GraspDeclaration.search_box. A reference that grounds to nothing, or to more than one instance with no object_id, is refused and the goal is not sent. Naming arms nothing: the kernel only ever trusts the region the producer measured inside the seed.

Attributes:

Name	Type	Description
label	str	Open-vocabulary object label, e.g. "box"; matched case-insensitively against the live lifted detections' labels.
object_id	str | None	A spatial-memory node_id from recall_object when the reasoner already knows the instance; None = ground by label.
contact_links	list[str]	Gripper links the grasp declaration names, all child_links of role: gripper joints of ONE hand (the gripper joints hanging off one arm); empty = the hand of a single-hand robot. A robot with several hands refuses an empty list (the grounding would otherwise exempt every hand), and links of two hands, a non-gripper link, or only part of a hand's links (that would exempt one finger and the kernel would stop the grasp) are refused.

Example

GraspTargetRef(label="box").contact_links []

GripperClosureCalibration

Bases: BaseModel

How a role: gripper joint's position reads "closed on an object".

The position-stall grasp trigger (openral_hal._grasp_trigger) needs no effort channel: a jaw commanded toward closed that settles short of the command by more than its empty-close error is stalled on something. These are the numbers that turn that into thresholds, in the joint's own position units: robot-type constants of the gripper mechanism, declared in the robot manifest and derived from teleop data; not per cell (CLAUDE.md §1.2: cited measurements, never a library default).

Attributes:

Name	Type	Description
closed_position	float	The commanded position of a fully closed jaw. Must be an end of the joint's position_limits: "closer to closed" is read as the distance to it.
closed_rest_offset	float	How far from closed_position the jaw rests when it closes on nothing (mechanical stop, servo deadband). Measured from teleop data.
stall_gap	float	How much further from closed than the command (beyond closed_rest_offset) a settled jaw must sit to read as stalled on an object. Also the band a command must be within of closed_position to count as a close command, and (halved) the hysteresis for slip / release / re-seat.
settle_tolerance	float	Largest position span over the trigger's settle window for the jaw to count as stationary. A few position-encoder LSBs.

Example

GripperClosureCalibration( ... closed_position=0.0, ... closed_rest_offset=0.0086, ... stall_gap=0.08, ... settle_tolerance=0.001, ... ).stall_gap 0.08

GripperReadMode

Bases: str, Enum

How openral_hal.MujocoArmHAL reports the gripper qpos.

SUM_OVER_SCALE (default) — clip(sum(qpos[addrs]) / scale, 0, 1). Matches the Franka parallel gripper where two finger qpos are summed and divided by 2 * max_finger_extent (=0.08 m for Panda). Public surface is normalised [0, 1]. AFFINE_LOW_HIGH — (qpos[addrs[0]] - ctrl_range[0]) / (ctrl_range[1] - ctrl_range[0]). Used by the SO-100 menagerie Jaw joint, a 1-DoF revolute with a non-zero closed position (-0.174 rad). Public surface is normalised [0, 1]. PASSTHROUGH — qpos[addrs[0]] reported verbatim, in the same physical units as the MJCF (metres for Aloha prismatic fingers, radians for OpenArm revolute jaws). The public surface is not [0, 1] — Skills must accept the raw range.

GripperWriteMode

Bases: str, Enum

How openral_hal.MujocoArmHAL maps an Action's gripper value to ctrl.

NORMALISED (default) — input is [0, 1], mapped affinely to ctrl_range. low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). PASSTHROUGH — input is in the same physical units as ctrl_range (metres / radians), written directly to ctrl. MuJoCo's actuator ctrlrange does the clipping. Used by Aloha (positive-finger metres) and OpenArm (jaw radians).

GroundedSubtask

Bases: BaseModel

One subtask bound to exactly ONE specific object.

Makes the "smallest actionable unit" invariant a type: object_ref is the single concrete object (or place) this subtask acts on, and text is the instruction handed to the skill (the VLA prompt). The validator forbids a collective/quantified object_ref or text and requires text to name its object_ref — so an ungrounded "pick up the first batch of objects" is not a representable value and the provider's structured-output path re-prompts (the same mechanism that rejects a malformed ReasonerToolCall).

Attributes:

Name	Type	Description
object_ref	str	The single concrete object/place this subtask acts on ("milk", "the fridge"). Never a quantifier or generic plural.
text	str	The instruction the skill receives, naming object_ref ("pick up the milk and put it in the basket").

Example

GroundedSubtask(object_ref="milk", text="pick up the milk and bag it").render() 'pick up the milk and bag it' is_collective_target( ... GroundedSubtask(object_ref="milk", text="grab the milk").object_ref ... ) False

render() -> str

The instruction string handed to MissionState / the skill.

Source code in python/core/src/openral_core/schemas.py

def render(self) -> str:
    """The instruction string handed to ``MissionState`` / the skill."""
    return self.text.strip()

HalConfig

Bases: BaseModel

Configuration for a HAL adapter instantiation by the inference runner.

Picks which HAL adapter class to use (sim digital twin vs real hardware) and how to reach the robot (serial port, FCI URI, ROS 2 namespace). Adapter-specific fields are forwarded as params.

Attributes:

Name	Type	Description
adapter	str	HAL adapter id, e.g. "so100_follower", "so100_digital_twin", "franka_panda_real", "ur5e_real", "ros_control". Resolved against the HAL registry.
transport	dict[str, object]	Transport-layer parameters. Examples: {"port": "/dev/ttyACM0", "baud": 1_000_000} for SO-100; {"fci_uri": "172.16.0.2"} for Franka; {"namespace": "/ur5e"} for ros2_control.
params	dict[str, object]	Adapter-specific keyword arguments forwarded to the constructor (e.g. calibration overrides).

Example

HalConfig( ... adapter="so100_follower", ... transport={"port": "/dev/ttyACM0", "baud": 1_000_000}, ... ).adapter 'so100_follower'

HalEntrypoints

Bases: BaseModel

Per-robot simulation and real-hardware HAL import strings.

The two HALs a robot can expose, declared independently so the choice of HAL type lives in the manifest (never in environment config or runtime params). Each value is a "module:Attr" import string resolved by openral_hal.build_hal, or None when the robot has no HAL of that kind (sim-only / real-only / scene-only).

Attributes:

Name	Type	Description
sim	str | None	Import string for the simulation HAL. When None and RobotDescription.sim is populated, the resolver derives MujocoArmHAL.from_description — so every plain arm leaves this null. Set it explicitly only for a non-generic sim HAL (e.g. "openral_hal.panda_mobile:PandaMobileHAL", which has no sim: block to derive from).
real	str | None	Import string for the real-hardware HAL (e.g. "openral_hal.ur_real:UR5eRealHAL"). None for sim-only robots; the resolver raises ROSCapabilityMismatch if mode="real" is requested on a robot whose real is None.
parameters	HalParameters	Per-robot HAL construction defaults (serial port, robot_ip, …) merged into the constructor by openral_hal.build_hal. Empty by default.
real_bringup	str | None	Optional vendor ros2_control bringup that deploy run includes next to the real HAL, as "<ros_package>:<file>.launch.py" (resolved under that package's share/<pkg>/launch/). None means no bringup is included; there is no convention-based fallback.

Example

HalEntrypoints(real="openral_hal.ur_real:UR5eRealHAL").sim is None True HalEntrypoints(real_bringup="openral_hal_openarm:real_bringup.launch.py").real_bringup 'openral_hal_openarm:real_bringup.launch.py'

HalParameters

Bases: BaseModel

Per-robot HAL construction defaults declared in the manifest.

Carries the transport / constructor keyword arguments a robot's HAL needs — the SO-100's serial port + baud, a ros2_control arm's robot_ip / fci_ip — so the manifest is the single source of those defaults instead of a per-robot lifecycle-node subclass. This is the schema seam that lets the unified, robot.yaml-driven ManifestHALLifecycleNode serve a parameterised HAL without a bespoke _create_hal.

openral_hal.build_hal merges defaults underneath any explicit transport kwargs (so a deploy run override wins) and then drops every key the target HAL constructor does not accept — exactly the filtering it already applies to transport. Empty by default, so robots that need no construction kwargs (the derived MujocoArmHAL arms) are unaffected.

Attributes:

Name	Type	Description
defaults	dict[str, object]	HAL constructor / transport keyword defaults, e.g. {"port": "/dev/ttyACM0", "baud": 1_000_000} for the SO-100 or {"robot_ip": "192.168.1.10"} for a real UR arm.
can_bus_bindings	dict[str, str]	Maps a key of defaults to the role token that identifies which physical CAN bus fills it, e.g. {"left_can_interface": "left", "right_can_interface": "right"}. Declaring this opts the robot into having openral detect overwrite those defaults with the interface names actually found on the host, instead of shipping one lab's names to every user. A CAN interface name is a property of the host, not of the robot: the same arm is openarm_left on a machine with a udev rule and can1 on one without. The manifest therefore declares only which parameter each bus fills and how to recognise it; the value is discovered. Robots with no CAN bus leave this empty and are untouched. The token is matched case-insensitively as a substring of the interface name, so it works for any bus count — one entry for a single-bus arm, two for a bimanual station, N for an N-limb machine. Matching is never positional: alphabetical order is a coincidence, and binding the left arm's trajectory to the right arm's bus is exactly the failure this avoids. An entry that matches no interface, or more than one, is reported as a warning and leaves the manifest value untouched (CLAUDE.md §1.4).

Example

HalParameters(defaults={"port": "/dev/ttyACM0"}).defaults["port"] '/dev/ttyACM0' HalParameters(can_bus_bindings={"left_can_interface": "left"}).can_bus_bindings

Hand

Bases: str, Enum

Laterality of an end-effector.

HumanEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_HUMAN — a human triggered an intervention.

Attributes:

Name	Type	Description
kind	Literal['human']	Discriminator (always "human").
actor	str	Identifier of the operator (e.g. "slack:alice", "gui").
reason	str	Free-form reason given by the actor.

ImagePreprocessing

Bases: BaseModel

Per-rSkill image preprocessing contract.

Properties of how the checkpoint was trained against image frames, surfaced on the manifest so the sim adapter does not have to learn them from a YAML override. vla.extra overrides on the eval config still win — see openral_rskill._vla_core.resolve_image_preprocessing.

Attributes:

Name	Type	Description
flip_180	bool	Apply torch.flip(t, dims=[1, 2]) (H+W reversal) to every camera frame before the policy forward. SmolVLA / pi05 LIBERO checkpoints want this on; the RoboCasa checkpoints published by RoMALab / DAVIAN-Robotics want it off.
flip_vertical	bool	Apply img[::-1, :, :] (vertical-only, H-axis reversal) BEFORE any 180° rotation. Mirrors robocasa.wrappers.gym_wrapper.RoboCasaGymEnv.process_img, which the canonical openpi-robocasa eval applies before feeding frames to the policy. Required for robocasa/robocasa365_checkpoints/pi05_pretrain_human300 and its lerobot-converted siblings; the RoMALab MG_300 checkpoint does NOT want this flip.
input_template	str	Format string used to name image tensors in the policy input batch — for example "observation.images.{cam}" for SmolVLA / pi05 RoMALab vs "observation.image.{cam}" for ruiname/pi05-robocasa-10tasks-200k.
aliases	dict[str, str]	Per-checkpoint rename map from a VLA slot (the vla_feature_key suffix a sensors_required entry names, e.g. camera1) to the model's expected input feature name (e.g. image). Keys are slots — never sensor or scene camera names — so the map means the same thing on openral sim run, deploy sim and deploy run (see sensor_name_to_slot); unknown_alias_slots reports keys that break this rule. Empty means pass through unchanged. SmolVLA / pi05 LIBERO want {"camera1": "image", "camera2": "image2"}.
norm_tag	str | None	Normalization statistics tag for policies that use multiple checkpoints or training distributions (e.g. MolmoAct2 with norm_tag="so100_so101_molmoact2" for SO-100/101 finetuned weights). Optional; None means the adapter's default tag applies. Used by predict_action methods to select the right norm_stats entry at inference time.
image_max_crops	int | None	Cap on the number of image tiles a multi-crop image processor (Molmo / MolmoAct2 family) produces per camera frame. Each extra 378px crop adds ~182 pooled image tokens and attention cost is quadratic in the token count, so this is the primary activation-memory lever (the NF4 weights are fixed at ~3.5 GiB; what overflows an 8 GiB card is the crop activations). Optional; None keeps the checkpoint's own default (8 for MolmoAct2). A per-rollout vla.extra["image_max_crops"] (or the OPENRAL_MOLMOACT2_MAX_CROPS env) still overrides this per-checkpoint default. Must be >= 1 when set.
resize_width	int | None	Exact checkpoint input width. When set with resize_height, adapters resize the source frame before policy inference. This lets a high-resolution detector feed coexist with a policy trained at a lower native render size.
resize_height	int | None	Exact checkpoint input height; must be set together with resize_width.

unknown_alias_slots(sensors_required: list[SensorRequirement]) -> list[str]

Alias keys that are not a slot declared by sensors_required.

Example

req = SensorRequirement( ... modality=SensorModality.RGB, vla_feature_key="observation.images.camera1" ... ) ImagePreprocessing(aliases={"camera1": "image", "top": "x"}).unknown_alias_slots( ... [req] ... ) ['top']

Source code in python/core/src/openral_core/schemas.py

def unknown_alias_slots(self, sensors_required: list[SensorRequirement]) -> list[str]:
    """Alias keys that are not a slot declared by ``sensors_required``.

    Example:
        >>> req = SensorRequirement(
        ...     modality=SensorModality.RGB, vla_feature_key="observation.images.camera1"
        ... )
        >>> ImagePreprocessing(aliases={"camera1": "image", "top": "x"}).unknown_alias_slots(
        ...     [req]
        ... )
        ['top']
    """
    declared = _required_rgb_slots(sensors_required)
    return [key for key in self.aliases if key not in declared]

IntrinsicsPinhole

Bases: BaseModel

Pinhole camera intrinsics.

Attributes:

Name	Type	Description
width	int	Image width in pixels.
height	int	Image height in pixels.
fx	float	Focal length in x (pixels).
fy	float	Focal length in y (pixels).
cx	float	Principal point x (pixels).
cy	float	Principal point y (pixels).
distortion_model	Literal['plumb_bob', 'rational_polynomial', 'equidistant', 'none']	Distortion model name, in ROS CameraInfo naming: plumb_bob (Brown, [k1, k2, p1, p2, k3]), rational_polynomial (OpenCV rational, [k1, k2, p1, p2, k3, k4, k5, k6]: the radial term is (1 + k1 r^2 + k2 r^4 + k3 r^6) / (1 + k4 r^2 + k5 r^4 + k6 r^6)), equidistant (fisheye, [k1, k2, k3, k4]) or none.
distortion_coeffs	list[float]	Distortion coefficients, in the model's order.

Example

IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0) IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0, ...)

JointSpec

Bases: BaseModel

URDF-derived joint specification.

Attributes:

Name	Type	Description
name	str	Joint name matching the URDF.
joint_type	JointType	Kinematic joint type.
parent_link	str	Parent link name.
child_link	str	Child link name.
axis_xyz	tuple[float, float, float]	Rotation/translation axis unit vector.
position_limits	tuple[float, float] | None	(min, max) in the unit of the value that travels on this channel — radians or metres for arm joints, and (0.0, 1.0) for a gripper whose HAL contract is a normalised jaw fraction (i.e. one declared normalised in SimDescription.grippers). The safety kernel's envelope, the rSkill runner's pre-clamp and the supervisor's gripper_min / gripper_max all compare a commanded Action value against these bounds, so declaring a normalised gripper's mechanical radian range here stops the envelope from constraining that channel at all (issue #62). A mechanical range that differs from the channel unit belongs in SimGripperDescription.ctrl_range / the URDF.
velocity_limit	float | None	Maximum velocity.
effort_limit	float | None	Maximum effort (N or Nm).
has_position_sensor	bool	Whether position feedback is available.
has_velocity_sensor	bool	Whether velocity feedback is available.
has_torque_sensor	bool	Whether the joint has a dedicated joint torque sensor. A torque estimate derived from motor current does not count: declare False for current-derived estimates so force decisions do not over-trust them.
backlash_estimate	float | None	Estimated backlash in radians.
actuator_kind	Literal['dc', 'bldc', 'stepper', 'servo', 'tendon', 'hydraulic', 'pneumatic'] | None	Type of actuator.
sim_joint_name	str | None	Optional override carrying the MJCF/MuJoCo joint name as it appears in the simulator's compiled model when that name differs from name. The HAL uses name for the world-state contract (it's the URDF-shaped logical identifier the safety supervisor sees on every chunk); a separate sim_joint_name lets a sim-adapter look up mj_name2id without hardcoding robosuite / robocasa / MuJoCo naming conventions in backend modules. None means "the MJCF joint name matches name" — true for every fixed-base manipulator we ship; only mobile bases and humanoids whose MJCF auto-prefixes joints (robosuite's mobilebase0_… namespace, GR-1's robot0_…) need it.
role	JointRole	Structural classification. The downstream runner / safety / dataset-bridge code identifies grippers and base DoFs by this tag instead of substring-matching the joint name (which silently misclassifies any joint containing "gripper", e.g. "gripper_pose"). Default "unknown" keeps legacy manifests loadable; the fleet annotates incrementally as this rolls out.
closure_calibration	GripperClosureCalibration | None	Position-stall grasp-trigger calibration for a role: "gripper" joint (GripperClosureCalibration). None = uncalibrated: the HAL's vision attachment leg refuses to arm a trigger on that joint rather than guess thresholds.
origin_xyz	tuple[float, float, float]	Fixed translation (metres) of this joint's frame in its parent_link frame — the URDF <joint><origin xyz>. With origin_rpy and axis_xyz it gives the kernel everything it needs for forward kinematics. Default (0, 0, 0); populated by the offline lowering tool (from MJCF/URDF) only for robots that enable self-collision checking — legacy manifests are unaffected.
origin_rpy	tuple[float, float, float]	Fixed orientation (roll, pitch, yaw, radians) of this joint's frame in its parent_link frame — the URDF <joint><origin rpy>. Default (0, 0, 0).

JointState

Bases: BaseModel

Real-time joint state snapshot.

Attributes:

Name	Type	Description
name	list[str]	Ordered list of joint names.
position	list[float]	Joint positions (rad or m).
velocity	list[float]	Joint velocities (rad/s or m/s).
effort	list[float]	Joint efforts (Nm or N), all joints or none. Empty means not measured; a producer never zero-fills an unmeasured joint, since a torque observer reads 0.0 as an unloaded joint (#373).
stamp_ns	int	ROS 2 timestamp in nanoseconds.

JointType

Bases: str, Enum

Kinematic joint type following URDF conventions.

JointUnits

Bases: str, Enum

Angular units a joint-position checkpoint was trained in.

A joint-position VLA consumes state and emits actions in ONE angular convention (whatever its training dataset recorded). openral's JointState / Action contract is radians, so the skill_runner must convert deg<->rad at the policy boundary when the checkpoint is in degrees. Getting this wrong is not subtle: feeding a degrees-trained policy radians makes every observation ~57x too small (out-of-distribution -> garbage actions), and emitting its degree actions as radians makes the HAL command ~57x too large (the arm slams its limits). Declaring it on the manifest removes the fragile runtime guess.

LaunchInclude

Bases: BaseModel

A vendor ROS 2 launch file a deploy has to bring up alongside the graph.

Some sensors are not opened by OpenRAL at all — they are published by the vendor's own driver, and the scene's SensorDeployBinding merely subscribes to what that driver puts on the bus (backend: ros2_image). The binding names the topic but not who publishes it, so until this existed an operator had to start the driver by hand in a second terminal and every camera fed that way was silently empty when they forgot.

Same shape as the HAL's real_bringup.launch.py convention, but declared as data rather than by file name: a driver needs arguments (which camera model, which override YAML), and those are host-specific — which is exactly what a DeployScene is for.

Resolved through FindPackageShare, so the launch comes from the sourced overlay like any other vendor package, and a missing package fails loudly at launch-parse time rather than leaving a topic quietly unpublished.

Attributes:

Name	Type	Description
package	str	ROS 2 package providing the launch file.
launch_file	str	File name inside that package's launch/ directory.
args	dict[str, str]	Launch arguments, forwarded verbatim. Values are strings because that is what ros2 launch takes; "true"/"false" for booleans.

Example

LaunchInclude(package="zed_wrapper", launch_file="zed_camera.launch.py").args {}

LifecycleTransitionTool

Bases: _ReasonerToolBase

Tool variant — drive a ROS 2 lifecycle transition on a peer node.

Dispatch: service call on <node>/change_state with the matching Transition id (configure / activate / deactivate / cleanup). Used to bring a HAL back online, restart a faulted node, or stage a controlled shutdown.

Attributes:

Name	Type	Description
tool	Literal['lifecycle_transition']	Discriminator (always "lifecycle_transition").
node	str	Fully-qualified ROS node name (e.g. "/openral/hal/so100").
transition	Literal['configure', 'activate', 'deactivate', 'cleanup']	One of "configure", "activate", "deactivate", "cleanup". Other transitions ("shutdown", error-recovery) are deliberately omitted from the open-core palette per CLAUDE.md §6 Layer 6 — shutdown is owned by the safety supervisor, not the reasoner.

LinkCollisionGeometry

Bases: BaseModel

One convex collision volume rigidly attached to a robot link.

The lowered, kernel-facing form of a link's collision geometry. Authored by hand, or emitted by the offline lowering tool from a robot's MJCF or URDF + SRDF source; the kernel loads these into pre-sized buffers at on_configure and never parses the source geometry on the hot path. link_name is a JointSpec link (joints stays normative for the kinematic chain — this adds geometry only).

Attributes:

Name	Type	Description
link_name	str	The JointSpec.child_link (or JointSpec.parent_link) this volume is attached to.
shape	CollisionShape	The convex primitive (capsule, sphere, or box).
origin_xyz_rpy	tuple[float, float, float, float, float, float]	Pose of the primitive in the link frame — (x, y, z, roll, pitch, yaw) in metres and radians.
tight_geometry	TightCollisionGeometry | None	Optional tighter convex geometry the safety kernel uses in place of shape in two narrow phases: arm-link vs world voxel, and — when both links of a pair declare it — self-collision box vs box. shape remains the broad-phase bound, the containment proof, and the geometry every other check uses. Only valid on a BoxShape, and only when it fits inside it.

Example

g = LinkCollisionGeometry( ... link_name="link_1", ... shape=CapsuleShape(radius_m=0.04, length_m=0.3), ... ) g.origin_xyz_rpy (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)

LocateInViewTool

Bases: _ReasonerToolBase

Tool variant (read-only) — check if an object is in a live camera view.

The complement to RecallObjectTool: where recall_object recalls a remembered object from the scene-graph memory, locate_in_view asks a live camera-mounted VLM detector (e.g. LocateAnything) to look at the current frame right now and report whether the queried object is visible — and where. It runs the detector's open-vocabulary query on demand via the /openral/perception/locate_in_view ROS service and feeds the answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation — it only reads a frame; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['locate_in_view']	Discriminator (always "locate_in_view").
query	str	The object(s) to look for, as concrete object noun(s) — sent verbatim as the detector's open-vocabulary query. The fast default locator (omdet-turbo-locator) is a multi-label detector: it matches each comma-separated term as one object class, so query a single noun ("mug") or a comma-separated list for several ("cup, bowl, ketchup bottle"). A collective/quantified PHRASE ("the objects on the table", "everything", "all items") is treated as one literal class name and matches NOTHING — name the concrete objects instead. Referring expressions with relations ("the mug behind the bowl") need the locateanything-3b locator (set detector); omdet ignores the relation.
camera	str	Optional camera selector. Empty (default) uses the detector's primary camera; otherwise names one of the detector's configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name — the detector is camera-agnostic and maps the id to a topic.
detector	str	Optional on-demand locator selector. Empty (default) uses the deployment's default locator; otherwise an rSkill id / short alias of one of the on-demand locators in the graph (e.g. "omdet_turbo-any-locator-fp16" for fast simple "find X", "locateanything_3b-any-general-nf4" for complex referring expressions). The reasoner routes to /openral/perception/<detector>/locate_in_view. Still read-only — choosing a model does not grant actuation.

MemorySearchTool

Bases: _ReasonerToolBase

Tool variant (read-only) — search the archival memory log.

Pages in entries evicted from the bounded MEMORY.md core into the archival JSONL (MemGPT recall), so the LLM can recall an older fact — e.g. a stale object location used as a search prior — without loading the whole history. Read-only; no actuation.

Attributes:

Name	Type	Description
tool	Literal['memory_search']	Discriminator (always "memory_search").
query	str	Free-text query over archived memory.
section	MemorySection | None	Optional MemorySection filter.
limit	int	Maximum number of results to return.

MemoryWriteTool

Bases: _ReasonerToolBase

Tool variant (write) — edit the robot's self-maintained MEMORY.md.

The reasoner's first write-capable tool. It edits the persistent, human-readable semantic memory (preferences, corrections, lessons, durable home facts, object locations, open tasks) through an explicit operation — never a free-form rewrite (Mem0 ADD/UPDATE/DELETE + Zep temporal supersession). It writes ONLY to the advisory memory file and holds no authority over actuation: a wrong memory yields a bad plan the C++ kernel still vetoes (CLAUDE.md §1.1). The dispatch that applies the edit is wired in a later phase — this is the typed contract.

Attributes:

Name	Type	Description
tool	Literal['memory_write']	Discriminator (always "memory_write").
op	Literal['add', 'update', 'supersede', 'delete']	The edit operation — add / update / supersede / delete.
section	MemorySection	Which MemorySection to edit.
content	str	The fact, as a distilled NL rule/line. Required for every op except delete (validated).
importance	float	LLM-assigned salience in [0, 1] (Generative Agents), used by recency-importance-relevance retrieval when the section is capped.
target	str | None	The existing entry this op acts on. Required for update / supersede / delete; omitted for add (validated).

MimicJoint

Bases: BaseModel

A link posed by another joint's value: q = multiplier * q[mimics] + offset.

The URDF <mimic> / MuJoCo joint-equality relation, for a link that moves but has no command channel of its own — the OpenArm's second (follower) finger, driven through the first by the gripper's linkage. RobotDescription.joints is consumed unfiltered as the action width, the sim qpos map and the runner's permutation, so a follower cannot be a JointSpec; and a FixedAttachment cannot move. This is the third edge kind of the collision tree, and nothing else: no limits, no sensor, no DoF, invisible to every joints consumer. The safety kernel poses child_link rigidly at the mimicked value every check (JointKind::kMimicRevolute / kMimicPrismatic), so the follower finger is checked at the opening the gripper is commanded to, like every other link, instead of swept over its whole stroke (docs/reference/finger-mimic-geometry.md).

Every geometric value comes from the robot's real URDF/MJCF (openral collision lower re-injects origin_* / axis_xyz from the MJCF and refuses a multiplier / offset the MJCF equality disagrees with). Never hand-estimate one.

Attributes:

Name	Type	Description
name	str	Joint name, matching the source URDF <joint> where one exists.
parent_link	str	The link this joint hangs from.
child_link	str	The link it poses. Defined by no other edge.
mimics	str	The JointSpec (by name) whose value drives this one.
multiplier	float	q = multiplier * q[mimics] + offset; finite, non-zero.
offset	float	Same relation; finite, in the joint's own unit.
joint_type	JointType	revolute (or continuous) or prismatic.
axis_xyz	tuple[float, float, float]	This joint's own axis in its frame (the source model's, not the leader's: the OpenArm follower's axis is the leader's negated in the URDF, the same with multiplier = -1; the MJCF spells it as the same axis with multiplier = 1).
origin_xyz	tuple[float, float, float]	Translation (metres) of the joint frame in parent_link.
origin_rpy	tuple[float, float, float]	Orientation (roll, pitch, yaw), radians, same frame.
sim_joint_name	str | None	The MJCF joint this is, when the twin spells it differently (the OpenArm's openarm_left_finger_joint2).

Example

m = MimicJoint( ... name="openarm_left_finger_joint2", ... parent_link="openarm_left_link7", ... child_link="openarm_left_finger_follower", ... mimics="left_gripper", ... multiplier=1.0, ... axis_xyz=(1.0, 0.0, 0.0), ... origin_xyz=(-0.00143, 0.018, -0.068), ... ) m.value_at(0.5) 0.5

value_at(leader_value: float) -> float

This joint's value when the mimicked joint is at leader_value.

Source code in python/core/src/openral_core/schemas.py

def value_at(self, leader_value: float) -> float:
    """This joint's value when the mimicked joint is at ``leader_value``."""
    return self.multiplier * leader_value + self.offset

MotionMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/motion.

Emitted when a frame-difference detector measures sub-frame motion above a configured threshold. region_bbox is set when the detector localises the moving pixels (top-left origin in pixels); None means "motion magnitude only, no localisation."

Attributes:

Name	Type	Description
kind	Literal['motion']	Discriminator (always "motion").
magnitude	float	Mean absolute per-pixel difference, normalised to [0, 1] against the encoding's full-scale range.
threshold	float	Magnitude threshold that fired the event.
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the moving region in pixels.

ObjectDetection2D

Bases: BaseModel

A single 2D detection inside an ObjectsMetadata event.

The 3D DetectedObject lives in WorldState; the 2D form here is what a per-camera detector (nvinfer, tflite, cv2-CPU) emits before any pose lift / fusion.

Attributes:

Name	Type	Description
label	str	Semantic class label as produced by the model.
confidence	float	Detection confidence in [0, 1].
bbox_xyxy	tuple[int, int, int, int]	Axis-aligned bounding box in pixels (x_min, y_min, x_max, y_max); image origin top-left.
det_id	int	Stable per-detector, per-camera identity, assigned by a 2D-IoU tracker at detection time so an object can be referred to and de-duplicated even when the 3D lift cannot run (RGB-only / no depth). -1 means untracked (legacy detectors / single-shot). Propagated into DetectedObject.track_id by the lift so a physical object carries one id whether it appears in the camera-space in_view line (no depth) or the 3D scene_objects line (depth).

ObjectsMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/objects.

Emitted by the event leg's detector element (nvinfer on Jetson, tflite on CPU, etc.) when one or more objects are detected.

Attributes:

Name	Type	Description
kind	Literal['objects']	Discriminator (always "objects").
detections	list[ObjectDetection2D]	Per-object 2D detections, ordered by descending confidence.
model_id	str	Identifier of the detector that fired (e.g. "yolov8n", "nvinfer:resnet50").
frame_width	int	Pixel width of the frame the detector ran on; the bbox_xyxy of each detection is in this pixel space (the cross-frame lift scales it to the sensor's intrinsics resolution).
frame_height	int	Pixel height of that frame.
camera_frame_id	str | None	tf2 frame the source image's pixels live in (the camera's optical frame), from the driver's CameraInfo header. Set together with camera_intrinsics; None (no CameraInfo reached the detector) means a consumer falls back to the SensorSpec named by sensor_id, whose frame and intrinsics may be a sim stand-in on real hardware.
camera_intrinsics	IntrinsicsPinhole | None	The driver's live pinhole K for that frame, at the CameraInfo's own resolution (the lift rescales the box to it).

Example

md = ObjectsMetadata( ... sensor_id="top", ... detections=[], ... model_id="rtdetr", ... frame_width=1920, ... frame_height=1080, ... camera_frame_id="zed_left_camera_frame_optical", ... camera_intrinsics=IntrinsicsPinhole( ... width=1920, height=1080, fx=1498.18, fy=1498.18, cx=936.11, cy=541.81 ... ), ... ) md.camera_frame_id 'zed_left_camera_frame_optical'

ObservationSpec

Bases: BaseModel

VLA observation configuration for this robot.

Attributes:

Name	Type	Description
state_key	str	Observation dict key for the state vector.
state_shape	tuple[int, ...]	Shape of the state tensor, e.g. (6,) for 6-D EEF.
state_representation	StateRepresentation | None	How the state vector is encoded.
image_flip_180	bool	Deprecated; has no effect on image processing. The live knob is the per-checkpoint RSkillManifest.image_preprocessing.flip_180. Kept so existing manifests still load; setting it emits a FutureWarning.

OccupancyGridRef

Bases: BaseModel

Reference to a 2D occupancy grid for mobile-base world-collision.

Mirrors the nav_msgs/OccupancyGrid metadata that openral_runner.slam_bridge already decodes. The kernel consumes a bounded, fixed-capacity copy; a grid exceeding the configured capacity or older than the staleness deadline is treated as unavailable (fail-closed). The occupancy bytes are referenced by topic, not inlined.

Attributes:

Name	Type	Description
frame_id	str	tf2 frame the grid origin is expressed in (e.g. "map").
resolution_m	float	Edge length of one cell, in metres.
width	int	Grid width in cells.
height	int	Grid height in cells.
origin	Pose6D	Pose of cell (0, 0)'s lower-left corner.
data_topic	str	ROS 2 topic carrying the nav_msgs/OccupancyGrid.

OcrMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/ocr.

Attributes:

Name	Type	Description
kind	Literal['ocr']	Discriminator (always "ocr").
text	str	Recognised text (already stripped of leading / trailing whitespace by the detector).
confidence	float	Recogniser confidence in [0, 1].
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the recognised region in pixels.

PerceptionStaleEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_PERCEPTION — a sensor frame went stale.

Attributes:

Name	Type	Description
kind	Literal['perception']	Discriminator (always "perception").
sensor_id	str	SensorSpec.name of the stale sensor.
staleness_ms	float	Observed age of the last frame, in milliseconds.
threshold_ms	float	Staleness threshold that was crossed, in milliseconds.

PhysicsBackend

Bases: str, Enum

Physics / scene backend used to instantiate a SceneSpec.

Attributes:

Name	Type	Description
MUJOCO		Vanilla MuJoCo (CPU / single-env). Default for LIBERO, MetaWorld, and VLABench (native lerobot 0.6.0 VLABenchEnv, in-process).
MUJOCO_MJX		MuJoCo MJX (XLA, GPU-batched headless rollouts).
PYBULLET		PyBullet (legacy adapters, contact-rich tabletop).
SAPIEN		SAPIEN (Hillbot/UCSD physics + ray-traced rendering). The engine under ManiSkill3 and RoboTwin 2.0; the RoboTwin dual-arm benchmark backend runs it out-of-process via a py3.10 sidecar. ManiSkill3 scenes predate this slot and historically declared MUJOCO — new SAPIEN backends use this value.
ISAACSIM		NVIDIA Isaac Sim (Omniverse + PhysX + RTX, GPU) — the isaac_sim scene, driven out-of-process via a py3.11 sidecar.
COPPELIASIM		CoppeliaSim/PyRep — the RLBench benchmark backend, driven out-of-process via a py3.10 sidecar.
GENESIS		Genesis (physics-language unification). Future.
MOCK		In-process mock with no physics — used for wiring smoketests.

PlaceDeclaration

Bases: BaseModel

Dispatch's typed statement that a place phase is active for a payload.

The ADR-0097 distinction between "arrived at its declared destination" and "grazed a wall" mid-carry — never inferred from motion or simulator introspection. On its own it exempts nothing: it only lets a producer attest a place-phase SupportContactWitness on the declared target, which the kernel then exempts under the pick witness's own bounds/hysteresis (ADR-0092 D6). Scoped to one goal execution; dies on retraction, goal end/cancel, E-stop, or timeout_s after stamp_ns, whichever fires first (HZ-0097-3).

Drafted amendment (ADR-0097, default off): the real place producer (openral_hal._place_target_leg) measures the surface directly under the carried payload and attaches it as the region of dispatch's goal-scope declaration (target_id="surface", the runner's place_approach_enabled); it never declares on its own, and the region dies with the declaration, the payload and the region's age bound.

Attributes:

Name	Type	Description
target_id	str	Identity of the declared place target, e.g. "sim:cab_1_left_group_main"; an attested support_id must name it or a surface of it.
object_id	str	Payload the declaration scopes to (AttachedCollisionObject.object_id); empty = whichever payload is carried.
rskill_id	str	Dispatching skill, for attributability (HZ-0097-2).
trace_id	str	OTel trace id, for attributability (HZ-0097-2).
timeout_s	float	Backstop expiry window in seconds after stamp_ns; capped at MAX_TIMEOUT_S.
stamp_ns	int	Dispatcher timestamp; with active this is the whole liveness key.
active	bool	False retracts the declaration.
contact_force_threshold_n	float	Per-declaration contact-force bound (ADR-0100); 0.0 = no force gate. Never read from the robot-wide SafetyEnvelope.contact_force_threshold_n.
region	PlaceRegion | None	Producer-measured bounded region of the target (ADR-0097's 2026-08-14 amendment); None = no approach allowance (pre-amendment margins). Dies with the declaration.
search_box	PlaceRegion | None	Optional gravity-aligned box (in the voxel grid's base frame) naming the surface a place may use: a hint only. The real place producer measures the support directly under the carried payload with or without it; when set, it refuses a measured patch whose centre lies outside the box. The reasoner grounds it from an optional PlaceTargetRef (a lifted detection or a recalled memory node, padded); dispatch passes it through unchanged. It never arms anything — the allowance is always the producer-measured region. geometry must be empty.

Example

declaration = PlaceDeclaration( ... target_id="sim:cab_1_left_group_main", ... rskill_id="openral/pi05-robocasa", ... timeout_s=60.0, ... stamp_ns=1_000_000_000, ... ) declaration.is_live(now_ns=31_000_000_000) True declaration.is_live(now_ns=91_000_000_000) False

fill_idl(msg: object, *, primitive_factory: Callable[[], object] | None = None) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Parameters:

Name	Type	Description	Default
msg	object	A duck-typed openral_msgs/PlaceDeclaration to populate.	required
primitive_factory	Callable[[], object] | None	Forwarded to PlaceRegion.fill_idl; needed only when the region carries the declared target's own geometry (ADR-0098). Dispatch never measures geometry, so its publication path never needs one.	None

Source code in python/core/src/openral_core/schemas.py

def fill_idl(
    self, msg: object, *, primitive_factory: Callable[[], object] | None = None
) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

    Args:
        msg: A duck-typed ``openral_msgs/PlaceDeclaration`` to populate.
        primitive_factory: Forwarded to ``PlaceRegion.fill_idl``; needed
            only when the region carries the declared target's own geometry
            (ADR-0098). Dispatch never measures geometry, so its publication
            path never needs one.
    """
    msg.target_id = self.target_id  # type: ignore[attr-defined]
    msg.object_id = self.object_id  # type: ignore[attr-defined]
    msg.rskill_id = self.rskill_id  # type: ignore[attr-defined]
    msg.trace_id = self.trace_id  # type: ignore[attr-defined]
    msg.timeout_s = float(self.timeout_s)  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.active = bool(self.active)  # type: ignore[attr-defined]
    msg.contact_force_threshold_n = float(  # type: ignore[attr-defined]
        self.contact_force_threshold_n
    )
    msg.region_valid = self.region is not None  # type: ignore[attr-defined]
    if self.region is not None:
        self.region.fill_idl(
            msg.region,  # type: ignore[attr-defined]
            primitive_factory=primitive_factory,
        )
    # Never carries geometry (validated), so no primitive factory is needed.
    msg.search_box_valid = self.search_box is not None  # type: ignore[attr-defined]
    if self.search_box is not None:
        self.search_box.fill_idl(msg.search_box)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    return cls(
        target_id=str(msg.target_id),  # type: ignore[attr-defined]
        object_id=str(msg.object_id),  # type: ignore[attr-defined]
        rskill_id=str(msg.rskill_id),  # type: ignore[attr-defined]
        trace_id=str(msg.trace_id),  # type: ignore[attr-defined]
        timeout_s=float(msg.timeout_s),  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        active=bool(msg.active),  # type: ignore[attr-defined]
        contact_force_threshold_n=float(getattr(msg, "contact_force_threshold_n", 0.0)),
        region=(
            PlaceRegion.from_idl(msg.region)  # type: ignore[attr-defined]
            if bool(getattr(msg, "region_valid", False))
            else None
        ),
        search_box=(
            PlaceRegion.from_idl(msg.search_box)  # type: ignore[attr-defined]
            if bool(getattr(msg, "search_box_valid", False))
            else None
        ),
    )

is_live(*, now_ns: int) -> bool

Is this declaration still in force at now_ns?

Fails toward dead: a retracted declaration, one past its backstop, and one stamped in the future (a clock that jumped, which is not evidence of anything) are all dead. A dead declaration permits no place-witness attestation, which is the pre-ADR-0097 behaviour.

Clock-domain contract. now_ns and stamp_ns must be readings of the same clock, and that clock is the publishing graph's ROS clock: the dispatching rSkill runner stamps the declaration from node.get_clock().now(), the evidence producer re-stamps its publications from the same clock, and the safety kernel compares against its own. Under use_sim_time that domain is simulator time, whose readings are small (order 1e9 ns) where wall time is order 1.79e18 ns. A consumer that has no such reading of its own must use the newest stamp observed on the message stream carrying the declaration — never time.time_ns, which reports a wall epoch that puts every sim-stamped declaration ~57 years past its backstop and silently kills it.

Parameters:

Name	Type	Description	Default
now_ns	int	Consumer's current time, same clock as stamp_ns — see the clock-domain contract above.	required

Returns:

Type	Description
bool	True only while the declaration is active and inside its
bool	backstop window.

Source code in python/core/src/openral_core/schemas.py

def is_live(self, *, now_ns: int) -> bool:
    """Is this declaration still in force at ``now_ns``?

    Fails toward dead: a retracted declaration, one past its backstop, and
    one stamped in the future (a clock that jumped, which is not evidence
    of anything) are all dead. A dead declaration permits no place-witness
    attestation, which is the pre-ADR-0097 behaviour.

    **Clock-domain contract.** ``now_ns`` and ``stamp_ns`` must be
    readings of the *same* clock, and that clock is the publishing graph's
    ROS clock: the dispatching rSkill runner stamps the declaration from
    ``node.get_clock().now()``, the evidence producer re-stamps its
    publications from the same clock, and the safety kernel compares against
    its own. Under ``use_sim_time`` that domain is **simulator** time, whose
    readings are small (order 1e9 ns) where wall time is order 1.79e18 ns.
    A consumer that has no such reading of its own must use the newest stamp
    observed on the message stream carrying the declaration — never
    ``time.time_ns``, which reports a wall epoch that puts every sim-stamped
    declaration ~57 years past its backstop and silently kills it.

    Args:
        now_ns: Consumer's current time, same clock as ``stamp_ns`` — see
            the clock-domain contract above.

    Returns:
        ``True`` only while the declaration is active and inside its
        backstop window.
    """
    if not self.active:
        return False
    elapsed_ns = now_ns - self.stamp_ns
    return 0 <= elapsed_ns <= int(self.timeout_s * 1e9)

PlaceRegion

Bases: BaseModel

The bounded region of a declared place target (ADR-0097 amendment).

The volume inside which the declared payload's world-collision margin is reduced by min(1.5 x voxel, 40 mm) (ADR-0097's Second Amendment, 2026-08-15 — 37.5 mm at sim's 25 mm cells, 40 mm rather than 75 mm at a real 50 mm grid since the absolute ceiling still binds) so it can physically reach the support contact the place witness is earned by. Round-6 validation: without it the flagship counter->cabinet insertion cannot complete even with a correct declaration — passing through an enclosed target's opening means passing within margin distance of voxel-quantised walls, and at 25 mm cells that quantisation closes the opening's predicted clearance 22-30 mm before the payload arrives. A witness earned by touching can never arm if the payload is stopped before it can touch.

Producer-supplied, producer-specific. Sim computes it from the declared body's MuJoCo model subtree; real hardware measures the support slab directly under the carried payload from the live voxel map (openral_hal._place_target_leg, default off, drafted ADR amendments pending). The kernel is producer-agnostic: it consumes this box identically whoever measured it, and never derives one itself.

The box is oriented, not axis-aligned — the axis-aligned hull of a rotated receptacle is strictly larger, i.e. more permissive. Absent (PlaceDeclaration.region is None) means no allowance at all — the pre-amendment margins, unchanged. A degenerate or over-large region is rejected here and again in the kernel, both times toward "no allowance".

The same type also carries the producer-measured region of a GraspDeclaration (real pick-and-place design §2.1), under that declaration's tighter caps and with geometry always empty.

Attributes:

Name	Type	Description
frame_id	str	Frame pose is expressed in. Must be the robot base frame — the frame OccupancyVoxels is published in. The kernel refuses a region whose frame does not match the occupancy grid's, because a region measured in one frame and applied in another is a relaxation aimed at the wrong volume.
pose	Pose6D	Region box centre pose in frame_id.
half_extents	tuple[float, float, float]	Box half-extents along its own axes (m), each in (0, MAX_HALF_EXTENT_M].
evidence_ref	str	What the producer measured the box from, for attributability (HZ-0097-2/4). Free text, never parsed.
stamp_ns	int	Producer timestamp.
geometry	tuple[AttachedCollisionPrimitive, ...]	The declared target's own collision primitives, each posed in frame_id (ADR-0098, survey Path B). Empty — the default, and what every producer that cannot measure geometry publishes — leaves the region behaving exactly as it did before ADR-0098. Supplying it lets the kernel adjudicate a cell against the modelled surface instead of against the 25 mm cube that quantised it, bounded so that it can never be more permissive than the box alone: the kernel uses the modelled distance only while it is within the blanket allowance that cell would have been granted anyway.

Example

region = PlaceRegion( ... frame_id="base_link", ... pose=Pose6D( ... xyz=(0.6, 0.1, 1.1), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="base_link" ... ), ... half_extents=(0.20, 0.18, 0.12), ... evidence_ref="mujoco_body_subtree:cab_1_left_group_main", ... ) round(region.volume_m3(), 5) 0.03456

fill_idl(msg: object, *, primitive_factory: Callable[[], object] | None = None) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Parameters:

Name	Type	Description	Default
msg	object	A duck-typed openral_msgs/PlaceRegion to populate in place.	required
primitive_factory	Callable[[], object] | None	Builds one empty AttachedCollisionPrimitive message. Required only when geometry is non-empty — dispatch never measures geometry, so its publication path never needs one.	None

Raises:

Type	Description
ROSConfigError	geometry is non-empty and no primitive_factory was supplied. Dropping the geometry silently would leave the kernel adjudicating a declared target it was told nothing about, which reads on the wire exactly like a producer that measured nothing.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(
    self, msg: object, *, primitive_factory: Callable[[], object] | None = None
) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

    Args:
        msg: A duck-typed ``openral_msgs/PlaceRegion`` to populate in place.
        primitive_factory: Builds one empty ``AttachedCollisionPrimitive``
            message. Required only when ``geometry`` is non-empty —
            dispatch never measures geometry, so its publication path never
            needs one.

    Raises:
        ROSConfigError: ``geometry`` is non-empty and no
            ``primitive_factory`` was supplied. Dropping the geometry
            silently would leave the kernel adjudicating a declared target
            it was told nothing about, which reads on the wire exactly like
            a producer that measured nothing.
    """
    msg.frame_id = self.frame_id  # type: ignore[attr-defined]
    msg.pose.position.x = float(self.pose.xyz[0])  # type: ignore[attr-defined]
    msg.pose.position.y = float(self.pose.xyz[1])  # type: ignore[attr-defined]
    msg.pose.position.z = float(self.pose.xyz[2])  # type: ignore[attr-defined]
    msg.pose.orientation.x = float(self.pose.quat_xyzw[0])  # type: ignore[attr-defined]
    msg.pose.orientation.y = float(self.pose.quat_xyzw[1])  # type: ignore[attr-defined]
    msg.pose.orientation.z = float(self.pose.quat_xyzw[2])  # type: ignore[attr-defined]
    msg.pose.orientation.w = float(self.pose.quat_xyzw[3])  # type: ignore[attr-defined]
    msg.half_extents.x = float(self.half_extents[0])  # type: ignore[attr-defined]
    msg.half_extents.y = float(self.half_extents[1])  # type: ignore[attr-defined]
    msg.half_extents.z = float(self.half_extents[2])  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]
    msg.geometry = []  # type: ignore[attr-defined]
    if self.geometry and primitive_factory is None:
        raise ROSConfigError(
            f"PlaceRegion for {self.frame_id!r} carries {len(self.geometry)} target "
            "primitives but fill_idl was given no primitive_factory to encode them. "
            "Publishing the region without its geometry is indistinguishable on the wire "
            "from a producer that measured none."
        )
    for primitive in self.geometry:
        item = primitive_factory()  # type: ignore[misc]  # reason: guarded above
        primitive.fill_idl(item)
        msg.geometry.append(item)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    pose = msg.pose  # type: ignore[attr-defined]
    half = msg.half_extents  # type: ignore[attr-defined]
    return cls(
        frame_id=str(msg.frame_id),  # type: ignore[attr-defined]
        pose=Pose6D(
            xyz=(float(pose.position.x), float(pose.position.y), float(pose.position.z)),
            quat_xyzw=(
                float(pose.orientation.x),
                float(pose.orientation.y),
                float(pose.orientation.z),
                float(pose.orientation.w),
            ),
            frame_id=str(msg.frame_id),  # type: ignore[attr-defined]
        ),
        half_extents=(float(half.x), float(half.y), float(half.z)),
        evidence_ref=str(msg.evidence_ref),  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
        geometry=tuple(
            AttachedCollisionPrimitive.from_idl(item, object_id=str(msg.frame_id))  # type: ignore[attr-defined]
            for item in msg.geometry  # type: ignore[attr-defined]
        ),
    )

volume_m3() -> float

Volume of the region box in cubic metres.

Source code in python/core/src/openral_core/schemas.py

def volume_m3(self) -> float:
    """Volume of the region box in cubic metres."""
    hx, hy, hz = self.half_extents
    return 8.0 * hx * hy * hz

PlaceTargetRef

Bases: BaseModel

An OPTIONAL hint naming the surface an ExecuteRskillTool goal places onto.

Where to set the payload down is the policy's job, and the real place producer measures the support directly under the carried payload without being told (real pick-and-place design §2.3), so a goal never needs this. Set, it restricts the place to one labelled surface ("shelf", "table"): perception grounds it at dispatch to a padded, gravity-aligned box in the robot base frame — a recalled spatial-memory node's 3D box when object_id / place_node_id is set, else the one live lifted detection whose label matches — sent as PlaceDeclaration.search_box, and the producer refuses a patch measured outside it. Nothing about the cell is surveyed or predeclared. A reference that grounds to nothing, or to more than one instance with no node id, is refused and the goal is not sent. Naming arms nothing: the kernel only ever trusts the measured region.

Attributes:

Name	Type	Description
label	str	Open-vocabulary label of the surface, e.g. "shelf"; matched case-insensitively against the live lifted detections' labels.
object_id	str | None	A spatial-memory node_id from recall_object naming the surface's instance; None = ground by label (or place_node_id).
place_node_id	str | None	A spatial-memory place node id, for a remembered place that carries a 3D box; None = ground by label (or object_id). At most one of object_id / place_node_id.

Example

PlaceTargetRef(label="shelf").object_id is None True

PlaybookContract

Bases: BaseModel

Manifest contract for kind: "playbook" rSkills.

A playbook is a human-authored standard-operating-procedure — a structured Markdown document describing how the S2 Reasoner should approach a class of task: its trigger, the ordered decision steps it composes from existing reasoner tools, the bound on those steps, and its verifiable acceptance (done) predicate. It is content the reasoner reads, never code it executes, and carries no weights, actuators, ROS server, or Action contract — the symbolic counterpart to a vla policy.

Required when RSkillManifest.kind is "playbook"; forbidden for all other kinds (enforced by RSkillManifest._check_kind_consistency). The body_uri Markdown body is what the reasoner injects into its system prompt (or retrieves by trigger similarity at scale); the manifest description + trigger drive selection. Advisory only — a playbook changes what the LLM decides, never its actuation authority: every motion it triggers is still an ExecuteRskill → Action chunk → C++ safety kernel (CLAUDE.md §1.1).

Attributes:

Name	Type	Description
trigger	str	Natural-language description of when this playbook applies. Used as the retrieval key when more playbooks are installed than fit the system prompt (the deferred load_playbook tool).
body_uri	str	Repo-relative path to the Markdown SOP (e.g. "./PLAYBOOK.md").
composes_tools	list[str]	The ReasonerToolCall discriminators the SOP uses (execute_rskill, recall_object, locate_in_view, query_scene, query_task_progress, memory_write, memory_search …). Advisory — a manifest-level hint of the playbook's tool surface; at least one entry is required.
done_predicate	str	Natural-language acceptance test the playbook verifies before declaring success (e.g. "the target object is in the gripper").
max_steps	int	Hard bound on the number of tool calls before the playbook must terminate (no hidden default — CLAUDE.md §1.4). Must be > 0.

Example

p = PlaybookContract( ... trigger="the goal names a physical object whose location is not given", ... body_uri="./PLAYBOOK.md", ... composes_tools=["recall_object", "locate_in_view", "execute_rskill"], ... done_predicate="the target object is confirmed in view at a known pose", ... max_steps=12, ... ) p.max_steps 12

Pose6D

Bases: BaseModel

6D pose: position + quaternion.

Attributes:

Name	Type	Description
xyz	tuple[float, float, float]	Position (x, y, z) in meters.
quat_xyzw	tuple[float, float, float, float]	Quaternion (x, y, z, w).
frame_id	str	tf2 reference frame.

ProtocolSpec

Bases: BaseModel

Stand-alone eval-protocol descriptor.

Each BenchmarkScene now carries its own n_episodes / seed / task.success_key / task.max_steps; ProtocolSpec is retained as a public schema for describing a protocol independently of a suite (ADR drafts, benchmark-report tooling).

Pins the methodology so two rSkills evaluated under the same benchmark produce apples-to-apples numbers (e.g. LIBERO: 10 episodes per task, fixed seed range, is_success).

Attributes:

Name	Type	Description
n_episodes	int	Number of independent episodes per task. Honest success rates need >= 10 per task for the published LIBERO / MetaWorld protocols.
seeds	list[int]	Seed list applied per task (paired index-wise with the episode index). Length MUST be >= n_episodes; runners slice seeds[:n_episodes] so re-runs are reproducible.
success_key	str	info key on the gym step() return whose truthy value marks task success. Default "is_success" (LIBERO convention); MetaWorld uses "success".
max_steps	int	Per-task episode budget. Adapters may clip to scene-internal limits; the value here is the protocol ceiling.
min_reps	int | None	Minimum number of completed episodes per task before the result JSON is written. Guards against a crash mid-suite producing a misleadingly partial roll-up. None means "require every task x episode".

Example

p = ProtocolSpec(n_episodes=10, seeds=list(range(10)), max_steps=280) p.success_key 'is_success'

model_post_init(_context: object) -> None

Cross-field validation: seeds list must cover n_episodes.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: seeds list must cover n_episodes."""
    if len(self.seeds) < self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.seeds has {len(self.seeds)} entries but "
            f"n_episodes={self.n_episodes}; provide at least n_episodes "
            f"seeds so re-runs are reproducible."
        )
    if self.min_reps is not None and self.min_reps > self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.min_reps ({self.min_reps}) exceeds "
            f"n_episodes ({self.n_episodes}); a benchmark cannot require "
            f"more completed reps than it schedules."
        )

QuantizationBackend

Bases: str, Enum

Inference backend that will execute the quantized model.

Attributes:

Name	Type	Description
PYTORCH		torch.quantization / bitsandbytes / torchao.
ONNX		onnxruntime with quantization pre-applied at export.
TENSORRT		NVIDIA TensorRT engine (INT8 / FP8 calibrated).
GGUF		llama.cpp / ggml GGUF format (CPU and Metal).
MLX		Apple MLX framework (Apple Silicon only).

Example

QuantizationBackend.PYTORCH.value 'pytorch'

QuantizationConfig

Bases: BaseModel

Full specification of how to quantize a skill's model weights.

Attributes:

Name	Type	Description
dtype	QuantizationDtype	Target numeric format.
backend	QuantizationBackend	Inference backend that will execute the model.
per_channel	bool	Use per-channel (vs per-tensor) quantization.
calibration_dataset	str | None	HuggingFace dataset ID or local path used for post-training calibration (INT8 / TensorRT only).
extra	dict[str, object]	Backend-specific overrides (e.g. {"calibration_steps": 128}).

Example

cfg = QuantizationConfig(dtype=QuantizationDtype.INT8) cfg.backend

QuantizationDtype

Bases: str, Enum

Numeric format used to represent model weights and activations.

Attributes:

Name	Type	Description
FP32		32-bit float (full precision, reference).
FP16		16-bit float (GPU-native, common default).
BF16		Brain-float16 (Ampere+ GPUs, TPUs; better dynamic range than FP16).
INT8		8-bit integer (CPU/GPU; good accuracy/speed trade-off).
FP8		8-bit float, E4M3 (Ada/Hopper+; keeps floating-point dynamic range at INT8's memory cost, which matters for transformer activations that INT8's fixed scale clips). Carried in a TensorRT engine as explicit QuantizeLinear/DequantizeLinear node pairs in the ONNX graph, not as a builder precision flag.
INT4		4-bit integer (edge / memory-constrained; some accuracy loss).
FP4_NVFP4		NVIDIA FP4 format (Blackwell; highest throughput). Block size 16 with an FP32/FP16/BF16 scale — distinct from the MX formats, which scale per 32 elements with an E8M0 exponent.

Example

QuantizationDtype.INT8.value 'int8' QuantizationDtype.FP8.value 'fp8'

QuerySceneTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask a scene VLM a question about the live view.

Backed by a kind: "vlm" rSkill (e.g. Qwen3.5-4B NF4) running in an out-of-process ZMQ sidecar. Where LocateInViewTool answers where an object is (open-vocabulary localization via the detector), query_scene answers open-ended questions about the scene's state — task-progress and success/failure verification the reasoner needs for its replanning ladder: "has the robot grasped the mug?", "is the bowl on the shelf?", "did we drop the object?", "is the table clear?".

It captures the current frame of the requested camera, sends it plus the question to the VLM over the /openral/perception/query_scene ROS service, and feeds the free-text answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation — it only reads a frame; the dispatch never gates the safety kernel. It is not a localizer: use LocateInViewTool to find objects.

Attributes:

Name	Type	Description
tool	Literal['query_scene']	Discriminator (always "query_scene").
question	str	Natural-language question about the current scene (e.g. "Has the robot grasped the red mug?"). Sent verbatim to the VLM as the textual prompt alongside the frame.
camera	str	Optional camera selector. Empty (default) uses the perception node's primary camera; otherwise names one of its configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name.

QueryTaskProgressTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask the reward monitor how the task is going.

Backed by a kind: "reward" rSkill (Robometer-4B NF4) running in parallel with the active VLA in an out-of-process ZMQ sidecar. Where QuerySceneTool answers open-ended scene questions as free text, query_task_progress returns a quantitative windowed assessment of the current task: normalized progress and success over the last window_s seconds, plus their trends and a stalled flag.

It calls the /openral/perception/query_task_progress ROS service, which scores the monitor's buffered camera frames against the task instruction and returns progress_now / success_now / progress_trend / success_trend / stalled / succeeded. The reasoner uses it to decide whether to continue, escalate to QuerySceneTool, advance, or enter the replanning ladder. Like every variant it holds no authority over actuation — the reward signal is advisory; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['query_task_progress']	Discriminator (always "query_task_progress").
window_s	float	How many seconds of recent frames to assess. Must be > 0; clamped to the monitor's configured frame_window_s.
task	str	Optional task-instruction override. Empty (default) reuses the instruction the monitor was co-activated with (the active VLA's goal).

RSkillAction

Bases: str, Enum

Closed vocabulary of high-level action verbs an rSkill can perform.

Declared on RSkillManifest.actions so the reasoner's LLM tool palette can present each skill with a structured "what does it do" label, in addition to the free-form description and the slug. The LLM scores tools primarily on natural-language description, but a closed verb vocabulary lets the palette pre-filter and lets the schema be unit-testable. New entries are additive.

Categories (descriptive only — not part of the wire format):

• Manipulation primitives: PICK, PLACE, PICK_AND_PLACE, TRANSFER, GRASP, RELEASE.
• Articulated / contact-rich: OPEN, CLOSE, PUSH, PULL, SLIDE, INSERT, POUR, WIPE, ROTATE.
• Motion: REACH; LOOK (aim a camera at a point).
• Mobile (for mobile-manipulator embodiments): NAVIGATE.
• Social / expressive: WAVE, SHAKE.
• Generalist marker: GENERALIST for foundation / multi-task checkpoints (e.g. RoboCasa-365, DROID, MetaWorld-MT50). The palette surfaces a generalist skill for goals that don't match a specific verb.
• Perception / reasoning (non-actuating S2 kinds): DETECT (detector), QUERY (scene VLM), MONITOR (reward monitor), PLAN (playbook decision procedure). These verbs are registry/discovery metadata only — their skills are reached via read-only reasoner tools or system-prompt injection, never an ExecuteRskill dispatch.

RSkillEvalBenchmark

Bases: BaseModel

The benchmark suite a RSkillEvalResult was measured against.

Attributes:

Name	Type	Description
name	str	Canonical suite name ("LIBERO", "MetaWorld", "ALOHA", "PushT", "RoboCasa" …). Used by openral benchmark report to group rows.
dataset	str | None	Optional HF dataset identifier.
protocol	str	Free-text description of the eval protocol.
robot	str	robot_id the benchmark was evaluated on.
simulator	str	Free-text simulator description.

RSkillEvalResult

Bases: BaseModel

Pydantic model of a rskills/<id>/eval/<benchmark>.json file.

The results and baselines blocks are intentionally benchmark-specific — LIBERO has four sub-suites with success rates, MetaWorld carries an MT50 average, ALOHA has a single cube-transfer rate. We require the metadata blocks (source, benchmark, eval_config) but leave results as a free-form dict so each benchmark can declare its own keys.

Attributes:

Name	Type	Description
schema_version	str	Semver-ish version string for this on-disk format.
source	RSkillEvalSource	Provenance — paper / reproduction state.
benchmark	RSkillEvalBenchmark	Suite identity (name, robot, simulator).
eval_config	dict[str, object]	Free-form configuration the benchmark was run with (chunk size, image size, denoising steps, …).
results	dict[str, object]	Free-form per-task / per-suite success rates.
baselines	dict[str, object]	Optional free-form comparison numbers from prior work.
trace_id	str | None	Hex OTel trace id (32 chars) for the rollout that produced this result. Set by openral benchmark run / benchmark scene from the cli.command root span so reviewers can deep-link from rskills/<id>/eval/<benchmark>.json straight to the trace tree in Jaeger / Tempo. None when there is no trace to point at: paper-cited numbers (reproduced_locally: false), and local runs with no OTLP endpoint configured (no --dashboard, no OTEL_EXPORTER_OTLP_ENDPOINT), where observability is in no-op mode.

Example

RSkillEvalResult.model_validate_json(

'{"schema_version": "0.1", "source": {...}, ...}')

from_json(path: str) -> RSkillEvalResult classmethod

Load and validate a rskills/<id>/eval/<benchmark>.json file.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the JSON file.	required

Returns:

Type	Description
RSkillEvalResult	A validated RSkillEvalResult.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the JSON fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_json(cls, path: str) -> RSkillEvalResult:
    """Load and validate a ``rskills/<id>/eval/<benchmark>.json`` file.

    Args:
        path: Filesystem path to the JSON file.

    Returns:
        A validated ``RSkillEvalResult``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the JSON fails schema validation.
    """
    import json as _json  # noqa: PLC0415  # reason: deferred import

    with open(path, encoding="utf-8") as fh:
        data = _json.load(fh)
    return cls.model_validate(data)

RSkillEvalSource

Bases: BaseModel

Provenance for a benchmark result block.

Used inside RSkillEvalResult so consumers know where the numbers came from (paper vs local reproduction) without having to read the _comment line.

Attributes:

Name	Type	Description
paper	str	Plain-text title of the source paper / report.
arxiv	str | None	Optional arxiv URL or id.
model_variant	str	Which checkpoint variant the numbers describe (e.g. "SmolVLA (0.45B)").
evaluated_by	str	Free-text — "upstream authors" or a contributor.
reproduced_locally	bool	True only when OpenRAL CI / contributor re-ran the benchmark and the listed numbers match.
reproduction_planned	str | None	Optional plan for closing the gap when reproduced_locally is False.
reproduction_cli	dict[str, object] | str | None	Either a single command string or a structured {description, single_suite_example, all_suites, suite_max_steps, notes} block (matches the LIBERO eval JSON shape).
table	str | None	Optional pointer at a specific table in the source paper.
status	str | None	Optional state marker — "in_progress", "deferred", "reproduced" — for benchmarks that are not yet final.

RSkillInfo

Bases: BaseModel

Snapshot of a Skill's runtime state — published on /skill/<name>/info.

Attributes:

Name	Type	Description
name	str	rSkill name (e.g. "noop_skill").
version	str	SemVer string.
state	RSkillState	Current primary lifecycle state.
weights_loaded	bool	Whether model weights have been loaded into memory.
quantized	bool	Whether weights have been quantized for the target runtime.
warmed_up	bool	Whether the model has been warmed up (first inference run).
embodiment_tags	list[str]	Embodiment tags from the skill manifest.
role	Literal['s0', 's1', 's2']	rSkill role: "s0" (cerebellar), "s1" (fast policy), "s2" (slow reasoning).
latency_budget_ms	float | None	Maximum allowed inference latency in milliseconds.
last_inference_ms	float | None	Actual latency of the most recent step() call.
error_msg	str | None	Human-readable error description when state == "error".
stamp_ns	int	Timestamp of this snapshot in nanoseconds.

Example

info = RSkillInfo(name="hello", version="0.1.0", state=RSkillState.UNCONFIGURED) info.weights_loaded False info.state

RSkillLatencyBudget

Bases: BaseModel

Per-stage latency budget declared in rskill.yaml.

Attributes:

Name	Type	Description
per_chunk_ms	float	End-to-end step() budget. CI fails if exceeded on the reference host (CLAUDE.md §2).
warmup_ms	float | None	Maximum allowed warm-up time during activate().
load_ms	float | None	Maximum allowed weight-load time during configure().
max_execution_s	float | None	Total execution budget for one execute_rskill goal, in graph-clock seconds under use_sim_time, else wall. A VLA policy never self-terminates (only wrapped-ROS skills raise ROSRskillGoalSatisfied), so a deploy dispatch with deadline_s=0 (the LLM's "use the manifest default" sentinel) would otherwise run forever. The skill_runner resolves deadline_s<=0 to this value (or a global default) and aborts the goal when it lapses so the reasoner re-evaluates the outcome — CLAUDE.md §3 "Deadline fallback mandatory". None = fall back to the runner's global default.

RSkillLicensePosture

Bases: str, Enum

License posture surfaced at install time (CLAUDE.md §3 VLA license matrix).

RSkillManifest

Bases: BaseModel

Pydantic model of the rskill.yaml package manifest (V1).

On-disk schema for an rSkill HF Hub repo; loaded by capability-checking against a RobotDescription, then constructing a runtime Skill. Commercial-use posture is derived from license via is_commercial_use_allowed.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk format version; "0.1" today (CLAUDE.md §1.6).
name	str	HF Hub identifier <owner>/<repo>, e.g. "openral/rskill-pick-cube-so100".
version	str	SemVer string of the rSkill package (not the wrapped weights).
license	RSkillLicensePosture	License posture surfaced at install time; drives is_commercial_use_allowed.
role	Literal['s0', 's1', 's2']	rSkill slot — "s1" fast policy (the only slot loaded today), "s0" cerebellar realtime (reserved), "s2" slow reasoning (reserved).
model_family	ModelFamily | None	Closed VLA / policy family; drives the runner's adapter dispatch.
embodiment_tags	list[EmbodimentTag]	Robot embodiments targeted; must intersect the robot's tags. "custom" requires embodiment_extra.
embodiment_extra	EmbodimentExtra | None	Sensor + actuator surface for "custom"; required iff "custom" is tagged, None otherwise.
capabilities_required	dict[str, bool | float | int | str]	Per-flag RobotCapabilities requirements the target robot must satisfy.
sensors_required	list[SensorRequirement]	Sensor inputs the policy expects. See SensorRequirement.
actuators_required	list[ActuatorRequirement]	At least one ActuatorRequirement; matched against the robot's action_spec.control_mode.
runtime	RSkillRuntime	Preferred inference runtime.
quantization	QuantizationConfig	Default QuantizationConfig for this rSkill.
weights_uri	str | None	HF Hub revision-pinned URI or local path to weights; production deployments pin a SHA (CLAUDE.md operating principle 8).
chunk_size	int	Action-chunk length emitted per Skill.step.
latency_budget	RSkillLatencyBudget	Latency contract enforced by CI on the reference host (CLAUDE.md §2).
min_vram_gb	dict[QuantizationDtype, float] | None	Optional minimum VRAM (GB) per quantization dtype; informational only.
fallback_skill_id	str | None	rSkill id to substitute on failure (RFC §8.3 replanning ladder).
benchmarks	dict[BenchmarkName, float]	Canonical suite id → success rate [0.0, 1.0]; full per-task breakdown lives in the matching eval/<key>.json.
evaluated_tasks	list[str]	Benchmark task ids / families trained or validated on; gates a scene's task.id against it (empty = permissive).
sim_env_control_mode	str | None	Optional simulator controller override ("relative"/"absolute"); None = backend default.
policy_extras	dict[str, object]	Adapter-owned runtime knobs copied into VLASpec.extra at CLI composition.
paper_url	str | None	Canonical paper URL for this skill / family.
dataset_uri	str | None	HF Hub URI for the training dataset.
source_repo	str | None	HF Hub URI for the upstream weights repo.
description	str	REQUIRED. Short (1-500 char) summary surfaced to operators and the reasoner's tool palette.
default_prompt	str | None	Optional verbatim task string the checkpoint was trained on; fallback when the goal's prompt is empty.
actions	list[RSkillAction]	REQUIRED. Closed-vocabulary action verbs (≥1); see RSkillAction.
objects	list[str]	Optional free-form object keywords (discriminative hints for the LLM).
scenes	list[str]	Optional free-form scene / environment keywords.
processors	RSkillProcessors | None	Per-file lerobot PolicyProcessorPipeline URIs; required for modern families (smolvla/pi05/xvla/diffusion/rldx).
image_preprocessing	ImagePreprocessing | None	Per-checkpoint image knobs (flip, camera renames); None = schema defaults.
state_contract	StateContract | None	Per-checkpoint proprioception layout.
action_contract	ActionContract | None	Per-checkpoint action vector contract (dim + optional representation).
n_action_steps	int | None	Replay cadence; omit when equal to chunk_size.

Example

m = RSkillManifest( ... name="openral/rskill-pick-cube-so100", ... version="0.1.0", ... license=RSkillLicensePosture.APACHE_2_0, ... role="s1", ... kind="vla", ... model_family="smolvla", ... embodiment_tags=["so100_follower"], ... runtime=RSkillRuntime.PYTORCH, ... weights_uri="hf://lerobot/smolvla_base", ... chunk_size=16, ... latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0), ... description="Pick a cube on the SO-100 follower arm.", ... actions=[RSkillAction.PICK], ... objects=["cube"], ... scenes=["tabletop"], ... actuators_required=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ), ... ], ... processors=RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json", ... ), ... ) m.is_commercial_use_allowed True m.schema_version '0.1'

is_commercial_use_allowed: bool property

Derive commercial-use posture from license.

apache-2.0 / mit / bsd → True. Every other posture (including unknown) → False, conservatively. The loader consults this in rSkill._check_license; users wanting to deploy a non-commercial skill in a research context set OPENRAL_ALLOW_NONCOMMERCIAL=1 per CLAUDE.md §1.9.

is_scaffold_placeholder: bool property

True when this is an unresolved rskills/template/ scaffold.

Checks the identity fields (name, weights_uri, source_repo) for the RSKILL_TEMPLATE_SENTINELS. The scaffold parses as a valid manifest (so tests can load it) but is not a real, loadable skill: the reasoner palette must not offer it as dispatchable, and the publish gate rejects it. One predicate so callers don't re-derive the sentinel check.

active_min_vram_gb() -> float | None

Declared VRAM (GB) for this skill at its active quantization dtype.

Reads min_vram_gb[quantization.dtype] — the footprint the skill will actually use at load, since quantization.dtype pins the runtime format . Returns None when min_vram_gb is unset or has no entry for the active dtype (the size is simply not declared — the caller decides whether that is an error). See assert_vla_reward_fits for the pair check that consumes this.

Source code in python/core/src/openral_core/schemas.py

def active_min_vram_gb(self) -> float | None:
    """Declared VRAM (GB) for this skill at its **active** quantization dtype.

    Reads ``min_vram_gb[quantization.dtype]`` — the footprint the skill will
    actually use at load, since ``quantization.dtype`` pins the runtime format
    . Returns ``None`` when ``min_vram_gb`` is unset or has no entry
    for the active dtype (the size is simply not declared — the caller decides
    whether that is an error). See ``assert_vla_reward_fits`` for the pair
    check that consumes this.
    """
    if self.min_vram_gb is None:
        return None
    return self.min_vram_gb.get(self.quantization.dtype)

from_yaml(path: str) -> RSkillManifest classmethod

Load and validate an rskill.yaml from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the manifest YAML.	required

Returns:

Type	Description
RSkillManifest	A validated RSkillManifest.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RSkillManifest:
    """Load and validate an ``rskill.yaml`` from disk.

    Args:
        path: Filesystem path to the manifest YAML.

    Returns:
        A validated ``RSkillManifest``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
    """
    return _load_yaml_model(cls, path)

RSkillProcessors

Bases: BaseModel

Explicit lerobot PolicyProcessorPipeline artefact pointers.

Closes Gap 1 + Gap 3 of the rSkill self-containment audit. Replaces the implicit snapshot_download(repo_id) → make_pre_post_processors(pretrained_path=...) fetch path used today by the SmolVLA / pi05 / xVLA / Diffusion / modern-ACT adapters. Two files are required because lerobot's pipeline ships them as a pair: a preprocessor that normalises the policy input batch and a postprocessor that un-normalises the action chunk.

Each URI must point at a specific file (the implicit-snapshot shape hf://owner/repo with no file tail is rejected). Two URI schemes accepted:

• hf://owner/repo[@rev]/path/to/file.json — file inside an HF Hub repo, revision-pinnable.

Attributes:

Name	Type	Description
preprocessor_uri	str	Per-file URI to the policy preprocessor JSON.
postprocessor_uri	str	Per-file URI to the policy postprocessor JSON.

Example

RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_libero/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_libero/policy_postprocessor.json", ... ) # doctest: +ELLIPSIS RSkillProcessors(preprocessor_uri='hf://...preprocessor.json', postprocessor_uri='hf://...postprocessor.json')

RSkillRuntime

Bases: str, Enum

Inference runtime hint declared in the manifest (RFC §7.3).

RSkillState

Bases: str, Enum

Primary lifecycle states for a Skill node.

Matches the ROS 2 Managed Node state machine with openral extensions.

States

unconfigured Initial state. Weights are not loaded. inactive Configured and weights loaded; not yet warmed up. Accepts no actions. active Ready to execute. step() may be called. finalized Terminal state after shutdown(). Cannot transition further. error Unrecoverable failure. Requires external intervention.

ReasonerEndpointPreset

Bases: NamedTuple

Everything a named endpoint implies beyond its URL.

dialect makes OPENRAL_REASONER_DIALECT unnecessary for a named endpoint — a preset knows its own wire format, and the escape hatch only has to ask when the operator supplies a bare URL nobody can classify.

Lives in openral_core next to REASONER_MODELS so the factory (openral_reasoner.tool_use) and openral doctor consume the SAME table — doctor used to hand-mirror it to avoid importing the optional reasoner package, and the mirror drifted twice.

ReasonerModel

Bases: BaseModel

A curated reasoner LLM — a member of REASONER_MODELS (ADR-0088).

Membership in REASONER_MODELS is the "tested for robotics tool calling" signal: every entry has been validated to reliably emit the discriminated ReasonerToolCall union (per-skill execute_rskill__* tools, nested GroundedSubtask, tool_choice honoured). There is deliberately no robotics_tool_calling flag — an untested model is simply not in the registry, and is reachable only through the uncurated escape hatch with a warning.

This model reorganises reasoner selection away from the location-leaking OPENRAL_REASONER_LLM_PROVIDER enum it replaced (ADR-0088; that spelling was removed in 0.3.0): the primary choice is the model; where it runs is the orthogonal, honest endpoint axis.

Attributes:

Name	Type	Description
id	str	Catalog key ("cosmos3-edge", "claude-opus-4-8"). The value of OPENRAL_REASONER_MODEL.
display_name	str	Human-facing label.
dialect	ReasonerDialect	Which wire client speaks to this model (anthropic uses the Messages API; openai uses Chat Completions — vLLM, OpenRouter, the Cosmos sidecar, etc. are all openai).
hosting	ReasonerHosting	cloud (vendor-run), managed_local (OpenRAL spawns the server), or byo_local (operator-run local server).
served_model_id	str	The id the endpoint expects, which may differ from id ("openai/gpt-5.5" on OpenRouter, "nvidia/Cosmos3-Edge" on the sidecar).
default_endpoint	str | None	Vendor URL, loopback URL, the REASONER_MANAGED_ENDPOINT sentinel, or None (the dialect's vendor default — e.g. the Anthropic API).
auth_required	bool	Whether the resolved endpoint needs an API key.
tool_choice	Literal['required', 'auto']	"required" forces one tool call per tick (the reasoner contract); "auto" for endpoints that reject "required".
max_tokens_default	int | None	Optional completion-token cap applied when the operator sets none — bounds a metered gateway's up-front reservation for reasoning models (GPT-5.x). None sends no cap.
min_gpu_vram_gb	float | None	BF16-resident VRAM floor for a comfortable local fit. None for cloud models (not a local-compute question).
required_dtype	QuantizationDtype | None	Dtype the local runtime must support, matched against ComputeSpec.gpu_supported_dtypes. None for cloud models.
weights_license	str | None	Informational weights-license posture for local models (§1.9), e.g. "OpenMDW-1.1". None for cloud API models (OpenRAL never holds their weights).

Example

m = REASONER_MODELS["cosmos3-edge"] m.dialect, m.hosting, m.auth_required ('openai', 'managed_local', False) REASONER_MODELS["claude-opus-4-8"].dialect 'anthropic'

is_local: bool property

True for managed_local / byo_local (needs local compute).

ReasonerTimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_REASONER_TIMEOUT — the LLM call missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['reasoner_timeout']	Discriminator (always "reasoner_timeout").
model	str	Model identifier (e.g. "claude-opus-4-7").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

RecallObjectMatch

Bases: BaseModel

One ranked match from a RecallObjectQuery.

Attributes:

Name	Type	Description
node_id	str	The matched node's id.
label	str	The matched node's label.
pose	Pose6D	The object's recalled pose (map frame).
score	float	Match score in [0, 1].
last_seen_ns	int	When the object was last observed, in nanoseconds.
approach	ApproachViewpoint | None	Optional computed camera-facing approach viewpoint.
inside_container_id	str | None	Set when the object is inside an occluding container that must be opened first; None otherwise.

RecallObjectQuery

Bases: BaseModel

Read-only query to recall a remembered object.

At least one of text / label must be non-empty. text is matched against node embeddings when an embedder is configured, otherwise matching falls back to label.

Attributes:

Name	Type	Description
text	str	Free-text query (open-vocabulary match).
label	str	Exact label match.
near	Pose6D | None	Optional pose to bias results toward (proximity).
max_age_ns	int | None	Optional recency filter — drop nodes whose last_seen_ns is older than this many nanoseconds before "now".
limit	int	Maximum number of matches to return.

RecallObjectResult

Bases: BaseModel

Result of a RecallObjectQuery.

Attributes:

Name	Type	Description
matches	list[RecallObjectMatch]	Ranked matches (possibly empty — an empty result is how the query reports "unknown"; callers raise / handle ROSObjectNotInMemory rather than fabricating a pose).

RecallObjectTool

Bases: _ReasonerToolBase

Tool variant (read-only) — recall a remembered object.

Queries the scene-graph spatial memory and returns the object's map-frame pose plus a camera-facing approach viewpoint and any occluding container to the reasoner's next reasoning step. Like every variant it holds no authority over actuation — it only reads memory. The dispatch that runs the query and feeds the result back to the LLM is wired separately (this is the typed contract).

Attributes:

Name	Type	Description
tool	Literal['recall_object']	Discriminator (always "recall_object").
query	str	Free-text or label naming the object to recall (e.g. "the red mug"). Mapped to a RecallObjectQuery at dispatch.
limit	int	Maximum number of ranked matches to return.

ReloadGstPipelineTool

Bases: _ReasonerToolBase

Tool variant — swap a sensor's GStreamer pipeline at runtime.

Dispatch: service call on /openral/sensors/<sensor_id>/reload_pipeline with the pipeline YAML payload. Lets the reasoner tune perception (switch resolution, enable an nvinfer leg, swap an RTSP source) without redeploying the runtime.

Attributes:

Name	Type	Description
tool	Literal['reload_gst_pipeline']	Discriminator (always "reload_gst_pipeline").
sensor_id	str	SensorSpec.name of the camera whose pipeline is being reloaded. Validated against the active runtime's sensor catalog at dispatch time.
pipeline_yaml	str	Full YAML body of the new SensorReaderConfig. The sensor node validates this against the Pydantic schema before accepting the swap.

ResolvePlaceQuery

Bases: BaseModel

Read-only query to resolve a place/room/agent reference to a goal.

Attributes:

Name	Type	Description
reference	str	Free-text, id, or label of the target (e.g. "kitchen", "where I was standing", a node id).
kind	SpatialNodeKind | None	Optional node-kind filter (room / place / agent).

ResolvePlaceResult

Bases: BaseModel

Result of a ResolvePlaceQuery.

Attributes:

Name	Type	Description
node_id	str	The resolved node's id.
goal	Pose6D	Navigation goal pose (map frame).
path_node_ids	list[str]	Ordered traversable_to path of node ids from the robot's current place to the goal (empty when no path is needed or known).

ResolvePlaceTool

Bases: _ReasonerToolBase

Tool variant (read-only) — resolve a place/room/agent to a goal.

Queries the scene-graph memory for a navigation goal pose plus a traversable_to path. Read-only; holds no authority over actuation. Dispatch + result-return are wired separately.

Attributes:

Name	Type	Description
tool	Literal['resolve_place']	Discriminator (always "resolve_place").
reference	str	Free-text, id, or label of the target (e.g. "the kitchen", "where I was standing"). Mapped to a ResolvePlaceQuery.

RewardContract

Bases: BaseModel

Manifest contract for kind: "reward" rSkills.

Carries the configuration a robotic reward / progress-monitor model (e.g. Robometer-4B, a Qwen3-VL-4B reward foundation model) needs to score a rollout. A reward skill runs in parallel with a kind: "vla" policy, continuously ingesting the VLA's camera frames into a rolling window, and emits per-frame progress ∈ progress_range and per-frame success probability. The Reasoner queries it on demand (QueryTaskProgressTool) to decide whether to continue, escalate to a scene VLM, advance, or replan. The signal is advisory only — it never actuates and never gates motors (CLAUDE.md §1.1).

Required when RSkillManifest.kind is "reward"; forbidden for all other kinds (enforced by RSkillManifest._check_kind_consistency). Like detector / vlm, a reward skill is a pure perception consumer: it emits no Action chunks and requires no actuators.

Attributes:

Name	Type	Description
progress_range	tuple[float, float]	(min, max) of the normalized per-frame progress scalar. Default (0.0, 1.0) — Robometer's discrete/binned mode (see num_bins) emits progress already in [0, 1].
success_threshold	float	Per-frame success probability at/above which the frame is considered a task success. In [0.0, 1.0]; default 0.5.
preference	bool	Whether the model also exposes a trajectory-preference head (Robometer does). Default False — the progress/success path is the Reasoner-facing contract; preference is future work.
backend	Literal['robometer', 'topreward']	Which reward runtime scores this skill. "robometer" (default) → the fine-tuned Robometer RewardModel in-process; "topreward" → the zero-shot TOPReward monitor (lerobot TOPRewardModel, per-frame progress from a prefix sweep of P("True" | video, instruction)), run in-process. Selects the backend in build_reward_monitor; existing manifests default to "robometer" so behavior is unchanged.
frame_window_s	float	Length of the rolling frame buffer in seconds. The node evicts frames older than this relative to the newest. Must be > 0.
target_fps	float	Frame-sampling rate fed to the model (Robometer's example uses 3 fps). Must be > 0. This is an S2-cadence monitor, not a per-control-step signal.
num_bins	int	Discrete-mode bin count for the progress head. Robometer's discrete mode yields per-frame normalized progress in [0, 1]; continuous mode yields raw regression values. Must be > 0; default 100.
instruction_required	bool	Whether a natural-language task instruction must accompany the frames (Robometer requires one). Default True.
check_floor	float	Below this progress value the attempt is clearly not done → no VLM adjudication, straight to the replanning ladder. Must be ≤ success_threshold; default 0.4 (calibrated for robometer's loose/general eval distribution that wanders ~0.55-0.78).
plateau_window_s	float	Trailing window (seconds) over which progress_trend ≈ 0 indicates the policy has stopped getting closer to the goal. Default 3.0.
plateau_tolerance	float	ε band absorbing critic noise when deciding whether the trend is flat. Default 0.05.
default_patience_s	float	Baseline execution ceiling (seconds) used as a backstop before the Reasoner escalates. Default 30.0.

Example

c = RewardContract(frame_window_s=8.0, target_fps=3.0) c.progress_range (0.0, 1.0) c.success_threshold 0.5 c.check_floor 0.4

RobotCapabilities

Bases: BaseModel

Physical capability flags for a robot body.

Describes what the robot can do (sensors, actuation, embodiment tags). Runtime compute / GPU properties live on ComputeSpec and are attached to the RobotDescription via its compute field.

Attributes:

Name	Type	Description
locomotion	list[LocomotionKind]	Locomotion types available.
can_lift_kg	float	Maximum payload in kg.
has_dexterous_hands	bool	Whether dexterous hands are present.
has_tactile	bool	Whether tactile sensing is available.
has_force_control	bool	Whether force/impedance control is supported.
has_vision	bool	Whether camera(s) are present.
has_lidar	bool	Whether LiDAR is present. Gates the 2D-lidar slam_toolbox backend.
has_vision_slam	bool	Whether the robot should run camera-based visual SLAM (cuVSLAM + nvblox) for localization/mapping, for robots that lack a lidar. Gates the visual SLAM backend. Independent of has_lidar; when both are set the lidar backend wins (it does not require an AI depth model).
has_audio	bool	Whether audio I/O is present.
bimanual	bool	Whether the robot has two arms.
supported_control_modes	list[ControlMode]	List of supported ControlMode values.
supported_vla_embodiments	list[str]	VLA embodiment IDs this robot can run.
embodiment_tags	list[str]	Short tags mapping to VLA heads / dataset splits.

RobotDescription

Bases: BaseModel

Top-level robot manifest — one per robot, published to HuggingFace Hub.

Attributes:

Name	Type	Description
name	str	Robot name, e.g. "so100_follower".
embodiment_kind	EmbodimentKind	Top-level kinematic class.
assets	AssetRefs	Unified URDF / MJCF / SRDF reference block (openral_core.assets.resolve_asset grammar). Empty by default.
base_frame	str	Base link tf2 frame name.
odom_frame	str	Odometry tf2 frame name.
map_frame	str	Map tf2 frame name.
joints	list[JointSpec]	List of joint specifications.
end_effectors	list[EndEffectorSpec]	List of end-effector specifications.
sensors	list[SensorSpec]	List of individual sensor specs.
sensor_bundles	list[SensorBundle]	List of multi-modal sensor bundles.
capabilities	RobotCapabilities	Capability flags for skill matching.
safety	SafetyEnvelope	Safety envelope constraints.
ros2_namespace	str	ROS 2 namespace prefix.
middleware	Literal['fastdds', 'cyclonedds', 'zenoh']	ROS 2 middleware selection.
onboard_compute	dict[str, object]	Onboard compute descriptors.
sdk_kind	Literal['open', 'closed_with_api', 'closed']	Whether the SDK is open or closed.
hal	HalEntrypoints	Simulation + real-hardware HAL import strings (deploy sim uses hal.sim, deploy run uses hal.real).
observation_spec	ObservationSpec | None	VLA observation configuration.
action_spec	ActionSpec | None	VLA action configuration.
sim	SimDescription | None	Optional MuJoCo wiring for openral_hal.MujocoArmHAL; lets the HAL build entirely from the manifest.
scene_defaults	SceneDefaults | None	Optional per-robot scene rendering defaults (TopCameraDefaults).
base_joints	list[str] | None	Optional ordered [forward, side, yaw] JointSpec.name refs for a planar mobile base; None for fixed-base manipulators.
collision_geometry	list[LinkCollisionGeometry]	Per-link convex collision primitives (capsules / spheres) for the safety kernel. Empty by default.
allowed_collision_pairs	list[tuple[str, str]]	Link-name pairs excluded from self-collision (from the SRDF disable_collisions block on real robots).
compute_edge	ComputeSpec | None	Compute spec for the on-robot accelerator; falls back to compute_local when None.
compute_local	ComputeSpec | None	Compute spec for the attached workstation / laptop.
compute_cloud	ComputeSpec | None	Optional remote compute endpoint (SSH or HTTPS).
schema_version	Literal['0.2']	On-disk schema version; only "0.2" loads. Any other version (including "0.1") is refused with a ROSConfigError.

Example

desc = RobotDescription( ... name="smoke_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["smoke"], ... ), ... safety=SafetyEnvelope(), ... ) assert desc.name == "smoke_robot"

control_rate_hz: float | None property

The robot's declared control rate, action_spec.control_freq_hz, or None.

The one place a robot's rate lives: the skill runner ticks at it, the HAL node publishes proprio at it, the real ros2_control HAL derives every trajectory deadline from it, and the recorder stamps it as fps.

Example

RobotDescription.from_yaml("robots/openarm/robot.yaml").control_rate_hz 30.0

lidar_sensor: SensorSpec | None property

The first declared 2-D LiDAR / scan sensor, or None.

The panda_mobile HAL synthesises a sensor_msgs/ LaserScan from MuJoCo ray-casts; its beam count (SensorSpec.n_channels), range (SensorSpec.range_min_m / SensorSpec.range_max_m) and rate (SensorSpec.rate_hz) live on this descriptor so openral deploy sim and the HAL lifecycle node read one source of truth from robot.yaml instead of each hardcoding scan params.

from_yaml(path: str) -> RobotDescription classmethod

Load and validate a RobotDescription YAML manifest from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the robot.yaml file.	required

Returns:

Type	Description
RobotDescription	A validated RobotDescription.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Example

RobotDescription.from_yaml("robots/franka_panda/robot.yaml")

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotDescription:
    """Load and validate a ``RobotDescription`` YAML manifest from disk.

    Args:
        path: Filesystem path to the ``robot.yaml`` file.

    Returns:
        A validated ``RobotDescription``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.

    Example:
        >>> # RobotDescription.from_yaml("robots/franka_panda/robot.yaml")
    """
    return _load_yaml_model(cls, path)

grasp_contact_links() -> tuple[str, ...]

The links a grasp may touch: every role: gripper jaw, follower fingers included.

The role: gripper joints' child_link first (the leader jaws the HAL names in a GraspDeclaration), then the child_link of every mimic_joints row following one of those joints — the other jaw of the same gripper, which the kernel must exempt in the grasp region too (ADR-0115). The launch passes this as the kernel's grasp_contact_links.

Example

d = RobotDescription.from_yaml("robots/openarm/robot.yaml") d.grasp_contact_links()[:2] ('openarm_left_finger_pair', 'openarm_right_finger_pair')

Source code in python/core/src/openral_core/schemas.py

def grasp_contact_links(self) -> tuple[str, ...]:
    """The links a grasp may touch: every ``role: gripper`` jaw, follower fingers included.

    The ``role: gripper`` joints' ``child_link`` first (the leader jaws the
    HAL names in a ``GraspDeclaration``), then the ``child_link`` of every
    ``mimic_joints`` row following one of those joints — the other jaw of
    the same gripper, which the kernel must exempt in the grasp region too
    (ADR-0115). The launch passes this as the kernel's ``grasp_contact_links``.

    Example:
        >>> d = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> d.grasp_contact_links()[:2]
        ('openarm_left_finger_pair', 'openarm_right_finger_pair')
    """
    grippers = [j for j in self.joints if j.role == "gripper"]
    names = {j.name for j in grippers}
    return tuple(j.child_link for j in grippers) + tuple(
        m.child_link for m in self.mimic_joints if m.mimics in names
    )

hand_links() -> tuple[str, ...]

The robot's hand links: the jaws, the link carrying them, and the wrist behind it.

Every role: gripper jaw and mimic follower (grasp_contact_links), then — once per hand — the mount the hand's leader jaws hang off (a chained finger climbed to its arm, as in gripper_hands): the link that carries the gripper; then the link that carrier hangs off, the wrist (never a root link: a gripper mounted on the robot's base adds nothing). The launch passes this as the safety kernel's workspace_contact_links (issue #349): the only links the manipulation-workspace contact rule ever relaxes. The wrist is in because a hand working at a tote's rim brings it within one voxel of the rim (8/8 deploy-sim stops, r23: PhysX 10 mm clear, the 15 mm map cells read -4 mm); the forearm and everything behind it never are. Derived from the manifest, never configured by name.

Example

d = RobotDescription.from_yaml("robots/openarm/robot.yaml") d.hand_links()[-2:] ('openarm_left_link6', 'openarm_right_link6')

Source code in python/core/src/openral_core/schemas.py

def hand_links(self) -> tuple[str, ...]:
    """The robot's hand links: the jaws, the link carrying them, and the wrist behind it.

    Every ``role: gripper`` jaw and mimic follower (``grasp_contact_links``), then —
    once per hand — the mount the hand's leader jaws hang off (a chained finger
    climbed to its arm, as in ``gripper_hands``): the link that carries the gripper;
    then the link that carrier hangs off, the wrist (never a root link: a gripper
    mounted on the robot's base adds nothing). The launch passes this as the safety
    kernel's ``workspace_contact_links`` (issue #349): the only links the
    manipulation-workspace contact rule ever relaxes. The wrist is in because a hand
    working at a tote's rim brings it within one voxel of the rim (8/8 deploy-sim
    stops, r23: PhysX 10 mm clear, the 15 mm map cells read -4 mm); the forearm and
    everything behind it never are. Derived from the manifest, never configured by
    name.

    Example:
        >>> d = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> d.hand_links()[-2:]
        ('openarm_left_link6', 'openarm_right_link6')
    """
    grippers = [j for j in self.joints if j.role == "gripper"]
    parent_of = {j.child_link: j.parent_link for j in grippers}
    joint_parent = {j.child_link: j.parent_link for j in self.joints}
    links = list(self.grasp_contact_links())
    carriers: list[str] = []
    for j in grippers:
        carrier = _gripper_mount(parent_of, j.parent_link)
        if carrier not in links:
            links.append(carrier)
            carriers.append(carrier)
    for carrier in carriers:
        wrist = joint_parent.get(carrier)
        # A root link (no joint of its own) is the robot's base, never a wrist.
        if wrist is not None and wrist in joint_parent and wrist not in links:
            links.append(wrist)
    return tuple(links)

nav2_footprint_param() -> str

This base's Nav2 footprint parameter string.

nav2_costmap_2d reads footprint as a string of [[x, y], ...] vertices and treats both "" and "[]" as "no polygon — use robot_radius". So a robot that declares footprint_polygon gets its measured outline, and one that declares only footprint_radius gets "[]", which keeps Nav2 on its cheaper circular path rather than silently inheriting the base param file's polygon (some other robot's shape).

A polygon matters beyond accuracy: only a polygon footprint can express a carried payload, which is what openral_nav2_bringup's footprint publisher pushes onto the costmaps' footprint topic while an object is attached.

Example

from openral_core import RobotDescription RobotDescription.from_yaml("robots/panda_mobile/robot.yaml").nav2_footprint_param() '[[0.35, 0.25], [-0.35, 0.25], [-0.35, -0.25], [0.35, -0.25]]'

Source code in python/core/src/openral_core/schemas.py

def nav2_footprint_param(self) -> str:
    """This base's Nav2 ``footprint`` parameter string.

    ``nav2_costmap_2d`` reads ``footprint`` as a string of
    ``[[x, y], ...]`` vertices and treats **both** ``""`` and ``"[]"``
    as "no polygon — use ``robot_radius``". So a robot that declares
    ``footprint_polygon`` gets its measured outline, and one that
    declares only ``footprint_radius`` gets ``"[]"``, which keeps
    Nav2 on its cheaper circular path rather than silently inheriting
    the base param file's polygon (some other robot's shape).

    A polygon matters beyond accuracy: only a polygon footprint can
    express a carried payload, which is what
    ``openral_nav2_bringup``'s footprint publisher pushes onto the
    costmaps' ``footprint`` topic while an object is attached.

    Example:
        >>> from openral_core import RobotDescription
        >>> RobotDescription.from_yaml("robots/panda_mobile/robot.yaml").nav2_footprint_param()
        '[[0.35, 0.25], [-0.35, 0.25], [-0.35, -0.25], [0.35, -0.25]]'
    """
    if not self.footprint_polygon:
        return "[]"
    inner = ", ".join(f"[{x:g}, {y:g}]" for x, y in self.footprint_polygon)
    return f"[{inner}]"

nav2_param_overrides() -> dict[str, str]

Nav2 param substitutions derived from this robot's base props.

Lets the Nav2 bringup stay generic: instead of hand-vendoring a per-robot nav2_<robot>.yaml, the launch rewrites a shared base param file with these key→value substitutions. Maps footprint_radius → robot_radius (collision envelope), the costmap inflation_radius (the circumscribed radius — footprint_radius or the farthest footprint_polygon vertex, whichever is larger — plus NAV2_INFLATION_CLEARANCE_M, so it stays ≥ the circumscribed radius Nav2 derives from the footprint it actually uses — otherwise Nav2 errors and falls back to slow full-footprint collision checks), footprint_polygon → the costmap footprint polygon (see nav2_footprint_param), and base_kinematics → the MPPI motion_model. Returns an empty dict for fixed-base arms (no mobile base → no Nav2). Velocity limits remain Nav2 tuning in the base param file; nav2.launch.py clamps them to safety.max_base_* separately (it needs the base file's values).

Source code in python/core/src/openral_core/schemas.py

def nav2_param_overrides(self) -> dict[str, str]:
    """Nav2 param substitutions derived from this robot's base props.

    Lets the Nav2 bringup stay generic: instead of
    hand-vendoring a per-robot ``nav2_<robot>.yaml``, the launch
    rewrites a shared base param file with these key→value
    substitutions. Maps ``footprint_radius`` → ``robot_radius``
    (collision envelope), the costmap ``inflation_radius`` (the
    circumscribed radius — ``footprint_radius`` or the farthest
    ``footprint_polygon`` vertex, whichever is larger — plus
    ``NAV2_INFLATION_CLEARANCE_M``, so it stays ≥ the circumscribed
    radius Nav2 derives from the footprint it actually uses —
    otherwise Nav2 errors and falls back to slow full-footprint
    collision checks), ``footprint_polygon`` →
    the costmap ``footprint`` polygon (see
    ``nav2_footprint_param``), and ``base_kinematics`` →
    the MPPI ``motion_model``. Returns an empty dict for fixed-base
    arms (no mobile base → no Nav2). Velocity limits remain Nav2
    tuning in the base param file; ``nav2.launch.py`` clamps them to
    ``safety.max_base_*`` separately (it needs the base file's values).
    """
    overrides: dict[str, str] = {}
    if self.footprint_radius is not None or self.footprint_polygon:
        # Emitted even for a radius-only robot, as Nav2's "[]" sentinel:
        # leaving the key unrewritten would hand that robot whatever
        # polygon the shared base file happens to ship.
        overrides["footprint"] = self.nav2_footprint_param()
        circumscribed = max(
            [self.footprint_radius or 0.0]
            + [math.hypot(x, y) for x, y in self.footprint_polygon or []]
        )
        overrides["inflation_radius"] = f"{circumscribed + NAV2_INFLATION_CLEARANCE_M:.3f}"
    if self.footprint_radius is not None:
        overrides["robot_radius"] = str(self.footprint_radius)
    if self.base_kinematics is not None:
        overrides["motion_model"] = {
            "omni": "Omni",
            "holonomic": "Omni",
            "differential": "DiffDrive",
            "ackermann": "Ackermann",
        }[self.base_kinematics]
    return overrides

validate_for_e2e_pipeline() -> None

Assert this manifest carries every field the e2e ROS graph needs.

The C++ safety kernel (cpp/openral_safety_kernel) reads per-joint position_limits / velocity_limit / effort_limit + the global safety: block. Per-joint limit fields are optional on JointSpec for sim-only robots, but they become mandatory the moment you ask openral deploy sim to bring up the kernel against this robot.

This method makes the e2e contract explicit and fails loud listing every missing field. Pair it with openral_safety.envelope_loader.compute_intersection(robot, skill=None) for the actual envelope synthesis — this method only validates.

Raises:

Type	Description
ROSConfigError	If any actuated joint is missing position_limits, velocity_limit, or effort_limit. The error lists every missing field at once so the operator does not have to fix one, re-run, fix the next.

Source code in python/core/src/openral_core/schemas.py

def validate_for_e2e_pipeline(self) -> None:
    """Assert this manifest carries every field the e2e ROS graph needs.

    The C++ safety kernel (``cpp/openral_safety_kernel``)
    reads per-joint ``position_limits`` / ``velocity_limit`` /
    ``effort_limit`` + the global ``safety:`` block. Per-joint
    limit fields are *optional* on ``JointSpec`` for sim-only
    robots, but they become mandatory the moment you ask
    ``openral deploy sim`` to bring up the kernel against this robot.

    This method makes the e2e contract explicit and fails loud
    listing every missing field. Pair it with
    ``openral_safety.envelope_loader.compute_intersection(robot,
    skill=None)`` for the actual envelope synthesis — this method
    only validates.

    Raises:
        ROSConfigError: If any actuated joint is missing
            ``position_limits``, ``velocity_limit``, or
            ``effort_limit``. The error lists every missing field
            at once so the operator does not have to fix one,
            re-run, fix the next.
    """
    missing: list[str] = []
    for joint in self.joints:
        if joint.joint_type not in (
            JointType.REVOLUTE,
            JointType.PRISMATIC,
            JointType.CONTINUOUS,
        ):
            continue
        if joint.position_limits is None:
            missing.append(f"joints[{joint.name}].position_limits")
        if joint.velocity_limit is None:
            missing.append(f"joints[{joint.name}].velocity_limit")
        if joint.effort_limit is None:
            missing.append(f"joints[{joint.name}].effort_limit")
    if missing:
        raise ROSConfigError(
            f"RobotDescription({self.name!r}) is missing fields required by "
            f"the e2e safety kernel: {missing}. Set them on the matching "
            "JointSpec(s) in robots/<robot_id>/robot.yaml."
        )

RobotUnit

Bases: BaseModel

One physical unit / host of a robot type: robots/<robot_id>/units/<unit>.yaml.

Selected by $OPENRAL_ROBOT_UNIT or DeployScene.robot_unit; see resolve_sensor_overlays.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk schema version.
robot_id	str	The robots/<robot_id> directory this unit belongs to.
unit	str	The unit name, equal to the file stem.
sensors	list[SensorOverlay]	Overlays for the robot manifest's sensors.

Example

RobotUnit(robot_id="so101_follower", unit="bench_laptop").sensors []

from_yaml(path: str) -> RobotUnit classmethod

Load and validate a unit overlay YAML file.

Example

RobotUnit.from_yaml("robots/openarm/units/orin.yaml").unit 'orin'

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotUnit:
    """Load and validate a unit overlay YAML file.

    Example:
        >>> RobotUnit.from_yaml("robots/openarm/units/orin.yaml").unit
        'orin'
    """
    return _load_yaml_model(cls, path)

RosIntegration

Bases: BaseModel

Wiring for an rSkill that wraps an existing ROS 2 action or service.

Populated when RSkillManifest.kind is "ros_action" or "ros_service". The ROSActionRskill adapter reads this block at configure time to build the right action / service client on the host LifecycleNode.

Attributes:

Name	Type	Description
package	str	IDL package name (e.g. "moveit_msgs", "nav2_msgs"). Imported lazily so manifests for un-installed ROS packages still parse — the loader raises ROSConfigError only when the skill is actually resolved.
interface_type	str	Action / service type name inside package (e.g. "MoveGroup", "NavigateToPose").
interface_name	str	Fully-qualified ROS path of the running server (e.g. "/move_action", "/navigate_to_pose").
result_trajectory_field	str | None	Dotted accessor pointing at a trajectory_msgs/JointTrajectory inside the action result (e.g. "planned_trajectory.joint_trajectory" for MoveIt's MoveGroup). When set, the adapter replays one waypoint per step call onto /openral/candidate_action (so the safety supervisor + HAL see every position). When None the action is treated as result-only: the adapter awaits the action result, raises ROSRskillGoalSatisfied on success, and never emits an Action chunk. This second mode covers wrapped ROS packages that drive actuators on their own (Nav2's behaviour tree publishes cmd_vel directly).
default_goal_json	str	JSON dict literal used to construct the goal message. v1 hard-codes the target here — the structured-prompt path that lowers LLM-emitted JSON into ExecuteRskill.Goal.prompt_metadata_json is a follow-up. REQUIRED so that a wrapped skill is always invocable end-to-end without per-call schema work.
ros_dependencies	list[str]	Apt / colcon packages the operator must have installed for the wrapped server to be reachable (e.g. "ros-${ROS_DISTRO}-moveit"). Surfaced by ral skill check and quoted in ROSConfigError messages when the action client fails to connect.

Example

ri = RosIntegration( ... package="moveit_msgs", ... interface_type="MoveGroup", ... interface_name="/move_action", ... result_trajectory_field="planned_trajectory.joint_trajectory", ... default_goal_json='{"target_joint_positions": [0, 0, 0, 0, 0, 0, 0]}', ... ros_dependencies=["ros-${ROS_DISTRO}-moveit"], ... ) ri.package 'moveit_msgs'

goal_builder: Literal['joint', 'pose', 'look_at'] | None = None class-attribute instance-attribute

Optional goal-lowering adapter over the shared ROSActionRskill MoveGroup engine. None (the default) sends default_goal_json + LLM overrides verbatim (the raw-IDL escape hatch). The named builders consume a typed block from the merged goal and lower it into MoveGroup constraints:

• "joint" — a joint block (positions, joint_names) → joint_constraints (JointGoalRskill).
• "pose" — a pose block (position, orientation as a quaternion array with quaternion_order, link_name) → MoveGroup position + orientation constraints for a Cartesian end-effector goal (PoseGoalRskill).
• "look_at" — a look_at block (target_xyz, camera, …) → the same pose-constraint lowering, with the gaze pose computed from the camera's live TF pose (LookAtRskill, a specialisation of the pose builder).

RunResult

Bases: BaseModel

Aggregated summary returned by InferenceRunner.run.

Attributes:

Name	Type	Description
n_ticks	int	Total ticks executed.
success	bool | None	Task success when the runner has a success signal (sim, scripted goal). None for open-ended hardware runs.
budget_violations	int	Count of ticks whose tick_ms exceeded the rSkill manifest's RSkillLatencyBudget.per_chunk_ms.
avg_inference_ms	float	Mean of TickResult.inference_ms.
p99_inference_ms	float	99th-percentile of TickResult.inference_ms.
avg_tick_ms	float	Mean of TickResult.tick_ms.
p99_tick_ms	float	99th-percentile of TickResult.tick_ms.
trace_id	str | None	OTel trace id (hex) of the run's root span. None when tracing is not configured.
save_dir	str | None	Directory where artefacts were written, when configured.
metadata	dict[str, object]	Free-form metadata (runner id, host, git SHA, …).

SafetyEnvelope

Bases: BaseModel

Safety constraints handed to the C++ safety kernel.

Not every field is enforced yet. Several are declared, not yet enforced by the kernel: max_force_n (issue #373), max_ee_accel_m_s2, contact_force_threshold_n and deadman_required are passed to the kernel as ROS parameters but no check consults them, and no_go_zones, e_stop_topic, e_stop_qos, cycle_time_violation_threshold_ms and human_in_loop_required are not read by any runtime component. Each attribute below says which case it is in.

Attributes:

Name	Type	Description
workspace_box_min_xyz	tuple[float, float, float] | None	Lower corner of allowed workspace (m). The kernel checks it only against CARTESIAN_POSE chunk positions.
workspace_box_max_xyz	tuple[float, float, float] | None	Upper corner of allowed workspace (m). Same scope as workspace_box_min_xyz.
no_go_zones	list[dict[str, object]]	List of polygon definitions (dicts with 'vertices'). Declared, not yet enforced: not passed to the kernel and not read by any runtime component.
max_ee_speed_m_s	float	Maximum end-effector linear speed in m/s. Also the C++ safety kernel's CARTESIAN_TWIST linear bound.
max_ee_accel_m_s2	float	Maximum end-effector acceleration in m/s². Declared, not yet enforced by the kernel: passed as a ROS parameter, but no check compares an acceleration against it.
max_joint_speed_factor	float	Fraction of joint velocity_limit allowed.
max_force_n	float	Declared maximum contact force in Newtons. Not enforced yet: it is validated here and passed to the C++ safety kernel as a parameter, but no check compares any measured or estimated force against it (issue #373).
max_torque_nm	float	Maximum joint torque in Nm. The kernel caps JOINT_TORQUE chunks at min(joint.torque_limit, this).
deadman_required	bool	Whether a deadman switch is required. Declared, not yet enforced by the kernel: passed as a ROS parameter but never consulted. The deploy launch starts the deadman watchdog unconditionally, whatever this says.
e_stop_topic	str	ROS 2 topic for E-stop commands. Declared, not read: the kernel latches on its hardcoded /openral/estop topic.
e_stop_qos	Literal['reliable']	QoS profile for E-stop topic. Declared, not read.
contact_force_threshold_n	float	Force threshold for contact detection. Declared, not yet enforced by the kernel: passed as a ROS parameter but never consulted. The kernel's place-contact force gate takes its threshold from each placement declaration instead.
cycle_time_violation_threshold_ms	float	Control cycle time violation threshold. Declared, not read by any runtime component.
human_in_loop_required	list[str]	rSkill names requiring human supervision. Declared, not read by any runtime component.
max_cartesian_step_m	float | None	Per-control-mode bound the C++ safety kernel enforces — per-step magnitude bound on CARTESIAN_DELTA's xyz triplet (Euclidean, after cartesian_delta_scale). None means "no per-mode check declared, skip". It is a norm, so a controller with a per-axis output limit a needs at least a·√3 here or in-distribution diagonal moves E-stop.
max_cartesian_step_rad	float | None	Per-control-mode kernel bound — per-step magnitude bound on CARTESIAN_DELTA's axis-angle triplet (Euclidean). None skips the check.
max_ee_angular_speed_rad_s	float | None	Per-control-mode kernel bound — angular component bound for CARTESIAN_TWIST (the linear bound reuses max_ee_speed_m_s). None skips the check.
max_base_linear_speed_m_s	float | None	Per-control-mode kernel bound — BODY_TWIST linear bound (Euclidean over vx,vy,vz). None skips the check; mobile manipulators / wheeled bases declare it. Must be finite and > 0 (inf would disable the kernel bound).
max_base_angular_speed_rad_s	float | None	Per-control-mode kernel bound — BODY_TWIST angular bound (Euclidean over wx,wy,wz; for planar bases only wz is non-zero). None skips the check.
self_collision_margin_m	float	Clearance margin (m) for the kernel's self/world/voxel geometric checks; a pair closer than this fires. Default 0.0 (collide on touch). A small negative value tolerates the grazing contact inherent to a compact arm's in-distribution operating envelope (e.g. the SO-101 pen VLA, whose links graze at true ≈ 0) while still catching a real jam (deeper penetration). Loosening this is a safety-WG decision and must be justified against the true (non-convex mesh) envelope clearance, not the conservative primitive distance.
starting_pose_max_joint_speed_rad_s	float | None	Approach speed (rad/s) of every non-prismatic joint on the skill runner's ramp from the live pose to a skill's starting_pose. Declared per robot, never derived: a rated velocity_limit is a ceiling, not an approach speed (issue 303 — deriving it put the OpenArm at 2 rad/s). The runner caps each joint at min(this, joint.velocity_limit). None = undeclared; a real robot with a non-prismatic joint must declare it.
starting_pose_tolerance_rad	float | None	Per-joint distance (rad) within which a non-prismatic joint counts as at its starting pose, both for the ramp and for verifying arrival before the policy starts. None = undeclared; a real robot with a non-prismatic joint must declare it.
starting_pose_max_joint_speed_m_s	float | None	The same approach speed for prismatic joints, in m/s (a gripper's 0.041 m stroke is not comparable to a radian). Also capped by joint.velocity_limit. A real robot with a prismatic joint must declare it.
starting_pose_tolerance_m	float | None	The per-joint arrival tolerance for prismatic joints, in m. A real robot with a prismatic joint must declare it.
joint_state_staleness_limit_s	float | None	Age (s) past which a joint-state reading is refused as stale — by the real HAL's read_state (ROSPerceptionStale) and by the skill runner's blocking waits. The one place a rig declares it: build_hal threads it into the HAL and deploy_e2e.launch.py into the runner in real mode. Must come from a measurement on the deploy host (tools/joint_state_staleness_probe.py). None = undeclared; a real robot must declare it.

SceneChangeMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/scene_change.

Emitted when a histogram / structural-similarity detector measures a frame-to-frame distance above a configured threshold — typically used to wake the reasoner when the scene has changed enough to warrant a re-plan.

Attributes:

Name	Type	Description
kind	Literal['scene_change']	Discriminator (always "scene_change").
distance	float	Frame-to-frame distance in the detector's native metric (e.g. cv2.HISTCMP_CHISQR_ALT distance, 1 - ssim).
threshold	float	Distance threshold that fired the event.
metric	str	Identifier of the distance metric (e.g. "chisqr_alt", "1-ssim", "hellinger").

SceneComposition

Bases: BaseModel

Declarative MJCF scene composition for a manifest-driven HAL.

Lets a robot whose sim HAL needs a composed MJCF (a bare arm spliced onto a tabletop + props) declare the composer in its manifest instead of a bespoke _create_hal lifecycle subclass (issue #191 Phase 3b). The manifest-driven node calls composer before constructing the HAL and threads the resulting MJCF path in as the HAL's mjcf_path.

The composer is a "module.path:function" import string. The function is called with **params and MUST return (xml: str, meshdir: Path) — the composed MJCF XML and the mesh directory it references (the node writes the XML next to meshdir so relative mesh paths resolve). Today's only composer is openral_sim.backends.openarm_robosuite._assets:compose_openarm_tabletop_mjcf.

Attributes:

Name	Type	Description
composer	str	"module.path:function" returning (xml, meshdir).
params	dict[str, object]	Keyword arguments for the composer (e.g. robot_lift_z, white_background).

Example

sc = SceneComposition( ... composer="pkg.scenes:compose_tabletop", ... params={"robot_lift_z": 0.36, "white_background": True}, ... ) sc.params["robot_lift_z"] 0.36

SceneDefaults

Bases: BaseModel

Per-robot scene rendering defaults.

These are values that scene composers may consult when the SimScene YAML does not override them. Today the only field is top_camera, which the openarm_tabletop_pnp backend consumes; future scenes can extend this submodel as new defaults are pulled out of backend hardcodes.

Attributes:

Name	Type	Description
top_camera	TopCameraDefaults | None	Default placement for the scene-overview camera.

Example

sd = SceneDefaults( ... top_camera=TopCameraDefaults( ... pos=(0.2, 0.0, 0.95), ... target=(0.65, 0.0, 0.05), ... fovy=65.0, ... ), ... ) sd.top_camera.pos (0.2, 0.0, 0.95)

SceneGraph

Bases: BaseModel

Persistent hierarchical scene-graph spatial memory.

The durable, queryable world model the S2 Reasoner consults to recall where objects/places/agents are and how to navigate to them. Distinct from the ephemeral collision grid and advisory only — never a safety input (CLAUDE.md §1.1).

Invariants (enforced): node ids are unique; every edge references existing nodes.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk schema version ("0.1"; no backward-incompatible change yet). Now the repo is published it is versioned for real (CLAUDE.md §1.6): an incompatible change bumps it and ships a migrator.
nodes	list[SpatialNode]	All scene-graph nodes.
edges	list[SpatialEdge]	All directed relations between nodes.

SceneSpec

Bases: BaseModel

A physics scene declaration — the WORLD the robot acts in.

A scene is a deterministic, reproducible MuJoCo / MJX / etc. world: assets, lighting, cameras, fixed objects. Tasks (TaskSpec) are evaluated INSIDE a scene; multiple tasks can share one scene (e.g. libero_spatial has 10 tasks per suite).

Attributes:

Name	Type	Description
id	str	Stable scene identifier used by the eval registry, e.g. "libero_spatial", "metaworld_mt50", "so100_tabletop".
backend	PhysicsBackend	Physics backend used to instantiate the scene.
assets_uri	str | None	Optional URI pointing at scene assets. When None, the registered adapter resolves assets internally (LIBERO / MetaWorld pull theirs from their own packages). Read by the isaac_sim scene as its environment USD (a local path, an http(s):// / omniverse:// URL, or isaac:<path> under the installed Isaac Sim asset root); other adapters ignore it.
observation_height	int	Default render height in pixels for camera obs.
observation_width	int	Default render width in pixels for camera obs.
cameras	list[str]	List of camera names the scene exposes. Adapters use this both to render and to map sensors to VLA feature keys.
backend_options	dict[str, object]	Backend-specific overrides (e.g. {"render_modes": ["rgb_array"]}). Opaque to the core schema: each sim backend owns the model that validates it, declared via SCENES.register(..., options_model=...) and run at load time by SCENES.validate_options.

Example

s = SceneSpec(id="libero_spatial", backend=PhysicsBackend.MUJOCO) s.observation_width 256

SceneTaskSpace

Bases: BaseModel

The action interface a scene-adapter family executes.

The scene leg of the cross-layer task-space contract. A scene picks a backend adapter (LIBERO OSC, RoboCasa composite, gym-aloha joints, the RLBench PyRep sidecar, …) and that adapter — not the coarse PhysicsBackend and not the individual scene instance — fixes which ControlMode s the policy's flat action vector is interpreted as. Many scenes share one adapter (the four LIBERO suites, RoboCasa's ~100 prebuilt PnP tasks), so the executed task space is declared once per family and keyed by the same vocabulary rSkills already use in RSkillManifest.evaluated_tasks (the leading token before any "/": "libero_spatial", "rlbench", "robocasa", …).

Attributes:

Name	Type	Description
modes	frozenset[ControlMode]	The ControlMode s the adapter executes. An rSkill is scene-compatible when its TaskSpace control modes are a subset of this set.
action_dim	int | None	The flat action-vector width the adapter steps, when fixed (LIBERO 7, MetaWorld 4, ALOHA 14). None when it varies by task or robot composition (RoboCasa 11-vs-12 base skew, ManiSkill per-task), in which case only modes is checked.
runs_via_default_packers	bool	True when every mode is in SIM_EXECUTABLE_CONTROL_MODES (the deploy-sim default packers could also drive it). False for adapters that drive a dedicated controller path outside the default packers — RLBench's CARTESIAN_POSE motion planner, RoboCasa-GR1's whole-body BASIC composite — mirroring KNOWN_SIM_GAPS in the sweep.

SegmenterContract

Bases: BaseModel

Manifest contract for kind: "segmenter" rSkills.

The sibling of DetectorContract for models that answer a geometric prompt rather than a semantic one: a detector is asked "where are the cups?" and replies with labelled, scored boxes; a segmenter is asked "what is the extent of the thing under this pixel?" and replies with a mask. It carries no labels and no score_threshold — it never classifies, and its consumer never gates on model confidence.

That is a safety property, not an oversight: the mask feeds the HAL's vision attachment-evidence producer, which turns masked depth into the collision payload a manipulator plans around. Only geometry (containment between the jaws, payload extent, depth validity) can reject a bad mask; a measured instance of a 59.8%-of-frame mask returned at the model's own top score of 0.977 is what removed score_threshold from this contract.

Required when RSkillManifest.kind is "segmenter"; forbidden for all other kinds (RSkillManifest._check_kind_consistency). Like detector / vlm, a pure perception producer: no Action chunks, no actuators.

Attributes:

Name	Type	Description
engine	SegmenterEngine	Backend that executes the model (SegmenterEngine). REQUIRED — there is no runtime-keyed fallback for this kind.
max_prompt_points	int	Upper bound on how many prompt points one request may carry (positive TCP point plus optional negative jaw-tip points). Bounds the request so a caller cannot make one segmentation unboundedly expensive. Default 8.
multimask	bool	Ask the backend for its multi-hypothesis head (SAM 2 returns three nested candidates per point prompt) instead of a single mask. The consumer picks among candidates on geometry, never on the model's per-candidate score. Default True.
min_mask_area_px	int	Masks smaller than this many set pixels are discarded as degenerate before they ever reach the depth intersection — a handful of pixels cannot bound a payload. Default 64.

Example

c = SegmenterContract(engine=SegmenterEngine.SAM2_HF) c.engine.value 'sam2_hf' (c.max_prompt_points, c.multimask, c.min_mask_area_px) (8, True, 64)

SegmenterEngine

Bases: str, Enum

Backend that executes a kind: "segmenter" rSkill.

The segmenter counterpart of DetectorEngine. There is no legacy runtime-keyed fallback to preserve here, so SegmenterContract.engine is REQUIRED — a segmenter manifest always names its backend outright.

Attributes:

Name	Type	Description
SAM2_HF		In-process Transformers Sam2Model promptable segmenter (SAM 2.1 Hiera). Consumes positive/negative point prompts in pixel coordinates and emits binary masks at the frame's own resolution.

Example

SegmenterEngine.SAM2_HF.value 'sam2_hf'

SelfVerifyEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SELFVERIFY — a self-check failed.

Attributes:

Name	Type	Description
kind	Literal['selfverify']	Discriminator (always "selfverify").
check	str	Short check identifier (e.g. "action_chunk.shape").
expected	str	Expected value as a string.
observed	str	Observed value as a string.

SensorBundle

Bases: BaseModel

Multi-modal sensor group, e.g. RealSense D455 = (RGB, depth, IMU).

Attributes:

Name	Type	Description
bundle_name	str	Unique name for this bundle.
sensors	list[SensorSpec]	List of constituent sensor specs.
sync	Literal['hardware', 'approximate', 'none']	Synchronization strategy.
sync_tolerance_ms	float	Tolerance for approximate synchronization.

SensorDeployBinding

Bases: BaseModel

Real-device binding that lets openral deploy run open a sensor.

The runtime counterpart to SensorSpec.sim_placement: sim_placement says how the sim renders a camera; deploy_binding says how a real deploy opens it — which SensorReaderBackend and its /dev/video* / pipeline parameters. It is host/site-specific (a /dev/video* index differs per machine).

A robot camera (wrist / head / the robot's own overview) takes it from its host's unit overlay, robots/<id>/units/<unit>.yaml (SensorOverlay), or from robots/<id>/robot.yaml for a robot with a single reference host; a deploy scene never redefines a robot sensor. A workcell-mounted camera (overhead / front) declares it on its own, distinctly named DeployScene.sensors entry. Either way the deploy sensor leg (openral_rskill_ros.sensor_leg) opens every SensorSpec that carries one and publishes it on /openral/cameras/<name>/image.

Attributes:

Name	Type	Description
backend	SensorReaderBackend	Which SensorReaderBackend opens the device.
backend_params	dict[str, object]	Backend-specific reader kwargs, e.g. {"device": "/dev/video0", "fps": 30} for opencv_thread or a {"pipeline": ...} string for gstreamer.
max_age_ms	int	How stale a frame may be before the reader raises.

Example

SensorDeployBinding( ... backend=SensorReaderBackend.OPENCV_THREAD, ... backend_params={"device": "/dev/video0", "fps": 30}, ... ).backend.value 'opencv_thread'

SensorFrame

Bases: BaseModel

A single sensor frame — metadata plus an optional inline payload.

A SensorFrame is the carrier passed from a SensorReader into WorldState.image_frames. Exactly one of data, topic, or handle is populated:

• data carries the pixel bytes for in-process delivery and for trace capture. JSON-serialized as base64 by Pydantic, so payloads round-trip through model_dump_json / model_validate_json.
• topic points at a ROS 2 topic — the bytes live on the ROS 2 bus and the consumer subscribes for them, which is what WorldState.images carries. Ros2ImageSensorReader does NOT return this shape: it subscribes on the caller's behalf and inlines the decoded pixels as data, because every read_latest consumer (the VLA observation builder, trace capture, the republishing sensor leg) needs the bytes, not a second subscription.
• handle is an opaque integer (e.g. CUDA NVMM pointer, DMA-BUF file descriptor) — in-process only, never serialized. Set by the GStreamerSensorReader when frames stay on the GPU.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; matches SensorSpec.name.
stamp_monotonic_ns	int	Capture-time monotonic timestamp in nanoseconds.
stamp_wall_ns	int	Capture-time wall-clock timestamp in nanoseconds.
encoding	FrameEncoding	How to interpret the bytes.
width	int	Image width in pixels.
height	int	Image height in pixels.
channels	int	Number of channels (3 for RGB/BGR, 1 for MONO/DEPTH).
data	bytes | None	Optional inline pixel payload (or compressed bytes for JPEG / PNG). Mutually exclusive with topic / handle.
topic	str | None	Optional ROS 2 topic reference. Mutually exclusive with data / handle.
handle	int | None	Optional opaque in-process handle (NVMM / DMA-BUF). Mutually exclusive with data / topic.
metadata	dict[str, object]	Free-form per-frame metadata (gain, exposure, …).

Example

SensorFrame( ... sensor_id="wrist_rgb", ... stamp_monotonic_ns=1, ... stamp_wall_ns=2, ... encoding=FrameEncoding.RGB8, ... width=640, ... height=480, ... topic=camera_topic("wrist_rgb"), ... ).channels 3

model_post_init(_context: object) -> None

Exactly one of (data, topic, handle) must be set.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Exactly one of (data, topic, handle) must be set."""
    populated = [bool(self.data), self.topic is not None, self.handle is not None]
    n = sum(populated)
    if n != 1:
        raise ValueError(
            f"SensorFrame({self.sensor_id!r}): exactly one of "
            f"(data, topic, handle) must be set; got {n}."
        )

SensorModality

Bases: str, Enum

Physical sensing modality.

SensorOverlay

Bases: BaseModel

Per-host / per-unit values for ONE sensor the robot manifest declares.

The robot manifest owns a sensor's identity and semantics (name, modality, frame_id, parent_frame); what differs between two physical units of the same robot type — the device path or serial in deploy_binding, the vendor driver topic, the calibrated mount pose and intrinsics — goes here. Any other field is refused (extra="forbid"). Applied by apply_sensor_overlays.

Attributes:

Name	Type	Description
name	str	The robot-manifest sensor this overlays.
deploy_binding	SensorDeployBinding | None	This host's real-device binding.
ros2_topic	str | None	This host's vendor-driver topic.
static_transform_xyz_rpy	tuple[float, float, float, float, float, float] | None	This unit's calibrated mount, in the manifest's parent_frame.
intrinsics	IntrinsicsPinhole | None	This unit's calibrated intrinsics.

Example

SensorOverlay(name="top", ros2_topic="/cam/image").ros2_topic '/cam/image'

SensorReaderBackend

Bases: str, Enum

Which SensorReader implementation to instantiate.

opencv_thread is the default and mirrors lerobot's per-camera background-thread pattern. galaxea_a1_camera_bridge consumes the versioned paired raw-frame service owned by an external A1 Runtime process without importing Runtime or opening either RealSense device. ros2_image subscribes to a ROS 2 image topic published by a vendor driver — the path for streams that exist only because an SDK computed them (ZED stereo depth, RealSense aligned depth), which no /dev/video* read can reach. gstreamer runs a GStreamer pipeline whose appsink delivers frames (NVMM / DMA-BUF on Jetson; CPU bytes on x86).

holoscan is reserved-but-unimplemented: a 2026-05-12 evaluation of NVIDIA Holoscan SDK as a parallel ingest backbone deferred adoption (lean GStreamer with custom NvBufSurface glue won the comparison). The enum value exists so a future PR can add the backend additively without bumping the schema again; configs that select it today raise ROSConfigError at factory time.

SensorReaderConfig

Bases: BaseModel

Per-sensor backend configuration for the inference runner.

Picks which SensorReader backend services a sensor and how the pipeline is parameterised. The optional publish_to_ros tee lets a GStreamer pipeline publish a downsampled stream to a ROS 2 topic for observability (rosbag2 / rqt_image_view) without putting the hot path through ROS.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; MUST match a SensorSpec.name in the target RobotDescription.
backend	SensorReaderBackend	Which SensorReaderBackend to instantiate.
backend_params	dict[str, object]	Backend-specific keyword arguments forwarded to the reader constructor. For gstreamer typically a single "pipeline" string. For opencv_thread typically {"device": "/dev/video0", "fps": 30}.
max_age_ms	int	How stale a frame may be before SensorReader.read_latest raises. Defaults to ~3 frames at 30 Hz.
publish_to_ros	bool	If True, the reader tees a downsampled stream to publish_topic at publish_rate_hz. gstreamer backend only; every other backend refuses it.
publish_topic	str | None	ROS 2 topic to publish to when publish_to_ros is True. Required iff publish_to_ros.
publish_rate_hz	float | None	Downsample rate for the ROS tee.
publish_frame_id	str | None	TF frame stamped on the tee's Image and CameraInfo headers (SensorSpec.frame_id). None stamps sensor_id. Only valid with publish_to_ros.
publish_camera_info	IntrinsicsPinhole | None	Manifest intrinsics (SensorSpec.intrinsics). When set, the tee also publishes sensor_msgs/CameraInfo on the image topic's sibling (.../<name>/image → .../<name>/camera_info), scaled to each published frame. None publishes images only. Only valid with publish_to_ros.

Example

SensorReaderConfig( ... sensor_id="wrist_rgb", ... backend=SensorReaderBackend.GSTREAMER, ... backend_params={ ... "pipeline": "v4l2src device=/dev/video0 ! " ... "nvv4l2decoder ! nvvideoconvert ! appsink" ... }, ... publish_to_ros=True, ... publish_topic=camera_topic("wrist_rgb"), ... publish_rate_hz=5.0, ... ).backend.value 'gstreamer'

model_post_init(_context: object) -> None

Cross-field validation for the ROS tee.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation for the ROS tee."""
    if not self.publish_to_ros and (
        self.publish_frame_id is not None or self.publish_camera_info is not None
    ):
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_frame_id / "
            f"publish_camera_info are set but publish_to_ros is False; they "
            f"only configure the ROS tee."
        )
    if self.publish_to_ros and self.backend != SensorReaderBackend.GSTREAMER:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_to_ros needs the "
            f"gstreamer backend (got {self.backend.value!r}); only its in-pipeline "
            f"tee publishes to ROS. Other backends are published by the deploy "
            f"sensor leg's SensorRosPublisher, not by this flag."
        )
    if self.publish_to_ros and self.publish_topic is None:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_to_ros is "
            f"True but publish_topic is unset; a ROS tee needs a topic."
        )
    if self.publish_topic is not None and not self.publish_to_ros:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_topic is "
            f"set but publish_to_ros is False; enable publish_to_ros "
            f"explicitly or drop the topic."
        )

SensorRequirement

Bases: BaseModel

One sensor an rSkill needs the robot to provide.

Used by RSkillManifest.sensors_required to declare the inputs the policy expects. The compatibility check resolves each entry against a RobotDescription's sensors list and rejects the pairing if no robot sensor satisfies the requirement.

Resolution rules (in order):

1. If vla_feature_key is set, the robot MUST expose exactly one sensor with that vla_feature_key. The check then verifies modality matches and (if specified) the sensor's intrinsics meet min_width / min_height.
2. Otherwise, the robot must expose at least count sensors of the requested modality (each meeting any specified resolution minimum).

Attributes:

Name	Type	Description
modality	SensorModality	Required physical modality (rgb, depth, imu…).
vla_feature_key	str | None	Optional exact key the VLA expects, e.g. "observation.images.camera1". When set, the robot's matching SensorSpec.vla_feature_key must be identical.
min_width	int | None	Minimum image width in pixels (RGB/depth/IR/stereo only).
min_height	int | None	Minimum image height in pixels.
count	int	Number of robot sensors of this modality required when vla_feature_key is unset. Ignored when vla_feature_key is provided (a key uniquely identifies one sensor).

Example

SensorRequirement( ... modality=SensorModality.RGB, ... vla_feature_key="observation.images.camera1", ... min_width=224, ... min_height=224, ... ) # doctest: +ELLIPSIS SensorRequirement(modality=, ...)

SensorSpec

Bases: BaseModel

Generalizable sensor descriptor — covers all modalities.

Attributes:

Name	Type	Description
name	str	Human-readable sensor name, e.g. "head_rgb", "left_wrist_depth".
modality	SensorModality	Physical sensing modality.
frame_id	str	tf2 frame name.
parent_frame	str | None	tf2 parent frame (for static transform).
static_transform_xyz_rpy	tuple[float, float, float, float, float, float] | None	Static transform from parent to this sensor.
rate_hz	float	Expected publishing rate in Hz.
intrinsics	IntrinsicsPinhole | None	Pinhole camera intrinsics (if applicable).
encoding	str | None	Image encoding, e.g. "rgb8", "16UC1".
vla_feature_key	str | None	VLA observation dict key this sensor maps to, e.g. 'observation.images.camera1'. Used by skill loaders to auto-wire sensors to VLA input_features.
shares_mount_with	str | None	Another sensor of this robot that is the same physical device (a stereo camera's colour stream and its depth): this one sits at THAT sensor's mount (parent_frame + static_transform_xyz_rpy, a unit's calibration included), so a sim renders both from one pose. None: the sensor declares its own mount, if any.
ros2_topic	str | None	The image topic an external ROS 2 driver publishes (/camera/color/image_raw, /zed/zed_node/...). None (every in-tree manifest) means OpenRAL itself produces the sensor (sim bridge or sensor leg) under the canonical camera_topic(name, kind) layout. Today it is read only by the RealSense calibration command. Nothing remaps it onto camera_topic(name) yet (ADR-0108 Decision 2, not implemented), so a camera served only by a vendor driver does not reach the canonical topics: bind it through a deploy_binding (e.g. the ros2_image backend) to publish it there.
ros2_msg_type	str | None	ROS 2 message type, e.g. "sensor_msgs/Image". None for non-ROS robots (USB, sim-only).
qos_profile	Literal['sensor_data', 'reliable', 'transient_local', 'parameters']	QoS profile key.
catalog_id	str | None	Optional sensor catalog id used as provenance for a catalog-backed physical device. The spec remains fully materialized; calibrated intrinsics, placement, feature keys, and serials stay on this instance.
vendor	str | None	Sensor vendor name.
model	str | None	Sensor model, e.g. "RealSense D455".
driver_pkg	str | None	ROS 2 driver package name.
metadata	dict[str, object]	Additional key-value metadata.

is_cloud_source: bool property

A depth camera or 3D lidar whose driver can feed octomap_server a cloud.

Wider than :attr:is_depth_camera: a real driver builds its own PointCloud2, so no pinhole intrinsics are needed (a 3D lidar has none). On a real deploy any such sensor auto-enables the octomap leg, which then needs the driver's topic pinned (:func:deploy_cloud_topic).

Example

SensorSpec( ... name="lidar", modality="point_cloud", frame_id="lidar", rate_hz=10.0 ... ).is_cloud_source True

is_depth_camera: bool property

A depth / point-cloud camera with intrinsics.

The one test for "can this be back-projected into a cloud": the sim bridge's depth synth, the sim cloud topic (deploy_cloud_topic), the Nav2-over-visual-SLAM guard and the launch's depth-camera pick all use it.

Example

desc = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml") ", ".join(s.name for s in desc.sensors if s.is_depth_camera) 'front_depth'

SimDescription

Bases: BaseModel

MuJoCo wiring for a single-arm robot, consumed by MujocoArmHAL.

The MJCF itself is named by RobotDescription.assets.mjcf; this block carries only the joint↔qpos/qvel/actuator plumbing. All fields are optional with defaults derived from RobotDescription.joints. The default mapping is "1:1 in joint order, offset by 7 (qpos) / 6 (qvel) if floating_base is True" — which is correct for every robot in the open core today bar minor gripper bookkeeping.

Attributes:

Name	Type	Description
floating_base	bool	If True, the robot's MJCF has a 6-DoF free joint before the actuated joints (humanoids). In that case the default joint_qpos_addr[joints[i].name] = 7 + i and joint_qvel_addr[joints[i].name] = 6 + i.
joint_qpos_addr	dict[str, int] | None	Optional override of the joint→qpos-index map. When omitted, the default 1:1 mapping is used.
joint_qvel_addr	dict[str, int] | None	Optional override of the joint→qvel-index map. When omitted, defaults to joint_qpos_addr (or its default).
actuator_index	dict[str, int] | None	Optional override of the joint→actuator-index map. When omitted, defaults to the 1:1 mapping joints[i].name → i.
grippers	list[SimGripperDescription]	Optional list of SimGripperDescription entries. Each must reference a joint by name that is also present in RobotDescription.joints. Single-arm robots have one entry (or none); bimanual robots have two (left + right).
settle_steps_default	int | None	Pinned number of mj_step calls executed per MujocoArmHAL.send_action. None (default) derives it at connect so one action is one control period of action_spec.control_freq_hz (issue #358); a pinned count is honoured and logged as an override that leaves the twin wall-paced.
keyframe_index	int | None	When set, MujocoArmHAL.connect calls mj_resetDataKeyframe(model, data, keyframe_index) before mj_forward. Required for MJCFs whose default MjData.qpos (zeros) sits outside the actuator ctrlrange — e.g. the gym-aloha parallel jaws, whose fingers have ctrlrange=[0.021, 0.057] and never recover from qpos = 0.
seed_ctrl_from_qpos	bool	When True, MujocoArmHAL.connect seeds data.ctrl[actuator] = data.qpos[joint_qpos_addr] for every controllable joint so position actuators hold the initial pose on the first mj_step. Required by the OpenArm v2 MJCF (its position actuators with per-class PD gains will drive qpos to ctrl == 0 otherwise).

Example

SimDescription(floating_base=True).floating_base True

SimEnvironment

Bases: BaseModel

A full sim configuration: the swappable (robot x scene x task x VLA) tuple.

The composed runtime form of a SimScene plus an RSkillManifest. openral_sim's CLI builds it; adapters consume it. Not loaded from YAML directly — from_yaml raises.

Attributes:

Name	Type	Description
robot_id	str	ID into the eval ROBOTS registry. Matches a robot's RobotDescription.name or an embodiment shortcut (e.g. "so100_follower", "franka_panda").
scene	SceneSpec	Scene the robot acts in.
task	TaskSpec	Task to evaluate. task.scene_id MUST equal scene.id.
vla	VLASpec	Policy that drives the robot.
base_pose	Pose6D | None	Optional per-episode mounting pose for the robot in the scene's world frame. None means "use the scene adapter's default" (e.g. an URDF's identity placement). Only honoured by free-axis scene adapters; setting it on a scene that registers with a fixed_robot= constraint (LIBERO, MetaWorld, RoboCasa, PushT, ALOHA) is rejected at CLI compose-time. The pose's frame_id is expected to be "world"; adapters anchor on the robot's RobotDescription.base_frame.
seed	int	Global random seed (env reset, action sampling, torch RNG).
n_episodes	int	Number of episodes to run. 1 for a smoketest; ≥ 50 for an honest single-task success rate.
record_video	bool	Whether to ask the runner to record video frames.
save_dir	str | None	Optional directory to write artefacts (video, traces, json summary). None means "discard outputs".
metadata	dict[str, object]	Free-form notes (commit SHA, run owner, etc.).

Example

env = SimEnvironment( ... robot_id="franka_panda", ... scene=SceneSpec(id="libero_spatial"), ... task=TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the cube", ... ), ... vla=VLASpec(id="smolvla", weights_uri="hf://lerobot/smolvla_libero"), ... ) env.task.scene_id == env.scene.id True

model_post_init(_context: object) -> None

Cross-field validation: task.scene_id must match scene.id.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: task.scene_id must match scene.id."""
    if self.task.scene_id != self.scene.id:
        raise ValueError(
            f"SimEnvironment.task.scene_id ({self.task.scene_id!r}) does not "
            f"match scene.id ({self.scene.id!r}); a task can only run in its "
            f"declared scene."
        )

SimGripperDescription

Bases: BaseModel

Gripper wiring inside a MuJoCo MJCF.

Attributes:

Name	Type	Description
joint	str	Name of the gripper joint as it appears in RobotDescription.joints (the public, lerobot-style name — not the menagerie MJCF name).
ctrl_range	tuple[float, float]	(low, high) raw control range for the gripper actuator. In NORMALISED write mode, low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). In PASSTHROUGH write mode this is informational — MuJoCo clips on its own.
qpos_addrs	tuple[int, ...]	qpos indices used to compute the reported gripper position. When the gripper has multiple finger joints (Franka), list every finger's qpos index.
qpos_scale	float	Span used to normalise the summed/raw gripper qpos to [0, 1]. For Franka with two fingers each in [0, 0.04] m, qpos_scale = 0.08. Ignored by AFFINE_LOW_HIGH and PASSTHROUGH read modes but kept for symmetry with the base-class invariant.
read_mode	GripperReadMode	How to report the qpos — see GripperReadMode.
write_mode	GripperWriteMode	How to map an Action's gripper value to ctrl — see GripperWriteMode.
actuator_index	int | None	Explicit MJCF actuator index that receives the (mapped) gripper command. When omitted, defaults to the position of joint in RobotDescription.joints (i.e. the 1:1 mapping derived by SimDescription for arm joints).
mirror_actuator_index	int | None	Optional second actuator that receives the negation of the (mapped) gripper command. Models the Aloha parallel jaws where one motor drives +x and the other -x to keep the fingers symmetric. None for single-actuator grippers.

SimScene

Bases: DeployScene

Scene + task for openral sim run.

All task fields are overridable at the CLI level. max_steps and success_key are optional — omit them for open-ended experiments. Accepts a BenchmarkScene YAML transparently (the eval-specific fields n_episodes, seed, and metadata simply fill the defaults).

SpatialEdge

Bases: BaseModel

A directed relation between two scene-graph nodes.

Attributes:

Name	Type	Description
src	str	node_id of the source node.
dst	str	node_id of the destination node.
kind	SpatialRelationKind	Relation kind.

SpatialNode

Bases: BaseModel

A persistent, typed node in the scene-graph spatial memory.

A superset of DetectedObject for kind == OBJECT; also used for places, rooms, and agents. The pose is anchored in a durable, drift-corrected frame (typically the tf2 map frame); consumers resolve it to the live base frame at query time via tf2 — the node never stores a raw transform.

This is advisory world-model state consumed by the S2 Reasoner. It is never a safety input (CLAUDE.md §1.1): the safety kernel gates only on the live, bounded geometric world.

Attributes:

Name	Type	Description
node_id	str	Stable identifier, unique within a SceneGraph.
kind	SpatialNodeKind	Node kind (object / place / room / agent).
pose	Pose6D	6D pose, anchored in the durable map frame.
label	str	Semantic label or name (e.g. "fridge", "kitchen").
confidence	float	Confidence in [0, 1] for a perceived node.
bbox_3d	tuple[float, float, float, float, float, float] | None	Optional 3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
embedding_ref	str | None	Optional handle into the vector store for open-vocabulary matching; None → label-only.
is_container	bool	Whether the node can hold other nodes (fridge, cabinet).
occludes_contents	bool	Whether contents are unobservable until the container is opened. Requires is_container.
first_seen_ns	int	Timestamp of first observation, in nanoseconds.
last_seen_ns	int	Timestamp of the most recent observation, in nanoseconds; must be >= first_seen_ns.
observation_count	int	Number of times this node has been observed.

Example

node = SpatialNode( ... node_id="fridge", ... kind=SpatialNodeKind.OBJECT, ... pose=Pose6D(xyz=(3.0, 1.0, 0.9), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"), ... label="fridge", ... is_container=True, ... occludes_contents=True, ... first_seen_ns=1, ... last_seen_ns=2, ... ) node.is_container True

SpatialNodeKind

Bases: str, Enum

Kind of node in the persistent scene-graph spatial memory.

OBJECT is the foundation (an accumulated DetectedObject); PLACE is a standable navigation waypoint; ROOM is a semantic area grouping places/objects; AGENT is a person or robot with a pose — the requester of a task is an AGENT so "bring it back to me" resolves to a concrete goal.

SpatialRelationKind

Bases: str, Enum

Kind of directed edge between scene-graph nodes.

CONTAINS links a room/container to what is inside it (a fridge CONTAINS a wine bottle); AT_PLACE links an object/agent to the waypoint to stand at to reach it; TRAVERSABLE_TO is the topological navigation graph between places/rooms; ON / NEAR are incidental object-to-object spatial relations.

SphereShape

Bases: BaseModel

Sphere collision primitive — the simplest convex link/obstacle volume.

Attributes:

Name	Type	Description
shape	Literal['sphere']	Discriminator (always "sphere").
radius_m	float	Sphere radius in metres.

Example

SphereShape(radius_m=0.05).shape 'sphere'

StateContract

Bases: BaseModel

Per-rSkill state-vector contract.

Surfaces the proprioception layout the checkpoint was trained against so the runtime adapter does not have to learn it from a YAML override.

Attributes:

Name	Type	Description
layout	StateLayout | None	Named proprioception layout — see StateLayout. LIBERO / MetaWorld / pusht / aloha leave this None and consume the raw joint-position vector directly.
dim	int | None	Explicit state dimension override. The runtime adapter clips or pads the env state vector to this width before handing it to the policy.
bindings	StateContractBindings | None	Per-robot source bindings — TF frame names + JointState entries the runtime adapter pulls values from. REQUIRED when layout is in WRAPPED_TASK_SPACE_LAYOUTS, FORBIDDEN otherwise (joint-space layouts have no source to parameterise).

StateContractBindings

Bases: BaseModel

Per-robot source bindings for an rSkill's state_contract.layout.

Symmetric to ControlModeSemantics on the action side (joint_order + reference_frame + gripper_convention): the rSkill manifest names the shape via StateContract.layout; these bindings name the sources on the deploying robot. The layout-adapter registry (openral_state_adapter) joins shape + bindings + live sensor_msgs/JointState + live /tf into the per-checkpoint state vector.

Bindings are required for layouts in WRAPPED_TASK_SPACE_LAYOUTS and forbidden otherwise (the joint-space layouts have no source to parameterise).

Attributes:

Name	Type	Description
eef_frame	str | None	tf2 link name of the end effector (e.g. "panda_hand"). Required for any layout reading EE position / orientation.
base_frame	str | None	tf2 link name of the mobile base (e.g. "base_link"). Required for any layout reading base position / orientation OR base-relative EE poses.
world_frame	str | None	tf2 root frame the base pose is expressed in (default "map" — slam_toolbox publishes map → odom → base_link). Set to "odom" for deployments without SLAM.
gripper_qpos_joints	list[str]	JointState.name entries whose positions populate the gripper-width slot(s) of the layout, in the order the policy expects (e.g. ["panda_finger_joint1", "panda_finger_joint2"]). Empty for grasper-less robots or layouts without a gripper slot.
quaternion_convention	Literal['xyzw', 'wxyz']	Component order of any quaternion in the assembled vector. ROS / TF2 default is "xyzw" (the geometry_msgs/Quaternion field order). Some upstream checkpoints expect "wxyz" — declare it here so the assembler permutes once, at the boundary.

StateRepresentation

Bases: str, Enum

State vector representation format.

StaticBox

Bases: BaseModel

One declared static fixture as an oriented box in the robot's base_frame.

Example

StaticBox( ... source="/World/Cell/LeftWall", ... xyz=(0.3, 0.55, 0.08), ... half_extents=(0.675, 0.018, 0.82), ... ).kind 'box'

StaticGeometry

Bases: BaseModel

A cell's declared static geometry (DeployRuntime.static_geometry_file).

Example

g = StaticGeometry.model_validate( ... { ... "static_geometry": [ ... {"kind": "box", "xyz": [0.9, 0.0, 0.1], "half_extents": [0.02, 0.55, 0.8]}, ... {"kind": "mesh", "file": "Rack.npz"}, ... ] ... } ... ) " ".join(e.kind for e in g.static_geometry) 'box mesh'

StaticMesh

Bases: BaseModel

One declared static fixture as a triangle mesh in the robot's base_frame.

file is an .npz with vertices (N, 3) metres and triangles (M, 3) indices, relative to the StaticGeometry YAML that lists it.

Example

StaticMesh(source="/World/OMAR/Rack", file="Rack.npz").kind 'mesh'

SupportContactWitness

Bases: BaseModel

Bounded attestation that a payload rests on a named support surface.

Attachment says what the robot is carrying; it does not say the payload is free of its environment (ADR-0092 D6). A grasped baguette still lies on the counter while the gripper closes, and the safety kernel correctly sees that as a payload-vs-world penetration. This witness is World State's statement that one specific contact is legitimate — the kernel exempts nothing else. Absent witness, absent exemption: the kernel stops.

Geometry is expressed in the attached object's own frame, deliberately not the base frame and not voxel indices: indices decorrelate as a mobile base drives and the occupancy lattice re-phases underneath the robot, while the physical contact persists — the object frame moves with the payload, so the attested plane keeps naming the same physical support face.

max_penetration_m is a physical depth. A 25 mm occupancy lattice reads a 1 mm physical contact as up to ~15 mm of penetration because the surface cell's cube overshoots the true surface; the kernel accounts for that discretisation separately and geometrically, so inflating this field to cover it would license real penetration.

Attributes:

Name	Type	Description
support_id	str	Identity of the supporting environment surface.
contact_point_in_object	tuple[float, float, float]	A point on the support plane, object frame.
contact_normal_in_object	tuple[float, float, float]	Unit outward support normal in the object frame, pointing from the supporting solid toward the payload.
patch_radius_m	float	Lateral radius of the supported patch about the contact point. Contact outside it is never exempt.
max_penetration_m	float	Attested bound on the physical penetration depth.
confidence	float	Witness confidence in [0, 1].
evidence_kind	AttachmentEvidenceKind	Source that measured the support contact. The MuJoCo producer attests SIM_GEOM_DISTANCE — a signed closest-distance probe, not the solver's contact list, which contype / conaffinity suppression can leave empty under a payload that is demonstrably resting on a surface.
evidence_ref	str | None	Optional trace/contact reference.
stamp_ns	int	Producer timestamp. With support_id this keys the safety kernel's re-arm check, so a witness that already died after separation stays dead until a genuinely new one is attested.

Example

witness = SupportContactWitness( ... support_id="sim:counter_main", ... contact_point_in_object=(0.0, 0.0, -0.025), ... contact_normal_in_object=(0.0, 0.0, 1.0), ... patch_radius_m=0.1, ... max_penetration_m=0.00118, ... confidence=1.0, ... evidence_kind=AttachmentEvidenceKind.SIM_GEOM_DISTANCE, ... stamp_ns=1, ... ) witness.patch_radius_m 0.1

fill_idl(msg: object) -> None

Populate a duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

def fill_idl(self, msg: object) -> None:
    """Populate a duck-typed OpenRAL ROS IDL message without importing ROS."""
    msg.support_id = self.support_id  # type: ignore[attr-defined]
    point = msg.contact_point_in_object  # type: ignore[attr-defined]
    point.x, point.y, point.z = self.contact_point_in_object
    normal = msg.contact_normal_in_object  # type: ignore[attr-defined]
    normal.x, normal.y, normal.z = self.contact_normal_in_object
    msg.patch_radius_m = float(self.patch_radius_m)  # type: ignore[attr-defined]
    msg.max_penetration_m = float(self.max_penetration_m)  # type: ignore[attr-defined]
    msg.confidence = float(self.confidence)  # type: ignore[attr-defined]
    msg.evidence_kind = self.evidence_kind.value  # type: ignore[attr-defined]
    msg.evidence_ref = self.evidence_ref or ""  # type: ignore[attr-defined]
    msg.stamp_ns = int(self.stamp_ns)  # type: ignore[attr-defined]

from_idl(msg: object) -> Self classmethod

Decode the duck-typed OpenRAL ROS IDL message without importing ROS.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_idl(cls, msg: object) -> Self:
    """Decode the duck-typed OpenRAL ROS IDL message without importing ROS."""
    point = msg.contact_point_in_object  # type: ignore[attr-defined]
    normal = msg.contact_normal_in_object  # type: ignore[attr-defined]
    return cls(
        support_id=str(msg.support_id),  # type: ignore[attr-defined]
        contact_point_in_object=(float(point.x), float(point.y), float(point.z)),
        contact_normal_in_object=(float(normal.x), float(normal.y), float(normal.z)),
        patch_radius_m=float(msg.patch_radius_m),  # type: ignore[attr-defined]
        max_penetration_m=float(msg.max_penetration_m),  # type: ignore[attr-defined]
        confidence=float(msg.confidence),  # type: ignore[attr-defined]
        evidence_kind=AttachmentEvidenceKind(str(msg.evidence_kind)),  # type: ignore[attr-defined]
        evidence_ref=str(msg.evidence_ref) or None,  # type: ignore[attr-defined]
        stamp_ns=int(msg.stamp_ns),  # type: ignore[attr-defined]
    )

SuppressedSummaryEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SUPPRESSED_SUMMARY — rolling rate-limit roll-up.

Emitted at ~1 Hz by openral_observability.FailureBusPublisher when one or more (kind, severity) buckets dropped events during the past window.

Attributes:

Name	Type	Description
kind	Literal['suppressed_summary']	Discriminator (always "suppressed_summary").
window_s	float	Length of the summarized window, in seconds.
kinds	list[int]	Parallel array of suppressed KIND_* values.
severities	list[int]	Parallel array of suppressed SEVERITY_* values.
counts	list[int]	Parallel array of dropped-event counts per bucket.

TaskSpace

Bases: BaseModel

Layer-neutral view of an action interface as ordered typed segments.

Produced from an rSkill's ActionContract (and, in a later phase, declared by a scene) so the three asset layers — robots, rSkills, scenes — share one comparable object instead of three implicit encodings (a flat supported_control_modes set, an ActionContract, and a runtime-probed action_dim). It is derived, never a hand-authored manifest field, so it cannot drift from the primitives it is built on.

Attributes:

Name	Type	Description
segments	list[TaskSpaceSegment]	Ordered, non-empty list of TaskSpaceSegment. Order matches the flat action-vector layout for skill-derived spaces.
representation	ActionRepresentation | None	The source ActionRepresentation when known (carried through from ActionContract), for diagnostics.

Example

ts = TaskSpace( ... segments=[ ... TaskSpaceSegment( ... family=TaskSpaceFamily.CARTESIAN, ... control_mode=ControlMode.CARTESIAN_DELTA, ... width=6, ... target="panda_hand", ... ), ... TaskSpaceSegment( ... family=TaskSpaceFamily.GRIPPER, ... control_mode=ControlMode.GRIPPER_POSITION, ... width=1, ... target="panda_hand", ... ), ... ], ... ) ts.total_dim 7 sorted(m.value for m in ts.control_modes) == ["cartesian_delta", "gripper_position"] True

control_modes: set[ControlMode] property

The distinct ControlMode s this space drives.

total_dim: int property

Sum of segment widths — the flat action-vector dimensionality.

from_action_contract(action: ActionContract, robot: RobotDescription) -> TaskSpace classmethod

Build the task space an rSkill emits, expanding slots / representation.

Resolution order:

1. action.slots set → one segment per non-discard slot.
2. else action.representation set → expand via canonical_slots_for_representation (returns None for joint representations, falling through to step 3).
3. else → a single whole-vector JOINT_POSITION segment (legacy path).

Parameters:

Name	Type	Description	Default
action	ActionContract	The rSkill's per-checkpoint action contract.	required
robot	RobotDescription	The robot the skill is being matched against — used by the representation expander to resolve the primary end-effector.	required

Returns:

Type	Description
TaskSpace	The normalized TaskSpace the checkpoint's action vector
TaskSpace	occupies.

Example

from openral_core.schemas import ( ... ActionSlot, ... EmbodimentKind, ... EndEffectorSpec, ... JointSpec, ... JointType, ... RobotCapabilities, ... SafetyEnvelope, ... ) robot = RobotDescription( ... name="franka_panda", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["franka_panda"], ... ), ... safety=SafetyEnvelope(), ... ) ac = ActionContract( ... dim=7, ... slots=[ ... ActionSlot( ... range=(0, 5), ... control_mode=ControlMode.CARTESIAN_DELTA, ... ee="panda_hand", ... frame="panda_hand", ... ), ... ActionSlot( ... range=(6, 6), control_mode=ControlMode.GRIPPER_POSITION, ee="panda_hand" ... ), ... ], ... ) ts = TaskSpace.from_action_contract(ac, robot) ts.total_dim 7 [seg.family.value for seg in ts.segments] == ["cartesian", "gripper"] True

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_action_contract(cls, action: ActionContract, robot: RobotDescription) -> TaskSpace:
    """Build the task space an rSkill emits, expanding slots / representation.

    Resolution order:

    1. ``action.slots`` set → one segment per non-discard slot.
    2. else ``action.representation`` set → expand via
       ``canonical_slots_for_representation`` (returns ``None`` for joint
       representations, falling through to step 3).
    3. else → a single whole-vector ``JOINT_POSITION`` segment (legacy path).

    Args:
        action: The rSkill's per-checkpoint action contract.
        robot: The robot the skill is being matched against — used by the
            representation expander to resolve the primary end-effector.

    Returns:
        The normalized ``TaskSpace`` the checkpoint's action vector
        occupies.

    Example:
        >>> from openral_core.schemas import (
        ...     ActionSlot,
        ...     EmbodimentKind,
        ...     EndEffectorSpec,
        ...     JointSpec,
        ...     JointType,
        ...     RobotCapabilities,
        ...     SafetyEnvelope,
        ... )
        >>> robot = RobotDescription(
        ...     name="franka_panda",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(
        ...         supported_control_modes=[ControlMode.JOINT_POSITION],
        ...         embodiment_tags=["franka_panda"],
        ...     ),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> ac = ActionContract(
        ...     dim=7,
        ...     slots=[
        ...         ActionSlot(
        ...             range=(0, 5),
        ...             control_mode=ControlMode.CARTESIAN_DELTA,
        ...             ee="panda_hand",
        ...             frame="panda_hand",
        ...         ),
        ...         ActionSlot(
        ...             range=(6, 6), control_mode=ControlMode.GRIPPER_POSITION, ee="panda_hand"
        ...         ),
        ...     ],
        ... )
        >>> ts = TaskSpace.from_action_contract(ac, robot)
        >>> ts.total_dim
        7
        >>> [seg.family.value for seg in ts.segments] == ["cartesian", "gripper"]
        True
    """
    slots = action.slots
    if slots is None and action.representation is not None:
        slots = canonical_slots_for_representation(
            action.representation, dim=action.dim, description=robot
        )
    if slots is None:
        # Legacy whole-vector path: an undeclared layout is joint-position.
        return cls(
            segments=[
                TaskSpaceSegment(
                    family=TaskSpaceFamily.JOINT,
                    control_mode=ControlMode.JOINT_POSITION,
                    width=action.dim,
                )
            ],
            representation=action.representation,
        )
    segments: list[TaskSpaceSegment] = []
    for slot in slots:
        mode = slot.control_mode
        if mode is None:  # discard slot — occupies range, emits no Action.
            continue
        lo, hi = slot.range
        segments.append(
            TaskSpaceSegment(
                family=_FAMILY_FOR_MODE[mode],
                control_mode=mode,
                width=hi - lo + 1,
                target=slot.ee,
            )
        )
    return cls(segments=segments, representation=action.representation)

TaskSpaceFamily

Bases: str, Enum

Coarse classification of a ControlMode for task-space views.

Redundant with ControlMode (derivable via _FAMILY_FOR_MODE) but stored on TaskSpaceSegment so the object reads cleanly in logs / dashboards and so family↔mode consistency is validated once at construction.

TaskSpaceMatch

Bases: BaseModel

Result of task_space_compatible.

Attributes:

Name	Type	Description
ok	bool	True when every skill segment is executable on the robot.
reasons	list[str]	Human-readable explanation of each incompatibility; empty when ok is True.

TaskSpaceSegment

Bases: BaseModel

One typed slice of an action vector, layer-neutral.

The normalized form of an ActionSlot (rSkill side) or a robot's advertised control mode (robot side), stripped of the slot's absolute [start, end] indices so robot capabilities and scene expectations — which are not policy vectors — can be expressed in the same vocabulary.

Attributes:

Name	Type	Description
family	TaskSpaceFamily	Coarse classification, derived from control_mode and validated to match _FAMILY_FOR_MODE.
control_mode	ControlMode	The ControlMode this slice routes to.
width	int	Number of scalar dimensions the slice occupies (> 0). For the gripper this is the audit's "is the gripper a dimension?" answered structurally — a 1-D GRIPPER_* segment.
target	str | None	End-effector name for CARTESIAN_* / GRIPPER_* / DEX_HAND_JOINT modes (the frame / actuator the slice drives); None for joint / base / composite modes. May be None even for EE modes when the producer could not resolve an end-effector (e.g. a representation-expanded skill on a robot with no declared EE) — the matcher reports that as a reason rather than failing construction.

TaskSpec

Bases: BaseModel

A task declaration — WHAT the robot must achieve inside a scene.

Tasks decouple scene assets (SceneSpec) from goal-conditioning. The same scene can host many tasks; the same task can occasionally be run in multiple compatible scenes (rare; usually 1:1 with scene_id).

Success is evaluated by the scene adapter — the reward / success signal comes from the underlying gym env (info['is_success'], info['success'], terminal reward, …). success_key lets manifest authors override which info field the runner reads.

Attributes:

Name	Type	Description
id	str	Stable task identifier, e.g. "libero_spatial/task_0", "metaworld/push-v3". Adapters split on / to resolve.
scene_id	str	ID of the SceneSpec this task runs in.
instruction	str	Natural-language goal handed to the VLA as the "task" text input. Some adapters override this with a description baked into the underlying suite (LIBERO, MetaWorld).
max_steps	int | None	Episode budget. Adapters may clip to scene-internal limits. None means unset; BenchmarkScene enforces a concrete value via its model validator.
success_key	str | None	Key inside info returned by env.step() whose truthy value marks task success. None means unset; BenchmarkScene enforces a concrete value via its model validator (required for paper-comparison benchmarks).
metadata	dict[str, object]	Free-form per-task notes (paper reference, dataset split, …).

Example

t = TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the black bowl", ... max_steps=200, ... success_key="is_success", ... ) t.max_steps 200 t.success_key 'is_success' TaskSpec(id="libero_spatial/task_0", scene_id="libero_spatial").max_steps is None True

TickResult

Bases: BaseModel

One tick's record returned by InferenceRunner.tick.

Carries the timing breakdown so the parent OTel span can attach exact sub-stage durations and the latency budget enforcement can flag violations without re-instrumenting.

The hardware fields (sensors_ms..hal_ms, safety_violations, action_applied) are the original (v1) surface used by DeployRunner. The sim-specific fields (step_idx..truncated) were added when SimRunner adopted per-step tick semantics; hardware leaves them at their defaults (None), so a hardware tick serialises identically to v1 under model_dump(exclude_none=True).

Attributes:

Name	Type	Description
stamp_ns	int	Tick wall-clock timestamp in nanoseconds (tick start).
tick_idx	int	0-indexed tick counter within a run.
sensors_ms	float	SensorReader.read_latest total wall-time.
world_state_ms	float	WorldStateAggregator.snapshot wall-time.
inference_ms	float	Skill.step wall-time (the chunk dispatch cost, not the full chunked inference — see ChunkedExecutor).
safety_ms	float	Safety check wall-time.
hal_ms	float	HAL.send_action wall-time.
tick_ms	float	End-to-end tick wall-time including the rate-limiter overhead.
chunk_index	int | None	Index of the action played out from a chunked executor. None for non-chunked skills.
safety_violations	list[str]	List of safety-violation reason strings emitted during this tick. Non-empty implies the action was either clamped or dropped per the safety policy.
action_applied	bool	False when the DeadlineOverrunPolicy was drop and the runner elected not to publish this tick's action, or when a sim tick is the reset-tick between episodes (no inference / env step happened).
step_idx	int | None	0-indexed step within the current episode. Set by SimRunner on step-ticks; None on hardware ticks and on sim reset-ticks.
episode_idx	int | None	0-indexed episode within the current run. Set by SimRunner on every tick (including reset-ticks); None on hardware ticks.
reward	float | None	Env step reward for this tick. Set by SimRunner on step-ticks; None on hardware ticks and reset-ticks.
terminated	bool | None	Whether the env signalled natural termination this tick. Set by SimRunner; None elsewhere.
truncated	bool | None	Whether the env hit its step budget this tick. Set by SimRunner; None elsewhere.
trace_context	str | None	Full W3C traceparent for this tick's rskill.tick span, in the form 00-<trace_id_hex>-<span_id_hex>-<flags_hex>. Optional — set by the runner when an OTel context is active so offline consumers (dataset writers, post-hoc analysers) can resume the trace without re-deriving it from the live span. Default None for byte-identical v1 JSON under model_dump(exclude_none=True).

TightCollisionGeometry

Bases: BaseModel

Tight convex geometry refining a link's BoxShape for the kernel's narrow phases.

The safety kernel's arm-link-vs-world-voxel check runs a staged narrow phase when a link declares this: a 26-DOP separating-axis bound on every occupied cell, then GJK on the link's exact convex hull for the cells the DOP cannot clear. Self-collision does the same for a box pair the OBBs cannot separate, when both links declare a stage-2 hull — which retired the panda_link5/panda_link7 allowed-collision exemption (issue

191). Both stages are strict subsets of the box they refine, so the

broad-phase window (sized from half_extents_m alone) does not move. Expressed in the box's own local frame (LinkCollisionGeometry.origin_xyz_rpy).

Containment is checked here, again by the kernel at on_configure (validate_tight_geometry), and once more against the real source mesh by tests/unit/test_collision_tight_geometry.py — definitional rather than fitted: the slabs are tangent halfspaces u·x <= max over the mesh of u·x, the hull is conv(vertices) of the mesh, no optimiser tolerance anywhere.

Generated by tools/generate_tight_geometry.py; not hand-authored.

Attributes:

Name	Type	Description
dop_lo_m	tuple[float, ...]	Per-axis lower slab bound along DOP_AXES, 13 entries.
dop_hi_m	tuple[float, ...]	Per-axis upper slab bound, 13 entries.
hull_vertices_m	tuple[tuple[float, float, float], ...]	Exact convex-hull vertices in the box frame, at most MAX_TIGHT_HULL_VERTICES of them. Empty means stage 1 only — the DOP still tightens the check, but no exact test runs. That is the representation a link whose hull is over the ceiling gets, and it is a cost decision, not a safety one: the DOP is a containing solid either way.
hull_overhang_m	float | None	How far the hull's surface reaches past the real source mesh it was built from, sampled by tools/generate_tight_geometry.py (issue #221). Containment (mesh ⊆ hull) is proved elsewhere; this is the other direction, and it is what an offline adjudicator needs to charge a hull-fidelity self-collision stop an admissible gap instead of leaving it unscorable. None when hull_vertices_m is empty, or when the hull has no source mesh on disk to measure against — never defaulted to 0, since that would silently claim a perfect fit.

Example

t = TightCollisionGeometry( ... dop_lo_m=(-0.05,) * 13, ... dop_hi_m=(0.05,) * 13, ... ) t.hull_vertices_m ()

TimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_TIMEOUT — an operation missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['timeout']	Discriminator (always "timeout").
operation	str	Short name of the operation that timed out (e.g. "skill.step", "hal.read_state", "reasoner.tick").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

TopCameraDefaults

Bases: BaseModel

Default placement for the scene-level "top" (a.k.a. "base") camera.

Per-robot scene defaults consumed by sim backends that render an overview camera (today: the openarm_tabletop_pnp scene). The values describe a look-at camera pointed from pos toward target with vertical field-of-view fovy in degrees.

Backend YAML overrides (scene.backend_options.top_camera_*) still win — this submodel is the default fed to the composer when no override is set, replacing the previous module-level constants in openral_sim.backends.openarm_robosuite._assets.

Attributes:

Name	Type	Description
pos	tuple[float, float, float]	(x, y, z) world-frame camera position in metres.
target	tuple[float, float, float]	(x, y, z) world-frame look-at point in metres.
fovy	float	Vertical field-of-view in degrees.

Example

TopCameraDefaults(pos=(0.2, 0.0, 0.95), target=(0.65, 0.0, 0.05), fovy=65.0).fovy 65.0

UrdfAsset

Bases: BaseModel

A URDF asset reference plus its robot_state_publisher wiring.

The ref is resolved by openral_core.assets.resolve_asset; root_frame and base_to_root_xyz_rpy carry the static transform that bridges a URDF whose root link differs from the robot's base_frame (e.g. Franka's panda_link0 mounted onto a base_link mobile platform).

Attributes:

Name	Type	Description
ref	str	Asset reference (rd:<module>, file:<relpath>, or ros2://robot_description for runtime topic-supplied URDFs).
root_frame	str | None	The URDF's root link name when it differs from RobotDescription.base_frame. None → the URDF root equals base_frame (no static transform needed).
base_to_root_xyz_rpy	tuple[float, float, float, float, float, float] | None	The 6-DoF transform [x, y, z, roll, pitch, yaw] (metres + radians) published via static_transform_publisher to bridge base_frame to root_frame. None when root_frame is None.

Example

UrdfAsset(ref="rd:panda_description", root_frame="panda_link0").root_frame 'panda_link0'

VLASpec

Bases: BaseModel

A VLA / policy declaration — the BRAIN driving the robot.

Lightweight pointer to a policy: either an installed rSkill (referenced by manifest name) or a raw HF Hub repo. The eval registry resolves this to a runtime adapter that returns Action objects.

Attributes:

Name	Type	Description
id	str	Adapter id in the eval registry, e.g. "smolvla", "pi05", "xvla", "random", "zero". Picks the loader.
weights_uri	str	Where to fetch weights from. Pass a bare rSkill reference: a name (smolvla-libero), a path (rskills/smolvla-libero), or a bare HF repo ID (OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16). The openral_sim.SimRunner requires a locally-resolvable reference — raw "hf://" URIs are rejected. Other URI shapes (e.g. "mock://") are still parsed by the schema so unit tests of the eval registries can run without an rSkill, but they will fail-fast if used with the runner.
device	str	Torch device override. "auto" picks cuda:0 if available, otherwise cpu.
quantization	QuantizationConfig | None	Optional quantization override for this run.
deterministic	bool	When True, set torch.use_deterministic_algorithms and disable cuDNN benchmarking.
extra	dict[str, object]	Adapter-specific options, e.g. {"chunk_size": 16}.

Example

v = VLASpec(id="smolvla", weights_uri="rskills/smolvla-libero") v.device 'auto'

ValidationGroundTruthAdjudication

Bases: BaseModel

What the simulator's own probe says about a kernel stop.

Derived from the sim.estop_ground_truth_snapshot line the sim HAL emits per stop. The probe measures against solid world geometry only and carries its own caveat that a zero contact count is not an emptiness test, so the adjudication is driven by the distance probes rather than the contact list.

Attributes:

Name	Type	Description
verdict	GroundTruthAdjudication	The adjudication.
stop_class	str | None	The probe's own class for the stop ("robot_world", "attached_payload").
sim_time_s	float | None	Simulator time at the stop.
grid_resolution_m	float | None	Occupancy-grid edge length; sets the quantization budget.
quantization_budget_m	float | None	Half the grid cell's body diagonal — the voxel-only term, not the whole admissible gap.
admissible_gap_m	float | None	Budget applied to discrepancy_m; prefers the HAL's own adjudication_budget.admissible_gap_m (88.2 mm vs. a 25 mm grid's 21.7 mm voxel-only term on the 2026-08-23 rounds), else quantization_budget_m for a pre-budget snapshot.
budget_source	str	Where admissible_gap_m came from — "hal-adjudication-budget", "grid-quantization" (a lower bound only: can prove within-quantization, never false-positive), or "" (no budget; rounds before #144 published none and two verdicts were over-convicted on that alone).
probe_collidability_filtered	bool | None	Whether both probe sides were solid-geom-restricted; False on pre-filter snapshots (a 0 m pair may be a visual mesh, so cannot support real-contact).
probe_distance_certified	bool | None	Whether every probe distance is proof-certified (openral_hal.convex_distance); False on mujoco.mj_geomDistance snapshots, which read confidently wrong 0.000 m for pairs whose certified value is +14.8/+82.2/ +107.9 mm — such a probe withdraws the verdict to unadjudicated rather than reversing it.
unadjudicated_reason	str	Why the verdict is unadjudicated (empty otherwise) — distinguishes a harness fault from a run fact.
nearest_any_m	float | None	Distance of the closest probed pair of any kind; None when nothing was within distmax_m.
nearest_tripping_party_m	float | None	Distance of the closest pair naming the kernel's tripping party; None when out of range.
nearest_pair	dict[str, object]	Bodies/geoms of the closest probed pair, verbatim.
discrepancy_m	float | None	nearest_tripping_party_m minus the kernel's min_distance_m — clearance the kernel did not see.
probed_pairs	int | None	Pairs the probe actually measured.
probe_truncated	bool | None	Whether the probe hit its call budget (a truncated probe can never support false-positive).
distmax_m	float | None	The probe's cutoff distance.
payload_contacts	int | None	MuJoCo contact count involving the payload.

Example

adj = ValidationGroundTruthAdjudication( ... verdict="real-contact", stop_class="attached_payload", sim_time_s=17.5 ... ) adj.verdict 'real-contact'

ValidationRoundDiff

Bases: BaseModel

Round-over-round comparison — the "what changed" artifact.

Attributes:

Name	Type	Description
schema_version	str	On-disk format version.
round_id	str	The newer round.
baseline_round_id	str	The round it is compared against.
executed_sha	str	SHA the newer round ran.
baseline_executed_sha	str	SHA the baseline ran.
seed	int | None	Scene seed the newer round pinned.
baseline_seed	int | None	Scene seed the baseline pinned. The SHA alone does not make a comparison a reproducibility one: the scene seed decides the initial configuration, so two rounds at one SHA on different seeds are a before/after of two different scenes. A seed-1-vs-seed-2 diff was labelled reproducibility for exactly that reason.
scenes	list[ValidationSceneDelta]	Per-scene deltas, in the newer round's order.

Example

d = ValidationRoundDiff( ... round_id="b", ... baseline_round_id="a", ... executed_sha="0" * 40, ... baseline_executed_sha="0" * 40, ... seed=1, ... baseline_seed=2, ... ) d.same_sha, d.is_reproducibility (True, False)

changed_scenes: list[str] property

Scene keys whose outcome moved between the two rounds.

is_reproducibility: bool property

Whether this compares one configuration with itself.

Same code and same seed. Anything else is a before/after: a seed change moves the scene's initial configuration, so the two rounds are not repeats of one another however equal their SHAs are.

same_seed: bool property

Whether both rounds pinned the same scene seed.

same_sha: bool property

Whether both rounds ran the same code.

ValidationRoundMetadata

Bases: BaseModel

What was executed, on what, from what — the reproducibility half.

Every field answers a "why couldn't we reproduce a past round" question: which code ran, whether the overlay was built from it, which dependency set, which entry point.

Attributes:

Name	Type	Description
round_id	str	Directory name of the round, e.g. "2026-08-22-master-1".
started_at	str	ISO-8601 UTC timestamp the round began.
host	str	Hostname of the validation runner.
executed_sha	str	Full git SHA of the checkout that ran.
worktree_clean	bool | None	Whether git status --porcelain was empty; None only for pre-harness imports (run refuses a dirty worktree).
overlay_built_at_ns	int | None	Modification time of the built ROS overlay, to prove it isn't older than the sources.
launcher_path	str	Absolute path of the openral entry point invoked.
repo_root	str	Absolute path exported as OPENRAL_REPO_ROOT.
robot_manifest_path	str | None	Repo-relative path of the resolved robot manifest.
robot_id	str | None	The robot id the scenes resolved to.
sync_groups	list[str]	Dependency groups the environment was synced with.
stack_argv	list[str]	The launch argv shared by every scene, verbatim.
safety_overrides_absent	bool	True only when the composed argv was checked and carries no safety-knob override; a round is refused rather than recorded False.
collision_scale	dict[str, float]	The #188 graded-velocity band the kernel ran with, when armed; empty = shipped default (disabled). Arrives via OPENRAL_COLLISION_SCALE_* env vars, invisible to argv inspection.
gpu_name	str | None	GPU the round ran on, when nvidia-smi was available.
notes_path	str | None	Round-relative path of the human-readable notes.
scene_dirs	dict[str, str]	Scene key → round-relative artifact directory; empty = same name as the scene key.
seed	int	The scene seed the round pinned; read with executed_sha for a true reproducibility pair (a seed mismatch is a before/after).
artifact_stem	str	Filename stem of every per-scene artifact; pre-harness rounds used "seed1".
imported_from	str | None	Absolute path of the pre-harness round this was imported from, if any.
scene_pins	dict[str, bool]	Scene runtime: keys the round pinned (no CLI flag exists for them); read with stack_argv for the whole executed stack.

Example

m = ValidationRoundMetadata( ... round_id="2026-08-22-master-1", ... started_at="2026-08-22T10:02:00Z", ... executed_sha="2edcf67" + "0" * 33, ... worktree_clean=True, ... ) m.safety_overrides_absent True

ros_automatic_discovery_range: str = '' class-attribute instance-attribute

ROS_AUTOMATIC_DISCOVERY_RANGE; "" means unset, i.e. SUBNET — discovery across the whole subnet, which is how a robot on another host reaches a simulation.

ros_domain_id: str = '' class-attribute instance-attribute

ROS_DOMAIN_ID the round ran on; "" means unset, i.e. domain 0.

Recorded from #227 onward. Before it, no round's DDS scope was captured, so it is not knowable after the fact whether one shared a ROS graph with another machine — which is exactly what happened on 2026-09-05, when a sim on the default scope consumed a live OpenArm's /joint_states.

ValidationRoundVerdicts

Bases: BaseModel

One validation round's machine-readable result — verdicts.json.

This is the on-disk contract tools/validation_matrix.py writes and docs/reference/collision-validation-evidence.md is fed from.

Attributes:

Name	Type	Description
schema_version	str	On-disk format version.
metadata	ValidationRoundMetadata	Reproducibility metadata for the round.
scenes	list[ValidationSceneVerdict]	One verdict per scene, in execution order.

Example

r = ValidationRoundVerdicts( ... metadata=ValidationRoundMetadata( ... round_id="r1", ... started_at="2026-08-22T10:02:00Z", ... executed_sha="0" * 40, ... worktree_clean=True, ... ), ... scenes=[], ... ) r.schema_version '0.1'

from_json(path: str) -> ValidationRoundVerdicts classmethod

Load and validate a round's verdicts.json.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the JSON file.	required

Returns:

Type	Description
ValidationRoundVerdicts	A validated ValidationRoundVerdicts.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the JSON fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_json(cls, path: str) -> ValidationRoundVerdicts:
    """Load and validate a round's ``verdicts.json``.

    Args:
        path: Filesystem path to the JSON file.

    Returns:
        A validated ``ValidationRoundVerdicts``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the JSON fails schema validation.
    """
    import json as _json  # noqa: PLC0415  # reason: deferred import

    with open(path, encoding="utf-8") as fh:
        data = _json.load(fh)
    return cls.model_validate(data)

scene(name: str) -> ValidationSceneVerdict | None

Return the verdict for name, or None when absent.

Parameters:

Name	Type	Description	Default
name	str	The scene key.	required

Returns:

Type	Description
ValidationSceneVerdict | None	The matching ValidationSceneVerdict, if the round ran it.

Source code in python/core/src/openral_core/schemas.py

def scene(self, name: str) -> ValidationSceneVerdict | None:
    """Return the verdict for ``name``, or ``None`` when absent.

    Args:
        name: The scene key.

    Returns:
        The matching ``ValidationSceneVerdict``, if the round ran it.
    """
    return next((s for s in self.scenes if s.scene == name), None)

ValidationSceneDelta

Bases: BaseModel

How one scene's verdict moved between two rounds.

Attributes:

Name	Type	Description
scene	str	The scene key.
baseline_outcome	ValidationOutcome | None	Outcome in the baseline round, None if the scene is new.
outcome	ValidationOutcome | None	Outcome in the current round, None if the scene was dropped.
changed	bool	Whether the outcome differs.
changed_fields	dict[str, dict[str, object]]	Field-by-field deltas, as {field: {"from": ..., "to": ...}}, for the comparable subset (outcome, task success, tripping pair, distances, allowance).

Example

d = ValidationSceneDelta( ... scene="baguette", ... baseline_outcome="estop-collision-real", ... outcome="estop-collision-false-positive", ... changed=True, ... ) d.changed True

ValidationSceneVerdict

Bases: BaseModel

One scene of one validation round, as a queryable record.

Attributes:

Name	Type	Description
scene	str	Short scene key ("baguette", "sink_cup", "fridge", "utensil").
config_path	str	Repo-relative path of the DeployScene YAML that ran.
config_sha256	str | None	Digest of that YAML's bytes as executed, so a round can be reproduced even if the scene later changes.
seed	int	The scene seed in force.
prompt	str	The instruction dispatched to the policy.
rskill_id	str	The rSkill the run dispatched.
outcome	ValidationOutcome	The scene's outcome.
task_success_final	bool | None	success from the terminal sim.task_success_final line. None when the run never emitted one.
task_success_ever	bool | None	ever_succeeded from the same line.
task_success_steps	int | None	Simulator steps the episode ran.
task_success_transitions	int | None	Success-predicate transitions observed.
stop	ValidationStopEvidence | None	The kernel's verdict, when the run was stopped.
ground_truth	ValidationGroundTruthAdjudication | None	Adjudication of that stop against the simulator.
witness	ValidationWitnessTimeline	The attach / witness / declaration lifecycle.
monitor_records	int | None	How many records the run's monitor wrote from a ROS subscription — every line except its own monitor_started / monitor_stopped pair. None when no monitor JSONL exists. Zero means the monitor received nothing, which is a harness fault (its DDS participant never saw the graph) and must not be read as "the run stopped before there was anything to see".
dispatch_failure_reason	str	failure_reason reported by the action client, verbatim — or, when the dispatcher died before writing a status line at all, why it did.
wall_s	float | None	Wall-clock seconds the dispatch took.
artifacts	dict[str, str]	Round-relative paths of the files this verdict was derived from, so every field is traceable to a file.
harness_error_reason	str	Why the run produced no usable artifact set — the graph never came up, or the deploy CLI rejected its own argv. Non-empty exactly when outcome is harness-error, so a failed launch can never be read as a clean deadline.

Example

v = ValidationSceneVerdict( ... scene="fridge", ... config_path="scenes/deploy/robocasa_fridge_drawer.yaml", ... seed=1, ... outcome="estop-initial-configuration", ... ) v.outcome 'estop-initial-configuration'

ValidationStopEvidence

Bases: BaseModel

The kernel's safety.collision verdict for one scene, transcribed.

Every field is read verbatim out of the safety.collision log line the C++ kernel emits; nothing is recomputed. min_distance_m is the depth of the cell the kernel reported, sweep_min_distance_m the depth of the deepest cell its sweep found — they differ only when a support-contact witness exempted a deeper cell, which makes their inequality the authoritative "was an exemption live at the trip" evidence.

Attributes:

Name	Type	Description
kind	str	The kernel's collision kind ("world", "self").
party_a	str	First party — a robot link, or attached:<object_id> when the declared payload is the subject.
party_b	str	Second party — usually voxel_<index> for a world stop.
horizon_step	int	Predicted-horizon index the verdict was evaluated at. -1 is the reactive (measured-state) check.
min_distance_m	float	Reported penetration depth, negative when inside.
sweep_min_distance_m	float | None	Deepest cell the sweep found, negative when inside. None when the kernel build did not report it.
place_allowance_active	bool	Whether the ADR-0097 place-approach allowance was applied on the trip itself.
place_target	str	Declared place target at the trip, empty when none.
depth_is_box_bound	bool	Whether the reported min_distance_m is the OBB's bound rather than a measurement of the geometry the evidence names (#213). Set by the kernel only for a self-collision pair whose two links both ship a stage-2 hull and whose hulls the GJK found overlapping — GJK proves an overlap but does not size one. Sound (the hull is inside its box, so the box can only report more penetration) but loose: -31.97 mm for a ~1.5 mm hull interpenetration on panda_link5/panda_link7. An adjudicator must charge the box budget for such a stop, never treat the number as a hull measurement. False on a snapshot recorded before 213.
exemption_active	bool | None	True when sweep_min_distance_m is strictly deeper than min_distance_m — a deeper cell was exempted. None when the sweep depth was not reported.

Example

ev = ValidationStopEvidence( ... kind="world", ... party_a="panda_link5", ... party_b="voxel_170781", ... horizon_step=0, ... min_distance_m=-0.0209178, ... sweep_min_distance_m=-0.0209178, ... ) ev.exemption_active False

exemption_active: bool | None property

Whether the sweep found a deeper cell than the one reported.

involves_payload: bool property

Whether the declared payload — not a bare robot link — tripped.

ValidationWitnessTimeline

Bases: BaseModel

The attach / witness / place-declaration lifecycle of one scene.

Reconstructed from the run's monitor JSONL (producer side) and the kernel's own safety.support_witness_* / safety.place_region_* log lines (consumer side). Both are recorded because a disagreement between them is itself a finding.

Attributes:

Name	Type	Description
attach_t_s	float | None	Monitor time of the first payload attach, None if the run never grasped.
detach_t_s	float | None	Monitor time of the payload detach, if any.
support_id	str | None	The support body the witness last attested against.
kernel_witness_armed	int	safety.support_witness_armed line count.
kernel_witness_separated	int	safety.support_witness_separated count.
place_declaration_seen	bool	Whether a place declaration with a producer-measured region reached the monitor.
place_region_armed	bool	Whether the kernel armed the declared place region.
place_allowance_active_lines	int	How many lines disclosed place_allowance_active=1. 0 means the approach allowance never applied anywhere in the run.

Example

ValidationWitnessTimeline(attach_t_s=176.32).place_allowance_active_lines 0

VisionAttachmentRuntime

Bases: BaseModel

The real-hardware vision attachment leg of a deploy (DeployRuntime.vision_attachment).

When enabled, openral deploy launches the SAM 2.1 segmenter lifecycle node (openral_perception_ros segmenter_node) and turns on the HAL's vision attachment-evidence bridge (the vision_attachment_* HAL parameters), which publishes the grasped payload on /openral/attachment_state.

Coupling rule: on real hardware the leg ALWAYS turns the safety kernel's attached-payload check on with it (attached_collision_enabled, 1000 ms deadline), never the leg alone. The octomap bridge clears a published payload from the map and the robot self-filter removes it from the cloud regardless of the kernel flag, so a leg without the kernel check would make the payload invisible to every collision check.

Default off. Turning it on for a real cell is a Safety-WG decision (hazard log): the grasp trigger is the gripper's position stalling short of a close command (the OpenArm gripper reports no effort), its thresholds the manifest's robot-type constants (JointSpec.closure_calibration) derived from teleop data, not per cell.

Attributes:

Name	Type	Description
enabled	bool	Bring the leg up. Requires all four topics below.
camera	str	Manifest sensor name the segmenter and the bridge use (e.g. head_zed).
rgb_topic	str | None	The camera driver's RGB Image topic the segmenter caches.
rgb_camera_info_topic	str | None	The driver's CameraInfo for that RGB stream.
depth_topic	str | None	The driver's metric depth Image the bridge back-projects.
depth_camera_info_topic	str | None	The driver's CameraInfo for the depth stream.
segmenter_manifest	str	kind: segmenter rSkill manifest (repo- or scene-relative).
device	str	Segmenter torch device (auto / cuda / cpu).
deadline_s	float	How long the bridge waits for one segmenter reply.
evidence_timeout_s	float	How old the newest joint-state sample carrying a jaw position for every gripper may be before the attachment heartbeat stops (HAL param vision_attachment_evidence_timeout_s).
tf_frames	dict[str, str]	{manifest link: TF frame} renames where the live TF tree's frame names differ from the manifest's links.
grasp_target_enabled	bool	Run the pre-grasp target producer on the bridge (HAL param vision_attachment_grasp_target_enabled): it measures the region the kernel's grasp-target exemption arms with, so deploy run refuses DeployRuntime.grasp_allowance_enabled without it. Default off.
grasp_target_approach_m	float | None	Approach-armed grasp target (HAL param vision_attachment_grasp_target_approach_m; needs grasp_target_enabled): with no named target, a hand whose TCP comes within this many metres of occupied voxels arms a one-hand declaration measured around its jaws, and the rSkill runner arms a goal-scope declaration for every goal so the policy, not the reasoner, picks what to grasp. None = off (default). Calibration point; at most GraspDeclaration.MAX_HALF_EXTENT_M.
grasp_target_premeasure	bool	Pre-measure every object instance near a hand (or in a named search box) from head-camera segmentation before any hand arms, each camera box confirmed by the voxel map, and arm the region from the tracked instance nearest the hand on the arming tick (HAL param vision_attachment_grasp_target_premeasure; needs grasp_target_enabled), so the region exists before contact without the policy pausing for an unoccluded view. Source order: camera instance first, the map confirms it and holds it while the declared hand occludes it (docs/reference/object-primitives-design.md, issue #349). Prototype; default off.
grasp_target_margin_m	float	How far the grasp target is bloated, metres (HAL param vision_attachment_grasp_target_margin_m; design note §2.1 "Grasp-target margin"). Before the handover the region the kernel exempts for the declared finger links is the measured target grown by this margin on every face, downward too — the finger links may press up to this far into the support under the target (and into anything else within it) without a stop. From the handover on, the attached payload — the target lowered onto the measured support top and grown by it on the four sides and the top, never down — is also the region the kernel latches and keeps exempting the finger links over until the payload leaves it, carried with the hand and checked against the environment. The producer's own gates keep the measured region. Default 0.0 (no bloat, the safer side): a scene that wants the wider exemption names it (the real OpenArm cell's 20 mm octomap needs >= 0.03 to cover the support layer, design note §2.1). In [0, MAX_GRASP_TARGET_MARGIN_M] (0.05 m); the margin, not the measurement, shrinks when a bloated box would exceed the declaration's caps (logged). Calibration point, task-dependent; a Safety-WG setting (hazard row HZ-0115-32).
place_target_enabled	bool	Run the real place producer (HAL param vision_attachment_place_target_enabled): while a payload is held it measures the support surface directly under it from the voxel map, arms a place region for it and attests the map-support proximity witness. No place target needs naming. Default off.
release_timeout_s	float	How long the bridge's release window waits for the payload to clear the jaws (HAL param vision_attachment_release_timeout_s). Calibration point. The window's grid age and clearance are not scene knobs: the deploy launch derives them from the kernel's voxel deadline, world margin and octree resolution.

Example

VisionAttachmentRuntime(camera="head_zed").enabled False

WaitTool

Bases: _ReasonerToolBase

Tool variant — deliberate no-op: observe and wait for the next tick.

The reasoner's tool choice is forced (tool_choice="any" / "required"), so without this variant the LLM MUST act every tick even when the correct decision is "nothing to do yet" — e.g. a skill is mid- execution and progress is nominal, or the mission is finished and no new goal has arrived. Forcing an action in those states produces spurious read-only queries or operator-prompt spam. wait makes "do nothing this tick" an explicit, logged, traceable decision instead of a side effect.

Dispatch: none. The node records the call (rationale included) on the reasoner span and returns — no ROS traffic, no actuation, no state change.

Attributes:

Name	Type	Description
tool	Literal['wait']	Discriminator (always "wait").

WamEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WAM — a world-action-model discrepancy.

Attributes:

Name	Type	Description
kind	Literal['wam']	Discriminator (always "wam").
horizon	int	Rollout horizon at which the discrepancy was measured.
discrepancy	float	Scalar discrepancy in the WAM's native units.
wam_id	str	Identifier of the WAM that fired.

WorkspaceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WORKSPACE — EE pose left the safety AABB.

Attributes:

Name	Type	Description
kind	Literal['workspace']	Discriminator (always "workspace").
ee_name	str	End-effector that violated the box.
measured_xyz	tuple[float, float, float]	Measured position in metres.
box_min	tuple[float, float, float]	Box minimum corner (x_min, y_min, z_min).
box_max	tuple[float, float, float]	Box maximum corner (x_max, y_max, z_max).

WorldState

Bases: BaseModel

Snapshot consumed by Reasoner and Skills.

Attributes:

Name	Type	Description
stamp_ns	int	Snapshot timestamp in nanoseconds.
joint_state	JointState	Current joint state.
base_pose	Pose6D | None	Base link pose (mobile robots).
base_twist	tuple[float, float, float, float, float, float] | None	Base link twist (vx, vy, vz, wx, wy, wz).
policy_state	list[float] | None	Optional simulator-native proprioception vector for a checkpoint whose state includes more than joint positions/velocities. Populated through a typed HAL -> WorldState topic; never inferred.
ee_poses	dict[str, Pose6D]	End-effector poses keyed by EE name.
contact_forces	dict[str, tuple[float, ...]]	Contact forces keyed by contact name.
images	dict[str, str]	Sensor name → ROS 2 topic reference (not the raw image).
image_frames	dict[str, SensorFrame] | None	Optional per-sensor SensorFrame snapshot for no-ROS / in-process deployments. When None the consumer reads frames via images topic refs as before.
point_clouds	dict[str, str]	Sensor name → ROS 2 topic reference.
tactile	dict[str, str]	Sensor name → ROS 2 topic reference.
detected_objects	list[DetectedObject]	List of detected objects.
battery_pct	float | None	Battery percentage in [0, 100].
diagnostics	dict[str, Literal['ok', 'warn', 'error', 'stale']]	Per-component diagnostic status.
attached_objects	list[AttachedCollisionObject]	Collision objects carried by robot links. These are absent from world occupancy and remain collision-active as payloads.
attachment_revision	int	Monotonic producer revision for atomic snapshots.
attachment_stamp_ns	int	Timestamp of the last accepted attachment update.
place_declaration	PlaceDeclaration | None	The place-phase declaration in force for the carried payload as the evidence producer resolved it (ADR-0097 and its 2026-08-14 amendment), including the producer-measured PlaceRegion. None means no place phase is declared, which is no place witness and no approach allowance. It rides here rather than on a channel of its own because the amendment extends the payload already crossing the dispatch → HAL → World State boundary, and because the kernel must apply the region and the attachment set it is scoped to from one and the same snapshot.
grasp_declaration	GraspDeclaration | None	The grasp-phase declaration in force as the evidence producer resolved it (real pick-and-place design §2.1), including the producer-measured target region. None means no grasp phase is declared, i.e. no exemption. Carried beside place_declaration for the same one-snapshot reason.
occupancy_grid	OccupancyGridRef | None	Optional 2D occupancy grid reference for mobile-base footprint checks. None until populated; an absent or stale grid is treated as unavailable (fail-closed).

apply_sensor_overlays(sensors: Iterable[SensorSpec], overlays: Iterable[SensorOverlay]) -> list[SensorSpec]

Robot-manifest sensors with each SensorOverlay's set fields laid over them.

An overlay only replaces the per-host / per-unit fields SensorOverlay allows (deploy_binding, ros2_topic, static_transform_xyz_rpy, intrinsics); name, modality and frames stay the manifest's. The result is re-validated as a SensorSpec.

Raises:

Type	Description
ROSConfigError	An overlay names no manifest sensor, or names one twice.

Example

desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml") fix = SensorOverlay(name="top", static_transform_xyz_rpy=(0, 0, 0.5, 0, 0, 0)) apply_sensor_overlays(desc.sensors, [fix])[0].static_transform_xyz_rpy (0.0, 0.0, 0.5, 0.0, 0.0, 0.0)

Source code in python/core/src/openral_core/schemas.py

def apply_sensor_overlays(
    sensors: Iterable[SensorSpec], overlays: Iterable[SensorOverlay]
) -> list[SensorSpec]:
    """Robot-manifest sensors with each ``SensorOverlay``'s set fields laid over them.

    An overlay only replaces the per-host / per-unit fields ``SensorOverlay`` allows
    (``deploy_binding``, ``ros2_topic``, ``static_transform_xyz_rpy``, ``intrinsics``);
    name, modality and frames stay the manifest's. The result is re-validated as a
    ``SensorSpec``.

    Raises:
        ROSConfigError: An overlay names no manifest sensor, or names one twice.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml")
        >>> fix = SensorOverlay(name="top", static_transform_xyz_rpy=(0, 0, 0.5, 0, 0, 0))
        >>> apply_sensor_overlays(desc.sensors, [fix])[0].static_transform_xyz_rpy
        (0.0, 0.0, 0.5, 0.0, 0.0, 0.0)
    """
    by_name: dict[str, SensorOverlay] = {}
    for overlay in overlays:
        if overlay.name in by_name:
            raise ROSConfigError(f"sensor overlay names {overlay.name!r} twice")
        by_name[overlay.name] = overlay
    out = list(sensors)
    unknown = sorted(set(by_name) - {s.name for s in out})
    if unknown:
        raise ROSConfigError(
            f"sensor overlay(s) {', '.join(repr(n) for n in unknown)} name no robot-manifest "
            f"sensor (manifest has: {', '.join(s.name for s in out)})"
        )
    return [
        SensorSpec.model_validate(
            {**s.model_dump(), **by_name[s.name].model_dump(exclude={"name"}, exclude_unset=True)}
        )
        if s.name in by_name
        else s
        for s in out
    ]

assert_vla_reward_fits(vla: RSkillManifest, reward: RSkillManifest, gpu_total_gb: float, *, margin_gb: float = 0.5) -> float

Verify a VLA + its paired reward model co-reside in GPU VRAM.

A VLA emits no success signal of its own, so the reasoner needs the reward model resident alongside the running policy. This is the pre-load gate: both sizes are declared in their manifests, so we check the pair fits before the VLA is ever loaded — failing fast and loud instead of a mid-run CUDA OOM. It checks the model pair footprint only (a necessary condition); the sim / ROS overhead is budgeted separately.

Parameters:

Name	Type	Description	Default
vla	RSkillManifest	The VLA manifest (kind == "vla") about to be loaded.	required
reward	RSkillManifest	Its paired reward manifest (kind == "reward").	required
gpu_total_gb	float	The GPU's total VRAM in GB.	required
margin_gb	float	Headroom reserved for fragmentation / framework overhead.	0.5

Returns:

Type	Description
float	The combined VRAM (GB) the pair requires (vla + reward).

Raises:

Type	Description
ROSConfigError	Either manifest does not declare min_vram_gb for its active dtype — co-residency we are about to require cannot be verified, so the operator must declare the size.
ROSGPUMemoryError	vla + reward + margin_gb > gpu_total_gb — the pair does not fit; the deploy must not run a VLA without its reward signal.

See tests/unit/test_vla_reward_pairing.py for fixture-backed coverage (smolvla-libero + robometer-4b-nf4 = 6.7 GB, fits 8 GB, OOMs a 4 GB card).

Source code in python/core/src/openral_core/schemas.py

def assert_vla_reward_fits(
    vla: RSkillManifest,
    reward: RSkillManifest,
    gpu_total_gb: float,
    *,
    margin_gb: float = 0.5,
) -> float:
    """Verify a VLA + its paired reward model co-reside in GPU VRAM.

    A VLA emits no success signal of its own, so the reasoner needs the reward
    model resident *alongside* the running policy. This is the
    pre-load gate: both sizes are declared in their manifests, so we check the
    pair fits before the VLA is ever loaded — failing fast and loud instead of a
    mid-run CUDA OOM. It checks the **model pair footprint** only (a necessary
    condition); the sim / ROS overhead is budgeted separately.

    Args:
        vla: The VLA manifest (``kind == "vla"``) about to be loaded.
        reward: Its paired reward manifest (``kind == "reward"``).
        gpu_total_gb: The GPU's total VRAM in GB.
        margin_gb: Headroom reserved for fragmentation / framework overhead.

    Returns:
        The combined VRAM (GB) the pair requires (``vla + reward``).

    Raises:
        ROSConfigError: Either manifest does not declare ``min_vram_gb`` for its
            active dtype — co-residency we are about to *require* cannot be
            verified, so the operator must declare the size.
        ROSGPUMemoryError: ``vla + reward + margin_gb > gpu_total_gb`` — the pair
            does not fit; the deploy must not run a VLA without its reward signal.

    See ``tests/unit/test_vla_reward_pairing.py`` for fixture-backed coverage
    (smolvla-libero + robometer-4b-nf4 = 6.7 GB, fits 8 GB, OOMs a 4 GB card).
    """
    vla_gb = vla.active_min_vram_gb()
    reward_gb = reward.active_min_vram_gb()
    missing = [m.name for m, gb in ((vla, vla_gb), (reward, reward_gb)) if gb is None]
    if missing:
        raise ROSConfigError(
            f"cannot verify VLA+reward VRAM co-residency: {missing!r} do not declare "
            f"min_vram_gb for their active dtype. Declare it so the pair can be "
            f"checked before load."
        )
    assert vla_gb is not None and reward_gb is not None  # narrowed by the guard above
    combined = vla_gb + reward_gb
    if combined + margin_gb > gpu_total_gb:
        raise ROSGPUMemoryError(
            f"VLA {vla.name!r} ({vla_gb:.2f} GB @ {vla.quantization.dtype.value}) + "
            f"reward {reward.name!r} ({reward_gb:.2f} GB @ {reward.quantization.dtype.value}) "
            f"= {combined:.2f} GB + {margin_gb:.2f} GB margin exceeds GPU VRAM "
            f"{gpu_total_gb:.2f} GB. A VLA must run with its reward model resident; "
            f"pick a smaller-footprint pair or a larger GPU."
        )
    return combined

camera_topic(name: str, kind: CameraTopicKind = CameraTopicKind.IMAGE, *, prefix: str = CAMERA_TOPIC_PREFIX) -> str

The canonical ROS topic of camera name's kind stream.

<prefix>/<name>/<kind> — the only place the camera layout is spelled. Producers (sim bridge, sensor leg) and consumers (world state, perception, reasoner, Foxglove, launch files) all build through it, so a camera's publisher and subscribers can never drift apart.

Parameters:

Name	Type	Description	Default
name	str	SensorSpec.name of the camera.	required
kind	CameraTopicKind	Which stream of that camera.	IMAGE
prefix	str	Topic root; override only for a namespaced graph.	CAMERA_TOPIC_PREFIX

Returns:

Type	Description
str	The absolute topic name.

Raises:

Type	Description
ROSConfigError	name is empty or contains /.

Example

camera_topic("wrist") '/openral/cameras/wrist/image' camera_topic("front_depth", CameraTopicKind.POINTS) '/openral/cameras/front_depth/points'

Source code in python/core/src/openral_core/schemas.py

def camera_topic(
    name: str,
    kind: CameraTopicKind = CameraTopicKind.IMAGE,
    *,
    prefix: str = CAMERA_TOPIC_PREFIX,
) -> str:
    """The canonical ROS topic of camera ``name``'s ``kind`` stream.

    ``<prefix>/<name>/<kind>`` — the only place the camera layout is spelled.
    Producers (sim bridge, sensor leg) and consumers (world state, perception,
    reasoner, Foxglove, launch files) all build through it, so a camera's
    publisher and subscribers can never drift apart.

    Args:
        name: ``SensorSpec.name`` of the camera.
        kind: Which stream of that camera.
        prefix: Topic root; override only for a namespaced graph.

    Returns:
        The absolute topic name.

    Raises:
        ROSConfigError: ``name`` is empty or contains ``/``.

    Example:
        >>> camera_topic("wrist")
        '/openral/cameras/wrist/image'
        >>> camera_topic("front_depth", CameraTopicKind.POINTS)
        '/openral/cameras/front_depth/points'
    """
    if not name or "/" in name:
        raise ROSConfigError(f"camera name must be a non-empty single path segment, got {name!r}")
    return f"{prefix}/{name}/{kind.value}"

canonical_slots_for_representation(rep: ActionRepresentation, *, dim: int, description: RobotDescription) -> list[ActionSlot] | None

Build the canonical ActionSlot layout for a representation.

The skill_runner calls this to expand a skill that declares only ActionContract.representation (no explicit slots) into a typed slot layout it can dispatch. Joint representations return None so the caller keeps the legacy whole-vector JOINT_POSITION path; cartesian / gripper representations get one slot per control mode, addressed at the robot's primary end-effector (description.end_effectors[0]).

EndEffectorSpec carries no explicit tf-frame field, so the end-effector's name is used as the slot frame (the EE name is the tf frame the cartesian pose/delta is expressed in for the canonical embodiments).

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required
dim	int	Dimensionality of the policy's flat action vector.	required
description	RobotDescription	The target robot — its primary end-effector names the cartesian/gripper slots.	required

Returns:

Type	Description
list[ActionSlot] | None	For joint representations: None (caller keeps the legacy
list[ActionSlot] | None	whole-vector JOINT_POSITION path). Otherwise a list of typed
list[ActionSlot] | None	ActionSlot s that satisfy the per-slot validators.

Raises:

Type	Description
ROSConfigError	If the representation needs an end-effector but description.end_effectors is empty, or if dim is too small for the layout (DELTA_EE_6D / CARTESIAN_POSE need dim >= 6; DELTA_EE_6D_PLUS_GRIPPER needs dim >= 7).

Example

desc = RobotDescription( ... name="ex_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")], ... capabilities=RobotCapabilities(), ... safety=SafetyEnvelope(), ... ) slots = canonical_slots_for_representation( ... ActionRepresentation.DELTA_EE_6D, dim=6, description=desc ... ) slots[0].control_mode is ControlMode.CARTESIAN_DELTA True

Source code in python/core/src/openral_core/schemas.py

def canonical_slots_for_representation(
    rep: ActionRepresentation,
    *,
    dim: int,
    description: RobotDescription,
) -> list[ActionSlot] | None:
    """Build the canonical ``ActionSlot`` layout for a representation.

    The skill_runner calls this to expand a skill that declares
    only ``ActionContract.representation`` (no explicit ``slots``) into a
    typed slot layout it can dispatch. Joint representations return
    ``None`` so the caller keeps the legacy whole-vector ``JOINT_POSITION``
    path; cartesian / gripper representations get one slot per control
    mode, addressed at the robot's primary end-effector
    (``description.end_effectors[0]``).

    ``EndEffectorSpec`` carries no explicit tf-frame field, so the
    end-effector's ``name`` is used as the slot ``frame`` (the EE name is
    the tf frame the cartesian pose/delta is expressed in for the
    canonical embodiments).

    Args:
        rep: The VLA's declared action-vector representation.
        dim: Dimensionality of the policy's flat action vector.
        description: The target robot — its primary end-effector names
            the cartesian/gripper slots.

    Returns:
        For joint representations: ``None`` (caller keeps the legacy
        whole-vector ``JOINT_POSITION`` path). Otherwise a list of typed
        ``ActionSlot`` s that satisfy the per-slot validators.

    Raises:
        ROSConfigError: If the representation needs an end-effector but
            ``description.end_effectors`` is empty, or if ``dim`` is too
            small for the layout (``DELTA_EE_6D`` / ``CARTESIAN_POSE``
            need ``dim >= 6``; ``DELTA_EE_6D_PLUS_GRIPPER`` needs
            ``dim >= 7``).

    Example:
        >>> desc = RobotDescription(
        ...     name="ex_robot",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> slots = canonical_slots_for_representation(
        ...     ActionRepresentation.DELTA_EE_6D, dim=6, description=desc
        ... )
        >>> slots[0].control_mode is ControlMode.CARTESIAN_DELTA
        True
    """
    if rep in (ActionRepresentation.JOINT_POSITIONS, ActionRepresentation.JOINT_VELOCITIES):
        # Legacy whole-vector JOINT path — the caller emits one implicit
        # JOINT_POSITION / JOINT_VELOCITY slot covering the whole vector.
        return None

    if not description.end_effectors:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} addresses "
            "an end-effector but the robot description declares no end_effectors; "
            "cannot build a cartesian/gripper slot layout."
        )
    primary = description.end_effectors[0]
    ee_name = primary.name
    # EndEffectorSpec has no explicit tf-frame field; the EE name is the
    # tf frame the cartesian pose/delta is expressed in.
    ee_frame = ee_name

    has_gripper = rep in (
        ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER,
        ActionRepresentation.DELTA_EE_3D_PLUS_GRIPPER,
    )
    # Translation-only representations carry a 3-wide cartesian slice; all
    # others (6-DoF delta / cartesian pose) carry the full 6-wide slice.
    cart_width = (
        _EE_3D_WIDTH if rep is ActionRepresentation.DELTA_EE_3D_PLUS_GRIPPER else _EE_6D_WIDTH
    )
    min_dim = cart_width + 1 if has_gripper else cart_width
    if dim < min_dim:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} requires "
            f"dim >= {min_dim} but action_contract.dim={dim} is too small for the layout."
        )

    cart_mode = (
        ControlMode.CARTESIAN_POSE
        if rep is ActionRepresentation.CARTESIAN_POSE
        else ControlMode.CARTESIAN_DELTA
    )
    slots: list[ActionSlot] = [
        ActionSlot(
            range=(0, cart_width - 1),
            control_mode=cart_mode,
            ee=ee_name,
            frame=ee_frame,
        )
    ]
    if has_gripper:
        slots.append(
            ActionSlot(
                range=(cart_width, dim - 1),
                control_mode=ControlMode.GRIPPER_POSITION,
                ee=ee_name,
            )
        )
    return slots

check_scene_sensor_overrides(manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec]) -> None

Refuse a scene sensor entry that names a sensor the robot manifest defines.

A sensor the robot manifest declares belongs to the robot: its name, modality and frames live in robot.yaml, so every scene on that robot sees the same camera. DeployScene.sensors only adds workcell cameras, under names the manifest does not use, and those carry their own geometry and binding. What differs per host or per physical unit (device path, driver topic, calibrated mount and intrinsics) goes in a RobotUnit overlay instead (resolve_sensor_overlays).

Parameters:

Name	Type	Description	Default
manifest_sensors	Iterable[SensorSpec]	The robot manifest's sensors.	required
scene_sensors	Iterable[SensorSpec]	The DeployScene.sensors entries.	required

Raises:

Type	Description
ROSConfigError	A scene entry reuses a manifest sensor's name.

Example

desc = RobotDescription.from_yaml("robots/openarm/robot.yaml") check_scene_sensor_overrides(desc.sensors, [])

Source code in python/core/src/openral_core/schemas.py

def check_scene_sensor_overrides(
    manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec]
) -> None:
    """Refuse a scene sensor entry that names a sensor the robot manifest defines.

    A sensor the robot manifest declares belongs to the robot: its name, modality and
    frames live in ``robot.yaml``, so every scene on that robot sees the same camera.
    ``DeployScene.sensors`` only adds workcell cameras, under names the manifest does
    not use, and those carry their own geometry and binding. What differs per host or
    per physical unit (device path, driver topic, calibrated mount and intrinsics) goes
    in a ``RobotUnit`` overlay instead (``resolve_sensor_overlays``).

    Args:
        manifest_sensors: The robot manifest's ``sensors``.
        scene_sensors: The ``DeployScene.sensors`` entries.

    Raises:
        ROSConfigError: A scene entry reuses a manifest sensor's name.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> check_scene_sensor_overrides(desc.sensors, [])
    """
    robot_names = {s.name for s in manifest_sensors}
    clashes = [entry.name for entry in scene_sensors if entry.name in robot_names]
    if clashes:
        raise ROSConfigError(
            f"scene sensor(s) {', '.join(repr(n) for n in clashes)} are defined by the robot "
            "manifest. A deploy scene never redefines a robot camera — its name, modality and "
            "frames live in robots/<robot_id>/robot.yaml. Put a per-host binding or per-unit "
            "calibration in a unit overlay (robots/<robot_id>/units/<unit>.yaml, selected by "
            "the scene's robot_unit or $OPENRAL_ROBOT_UNIT), or give a workcell camera its own "
            "name."
        )

check_tight_geometry_fits_box(shape: CollisionShape, tight: TightCollisionGeometry, owner: str) -> None

Prove a stage-2 refinement stays inside the box it refines.

The kernel's broad-phase window is sized from half_extents_m alone, so a tight representation that reached outside that box would make the kernel skip cells it should visit — a missed collision, not a lost conservatism. That is why this is checked at the manifest/producer boundary rather than deferred to the kernel, and why both owners of a TightCollisionGeometry (a robot link and a carried payload primitive) run the same check rather than two hand-copied ones.

It is cheap because the DOP's first three axes are the box's own: a 26-DOP lies inside its own first three slabs, so those three bounds prove containment for the entire polytope.

Parameters:

Name	Type	Description	Default
shape	CollisionShape	The primitive tight refines. Must be a BoxShape.	required
tight	TightCollisionGeometry	The refinement.	required
owner	str	How to name the offender in the error — "link 'panda_link1'" or "attached primitive 3".	required

Raises:

Type	Description
ValueError	shape is not a box, or a slab reaches outside it. No slack is allowed; the shipped boxes carry a real 0.055-0.132 mm inward margin.

Example

check_tight_geometry_fits_box( ... BoxShape(half_extents_m=(0.1, 0.1, 0.1)), ... TightCollisionGeometry(dop_lo_m=(-0.05,) * 13, dop_hi_m=(0.05,) * 13), ... "link 'demo'", ... )

Source code in python/core/src/openral_core/schemas.py

def check_tight_geometry_fits_box(
    shape: CollisionShape,
    tight: TightCollisionGeometry,
    owner: str,
) -> None:
    """Prove a stage-2 refinement stays inside the box it refines.

    The kernel's broad-phase window is sized from ``half_extents_m`` alone, so
    a tight representation that reached *outside* that box would make the
    kernel skip cells it should visit — a missed collision, not a lost
    conservatism. That is why this is checked at the manifest/producer
    boundary rather than deferred to the kernel, and why both owners of a
    ``TightCollisionGeometry`` (a robot link and a carried payload primitive)
    run the same check rather than two hand-copied ones.

    It is cheap because the DOP's first three axes are the box's own: a 26-DOP
    lies inside its own first three slabs, so those three bounds prove
    containment for the entire polytope.

    Args:
        shape: The primitive ``tight`` refines. Must be a ``BoxShape``.
        tight: The refinement.
        owner: How to name the offender in the error — ``"link 'panda_link1'"``
            or ``"attached primitive 3"``.

    Raises:
        ValueError: ``shape`` is not a box, or a slab reaches outside it. No
            slack is allowed; the shipped boxes carry a real 0.055-0.132 mm
            inward margin.

    Example:
        >>> check_tight_geometry_fits_box(
        ...     BoxShape(half_extents_m=(0.1, 0.1, 0.1)),
        ...     TightCollisionGeometry(dop_lo_m=(-0.05,) * 13, dop_hi_m=(0.05,) * 13),
        ...     "link 'demo'",
        ... )
    """
    if not isinstance(shape, BoxShape):
        msg = (
            f"{owner} declares tight_geometry on a {type(shape).__name__}; it refines a "
            "BoxShape only, because the box is the broad-phase bound the containment "
            "proof is stated against"
        )
        raise ValueError(msg)
    for k, half in enumerate(shape.half_extents_m):
        if tight.dop_lo_m[k] < -half or tight.dop_hi_m[k] > half:
            msg = (
                f"{owner} tight_geometry escapes its box on axis {k}: "
                f"slab [{tight.dop_lo_m[k]}, {tight.dop_hi_m[k]}] is not inside "
                f"[{-half}, {half}]; a tight representation that reaches outside the box "
                "would make the kernel's broad phase skip cells it must visit"
            )
            raise ValueError(msg)

contains_rskill_template_sentinel(text: str | None) -> bool

True when text carries an unresolved rSkill scaffold sentinel.

Example

contains_rskill_template_sentinel("TEMPLATE_ORG/rskill-TEMPLATE_ID") True contains_rskill_template_sentinel("OpenRAL/rskill-qwen35_4b-any-general-nf4") False contains_rskill_template_sentinel(None) False

Source code in python/core/src/openral_core/schemas.py

def contains_rskill_template_sentinel(text: str | None) -> bool:
    """True when ``text`` carries an unresolved rSkill scaffold sentinel.

    Example:
        >>> contains_rskill_template_sentinel("TEMPLATE_ORG/rskill-TEMPLATE_ID")
        True
        >>> contains_rskill_template_sentinel("OpenRAL/rskill-qwen35_4b-any-general-nf4")
        False
        >>> contains_rskill_template_sentinel(None)
        False
    """
    if not text:
        return False
    return any(s in text for s in RSKILL_TEMPLATE_SENTINELS)

control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]

Map an ActionRepresentation to the ControlMode s it drives.

Used by the reasoner's deploy-path palette gate: a skill is only offered when the target robot advertises every mode in the returned set.

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required

Returns:

Type	Description
set[ControlMode]	The set of ControlMode s the representation maps onto.

Example

control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == { ... ControlMode.CARTESIAN_DELTA, ... ControlMode.GRIPPER_POSITION, ... } True

Source code in python/core/src/openral_core/schemas.py

def control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]:
    """Map an ``ActionRepresentation`` to the ``ControlMode`` s it drives.

    Used by the reasoner's deploy-path palette gate: a skill is
    only offered when the target robot advertises *every* mode in the
    returned set.

    Args:
        rep: The VLA's declared action-vector representation.

    Returns:
        The set of ``ControlMode`` s the representation maps onto.

    Example:
        >>> control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == {
        ...     ControlMode.CARTESIAN_DELTA,
        ...     ControlMode.GRIPPER_POSITION,
        ... }
        True
    """
    if rep is ActionRepresentation.JOINT_POSITIONS:
        return {ControlMode.JOINT_POSITION}
    if rep is ActionRepresentation.JOINT_VELOCITIES:
        return {ControlMode.JOINT_VELOCITY}
    if rep is ActionRepresentation.DELTA_EE_6D:
        return {ControlMode.CARTESIAN_DELTA}
    if rep in (
        ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER,
        ActionRepresentation.DELTA_EE_3D_PLUS_GRIPPER,
    ):
        return {ControlMode.CARTESIAN_DELTA, ControlMode.GRIPPER_POSITION}
    # CARTESIAN_POSE — the only remaining enum member.
    return {ControlMode.CARTESIAN_POSE}

deploy_cloud_topic(manifest_sensors: Iterable[SensorSpec], *, pinned: str | None, hal_mode: str, estimated_camera: str | None = None) -> str

The PointCloud2 topic a deploy's cloud consumers read, or "" when it has none.

octomap_server (cloud_in) and the world-state object lift's depth fallback both read it. A pinned DeployRuntime.octomap_cloud_topic wins. Next, a DeployRuntime.estimated_depth_camera: the depth provider publishes that RGB camera's DA3 cloud on camera_topic(<name>, POINTS), sim and real alike. Otherwise, in sim, the sim sensor bridge back-projects each :attr:SensorSpec.is_depth_camera onto camera_topic(<name>, POINTS), so exactly one such camera names the cloud. On a real deploy nothing in-tree publishes a cloud (the sensor leg publishes depth images), so the answer is "" until the driver's own topic is pinned. The one rule openral deploy (pre-launch refusal) and deploy_e2e.launch.py share.

Raises:

Type	Description
ROSConfigError	sim, nothing pinned, and several depth cameras: which one is mapped must be declared (pin runtime.octomap_cloud_topic), not "first one wins".

Example

desc = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml") deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="sim") '/openral/cameras/front_depth/points' deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real") '' deploy_cloud_topic(desc.sensors, pinned="/camera/depth/color/points", hal_mode="real") '/camera/depth/color/points' deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real", estimated_camera="head") '/openral/cameras/head/points'

Source code in python/core/src/openral_core/schemas.py

def deploy_cloud_topic(
    manifest_sensors: Iterable[SensorSpec],
    *,
    pinned: str | None,
    hal_mode: str,
    estimated_camera: str | None = None,
) -> str:
    """The ``PointCloud2`` topic a deploy's cloud consumers read, or ``""`` when it has none.

    ``octomap_server`` (``cloud_in``) and the world-state object lift's depth fallback
    both read it. A pinned ``DeployRuntime.octomap_cloud_topic`` wins. Next, a
    ``DeployRuntime.estimated_depth_camera``: the depth provider publishes that RGB
    camera's DA3 cloud on ``camera_topic(<name>, POINTS)``, sim and real alike.
    Otherwise, in sim, the sim sensor bridge back-projects each :attr:`SensorSpec.is_depth_camera`
    onto ``camera_topic(<name>, POINTS)``, so exactly one such camera names the cloud.
    On a real deploy nothing in-tree publishes a cloud (the sensor leg publishes depth
    *images*), so the answer is ``""`` until the driver's own topic is pinned. The one
    rule ``openral deploy`` (pre-launch refusal) and ``deploy_e2e.launch.py`` share.

    Raises:
        ROSConfigError: sim, nothing pinned, and several depth cameras: which one is
            mapped must be declared (pin ``runtime.octomap_cloud_topic``), not "first
            one wins".

    Example:
        >>> desc = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml")
        >>> deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="sim")
        '/openral/cameras/front_depth/points'
        >>> deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real")
        ''
        >>> deploy_cloud_topic(desc.sensors, pinned="/camera/depth/color/points", hal_mode="real")
        '/camera/depth/color/points'
        >>> deploy_cloud_topic(desc.sensors, pinned=None, hal_mode="real", estimated_camera="head")
        '/openral/cameras/head/points'
    """
    if pinned:
        return pinned
    if estimated_camera:
        return camera_topic(estimated_camera, CameraTopicKind.POINTS)
    if hal_mode == "real":
        return ""
    depth = [s.name for s in manifest_sensors if s.is_depth_camera]
    if len(depth) > 1:
        raise ROSConfigError(
            f"several depth cameras ({', '.join(depth)}) could feed the world map; pin "
            "runtime.octomap_cloud_topic to the one to map, e.g. "
            f"{camera_topic(depth[0], CameraTopicKind.POINTS)!r}"
        )
    return camera_topic(depth[0], CameraTopicKind.POINTS) if depth else ""

expected_repo_name(manifest: RSkillManifest) -> str

Suggest the canonical HF-repo name for an rSkill manifest.

Returns a name that satisfies repo_name_is_canonical for the manifest's kind — the value the publisher prints on a mismatch and --fix-name writes back. The <task> segment is a suggested default (see _task_slug_for_manifest); it is author-owned and the migration agent may refine it. The owner prefix is preserved from RSkillManifest.name (default _DEFAULT_RSKILL_OWNER).

Playbooks map to <owner>/rskill-playbook-<name>; ROS wrappers (ros_action / ros_service) to <owner>/rskill-<model>-<robot>-<task> (no <quant>); every other kind to <owner>/rskill-<model>-<robot>-<task>-<quant>.

Raises:

Type	Description
ValueError	If no canonical <model> token can be determined (see _model_token_for_manifest).

Example

m = RSkillManifest( ... name="OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16", ... version="0.1.0", ... license=RSkillLicensePosture.APACHE_2_0, ... role="s1", ... kind="vla", ... model_family="smolvla", ... embodiment_tags=["franka_panda"], ... runtime=RSkillRuntime.PYTORCH, ... weights_uri="hf://lerobot/smolvla_base", ... chunk_size=16, ... latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0), ... quantization=QuantizationConfig(dtype=QuantizationDtype.BF16), ... evaluated_tasks=["libero_spatial"], ... description="SmolVLA on LIBERO.", ... actions=[RSkillAction.GENERALIST], ... actuators_required=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ), ... ], ... processors=RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json", ... ), ... ) expected_repo_name(m) 'OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'

Source code in python/core/src/openral_core/schemas.py

def expected_repo_name(manifest: RSkillManifest) -> str:
    """Suggest the canonical HF-repo name for an rSkill manifest.

    Returns a name that satisfies ``repo_name_is_canonical`` for the
    manifest's ``kind`` — the value the publisher prints on a mismatch and
    ``--fix-name`` writes back. The ``<task>`` segment is a *suggested default*
    (see ``_task_slug_for_manifest``); it is author-owned and the migration
    agent may refine it. The owner prefix is preserved from
    ``RSkillManifest.name`` (default ``_DEFAULT_RSKILL_OWNER``).

    Playbooks map to ``<owner>/rskill-playbook-<name>``; ROS wrappers
    (``ros_action`` / ``ros_service``) to ``<owner>/rskill-<model>-<robot>-<task>``
    (no ``<quant>``); every other kind to
    ``<owner>/rskill-<model>-<robot>-<task>-<quant>``.

    Raises:
        ValueError: If no canonical ``<model>`` token can be determined (see
            ``_model_token_for_manifest``).

    Example:
        >>> m = RSkillManifest(
        ...     name="OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16",
        ...     version="0.1.0",
        ...     license=RSkillLicensePosture.APACHE_2_0,
        ...     role="s1",
        ...     kind="vla",
        ...     model_family="smolvla",
        ...     embodiment_tags=["franka_panda"],
        ...     runtime=RSkillRuntime.PYTORCH,
        ...     weights_uri="hf://lerobot/smolvla_base",
        ...     chunk_size=16,
        ...     latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0),
        ...     quantization=QuantizationConfig(dtype=QuantizationDtype.BF16),
        ...     evaluated_tasks=["libero_spatial"],
        ...     description="SmolVLA on LIBERO.",
        ...     actions=[RSkillAction.GENERALIST],
        ...     actuators_required=[
        ...         ActuatorRequirement(
        ...             kind=ControlMode.JOINT_POSITION,
        ...             control_mode_semantics=ControlModeSemantics(mode="absolute"),
        ...         ),
        ...     ],
        ...     processors=RSkillProcessors(
        ...         preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json",
        ...         postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json",
        ...     ),
        ... )
        >>> expected_repo_name(m)
        'OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'
    """
    owner = manifest.name.split("/", 1)[0] if "/" in manifest.name else _DEFAULT_RSKILL_OWNER
    if manifest.kind == "playbook":
        tail = _repo_tail_slug(manifest.name).removeprefix("playbook_").strip("_")
        name = tail if tail and _RSKILL_NAME_SEGMENT_RE.match(tail) else "main"
        return f"{owner}/rskill-playbook-{name}"
    model = _model_token_for_manifest(manifest)
    robot = _robot_token_for_manifest(manifest)
    task = _task_slug_for_manifest(manifest, model, robot)
    # Weightless wrappers (ros_action / ros_service) omit the <quant> segment.
    if manifest.kind in _WEIGHTLESS_KINDS:
        return f"{owner}/rskill-{model}-{robot}-{task}"
    quant = _quant_token_for_dtype(manifest.quantization.dtype)
    return f"{owner}/rskill-{model}-{robot}-{task}-{quant}"

extract_base_sim_joint_names(description: RobotDescription) -> tuple[str, str, str] | None

Return (forward, side, yaw) MJCF joint names from any mobile-base description.

Generic, robot-agnostic helper. Consumes RobotDescription.base_joints + each referenced joint's JointSpec.sim_joint_name override. Works for any robot whose robot.yaml declares both:

• base_joints: [<forward>, <side>, <yaw>] at the top level.
• Each of the three referenced joints carries a sim_joint_name: "..." mapping its URDF-shape name to the MJCF/MuJoCo joint name the simulator emits (typically auto-prefixed under a composed scene).

Returns None when the description has no base_joints block, fewer than three entries (the schema permits arbitrary list length for future non-planar bases — this helper specialises to the planar-base case), or any referenced joint lacks sim_joint_name. Callers should treat None as "fall back to module defaults" — the sim-side ray-cast helpers in openral_sim.backends.robocasa accept this.

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot's manifest, loaded via RobotDescription.from_yaml.	required

Returns:

Type	Description
tuple[str, str, str] | None	(forward, side, yaw) MJCF joint names, or None when
tuple[str, str, str] | None	the description doesn't carry a complete mobile-base block.

Example

See robots/panda_mobile/robot.yaml for a real fixture.

desc = RobotDescription( ... name="ex", ... embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR, ... joints=[ ... JointSpec( ... name="base_x", ... joint_type=JointType.PRISMATIC, ... parent_link="world", ... child_link="base_x_link", ... sim_joint_name="mobilebase0_joint_mobile_forward", ... ), ... JointSpec( ... name="base_y", ... joint_type=JointType.PRISMATIC, ... parent_link="base_x_link", ... child_link="base_y_link", ... sim_joint_name="mobilebase0_joint_mobile_side", ... ), ... JointSpec( ... name="base_yaw", ... joint_type=JointType.REVOLUTE, ... parent_link="base_y_link", ... child_link="base_link", ... sim_joint_name="mobilebase0_joint_mobile_yaw", ... ), ... ], ... capabilities=RobotCapabilities(embodiment_tags=["ex"]), ... safety=SafetyEnvelope(), ... base_joints=["base_x", "base_y", "base_yaw"], ... ) extract_base_sim_joint_names(desc)[0] 'mobilebase0_joint_mobile_forward'

Source code in python/core/src/openral_core/schemas.py

def extract_base_sim_joint_names(
    description: RobotDescription,
) -> tuple[str, str, str] | None:
    """Return ``(forward, side, yaw)`` MJCF joint names from any mobile-base description.

    Generic, robot-agnostic helper. Consumes
    ``RobotDescription.base_joints`` + each referenced joint's
    ``JointSpec.sim_joint_name`` override. Works for any robot whose
    ``robot.yaml`` declares both:

    * ``base_joints: [<forward>, <side>, <yaw>]`` at the top level.
    * Each of the three referenced joints carries a
      ``sim_joint_name: "..."`` mapping its URDF-shape ``name`` to the
      MJCF/MuJoCo joint name the simulator emits (typically auto-prefixed
      under a composed scene).

    Returns ``None`` when the description has no ``base_joints`` block,
    fewer than three entries (the schema permits arbitrary list length for
    future non-planar bases — this helper specialises to the planar-base
    case), or any referenced joint lacks ``sim_joint_name``. Callers should
    treat ``None`` as "fall back to module defaults" — the sim-side
    ray-cast helpers in ``openral_sim.backends.robocasa`` accept this.

    Args:
        description: The robot's manifest, loaded via
            ``RobotDescription.from_yaml``.

    Returns:
        ``(forward, side, yaw)`` MJCF joint names, or ``None`` when
        the description doesn't carry a complete mobile-base block.

    Example:
        >>> # See `robots/panda_mobile/robot.yaml` for a real fixture.
        >>> desc = RobotDescription(
        ...     name="ex",
        ...     embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="base_x",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="world",
        ...             child_link="base_x_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_forward",
        ...         ),
        ...         JointSpec(
        ...             name="base_y",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="base_x_link",
        ...             child_link="base_y_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_side",
        ...         ),
        ...         JointSpec(
        ...             name="base_yaw",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_y_link",
        ...             child_link="base_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_yaw",
        ...         ),
        ...     ],
        ...     capabilities=RobotCapabilities(embodiment_tags=["ex"]),
        ...     safety=SafetyEnvelope(),
        ...     base_joints=["base_x", "base_y", "base_yaw"],
        ... )
        >>> extract_base_sim_joint_names(desc)[0]
        'mobilebase0_joint_mobile_forward'
    """
    # Planar mobile bases carry exactly three holonomic axes (forward,
    # side, yaw); the schema doesn't constrain `base_joints` length so
    # future non-planar bases can declare richer surfaces, but this
    # helper specialises here.
    planar_base_dof = 3
    if description.base_joints is None or len(description.base_joints) < planar_base_dof:
        return None
    by_name = {j.name: j for j in description.joints}
    sim_names: list[str | None] = []
    for ref in description.base_joints[:planar_base_dof]:
        spec = by_name.get(ref)
        if spec is None:
            return None
        sim_names.append(spec.sim_joint_name)
    if any(n is None for n in sim_names):
        return None
    forward, side, yaw = sim_names
    # Narrow tuple[str | None, ...] → tuple[str, str, str] for type checkers.
    assert isinstance(forward, str)
    assert isinstance(side, str)
    assert isinstance(yaw, str)
    return (forward, side, yaw)

gripper_hands(description: RobotDescription) -> tuple[tuple[str, ...], ...]

The robot's hands: its role: gripper joints' child_links, grouped per hand.

A hand is the set of gripper joints that hang off one arm: joints sharing a parent_link, plus any gripper joint whose parent is another gripper joint's child (a chained finger). An SO-101's one jaw joint is one hand; an OpenArm's two finger-pair joints are two hands (one per arm); an R1 Pro's four finger joints are two hands of two fingers. The reasoner's grasp grounding makes a declaration name one hand; the HAL's approach-armed grasp target arms for one hand at a time.

Example

gripper_hands(RobotDescription.from_yaml("robots/openarm/robot.yaml")) (('openarm_left_finger_pair',), ('openarm_right_finger_pair',)) gripper_hands(RobotDescription.from_yaml("robots/r1pro/robot.yaml"))[0] ('left_gripper_finger_link1', 'left_gripper_finger_link2')

Source code in python/core/src/openral_core/schemas.py

def gripper_hands(description: RobotDescription) -> tuple[tuple[str, ...], ...]:
    """The robot's hands: its ``role: gripper`` joints' ``child_link``s, grouped per hand.

    A hand is the set of gripper joints that hang off one arm: joints sharing a
    ``parent_link``, plus any gripper joint whose parent is another gripper joint's
    child (a chained finger). An SO-101's one jaw joint is one hand; an OpenArm's two
    finger-pair joints are two hands (one per arm); an R1 Pro's four finger joints are
    two hands of two fingers. The reasoner's grasp grounding makes a declaration name
    one hand; the HAL's approach-armed grasp target arms for one hand at a time.

    Example:
        >>> gripper_hands(RobotDescription.from_yaml("robots/openarm/robot.yaml"))
        (('openarm_left_finger_pair',), ('openarm_right_finger_pair',))
        >>> gripper_hands(RobotDescription.from_yaml("robots/r1pro/robot.yaml"))[0]
        ('left_gripper_finger_link1', 'left_gripper_finger_link2')
    """
    grippers = [j for j in description.joints if j.role == "gripper"]
    parent_of = {j.child_link: j.parent_link for j in grippers}
    hands: dict[str, list[str]] = {}
    for j in grippers:
        hands.setdefault(_gripper_mount(parent_of, j.parent_link), []).append(j.child_link)
    return tuple(tuple(links) for links in hands.values())

is_collective_target(text: str) -> bool

True when text targets a set rather than one specific object.

A quantifier (all/every/each/both/everything) or a bare generic plural (objects/items/things). Narrow by design: a specific goal never trips it.

Example

is_collective_target("put all the objects in the basket") True is_collective_target("pick up the alphabet soup and put it in the basket") False

Source code in python/core/src/openral_core/schemas.py

def is_collective_target(text: str) -> bool:
    """True when ``text`` targets a *set* rather than one specific object.

    A quantifier (``all``/``every``/``each``/``both``/``everything``) or a bare
    generic plural (``objects``/``items``/``things``). Narrow by design: a
    specific goal never trips it.

    Example:
        >>> is_collective_target("put all the objects in the basket")
        True
        >>> is_collective_target("pick up the alphabet soup and put it in the basket")
        False
    """
    return _COLLECTIVE_TARGET_RE.search(text) is not None

load_robot_unit(robot_yaml: str | Path, unit: str) -> RobotUnit

Load <robot_yaml dir>/units/<unit>.yaml and check it belongs to that robot.

Raises:

Type	Description
ROSConfigError	The file is missing, or its robot_id / unit disagree with the manifest directory name / the requested unit.

Example

load_robot_unit("robots/openarm/robot.yaml", "thor").robot_id 'openarm'

Source code in python/core/src/openral_core/schemas.py

def load_robot_unit(robot_yaml: str | Path, unit: str) -> RobotUnit:
    """Load ``<robot_yaml dir>/units/<unit>.yaml`` and check it belongs to that robot.

    Raises:
        ROSConfigError: The file is missing, or its ``robot_id`` / ``unit`` disagree with
            the manifest directory name / the requested unit.

    Example:
        >>> load_robot_unit("robots/openarm/robot.yaml", "thor").robot_id
        'openarm'
    """
    manifest_dir = Path(robot_yaml).parent
    path = manifest_dir / "units" / f"{unit}.yaml"
    if not path.is_file():
        have = sorted(p.stem for p in (manifest_dir / "units").glob("*.yaml"))
        raise ROSConfigError(
            f"robot unit {unit!r} not found at {path} (available: {', '.join(have) or 'none'})"
        )
    loaded = RobotUnit.from_yaml(str(path))
    if loaded.robot_id != manifest_dir.name or loaded.unit != unit:
        raise ROSConfigError(
            f"{path} declares robot_id={loaded.robot_id!r} unit={loaded.unit!r}; "
            f"expected robot_id={manifest_dir.name!r} unit={unit!r}"
        )
    return loaded

merge_deploy_sensors(manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec]) -> list[SensorSpec]

The deploy's sensor set: the robot manifest's sensors, then the scene's workcell ones.

A scene only adds sensors (check_scene_sensor_overrides refuses one that reuses a manifest sensor's name), so this is a checked concatenation, in declaration order.

Raises:

Type	Description
ROSConfigError	A scene entry names a sensor the robot manifest defines.

Example

desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml") ", ".join(s.name for s in merge_deploy_sensors(desc.sensors, [])) 'top, wrist'

Source code in python/core/src/openral_core/schemas.py

def merge_deploy_sensors(
    manifest_sensors: Iterable[SensorSpec],
    scene_sensors: Iterable[SensorSpec],
) -> list[SensorSpec]:
    """The deploy's sensor set: the robot manifest's sensors, then the scene's workcell ones.

    A scene only adds sensors (``check_scene_sensor_overrides`` refuses one that reuses a
    manifest sensor's name), so this is a checked concatenation, in declaration order.

    Raises:
        ROSConfigError: A scene entry names a sensor the robot manifest defines.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml")
        >>> ", ".join(s.name for s in merge_deploy_sensors(desc.sensors, []))
        'top, wrist'
    """
    manifest = list(manifest_sensors)
    scene = list(scene_sensors)
    check_scene_sensor_overrides(manifest, scene)
    return [*manifest, *scene]

parse_reasoner_tool_choice(raw: str | None) -> str | None

Validate an OPENRAL_REASONER_TOOL_CHOICE value; None when unset.

Shared by the reasoner's client factory and openral doctor so the two cannot drift. Case-insensitive, like OPENRAL_REASONER_DIALECT.

Raises:

Type	Description
ROSConfigError	When set to anything but auto / required.

Example

parse_reasoner_tool_choice(" AUTO ") 'auto' parse_reasoner_tool_choice("") is None True

Source code in python/core/src/openral_core/schemas.py

def parse_reasoner_tool_choice(raw: str | None) -> str | None:
    """Validate an ``OPENRAL_REASONER_TOOL_CHOICE`` value; ``None`` when unset.

    Shared by the reasoner's client factory and ``openral doctor`` so the two
    cannot drift. Case-insensitive, like ``OPENRAL_REASONER_DIALECT``.

    Raises:
        ROSConfigError: When set to anything but ``auto`` / ``required``.

    Example:
        >>> parse_reasoner_tool_choice(" AUTO ")
        'auto'
        >>> parse_reasoner_tool_choice("") is None
        True
    """
    value = (raw or "").strip().lower()
    if not value:
        return None
    if value not in REASONER_TOOL_CHOICES:
        raise ROSConfigError(
            f"OPENRAL_REASONER_TOOL_CHOICE={raw!r}; expected one of {list(REASONER_TOOL_CHOICES)}."
        )
    return value

publishing_sensors(manifest_sensors: Iterable[SensorSpec], scene_sensors: Iterable[SensorSpec], hal_mode: str) -> list[SensorSpec]

The sensors that actually publish a camera topic on a deploy.

Sim: the manifest's sensors, which the sim sensor bridge renders (bound or not; a scene-only hardware camera has no sim publisher). Real: merge_deploy_sensors, keeping only sensors with a deploy_binding — an unbound sensor gets no reader and so no topic. The one rule both openral deploy (pre-launch decisions) and deploy_e2e.launch.py (which camera each consumer subscribes to) use, so they cannot disagree.

Example

arm = RobotDescription.from_yaml("robots/openarm/robot.yaml") sim = publishing_sensors(arm.sensors, [], "sim") ", ".join(s.name for s in sim if s.modality == "rgb") 'top, wrist_left, wrist_right' thor = resolve_sensor_overlays("robots/openarm/robot.yaml", "thor", required=True) bound = publishing_sensors(apply_sensor_overlays(arm.sensors, thor), [], "real") ", ".join(s.name for s in bound) 'head_zed, top, wrist_left, wrist_right'

Source code in python/core/src/openral_core/schemas.py

def publishing_sensors(
    manifest_sensors: Iterable[SensorSpec],
    scene_sensors: Iterable[SensorSpec],
    hal_mode: str,
) -> list[SensorSpec]:
    """The sensors that actually publish a camera topic on a deploy.

    Sim: the manifest's sensors, which the sim sensor bridge renders (bound or not; a
    scene-only hardware camera has no sim publisher). Real: ``merge_deploy_sensors``, keeping
    only sensors with a ``deploy_binding`` — an unbound sensor gets no reader and so no topic.
    The one rule both ``openral deploy`` (pre-launch decisions) and ``deploy_e2e.launch.py``
    (which camera each consumer subscribes to) use, so they cannot disagree.

    Example:
        >>> arm = RobotDescription.from_yaml("robots/openarm/robot.yaml")
        >>> sim = publishing_sensors(arm.sensors, [], "sim")
        >>> ", ".join(s.name for s in sim if s.modality == "rgb")
        'top, wrist_left, wrist_right'
        >>> thor = resolve_sensor_overlays("robots/openarm/robot.yaml", "thor", required=True)
        >>> bound = publishing_sensors(apply_sensor_overlays(arm.sensors, thor), [], "real")
        >>> ", ".join(s.name for s in bound)
        'head_zed, top, wrist_left, wrist_right'
    """
    if hal_mode != "real":
        return list(manifest_sensors)
    return [s for s in merge_deploy_sensors(manifest_sensors, scene_sensors) if s.deploy_binding]

repo_name_is_canonical(name: str, *, kind: RSkillKind, model_family: str | None = None) -> bool

Return True when name obeys the repo-naming grammar for kind.

Three shapes, chosen by kind:

• playbook → rskill-playbook-<name> (3 parts).
• ros_action / ros_service (_WEIGHTLESS_KINDS) → rskill-<model>-<robot>-<task> (4 parts) — no <quant> segment, since a ROS wrapper carries no weights.
• every other kind → rskill-<model>-<robot>-<task>-<quant> (5 parts) with <quant> ∈ CANONICAL_QUANT_TOKENS.

In all cases <robot> and <task> match the segment shape (robot ids are an open registry, vocabulary-checked in CI). The owner prefix (<owner>/) is ignored.

When model_family is given (a VLA skill), the <model> segment must be family-consistent — one of _MODEL_FAMILY_ALLOWED_TOKENS for that family, or the family id itself for a family with no entry — so a checkpoint can't be mislabelled with another family's token. When model_family is None (tool / ROS skills, or a pure name check) <model> ∈ CANONICAL_MODEL_TOKENS is required.

Example

repo_name_is_canonical( ... "OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16", kind="vla" ... ) True repo_name_is_canonical( ... "OpenRAL/rskill-moveit-multi-eef_pose", kind="ros_action" ... ) # ROS wrapper: no quant segment True repo_name_is_canonical( ... "OpenRAL/rskill-pi05-franka_panda-libero-bf16", ... kind="vla", ... model_family="smolvla", ... ) # wrong family token False repo_name_is_canonical("OpenRAL/rskill-playbook-find_object", kind="playbook") True repo_name_is_canonical("OpenRAL/rskill-smolvla-libero", kind="vla") # old 3-part name False

Source code in python/core/src/openral_core/schemas.py

def repo_name_is_canonical(name: str, *, kind: RSkillKind, model_family: str | None = None) -> bool:
    """Return ``True`` when ``name`` obeys the repo-naming grammar for ``kind``.

    Three shapes, chosen by ``kind``:

    * ``playbook`` → ``rskill-playbook-<name>`` (3 parts).
    * ``ros_action`` / ``ros_service`` (``_WEIGHTLESS_KINDS``) →
      ``rskill-<model>-<robot>-<task>`` (4 parts) — no ``<quant>`` segment, since
      a ROS wrapper carries no weights.
    * every other kind → ``rskill-<model>-<robot>-<task>-<quant>`` (5 parts) with
      ``<quant>`` ∈ ``CANONICAL_QUANT_TOKENS``.

    In all cases ``<robot>`` and ``<task>`` match the segment shape (robot ids
    are an open registry, vocabulary-checked in CI). The owner prefix
    (``<owner>/``) is ignored.

    When ``model_family`` is given (a VLA skill), the ``<model>`` segment must
    be family-consistent — one of ``_MODEL_FAMILY_ALLOWED_TOKENS`` for that
    family, or the family id itself for a family with no entry — so a
    checkpoint can't be mislabelled with another family's token. When
    ``model_family`` is ``None`` (tool / ROS skills, or a pure name check)
    ``<model>`` ∈ ``CANONICAL_MODEL_TOKENS`` is required.

    Example:
        >>> repo_name_is_canonical(
        ...     "OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16", kind="vla"
        ... )
        True
        >>> repo_name_is_canonical(
        ...     "OpenRAL/rskill-moveit-multi-eef_pose", kind="ros_action"
        ... )  # ROS wrapper: no quant segment
        True
        >>> repo_name_is_canonical(
        ...     "OpenRAL/rskill-pi05-franka_panda-libero-bf16",
        ...     kind="vla",
        ...     model_family="smolvla",
        ... )  # wrong family token
        False
        >>> repo_name_is_canonical("OpenRAL/rskill-playbook-find_object", kind="playbook")
        True
        >>> repo_name_is_canonical("OpenRAL/rskill-smolvla-libero", kind="vla")  # old 3-part name
        False
    """
    repo = name.split("/", 1)[1] if "/" in name else name
    parts = repo.split("-")
    if parts[0] != "rskill":
        return False
    if kind == "playbook":
        return (
            len(parts) == _RSKILL_PLAYBOOK_NAME_PARTS
            and parts[1] == "playbook"
            and bool(_RSKILL_NAME_SEGMENT_RE.match(parts[2]))
        )
    if kind in _WEIGHTLESS_KINDS:
        if len(parts) != _RSKILL_ROS_NAME_PARTS:
            return False
        _, model, robot, task = parts
        quant_ok = True
    else:
        if len(parts) != _RSKILL_NAME_PARTS:
            return False
        _, model, robot, task, quant = parts
        quant_ok = quant in CANONICAL_QUANT_TOKENS
    model_ok = (
        model in CANONICAL_MODEL_TOKENS
        if model_family is None
        else model in _allowed_model_tokens(model_family)
    )
    return (
        model_ok
        and bool(_RSKILL_NAME_SEGMENT_RE.match(robot))
        and bool(_RSKILL_NAME_SEGMENT_RE.match(task))
        and quant_ok
    )

required_vla_camera_slots(manifest: RSkillManifest, description: RobotDescription | None) -> tuple[str, ...]

RGB VLA slots an rSkill consumes on this robot, in robot manifest order.

The robot's slots (sensor_name_to_slot) trimmed to the ones the manifest's sensors_required names by vla_feature_key — a Franka exposes camera3 for VLABench while a LIBERO skill wants only camera1/camera2. All robot slots when the manifest names none.

Example

desc = RobotDescription.from_yaml("robots/franka_panda/robot.yaml") skill = RSkillManifest.from_yaml("rskills/smolvla-libero/rskill.yaml") required_vla_camera_slots(skill, desc) ('camera1', 'camera2')

Source code in python/core/src/openral_core/schemas.py

def required_vla_camera_slots(
    manifest: RSkillManifest, description: RobotDescription | None
) -> tuple[str, ...]:
    """RGB VLA slots an rSkill consumes on this robot, in robot manifest order.

    The robot's slots (``sensor_name_to_slot``) trimmed to the ones the
    manifest's ``sensors_required`` names by ``vla_feature_key`` — a Franka
    exposes ``camera3`` for VLABench while a LIBERO skill wants only
    ``camera1``/``camera2``. All robot slots when the manifest names none.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/franka_panda/robot.yaml")
        >>> skill = RSkillManifest.from_yaml("rskills/smolvla-libero/rskill.yaml")
        >>> required_vla_camera_slots(skill, desc)
        ('camera1', 'camera2')
    """
    slots = tuple(sensor_name_to_slot(description).values())
    required = _required_rgb_slots(manifest.sensors_required)
    return tuple(slot for slot in slots if slot in required) if required else slots

resolve_sensor_overlays(robot_yaml: str | Path, scene_unit: str | None, *, required: bool) -> list[SensorOverlay]

The sensor overlays of the robot unit this host runs.

The unit is $OPENRAL_ROBOT_UNIT if set, else the scene's robot_unit. With neither, there is nothing to overlay; but when required (a real deploy) and the robot ships a units/ directory, that is refused: its bindings and calibration are per unit, so running on the manifest's nominal values would be a silent guess.

Raises:

Type	Description
ROSConfigError	A required unit is not selected, or the selected one is invalid.

Example

resolve_sensor_overlays("robots/franka_panda/robot.yaml", None, required=True) []

Source code in python/core/src/openral_core/schemas.py

def resolve_sensor_overlays(
    robot_yaml: str | Path, scene_unit: str | None, *, required: bool
) -> list[SensorOverlay]:
    """The sensor overlays of the robot unit this host runs.

    The unit is ``$OPENRAL_ROBOT_UNIT`` if set, else the scene's ``robot_unit``. With
    neither, there is nothing to overlay; but when ``required`` (a real deploy) and the
    robot ships a ``units/`` directory, that is refused: its bindings and calibration
    are per unit, so running on the manifest's nominal values would be a silent guess.

    Raises:
        ROSConfigError: A required unit is not selected, or the selected one is invalid.

    Example:
        >>> resolve_sensor_overlays("robots/franka_panda/robot.yaml", None, required=True)
        []
    """
    unit = os.environ.get(ROBOT_UNIT_ENV) or scene_unit
    if not unit:
        units_dir = Path(robot_yaml).parent / "units"
        if required and units_dir.is_dir():
            have = sorted(p.stem for p in units_dir.glob("*.yaml"))
            raise ROSConfigError(
                f"{Path(robot_yaml).parent.name} has per-unit sensor overlays "
                f"({', '.join(have)}) but none is selected: set ${ROBOT_UNIT_ENV} or the "
                "scene's robot_unit to the unit this host drives"
            )
        return []
    return list(load_robot_unit(robot_yaml, unit).sensors)

scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole

Linearly rescale pinhole intrinsics to a new render resolution.

For a camera with a fixed field of view, fx/fy/cx/cy scale linearly with the image dimensions: a frame rendered at twice the width has twice the focal length (px) and twice the principal-point x. This is the consistency rule deploy-sim needs — the canonical manifest pins one nominal resolution, but a scene may render the same MuJoCo camera at another (scene.observation_width/height). Publishing the manifest's nominal intrinsics on a different-resolution render would back-project depth pixels and project occupancy voxels with the wrong focal length, corrupting the OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera model matched to whatever was actually rendered.

The distortion model and coefficients are preserved unchanged (coefficients are resolution-independent for the normalised plumb-bob model). When the target already equals base's resolution the input is returned as-is.

Parameters:

Name	Type	Description	Default
base	IntrinsicsPinhole	Nominal pinhole intrinsics (e.g. from a SensorSpec).	required
width	int	Target render width in pixels (> 0).	required
height	int	Target render height in pixels (> 0).	required

Returns:

Type	Description
IntrinsicsPinhole	Intrinsics scaled so (fx, fy, cx, cy) correspond to ``(width,
IntrinsicsPinhole	height)at the same field of view asbase``.

Raises:

Type	Description
ValueError	If width or height is not strictly positive.

Example

base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0) hi = scale_intrinsics_to(base, 640, 640) (hi.width, hi.fx, hi.cx) (640, 640.0, 320.0)

Source code in python/core/src/openral_core/schemas.py

def scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole:
    """Linearly rescale pinhole intrinsics to a new render resolution.

    For a camera with a *fixed* field of view, ``fx``/``fy``/``cx``/``cy`` scale
    linearly with the image dimensions: a frame rendered at twice the width has
    twice the focal length (px) and twice the principal-point x. This is the
    consistency rule deploy-sim needs — the canonical manifest pins one nominal
    resolution, but a scene may render the same MuJoCo camera at another
    (``scene.observation_width``/``height``). Publishing the manifest's nominal
    intrinsics on a different-resolution render would back-project depth pixels
    and project occupancy voxels with the wrong focal length, corrupting the
    OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera
    model matched to whatever was actually rendered.

    The distortion model and coefficients are preserved unchanged (coefficients
    are resolution-independent for the normalised plumb-bob model). When the
    target already equals ``base``'s resolution the input is returned as-is.

    Args:
        base: Nominal pinhole intrinsics (e.g. from a ``SensorSpec``).
        width: Target render width in pixels (``> 0``).
        height: Target render height in pixels (``> 0``).

    Returns:
        Intrinsics scaled so ``(fx, fy, cx, cy)`` correspond to ``(width,
        height)`` at the same field of view as ``base``.

    Raises:
        ValueError: If ``width`` or ``height`` is not strictly positive.

    Example:
        >>> base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0)
        >>> hi = scale_intrinsics_to(base, 640, 640)
        >>> (hi.width, hi.fx, hi.cx)
        (640, 640.0, 320.0)
    """
    if width <= 0 or height <= 0:
        raise ValueError(f"target resolution must be positive; got ({width}, {height})")
    if width == base.width and height == base.height:
        return base
    sx = width / base.width
    sy = height / base.height
    return IntrinsicsPinhole(
        width=width,
        height=height,
        fx=base.fx * sx,
        fy=base.fy * sy,
        cx=base.cx * sx,
        cy=base.cy * sy,
        distortion_model=base.distortion_model,
        distortion_coeffs=list(base.distortion_coeffs),
    )

scene_family(task_id: str) -> str

Reduce an evaluated_tasks entry to its scene-family key.

The family is the leading token before any "/" — "rlbench/open_drawer" → "rlbench", "libero_spatial" → "libero_spatial".

Example

scene_family("rlbench/open_drawer") 'rlbench' scene_family("metaworld") 'metaworld'

Source code in python/core/src/openral_core/schemas.py

def scene_family(task_id: str) -> str:
    """Reduce an ``evaluated_tasks`` entry to its scene-family key.

    The family is the leading token before any ``"/"`` — ``"rlbench/open_drawer"``
    → ``"rlbench"``, ``"libero_spatial"`` → ``"libero_spatial"``.

    Example:
        >>> scene_family("rlbench/open_drawer")
        'rlbench'
        >>> scene_family("metaworld")
        'metaworld'
    """
    return task_id.split("/", 1)[0]

scene_task_space_compatible(family: str, skill_space: TaskSpace) -> TaskSpaceMatch

Check an rSkill's TaskSpace is executable by a scene family.

The third leg of the cross-layer gate: a skill fits a scene when every ControlMode it emits is in the scene family's executed set, and — when the family fixes a width — its total dimensionality matches. Pairs with task_space_compatible (rSkill x robot); together they close the rSkill x robot x scene triangle the audit found unconnected.

Parameters:

Name	Type	Description	Default
family	str	The scene-family key (see scene_family).	required
skill_space	TaskSpace	The task space the rSkill emits (TaskSpace.from_action_contract).	required

Returns:

Type	Description
TaskSpaceMatch	A TaskSpaceMatch — ok plus a reason per incompatibility.
TaskSpaceMatch	An unknown family is reported as a single reason (not an exception)
TaskSpaceMatch	so a sweep can collect every gap in one pass.

Example

space = TaskSpace( ... segments=[ ... TaskSpaceSegment( ... family=TaskSpaceFamily.JOINT, ... control_mode=ControlMode.JOINT_POSITION, ... width=14, ... ) ... ] ... ) scene_task_space_compatible("robotwin", space).ok True

Source code in python/core/src/openral_core/schemas.py

def scene_task_space_compatible(family: str, skill_space: TaskSpace) -> TaskSpaceMatch:
    """Check an rSkill's ``TaskSpace`` is executable by a scene family.

    The third leg of the cross-layer gate: a skill fits a
    scene when every ``ControlMode`` it emits is in the scene family's
    executed set, and — when the family fixes a width — its total dimensionality
    matches. Pairs with ``task_space_compatible`` (rSkill x robot); together
    they close the rSkill x robot x scene triangle the audit found unconnected.

    Args:
        family: The scene-family key (see ``scene_family``).
        skill_space: The task space the rSkill emits
            (``TaskSpace.from_action_contract``).

    Returns:
        A ``TaskSpaceMatch`` — ``ok`` plus a reason per incompatibility.
        An unknown ``family`` is reported as a single reason (not an exception)
        so a sweep can collect every gap in one pass.

    Example:
        >>> space = TaskSpace(
        ...     segments=[
        ...         TaskSpaceSegment(
        ...             family=TaskSpaceFamily.JOINT,
        ...             control_mode=ControlMode.JOINT_POSITION,
        ...             width=14,
        ...         )
        ...     ]
        ... )
        >>> scene_task_space_compatible("robotwin", space).ok
        True
    """
    spec = SCENE_FAMILY_TASK_SPACE.get(family)
    if spec is None:
        return TaskSpaceMatch(
            ok=False,
            reasons=[
                f"scene family {family!r} has no declared task space in SCENE_FAMILY_TASK_SPACE"
            ],
        )
    reasons: list[str] = []
    extra = skill_space.control_modes - spec.modes
    if extra:
        reasons.append(
            f"control mode(s) {sorted(m.value for m in extra)} not executable by "
            f"scene family {family!r} (executes {sorted(m.value for m in spec.modes)})"
        )
    if spec.action_dim is not None and skill_space.total_dim != spec.action_dim:
        reasons.append(
            f"action dim {skill_space.total_dim} != scene family {family!r} "
            f"expected dim {spec.action_dim}"
        )
    return TaskSpaceMatch(ok=not reasons, reasons=reasons)

sensor_name_to_slot(description: RobotDescription | None) -> dict[str, str]

Map each RGB sensor NAME to its VLA slot, in manifest order.

The slot is the vla_feature_key suffix (observation.images.camera1 -> camera1), falling back to the sensor name when the sensor declares no key. Slots are the one camera namespace shared by openral sim run, deploy sim and deploy run: image_preprocessing.aliases keys and adapter camera_keys are slots, while topics and sim renders stay keyed by sensor name.

Example

desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml") sensor_name_to_slot(desc)

Source code in python/core/src/openral_core/schemas.py

def sensor_name_to_slot(description: RobotDescription | None) -> dict[str, str]:
    """Map each RGB sensor NAME to its VLA slot, in manifest order.

    The slot is the ``vla_feature_key`` suffix (``observation.images.camera1``
    -> ``camera1``), falling back to the sensor name when the sensor declares
    no key. Slots are the one camera namespace shared by ``openral sim run``,
    ``deploy sim`` and ``deploy run``: ``image_preprocessing.aliases`` keys
    and adapter ``camera_keys`` are slots, while topics and sim renders stay
    keyed by sensor name.

    Example:
        >>> desc = RobotDescription.from_yaml("robots/so101_follower/robot.yaml")
        >>> sensor_name_to_slot(desc)
        {'top': 'camera1', 'wrist': 'camera2'}
    """
    if description is None:
        return {}
    return {
        s.name: s.vla_feature_key.rsplit(".", 1)[-1] if s.vla_feature_key else s.name
        for s in description.sensors
        if s.modality == "rgb"
    }

task_space_compatible(skill_space: TaskSpace, robot: RobotDescription, *, hal_mode: Literal['sim', 'real'] = 'real') -> TaskSpaceMatch

Check an rSkill's TaskSpace is executable on a robot.

The single cross-layer gate replacing implicit embodiment_tags/dim matching. Compatible when every segment's control_mode is executable for hal_mode ("sim": SIM_EXECUTABLE_CONTROL_MODES; "real": the robot's capabilities.supported_control_modes), every CARTESIAN_*/GRIPPER_*/DEX_HAND_JOINT segment names a real end-effector, and the total joint-segment width fits the robot's joint count.

Parameters:

Name	Type	Description	Default
skill_space	TaskSpace	The task space the rSkill emits (TaskSpace.from_action_contract).	required
robot	RobotDescription	The target robot description.	required
hal_mode	Literal['sim', 'real']	"sim" (default-sim OSC packers) or "real" (hardware controllers).	'real'

Returns:

Type	Description
TaskSpaceMatch	A TaskSpaceMatch — ok plus a reason per incompatibility.

Example

from openral_core.schemas import ( ... EmbodimentKind, ... EndEffectorSpec, ... JointSpec, ... JointType, ... RobotCapabilities, ... SafetyEnvelope, ... ) robot = RobotDescription( ... name="franka_panda", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["franka_panda"], ... ), ... safety=SafetyEnvelope(), ... ) space = TaskSpace( ... segments=[ ... TaskSpaceSegment( ... family=TaskSpaceFamily.CARTESIAN, ... control_mode=ControlMode.CARTESIAN_DELTA, ... width=6, ... target="panda_hand", ... ) ... ] ... )

Real hardware needs a cartesian controller the Franka does not declare:

task_space_compatible(space, robot, hal_mode="real").ok False

In sim the robosuite OSC packer synthesises it from joint commands:

task_space_compatible(space, robot, hal_mode="sim").ok True

Source code in python/core/src/openral_core/schemas.py

def task_space_compatible(
    skill_space: TaskSpace,
    robot: RobotDescription,
    *,
    hal_mode: Literal["sim", "real"] = "real",
) -> TaskSpaceMatch:
    """Check an rSkill's ``TaskSpace`` is executable on a robot.

    The single cross-layer gate replacing implicit ``embodiment_tags``/dim
    matching. Compatible when every segment's ``control_mode`` is executable
    for ``hal_mode`` (``"sim"``: ``SIM_EXECUTABLE_CONTROL_MODES``;
    ``"real"``: the robot's ``capabilities.supported_control_modes``), every
    ``CARTESIAN_*``/``GRIPPER_*``/``DEX_HAND_JOINT`` segment names a real
    end-effector, and the total joint-segment width fits the robot's joint count.

    Args:
        skill_space: The task space the rSkill emits (``TaskSpace.from_action_contract``).
        robot: The target robot description.
        hal_mode: ``"sim"`` (default-sim OSC packers) or ``"real"`` (hardware controllers).

    Returns:
        A ``TaskSpaceMatch`` — ``ok`` plus a reason per incompatibility.

    Example:
        >>> from openral_core.schemas import (
        ...     EmbodimentKind,
        ...     EndEffectorSpec,
        ...     JointSpec,
        ...     JointType,
        ...     RobotCapabilities,
        ...     SafetyEnvelope,
        ... )
        >>> robot = RobotDescription(
        ...     name="franka_panda",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="panda_hand", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(
        ...         supported_control_modes=[ControlMode.JOINT_POSITION],
        ...         embodiment_tags=["franka_panda"],
        ...     ),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> space = TaskSpace(
        ...     segments=[
        ...         TaskSpaceSegment(
        ...             family=TaskSpaceFamily.CARTESIAN,
        ...             control_mode=ControlMode.CARTESIAN_DELTA,
        ...             width=6,
        ...             target="panda_hand",
        ...         )
        ...     ]
        ... )
        >>> # Real hardware needs a cartesian controller the Franka does not declare:
        >>> task_space_compatible(space, robot, hal_mode="real").ok
        False
        >>> # In sim the robosuite OSC packer synthesises it from joint commands:
        >>> task_space_compatible(space, robot, hal_mode="sim").ok
        True
    """
    reasons: list[str] = []
    if hal_mode == "sim":
        executable: set[ControlMode] = set(SIM_EXECUTABLE_CONTROL_MODES)
    else:
        executable = {ControlMode(m) for m in robot.capabilities.supported_control_modes}
    ee_names = {ee.name for ee in robot.end_effectors}
    n_joints = len(robot.joints)

    joint_width = 0
    for seg in skill_space.segments:
        if seg.control_mode not in executable:
            reasons.append(
                f"control_mode {seg.control_mode.value!r} not executable in "
                f"hal_mode={hal_mode!r} (segment family={seg.family.value!r}, "
                f"width={seg.width})"
            )
        if seg.family in (
            TaskSpaceFamily.CARTESIAN,
            TaskSpaceFamily.GRIPPER,
            TaskSpaceFamily.DEX_HAND,
        ):
            if seg.target is None:
                reasons.append(
                    f"{seg.family.value} segment ({seg.control_mode.value}) has no "
                    "target end-effector to address"
                )
            elif seg.target not in ee_names:
                reasons.append(
                    f"{seg.family.value} segment targets end-effector "
                    f"{seg.target!r}, not declared on robot {robot.name!r} "
                    f"(have: {sorted(ee_names)})"
                )
        if seg.family is TaskSpaceFamily.JOINT:
            joint_width += seg.width

    if joint_width > n_joints:
        reasons.append(
            f"joint-segment width {joint_width} exceeds robot joint count "
            f"{n_joints} on {robot.name!r}"
        )

    return TaskSpaceMatch(ok=not reasons, reasons=reasons)

openral exception hierarchy — use these, do not invent new base classes.

All exceptions derive from ROSError. ROSSafetyViolation and its subclasses must never be caught silently; they are only caught at the safety supervisor boundary where they trigger an E-stop and a structured incident log.

Example

try: ... raise ROSConfigError("missing URDF") ... except ROSError as exc: ... print(type(exc).name, str(exc)) ROSConfigError missing URDF

ROSCapabilityMismatch

Bases: ROSError

A skill requires a capability the target robot does not have.

ROSCollisionImminent

Bases: ROSSafetyViolation

A proposed motion would self-collide or strike a world obstacle.

Raised on the safety path when geometric checking finds a chunk whose forward-kinematic sweep brings a robot link within clearance of another link or a world primitive.

ROSConfigError

Bases: ROSError

Bad manifest, missing weights, or invalid YAML / URDF.

ROSDeadlineMissed

Bases: ROSFleetError

Cloud RTT exceeded the skill's deadline; fallback was not configured.

ROSDispatchUnavailable

Bases: ROSFleetError

No dispatcher (edge or cloud) is available for the requested skill.

ROSEStopRequested

Bases: ROSSafetyViolation

An emergency stop was requested; all actuation must cease immediately.

ROSError

Bases: Exception

Base class for all OpenRAL errors.

ROSFleetError

Bases: ROSError

A fleet-level or dispatch error.

ROSForceLimitExceeded

Bases: ROSSafetyViolation

Measured or predicted contact force exceeds the configured limit.

Not raised anywhere yet: no check currently compares a force against SafetyEnvelope.max_force_n. The class is exported so the force/current contact layer (issue #373) can raise it without an API change.

ROSGPUMemoryError

Bases: ROSRuntimeError

Out of GPU memory; use a smaller skill variant or quantize further.

ROSObjectNotInMemory

Bases: ROSPerceptionStale

A spatial-memory query matched no node, or only nodes that are stale.

Raised by the scene-graph query surface when a RecallObjectQuery / ResolvePlaceQuery cannot be satisfied. The caller degrades by treating the target as unknown (and may trigger active search) — it never fabricates a pose (CLAUDE.md §1.2, §1.4).

ROSPerceptionStale

Bases: ROSError

A sensor reading is older than the configured staleness deadline.

ROSPlanningError

Bases: ROSError

The reasoner failed to produce a valid plan.

ROSQuantizationError

Bases: ROSRuntimeError

Quantization failed or produced an incompatible engine.

ROSReasonerInvalidPlan

Bases: ROSPlanningError

The LLM returned a plan that failed schema or capability validation.

ROSRskillGoalSatisfied

Bases: ROSError

A wrapped-ROS rSkill has finished its goal successfully.

Used as a typed completion signal raised by openral_rskill.ros_action_rskill.ROSActionRskill._step_impl after the last waypoint of a one-shot planner (e.g. MoveIt) has been emitted, or after a result-only wrapped action (e.g. Nav2 NavigateToPose) reports success. The ExecuteRskill action server catches this specifically and closes the goal with success=True.

This is NOT an error — it inherits ROSError only to stay inside the OpenRAL exception surface so it is greppable and discoverable via the standard hierarchy. It must be caught ONLY at the rskill_runner_node execute-callback boundary; everywhere else it is a programming bug to raise it.

ROSRuntimeError

Bases: ROSError

General runtime failure during skill execution or HAL operation.

ROSSafetyViolation

Bases: ROSError

A safety constraint was violated.

NEVER catch this silently. Catch only at the safety supervisor boundary, trigger an E-stop, and emit a structured incident log entry.

ROSWorkspaceViolation

Bases: ROSSafetyViolation

An action would move the robot outside its allowed workspace.

CLI (openral)

openral CLI entry point — openral command.

Two modes of use:

• One-shot: openral <subcommand> [args...] runs a single command and exits. Use this in scripts and CI. openral --help lists the surface.
• Interactive REPL: openral with no arguments drops into a prompt where subcommands run bare (sim run --config … instead of openral sim run --config …). Type help for the menu, exit or Ctrl-D to leave.

Sub-commands

doctor Diagnose the host environment (Python, OS, ROS 2, GPU, USB). detect Probe hardware and write a full RobotDescription robot.yaml. connect Open a HAL connection to a robot and verify it responds. behavior serve Serve an rSkill to the official BEHAVIOR evaluator. calibrate camera Calibrate a camera sensor using ros2 camera_calibration. install Install opt-in dependency groups (sim, ros, libero, …). rskill search Find installable rSkills on the OpenRAL HF Hub org. rskill install Download an rSkill from the HF Hub and register it locally. rskill list List all locally installed rSkills. rskill check Report which installed rSkills will run on the current host. rskill new Scaffold a new local rSkill from rskills/template/. collision lower Lower a robot's URDF/SRDF into its self-collision model. collision check Fail if a manifest drifts from its lowered collision model. check Cross-validate every robot/skill/scene manifest in one pass.

Run openral --help for full usage.

CheckResult

Bases: NamedTuple

One row in the openral doctor output table.

Attributes:

Name	Type	Description
check	str	Short name of the thing being checked.
status	str	One of ok, fail, missing, absent, info, warn.
details	str	Human-readable detail string (path, version, device list, …).

behavior_serve(rskill: str = typer.Option(..., '--rskill', help='rSkill reference: local name/path or Hugging Face repo id.'), task: str = typer.Option(..., '--task', help='BEHAVIOR task name, e.g. turning_on_radio.'), instruction: str | None = typer.Option(None, '--instruction', help='Policy language instruction. Defaults to the task name with underscores replaced.'), host: str = typer.Option('127.0.0.1', '--host', help='WebSocket bind address.'), port: int = typer.Option(8000, '--port', min=1, max=65535, help='WebSocket port used by omnigibson.eval.eval.'), device: str = typer.Option('auto', '--device', help='Policy device override, e.g. auto, cuda:0, or cpu.'), state_dim: int = typer.Option(61, '--state-dim', min=1, help='Expected R1Pro proprioception width.'), action_dim: int = typer.Option(23, '--action-dim', min=1, help='Expected R1Pro action width.')) -> None

Serve one OpenRAL rSkill to BEHAVIOR's official evaluator.

Source code in python/cli/src/openral_cli/main.py

@behavior_app.command("serve")
def behavior_serve(
    rskill: str = typer.Option(
        ...,
        "--rskill",
        help="rSkill reference: local name/path or Hugging Face repo id.",
    ),
    task: str = typer.Option(
        ...,
        "--task",
        help="BEHAVIOR task name, e.g. turning_on_radio.",
    ),
    instruction: str | None = typer.Option(
        None,
        "--instruction",
        help="Policy language instruction. Defaults to the task name with underscores replaced.",
    ),
    host: str = typer.Option(
        "127.0.0.1",
        "--host",
        help="WebSocket bind address.",
    ),
    port: int = typer.Option(
        8000,
        "--port",
        min=1,
        max=65535,
        help="WebSocket port used by omnigibson.eval.eval.",
    ),
    device: str = typer.Option(
        "auto",
        "--device",
        help="Policy device override, e.g. auto, cuda:0, or cpu.",
    ),
    state_dim: int = typer.Option(
        61,
        "--state-dim",
        min=1,
        help="Expected R1Pro proprioception width.",
    ),
    action_dim: int = typer.Option(
        23,
        "--action-dim",
        min=1,
        help="Expected R1Pro action width.",
    ),
) -> None:
    """Serve one OpenRAL rSkill to BEHAVIOR's official evaluator."""
    from openral_cli.behavior import _serve_behavior_policy

    vla_spec = _parse_rskill_cli_arg(rskill).model_copy(update={"device": device})
    _serve_behavior_policy(
        vla_spec,
        task=task,
        instruction=instruction or task.replace("_", " "),
        host=host,
        port=port,
        state_dim=state_dim,
        action_dim=action_dim,
    )

benchmark_list(benchmarks_dir: Path = typer.Option(Path('benchmarks'), '--benchmarks-dir', help='Search directory for benchmark suite YAMLs.')) -> None

List every benchmark suite id available under benchmarks/*.yaml.

Each entry is a paste-able --suite value for openral benchmark run. No rollout, no GPU.

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("list")
def benchmark_list(
    benchmarks_dir: Path = typer.Option(
        Path("benchmarks"),
        "--benchmarks-dir",
        help="Search directory for benchmark suite YAMLs.",
    ),
) -> None:
    """List every benchmark suite id available under ``benchmarks/*.yaml``.

    Each entry is a paste-able ``--suite`` value for ``openral benchmark run``.
    No rollout, no GPU.
    """
    if not benchmarks_dir.is_dir():
        console.print(f"[red]No benchmarks dir at {benchmarks_dir}[/red]")
        raise typer.Exit(code=1)
    suites = sorted(p.stem for p in benchmarks_dir.glob("*.yaml") if p.is_file())
    if not suites:
        print("<none>")
        return
    for suite in suites:
        print(suite)

benchmark_report(rskills_dir: Path = typer.Option(Path('rskills'), '--rskills-dir', help='Directory containing rSkill packages (each with optional eval/*.json).'), json_output: bool = typer.Option(False, '--json', help='Emit a JSON dump instead of the rich-table summary.')) -> None

Walk every <skill>/eval/*.json and print a benchmark roll-up.

Validates each JSON against openral_core.RSkillEvalResult (the same schema the rSkill loader uses at install time) so a rotted file fails loudly instead of silently being skipped.

Example

openral benchmark report

openral benchmark report --json > /tmp/report.json

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("report")
def benchmark_report(
    rskills_dir: Path = typer.Option(
        Path("rskills"),
        "--rskills-dir",
        help="Directory containing rSkill packages (each with optional eval/*.json).",
    ),
    json_output: bool = typer.Option(
        False,
        "--json",
        help="Emit a JSON dump instead of the rich-table summary.",
    ),
) -> None:
    """Walk every ``<skill>/eval/*.json`` and print a benchmark roll-up.

    Validates each JSON against `openral_core.RSkillEvalResult`
    (the same schema the rSkill loader uses at install time) so a rotted
    file fails loudly instead of silently being skipped.

    Example:
        >>> # openral benchmark report
        >>> # openral benchmark report --json > /tmp/report.json
    """
    from openral_core import RSkillEvalResult
    from pydantic import ValidationError

    if not rskills_dir.is_dir():
        console.print(f"[red]rskills directory not found: {rskills_dir}[/red]")
        raise typer.Exit(code=1)

    rows: list[dict[str, object]] = []
    for skill_dir in sorted(p for p in rskills_dir.iterdir() if p.is_dir()):
        eval_dir = skill_dir / "eval"
        if not eval_dir.is_dir():
            continue
        for json_path in sorted(eval_dir.glob("*.json")):
            try:
                result = RSkillEvalResult.from_json(str(json_path))
            except (ValidationError, _json.JSONDecodeError) as exc:
                console.print(f"[red]invalid {json_path}:[/red] {exc}")
                raise typer.Exit(code=1) from exc
            rows.append(
                {
                    "rskill": skill_dir.name,
                    "benchmark": result.benchmark.name,
                    "robot": result.benchmark.robot,
                    "simulator": result.benchmark.simulator,
                    "reproduced_locally": result.source.reproduced_locally,
                    "model_variant": result.source.model_variant,
                    "status": result.source.status,
                    "results": result.results,
                    "path": (
                        str(json_path.relative_to(Path.cwd()))
                        if json_path.is_relative_to(Path.cwd())
                        else str(json_path)
                    ),
                }
            )

    if json_output:
        console.print_json(_json.dumps(rows, indent=2, default=str))
        return

    if not rows:
        console.print(f"[yellow]no rskills/<id>/eval/*.json files under {rskills_dir}[/yellow]")
        return

    rows.sort(key=lambda r: (str(r["benchmark"]), str(r["rskill"])))
    table = Table(title=f"rSkill benchmark report — {len(rows)} entries")
    table.add_column("Benchmark", style="cyan")
    table.add_column("rSkill", style="magenta")
    table.add_column("Variant", style="dim")
    table.add_column("Robot", style="dim")
    table.add_column("Repro local?", justify="center")
    table.add_column("Headline result", style="green")
    table.add_column("Status", style="dim")
    for row in rows:
        results = row["results"]
        headline = _summarize_results(results) if isinstance(results, dict) else "—"
        table.add_row(
            str(row["benchmark"]),
            str(row["rskill"]),
            str(row["model_variant"]),
            str(row["robot"]),
            "✓" if row["reproduced_locally"] else "✗",
            headline,
            str(row["status"] or ""),
        )
    console.print(table)

benchmark_run(suite: str = typer.Option(..., '--suite', help='Benchmark suite to evaluate — a bare ``list[BenchmarkScene]`` YAML. Either a built-in id (resolved to `benchmarks/<id>.yaml`) or a direct YAML path.'), rskill: str = typer.Option(..., '--rskill', help="rSkill reference — a bare name ('smolvla-libero'), a path ('rskills/smolvla-libero'), or an HF Hub repo id ('OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'). Raw hf:// is rejected (weights must come from a manifest). The policy adapter id is read from the manifest's `model_family` field."), out: Path | None = typer.Option(None, '--out', help='Output path for the RSkillEvalResult JSON. Defaults to rskills/<dir>/eval/<suite_id>.json derived from the rSkill ref.'), device: str | None = typer.Option(None, '--device', help='Torch device override for the policy (cpu, cuda:0, mps, auto).'), save_dir: Path | None = typer.Option(None, '--save-dir', help='Optional adapter-side artefact directory (videos, traces).'), benchmarks_dir: Path = typer.Option(Path('benchmarks'), '--benchmarks-dir', help='Search directory for built-in benchmark suite YAMLs.'), task: str | None = typer.Option(None, '--task', help="Run only this single task id from the suite (e.g. 'libero_spatial/3' or 'maniskill3/PushCube-v1'). Omit to run every task the rSkill supports (the suite is auto-filtered to the rSkill's evaluated_tasks)."), n_episodes: int | None = typer.Option(None, '--n-episodes', help='Override ``BenchmarkScene.n_episodes`` for every scene in the suite (lower for quick smoke runs). The published-protocol value lives in the suite YAML; this flag is for fast iteration, not for paper-comparison numbers.'), dry_run: bool = typer.Option(False, '--dry-run', help='Resolve the suite + VLA and print the planned (task x seed) matrix without running any rollouts. Useful in CI to validate config wiring.'), update_manifest: bool = typer.Option(True, '--update-manifest/--no-update-manifest', help='On success, write the avg_success_rate back into the rSkill manifest at `benchmarks.<suite_id>`. Surgical edit — preserves comments. Only fires for locally-resolvable rSkills. Disable for read-only paper-number runs.'), video: bool = typer.Option(True, '--video/--no-video', help='Record per-step world frames and write one MP4 per episode (named <task>[_seed<n>]_<rskill>_<success|fail>.mp4 plus a videos.json manifest). On by default; frames are written and freed per episode. Disable for allocation-light CI runs.'), video_dir: Path | None = typer.Option(None, '--video-dir', help='Destination directory for --video MP4s. Defaults to <eval JSON dir>/videos/<suite_id> next to the RSkillEvalResult.'), dashboard: bool = typer.Option(False, '--dashboard', help='Boot `openral dashboard` as a child process, point OTel at it, and shut it down on exit (same semantics as `openral sim run --dashboard`).'), dashboard_port: int = typer.Option(4318, '--dashboard-port', help='Port for the spawned dashboard when --dashboard is set.')) -> None

Run a benchmark suite and write a validated RSkillEvalResult JSON.

The runner iterates scenes x range(seed, seed + n_episodes), delegating each rollout to openral_sim.SimRunner so the rSkill compatibility check, OTel spans, and latency-budget reporting are identical to openral sim run. Each BenchmarkScene carries its own scene + task + robot; the BenchmarkSpec wrapper class was removed so the suite is a bare list of scenes whose id is the YAML filename stem.

Example

openral benchmark run --suite libero_spatial \

--rskill smolvla-libero

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("run")
def benchmark_run(
    suite: str = typer.Option(
        ...,
        "--suite",
        help=(
            "Benchmark suite to evaluate — a bare ``list[BenchmarkScene]`` YAML. "
            "Either a built-in id (resolved to "
            "`benchmarks/<id>.yaml`) or a direct YAML path."
        ),
    ),
    rskill: str = typer.Option(
        ...,
        "--rskill",
        help=(
            "rSkill reference — a bare name ('smolvla-libero'), a "
            "path ('rskills/smolvla-libero'), or an HF Hub repo id "
            "('OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'). "
            "Raw hf:// is rejected (weights must come from a manifest). "
            "The policy adapter id is read from the manifest's `model_family` "
            "field."
        ),
    ),
    out: Path | None = typer.Option(
        None,
        "--out",
        help=(
            "Output path for the RSkillEvalResult JSON. Defaults to "
            "rskills/<dir>/eval/<suite_id>.json derived from the rSkill ref."
        ),
    ),
    device: str | None = typer.Option(
        None,
        "--device",
        help="Torch device override for the policy (cpu, cuda:0, mps, auto).",
    ),
    save_dir: Path | None = typer.Option(
        None,
        "--save-dir",
        help="Optional adapter-side artefact directory (videos, traces).",
    ),
    benchmarks_dir: Path = typer.Option(
        Path("benchmarks"),
        "--benchmarks-dir",
        help="Search directory for built-in benchmark suite YAMLs.",
    ),
    task: str | None = typer.Option(
        None,
        "--task",
        help=(
            "Run only this single task id from the suite (e.g. "
            "'libero_spatial/3' or 'maniskill3/PushCube-v1'). Omit to run "
            "every task the rSkill supports (the suite is auto-filtered to "
            "the rSkill's evaluated_tasks)."
        ),
    ),
    n_episodes: int | None = typer.Option(
        None,
        "--n-episodes",
        help=(
            "Override ``BenchmarkScene.n_episodes`` for every scene in the "
            "suite (lower for quick smoke runs). The published-protocol value "
            "lives in the suite YAML; this flag is for fast iteration, not "
            "for paper-comparison numbers."
        ),
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help=(
            "Resolve the suite + VLA and print the planned (task x seed) "
            "matrix without running any rollouts. Useful in CI to "
            "validate config wiring."
        ),
    ),
    update_manifest: bool = typer.Option(
        True,
        "--update-manifest/--no-update-manifest",
        help=(
            "On success, write the avg_success_rate back into the rSkill "
            "manifest at `benchmarks.<suite_id>`. Surgical edit — "
            "preserves comments. Only fires for locally-resolvable rSkills. "
            "Disable for read-only paper-number runs."
        ),
    ),
    video: bool = typer.Option(
        True,
        "--video/--no-video",
        help=(
            "Record per-step world frames and write one MP4 per episode "
            "(named <task>[_seed<n>]_<rskill>_<success|fail>.mp4 plus a "
            "videos.json manifest). On by default; frames are written and "
            "freed per episode. Disable for allocation-light CI runs."
        ),
    ),
    video_dir: Path | None = typer.Option(
        None,
        "--video-dir",
        help=(
            "Destination directory for --video MP4s. Defaults to "
            "<eval JSON dir>/videos/<suite_id> next to the RSkillEvalResult."
        ),
    ),
    dashboard: bool = typer.Option(
        False,
        "--dashboard",
        help=(
            "Boot `openral dashboard` as a child process, point OTel at it, "
            "and shut it down on exit (same semantics as `openral sim run "
            "--dashboard`)."
        ),
    ),
    dashboard_port: int = typer.Option(
        4318,
        "--dashboard-port",
        help="Port for the spawned dashboard when --dashboard is set.",
    ),
) -> None:
    r"""Run a benchmark suite and write a validated `RSkillEvalResult` JSON.

    The runner iterates ``scenes x range(seed, seed + n_episodes)``,
    delegating each rollout to ``openral_sim.SimRunner`` so the
    rSkill compatibility check, OTel spans, and latency-budget reporting
    are identical to ``openral sim run``. Each ``BenchmarkScene``
    carries its own scene + task + robot; the ``BenchmarkSpec`` wrapper
    class was removed so the suite is a bare list of scenes whose id is
    the YAML filename stem.

    Example:
        >>> # openral benchmark run --suite libero_spatial \\
        >>> #     --rskill smolvla-libero
    """
    scenes, suite_id = _resolve_benchmark_suite(suite, benchmarks_dir)
    vla_spec = _parse_rskill_cli_arg(rskill)

    # --task selects a single explicit task from the suite. The rSkill's
    # evaluated_tasks auto-filter (in run_benchmark) still applies, so an
    # explicitly-picked task the rSkill was not trained for is rejected — same
    # contract as `openral benchmark scene`.
    if task is not None:
        matched = [s for s in scenes if s.task.id == task]
        if not matched:
            available = [s.task.id for s in scenes]
            console.print(
                f"[red]--task {task!r} is not in suite {suite_id!r}.[/red] "
                f"Available tasks: {available}"
            )
            raise typer.Exit(1)
        scenes = matched

    # Apply --n-episodes override to every scene before dry-run or real run.
    if n_episodes is not None:
        scenes = [s.model_copy(update={"n_episodes": n_episodes}) for s in scenes]

    if dry_run:
        _print_benchmark_run_plan(scenes, suite_id=suite_id, vla_spec=vla_spec)
        return

    out_path = out if out is not None else _default_benchmark_out_path(vla_spec, suite_id)
    out_path.parent.mkdir(parents=True, exist_ok=True)
    effective_video_dir: Path | None = None
    if video:
        effective_video_dir = (
            video_dir if video_dir is not None else (out_path.parent / "videos" / suite_id)
        )

    from openral_observability.dashboard import attached_dashboard
    from openral_sim.benchmark import run_benchmark

    # attached_dashboard is a no-op when enabled=False — wrap
    # unconditionally so the run_benchmark call is un-duplicated. It
    # re-opens the cli.command root span: _root's copy predates the
    # exporters the dashboard child provides, so without this the eval
    # JSON has no trace to point at.
    with attached_dashboard(
        enabled=dashboard,
        port=dashboard_port,
        subcommand="benchmark run",
        mode=semconv.RUN_MODE_BENCHMARK,
    ):
        result, episodes = run_benchmark(
            scenes,
            suite_id=suite_id,
            vla=vla_spec,
            device=device,
            save_dir=str(save_dir) if save_dir is not None else None,
            video_dir=str(effective_video_dir) if effective_video_dir is not None else None,
        )
    if effective_video_dir is not None:
        console.print(f"[cyan]videos[/cyan] {effective_video_dir}")

    out_path.write_text(result.model_dump_json(indent=2))
    avg = result.results.get("avg_success_rate", 0.0)
    console.print(
        f"[green]wrote {out_path}[/green] — avg success "
        f"= {float(avg) if isinstance(avg, (int, float)) else avg:.3f} "
        f"over {len(episodes)} episodes"
    )

    if update_manifest:
        from openral_rskill.loader import resolve_rskill_local_dir
        from openral_sim.benchmark import update_rskill_benchmarks

        # Resolve to the in-tree dir so the surgical write hits
        # rskills/<name>/rskill.yaml even when the user supplied a bare
        # name or a Hub-style repo id. Falls through to a cwd-relative
        # path for Hub-only references with no in-tree shim — the
        # update is then a no-op (FileNotFoundError handled below).
        local_dir = resolve_rskill_local_dir(vla_spec.weights_uri)
        skill_dir = str(local_dir) if local_dir is not None else vla_spec.weights_uri

        try:
            manifest_path = update_rskill_benchmarks(
                skill_dir,
                suite_id,
                float(avg) if isinstance(avg, (int, float)) else 0.0,
            )
            console.print(
                f"[green]updated {manifest_path}[/green] — "
                f"benchmarks.{suite_id} = "
                f"{float(avg) if isinstance(avg, (int, float)) else avg:.3f}"
            )
        except FileNotFoundError as exc:
            console.print(
                f"[yellow]skipped manifest update:[/yellow] {exc} (eval JSON was still written)"
            )

benchmark_scene(config: Path = typer.Option(..., '--config', help='Path to a BenchmarkScene YAML (`scenes/benchmark/<id>.yaml`). DeployScene and SimScene YAMLs are rejected with a redirect — `openral benchmark scene` accepts BenchmarkScene only (scene + task + `n_episodes` + `seed` + `metadata.paper` + `metadata.honest_scope`).'), rskill: str = typer.Option(..., '--rskill', help="rSkill reference — a bare name ('smolvla-libero'), a path ('rskills/smolvla-libero'), or an HF Hub repo id ('OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'). Raw hf:// is rejected (weights must come from a manifest). The policy adapter id is read from the manifest's `model_family` field."), out: Path | None = typer.Option(None, '--out', help='Output path for the RSkillEvalResult JSON. Defaults to rskills/<dir>/eval/scene_<scene_id>.json derived from the rSkill ref.'), device: str | None = typer.Option(None, '--device', help='Torch device override for the policy (cpu, cuda:0, mps, auto).'), save_dir: Path | None = typer.Option(None, '--save-dir', help='Optional adapter-side artefact directory (videos, traces).'), save_video: Path | None = typer.Option(None, '--save-video', help='Write a clean single-view world MP4 per episode to this directory, named <task>_<rskill>_<success|fail>.mp4, plus a videos.json manifest — for website hero clips (overlays are rendered by the page, not burned into pixels). The task slug keeps benchmark scenes sharing one backend from overwriting each other. Enables per-step frame capture. Pair with --n-episodes 1 for a single demo clip.'), video_size: int = typer.Option(1024, '--video-size', help="Square edge (px) for --save-video output. Frames are center-cropped to a square and resized to this size. Source sharpness is bounded by the scene's native render resolution."), n_episodes: int | None = typer.Option(None, '--n-episodes', help='Override `BenchmarkScene.n_episodes` (lower for quick smoke runs). The published-protocol value lives in the YAML; this flag is for fast iteration, not for paper-comparison numbers.'), view: bool | None = typer.Option(None, '--view/--no-view', help='Open a passive mujoco.viewer window and stream the rollout in real time (parity with `openral sim run --view`). Default (unset): headless — benchmark eval artefacts and CI/deploy runs are unaffected. Pass --view to require a window (errors loud if unsupported), or --no-view to force offscreen. Incompatible with MUJOCO_GL=egl.'), dry_run: bool = typer.Option(False, '--dry-run', help='Resolve the scene + rSkill, apply the evaluated_tasks gate, and print the planned (task x seed) matrix without running any rollouts or fetching weights. Useful in CI to validate config wiring; exits non-zero if the pairing would be rejected.'), update_manifest: bool = typer.Option(True, '--update-manifest/--no-update-manifest', help='On success, write the avg_success_rate back into the rSkill manifest at `benchmarks.<scene_id>`. Surgical edit — preserves comments. Only fires for locally-resolvable rSkills.'), write_eval: bool = typer.Option(True, '--write-eval/--no-write-eval', help='Persist the RSkillEvalResult JSON to --out (default rskills/<dir>/eval/scene_<scene_id>.json, a tracked path). Pass --no-write-eval for a fully non-mutating smoke run: the rollout still executes and prints its score, but nothing is written to the rSkill package (implies --no-update-manifest).'), dashboard: bool = typer.Option(False, '--dashboard', help='Boot `openral dashboard` as a child process, point OTel at it, and shut it down on exit (same semantics as `openral sim run --dashboard`).'), dashboard_port: int = typer.Option(4318, '--dashboard-port', help='Port for the spawned dashboard when --dashboard is set.')) -> None

Run a single-scene benchmark and write a validated RSkillEvalResult JSON.

Single-scene sibling of openral benchmark run --suite — accepts exactly one BenchmarkScene YAML and emits the same eval JSON shape so openral benchmark report does not need to distinguish the two entrypoints.

Example

openral benchmark scene \

--config scenes/benchmark/pusht.yaml \

--rskill diffusion-pusht

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("scene")
def benchmark_scene(  # noqa: PLR0915  # reason: linear CLI flow of guards and prints; splitting scatters the error surface
    config: Path = typer.Option(
        ...,
        "--config",
        help=(
            "Path to a BenchmarkScene YAML "
            "(`scenes/benchmark/<id>.yaml`). DeployScene and SimScene "
            "YAMLs are rejected with a redirect — `openral benchmark "
            "scene` accepts BenchmarkScene only (scene + task + "
            "`n_episodes` + `seed` + `metadata.paper` + "
            "`metadata.honest_scope`)."
        ),
    ),
    rskill: str = typer.Option(
        ...,
        "--rskill",
        help=(
            "rSkill reference — a bare name ('smolvla-libero'), a "
            "path ('rskills/smolvla-libero'), or an HF Hub repo id "
            "('OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'). "
            "Raw hf:// is rejected (weights must come from a manifest). "
            "The policy adapter id is read from the manifest's `model_family` "
            "field."
        ),
    ),
    out: Path | None = typer.Option(
        None,
        "--out",
        help=(
            "Output path for the RSkillEvalResult JSON. Defaults to "
            "rskills/<dir>/eval/scene_<scene_id>.json derived from the "
            "rSkill ref."
        ),
    ),
    device: str | None = typer.Option(
        None,
        "--device",
        help="Torch device override for the policy (cpu, cuda:0, mps, auto).",
    ),
    save_dir: Path | None = typer.Option(
        None,
        "--save-dir",
        help="Optional adapter-side artefact directory (videos, traces).",
    ),
    save_video: Path | None = typer.Option(
        None,
        "--save-video",
        help=(
            "Write a clean single-view world MP4 per episode to this "
            "directory, named <task>_<rskill>_<success|fail>.mp4, plus a "
            "videos.json manifest — for website hero clips (overlays are "
            "rendered by the page, not burned into pixels). The task slug "
            "keeps benchmark scenes sharing one backend from overwriting each "
            "other. Enables per-step frame capture. Pair with --n-episodes 1 "
            "for a single demo clip."
        ),
    ),
    video_size: int = typer.Option(
        1024,
        "--video-size",
        help=(
            "Square edge (px) for --save-video output. Frames are "
            "center-cropped to a square and resized to this size. Source "
            "sharpness is bounded by the scene's native render resolution."
        ),
    ),
    n_episodes: int | None = typer.Option(
        None,
        "--n-episodes",
        help=(
            "Override `BenchmarkScene.n_episodes` (lower for quick smoke "
            "runs). The published-protocol value lives in the YAML; this "
            "flag is for fast iteration, not for paper-comparison numbers."
        ),
    ),
    view: bool | None = typer.Option(
        None,
        "--view/--no-view",
        help=(
            "Open a passive mujoco.viewer window and stream the rollout in "
            "real time (parity with `openral sim run --view`). Default "
            "(unset): headless — benchmark eval artefacts and CI/deploy "
            "runs are unaffected. Pass --view to require a window (errors "
            "loud if unsupported), or --no-view to force offscreen. "
            "Incompatible with MUJOCO_GL=egl."
        ),
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help=(
            "Resolve the scene + rSkill, apply the evaluated_tasks gate, and "
            "print the planned (task x seed) matrix without running any "
            "rollouts or fetching weights. Useful in CI to validate config "
            "wiring; exits non-zero if the pairing would be rejected."
        ),
    ),
    update_manifest: bool = typer.Option(
        True,
        "--update-manifest/--no-update-manifest",
        help=(
            "On success, write the avg_success_rate back into the rSkill "
            "manifest at `benchmarks.<scene_id>`. Surgical edit — "
            "preserves comments. Only fires for locally-resolvable rSkills."
        ),
    ),
    write_eval: bool = typer.Option(
        True,
        "--write-eval/--no-write-eval",
        help=(
            "Persist the RSkillEvalResult JSON to --out (default "
            "rskills/<dir>/eval/scene_<scene_id>.json, a tracked path). "
            "Pass --no-write-eval for a fully non-mutating smoke run: the "
            "rollout still executes and prints its score, but nothing is "
            "written to the rSkill package (implies --no-update-manifest)."
        ),
    ),
    dashboard: bool = typer.Option(
        False,
        "--dashboard",
        help=(
            "Boot `openral dashboard` as a child process, point OTel at it, "
            "and shut it down on exit (same semantics as `openral sim run "
            "--dashboard`)."
        ),
    ),
    dashboard_port: int = typer.Option(
        4318,
        "--dashboard-port",
        help="Port for the spawned dashboard when --dashboard is set.",
    ),
) -> None:
    r"""Run a single-scene benchmark and write a validated `RSkillEvalResult` JSON.

    Single-scene sibling of ``openral benchmark run --suite`` — accepts
    exactly one ``BenchmarkScene`` YAML and emits the same eval JSON
    shape so ``openral benchmark report`` does not need to distinguish
    the two entrypoints.

    Example:
        >>> # openral benchmark scene \\
        >>> #   --config scenes/benchmark/pusht.yaml \\
        >>> #   --rskill diffusion-pusht
    """
    from openral_core import BenchmarkScene, load_scene_strict

    scene = load_scene_strict(str(config), BenchmarkScene)
    try:
        SCENES.validate_options(scene.scene.id, scene.scene.backend_options)
    except ROSConfigError as exc:
        console.print(f"[red]✗ config:[/red] {exc}")
        raise typer.Exit(code=1) from exc
    if n_episodes is not None:
        scene = scene.model_copy(update={"n_episodes": n_episodes})

    if dry_run:
        # Resolve the rSkill here rather than echoing it as-typed: a dry run
        # that never parses --rskill lets a broken manifest (or a non-VLA
        # kind) through the exact check it is run to perform. Manifest-only —
        # no weights are fetched. Built-in mock policies have no manifest
        # (same carve-out as openral_sim.benchmark._manifest_for_filter), so
        # they keep the as-typed echo.
        from openral_core.exceptions import ROSCapabilityMismatch  # reason: defer
        from openral_sim.sim_runner import _MOCK_PLACEHOLDER_URI, _MOCK_POLICY_IDS

        vla_line = f"rskill={rskill}"
        if rskill not in _MOCK_POLICY_IDS and rskill != _MOCK_PLACEHOLDER_URI:
            spec = _parse_rskill_cli_arg(rskill)
            vla_line = f"rskill={rskill} id={spec.id}"

            # Same task gate `run_benchmark_scene` applies, so a task-mismatched
            # pairing fails here instead of looking planned and then raising.
            from openral_rskill.loader import load_rskill_manifest
            from openral_sim.benchmark import check_benchmark_task_compatibility

            try:
                check_benchmark_task_compatibility(
                    load_rskill_manifest(spec.weights_uri),
                    task_id=scene.task.id,
                    scene_id=scene.scene.id,
                )
            except ROSCapabilityMismatch as exc:
                console.print(f"[red]✗ task gate:[/red] {exc}")
                raise typer.Exit(code=1) from exc

        console.print(
            f"[cyan]scene[/cyan] {scene.scene.id} — robot={scene.robot_id} "
            f"task={scene.task.id} n_episodes={scene.n_episodes} "
            f"seed={scene.seed}"
        )
        console.print(f"[cyan]vla[/cyan]   {vla_line}")
        console.print(
            f"[cyan]plan[/cyan]  {scene.n_episodes} episodes (seeds "
            f"{scene.seed}..{scene.seed + scene.n_episodes - 1})"
        )
        return

    vla_spec = _parse_rskill_cli_arg(rskill)

    out_path = out if out is not None else _default_benchmark_scene_out_path(vla_spec, scene)

    from openral_observability.dashboard import attached_dashboard
    from openral_sim.benchmark import run_benchmark_scene

    with attached_dashboard(
        enabled=dashboard,
        port=dashboard_port,
        subcommand="benchmark scene",
        mode=semconv.RUN_MODE_BENCHMARK,
    ):
        result, episodes = run_benchmark_scene(
            scene,
            vla_spec,
            device=device,
            save_dir=str(save_dir) if save_dir is not None else None,
            config_path=str(config),
            view=view,
            record_video=save_video is not None,
        )

    if save_video is not None:
        from openral_sim._website_video import write_world_videos

        write_world_videos(
            episodes,
            save_video,
            scene=scene.task.id,
            rskill=Path(rskill).name,
            section=Path(config).parent.name,
            size=video_size,
        )

    avg = result.results.get("avg_success_rate", 0.0)
    avg_f = float(avg) if isinstance(avg, (int, float)) else 0.0
    if _persist_scene_eval(result, out_path, write_eval=write_eval):
        console.print(
            f"[green]wrote {out_path}[/green] — avg success "
            f"= {avg_f:.3f} over {len(episodes)} episodes"
        )
    else:
        console.print(
            f"[yellow]--no-write-eval:[/yellow] not persisting result — avg success "
            f"= {avg_f:.3f} over {len(episodes)} episodes (nothing written to the rSkill)"
        )

    if not write_eval:
        # Non-mutating smoke run: skip the manifest writeback too.
        return

    if update_manifest and not _scene_id_is_benchmark_suite(scene.scene.id):
        # The rskill.yaml `benchmarks:` block holds canonical SUITE headlines
        # (RSkillManifest.benchmarks is keyed by BenchmarkName suite ids).
        # A single scene whose id is not itself a suite id (e.g. 'metaworld',
        # 'robocasa/PickPlaceCounterToCabinet') has no headline slot — writing
        # it would raise ROSConfigError. The per-scene result is already
        # captured in the eval JSON, so we skip the manifest write rather than
        # crash. (Scenes whose id IS a suite id — pusht, libero_spatial — still
        # update the headline below.)
        console.print(
            f"[yellow]skipped manifest update:[/yellow] scene id {scene.scene.id!r} "
            f"is not a canonical benchmark suite id; per-scene result written to "
            f"{out_path} only (rskill.yaml benchmarks: holds suite headlines)."
        )
    elif update_manifest:
        from openral_rskill.loader import resolve_rskill_local_dir
        from openral_sim.benchmark import update_rskill_benchmarks

        local_dir = resolve_rskill_local_dir(vla_spec.weights_uri)
        skill_dir = str(local_dir) if local_dir is not None else vla_spec.weights_uri
        try:
            manifest_path = update_rskill_benchmarks(
                skill_dir,
                scene.scene.id,
                float(avg) if isinstance(avg, (int, float)) else 0.0,
            )
            console.print(
                f"[green]updated {manifest_path}[/green] — "
                f"benchmarks.{scene.scene.id} = "
                f"{float(avg) if isinstance(avg, (int, float)) else avg:.3f}"
            )
        except FileNotFoundError as exc:
            console.print(
                f"[yellow]skipped manifest update:[/yellow] {exc} (eval JSON was still written)"
            )

calibrate_camera(sensor: str = typer.Option(..., '--sensor', '-s', help='Sensor name as it appears in robot.yaml (e.g. head_color).'), topic: str = typer.Option('', '--topic', help='Override the image topic (default: derived from sensor name).'), chessboard_size: str = typer.Option('8x6', '--chessboard-size', help='Internal corners COLSxROWS of the calibration target.'), square_size: float = typer.Option(0.025, '--square-size', help='Physical size of one chessboard square in metres.'), dry_run: bool = typer.Option(False, '--dry-run', help='Print the command instead of executing it.')) -> None

Calibrate a camera sensor using the ROS 2 camera_calibration package.

Builds and optionally runs::

ros2 run camera_calibration cameracalibrator \
    --size COLSxROWS --square SIZE \
    --ros-args -r image:=TOPIC -r camera_info:=INFO_TOPIC

Requires ros2_camera_calibration to be installed and ROS 2 sourced.

Example

openral calibrate camera --sensor head_color --chessboard-size 8x6 --square-size 0.025

openral calibrate camera --sensor head_color --dry-run

Source code in python/cli/src/openral_cli/main.py

@calibrate_app.command("camera")
def calibrate_camera(
    sensor: str = typer.Option(
        ...,
        "--sensor",
        "-s",
        help="Sensor name as it appears in robot.yaml (e.g. head_color).",
    ),
    topic: str = typer.Option(
        "",
        "--topic",
        help="Override the image topic (default: derived from sensor name).",
    ),
    chessboard_size: str = typer.Option(
        "8x6",
        "--chessboard-size",
        help="Internal corners COLSxROWS of the calibration target.",
    ),
    square_size: float = typer.Option(
        0.025,
        "--square-size",
        help="Physical size of one chessboard square in metres.",
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help="Print the command instead of executing it.",
    ),
) -> None:
    r"""Calibrate a camera sensor using the ROS 2 camera_calibration package.

    Builds and optionally runs::

        ros2 run camera_calibration cameracalibrator \
            --size COLSxROWS --square SIZE \
            --ros-args -r image:=TOPIC -r camera_info:=INFO_TOPIC

    Requires ``ros2_camera_calibration`` to be installed and ROS 2 sourced.

    Example:
        >>> # openral calibrate camera --sensor head_color --chessboard-size 8x6 --square-size 0.025
        >>> # openral calibrate camera --sensor head_color --dry-run
    """
    try:
        cols_str, rows_str = chessboard_size.lower().split("x")
        cols, rows = int(cols_str), int(rows_str)
    except ValueError:
        console.print(
            f"[red]Invalid --chessboard-size '{chessboard_size}'. "
            "Expected format: COLSxROWS, e.g. 8x6[/red]"
        )
        raise typer.Exit(code=1)  # noqa: B904

    image_topic = topic or f"/{sensor}/image_raw"
    info_topic = image_topic.replace("/image_raw", "/camera_info").replace(
        "/image_rect_raw", "/camera_info"
    )

    cmd = [
        "ros2",
        "run",
        "camera_calibration",
        "cameracalibrator",
        "--size",
        f"{cols}x{rows}",
        "--square",
        str(square_size),
        "--ros-args",
        "-r",
        f"image:={image_topic}",
        "-r",
        f"camera_info:={info_topic}",
    ]

    if dry_run:
        console.print("[bold]openral calibrate camera[/bold] — dry run:")
        console.print(" ".join(cmd))
        return

    ros2_bin = shutil.which("ros2")
    if ros2_bin is None:
        console.print(
            "[red]ros2 not found. Source your ROS 2 installation first:[/red]\n"
            "  source /opt/ros/<distro>/setup.bash"
        )
        raise typer.Exit(code=1)

    console.print(f"[bold]openral calibrate camera[/bold] — sensor: [cyan]{sensor}[/cyan]")
    console.print(f"  image topic : [dim]{image_topic}[/dim]")
    console.print(f"  camera info : [dim]{info_topic}[/dim]")
    console.print(f"  target size : [dim]{cols}x{rows} squares @ {square_size} m[/dim]")
    console.print("Running camera_calibration … (Ctrl+C to abort)")

    result = subprocess.run(cmd, check=False)
    if result.returncode != 0:
        console.print(f"[red]cameracalibrator exited with code {result.returncode}.[/red]")
        raise typer.Exit(code=result.returncode)

connect(robot: str = typer.Option(..., help='Robot type (so100, so101, g1, ur5e, …)'), port: str = typer.Option('', '--port', help='USB/serial port override, e.g. /dev/ttyUSB0')) -> None

Open a HAL connection to a robot, read one joint state, and disconnect.

Exits 0 on success; exits 1 with an error message on failure.

Supported robots: so100, so101 (both drive the shared SO100FollowerHAL Feetech serial bus — the SO-101 is the same controller as the SO-100).

Example

openral connect --robot so100

openral connect --robot so101 --port /dev/ttyUSB1

Source code in python/cli/src/openral_cli/main.py

@app.command()
def connect(
    robot: str = typer.Option(..., help="Robot type (so100, so101, g1, ur5e, …)"),
    port: str = typer.Option("", "--port", help="USB/serial port override, e.g. /dev/ttyUSB0"),
) -> None:
    """Open a HAL connection to a robot, read one joint state, and disconnect.

    Exits 0 on success; exits 1 with an error message on failure.

    Supported robots: so100, so101 (both drive the shared ``SO100FollowerHAL``
    Feetech serial bus — the SO-101 is the same controller as the SO-100).

    Example:
        >>> # openral connect --robot so100
        >>> # openral connect --robot so101 --port /dev/ttyUSB1
    """
    # so100 and so101 share the Feetech SO100FollowerHAL (identical USB
    # controller + driver); the label only changes the console banner.
    so_follower_labels = {"so100": "SO-100", "so101": "SO-101"}
    if robot in so_follower_labels:
        _connect_so_follower(so_follower_labels[robot], port or "/dev/ttyUSB0")
    else:
        console.print(f"[red]Unknown robot '{robot}'. Supported: so100, so101[/red]")
        raise typer.Exit(code=1)

dashboard(host: str = typer.Option('127.0.0.1', '--host', help='Bind address. Loopback by default; no auth.'), port: int = typer.Option(4318, '--port', help='HTTP port; serves UI, /api/state, /api/stream, and OTLP/HTTP receiver. Defaults to 4318 (the OTLP/HTTP standard port) rather than 8000 (issue #132) — `mkdocs serve` and most FastAPI demos already squat on 8000.'), log_level: str = typer.Option('warning', '--log-level', help='uvicorn log level (debug | info | warning | error).'), inprocess: str | None = typer.Option(None, '--inprocess', help="Optional shell-quoted command to spawn as a child process with OTEL_EXPORTER_OTLP_ENDPOINT pointed at this dashboard. Pass the whole command as one string (shlex-tokenised), e.g. `--inprocess 'openral sim run --config scenes/sim/pusht.yaml --rskill diffusion-pusht'`.")) -> None

Serve the OpenRAL live dashboard.

Source code in python/cli/src/openral_cli/main.py

@app.command(
    "dashboard",
    help=(
        "Serve a live debugging dashboard (read-only) at the given port. "
        "The same port also acts as an OTLP/HTTP receiver, so any "
        "`openral sim run` / `openral deploy run` pointed at "
        "OTEL_EXPORTER_OTLP_ENDPOINT=http://<host>:<port> + "
        "OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf streams in live. "
        "Works without Jaeger/Tempo running."
    ),
)
def dashboard(
    host: str = typer.Option(
        "127.0.0.1",
        "--host",
        help="Bind address. Loopback by default; no auth.",
    ),
    port: int = typer.Option(
        4318,
        "--port",
        help=(
            "HTTP port; serves UI, /api/state, /api/stream, and OTLP/HTTP "
            "receiver. Defaults to 4318 (the OTLP/HTTP standard port) "
            "rather than 8000 (issue #132) — `mkdocs serve` and most "
            "FastAPI demos already squat on 8000."
        ),
    ),
    log_level: str = typer.Option(
        "warning",
        "--log-level",
        help="uvicorn log level (debug | info | warning | error).",
    ),
    inprocess: str | None = typer.Option(
        None,
        "--inprocess",
        help=(
            "Optional shell-quoted command to spawn as a child process with "
            "OTEL_EXPORTER_OTLP_ENDPOINT pointed at this dashboard. Pass the "
            "whole command as one string (shlex-tokenised), e.g. "
            "`--inprocess 'openral sim run --config scenes/sim/pusht.yaml"
            " --rskill diffusion-pusht'`."
        ),
    ),
) -> None:
    """Serve the OpenRAL live dashboard."""
    import shlex

    from openral_observability.dashboard import run_dashboard

    inprocess_cmd = shlex.split(inprocess) if inprocess else None
    run_dashboard(
        host=host,
        port=port,
        inprocess_cmd=inprocess_cmd,
        log_level=log_level,
    )

deploy_list() -> None

List every deploy scene under scenes/deploy/*.yaml.

Each entry is a paste-able --config path for openral deploy run or deploy sim. Operator copies under a local/ directory (gitignored scenes/deploy/local/) are skipped. No hardware touch, no GPU.

Source code in python/cli/src/openral_cli/main.py

@deploy_app.command("list")
def deploy_list() -> None:
    """List every deploy scene under `scenes/deploy/*.yaml`.

    Each entry is a paste-able `--config` path for `openral deploy run` or `deploy sim`.
    Operator copies under a `local/` directory (gitignored `scenes/deploy/local/`) are
    skipped. No hardware touch, no GPU.
    """
    from openral_rskill.loader import find_repo_root_from

    repo_root = find_repo_root_from(Path(__file__))
    if repo_root is None:
        console.print("[red]Could not locate repo root.[/red]")
        raise typer.Exit(code=1)
    deploy_scenes = repo_root / "scenes" / "deploy"
    if not deploy_scenes.is_dir():
        print("<none>")
        return
    # scenes/deploy/local/ holds gitignored operator copies; never advertise them.
    configs = sorted(
        p
        for p in deploy_scenes.rglob("*.yaml")
        if "local" not in p.relative_to(deploy_scenes).parts
    )
    if not configs:
        print("<none>")
        return
    for cfg in configs:
        print(cfg.relative_to(repo_root))

deploy_run(config: Path = typer.Option(..., '--config', '-c', exists=True, readable=True, dir_okay=False, help='Path to a DeployScene YAML; its robot_id selects the real robot workcell.'), robot: str | None = typer.Option(None, '--robot', help='Override the robot_id resolved from --config.'), hal: list[str] | None = typer.Option(None, '--hal', help='Override HAL node params, key=value (repeatable), e.g. --hal port=/dev/ttyUSB1.'), dashboard: bool = typer.Option(True, '--dashboard/--no-dashboard', help='Spawn the live dashboard (default on).'), dashboard_port: int = typer.Option(4318, '--dashboard-port', help='Dashboard OTLP port.'), foxglove: bool = typer.Option(False, '--foxglove/--no-foxglove', help='Spawn the read-only Foxglove scene view.'), foxglove_port: int = typer.Option(8765, '--foxglove-port', help='Foxglove WebSocket port.'), initial_task: str | None = typer.Option(None, '--initial-task', help='Operator goal delivered to the reasoner at startup.'), enable_reward_monitor: bool | None = typer.Option(None, '--enable-reward-monitor/--no-enable-reward-monitor', help="Bring up the Robometer reward monitor parallel to the VLA (same leg `deploy sim` exposes): it scores the robot's first RGB camera topic and serves /openral/perception/query_task_progress. The manifest is auto-paired from the VLA palette's reward_rskill_name; override with --reward-monitor-manifest. Unset = the scene's runtime.enable_reward_monitor, else off."), reward_monitor_manifest: str | None = typer.Option(None, '--reward-monitor-manifest', help='Path to a kind:reward rSkill manifest. Empty auto-pairs from the VLA palette, falling back to rskills/robometer-4b. Ignored unless --enable-reward-monitor.'), dataset_out: str | None = typer.Option(None, '--dataset-out', help='record the session (proprio + action + camera frames + episode markers) to this rosbag2 mcap FILE (not a bag directory): its parent must exist and the file must not. Recording is segmented by episode markers, which only an EXECUTING rSkill emits — a session where every dispatch is rejected writes no file. Convert to a LeRobotDataset v3 offline with `openral dataset from-bag`. Same flag, same recorder and same launch args as `deploy sim`; on real hardware it is what leaves a supervised run replayable (CLAUDE.md §1.8). Empty disables recording.'), dataset_repo_id: str | None = typer.Option(None, '--dataset-repo-id', help='repo_id for the recorded dataset (default openral/dataset-<robot>).'), dataset_license: str | None = typer.Option(None, '--dataset-license', help='SPDX license carried into `openral dataset from-bag` (default CC-BY-4.0).'), dry_run: bool = typer.Option(False, '--dry-run', help='Print the resolved real-mode launch argv and exit without shelling out.')) -> None

Run an rSkill on REAL hardware via the production ROS graph.

Unlike openral deploy sim, this drives the real hardware HAL: it resolves the robot from --config (a DeployScene) and shells the SAME deploy_e2e.launch.py graph with hal_mode:=real — the HAL lifecycle node + C++ safety kernel + reasoner + world state (+ SLAM/Nav2 when the robot declares a lidar). The HAL's connect() fails loudly if no hardware is attached; a simulation-only robot raises ROSCapabilityMismatch (use openral deploy sim). Robot HAL defaults come from robot.yaml; --hal wins.

Source code in python/cli/src/openral_cli/main.py

@deploy_app.command("run")
def deploy_run(
    config: Path = typer.Option(  # reason: typer Option idiom
        ...,
        "--config",
        "-c",
        exists=True,
        readable=True,
        dir_okay=False,
        help="Path to a DeployScene YAML; its robot_id selects the real robot workcell.",
    ),
    robot: str | None = typer.Option(
        None,
        "--robot",
        help="Override the robot_id resolved from --config.",
    ),
    hal: list[str] | None = typer.Option(
        None,
        "--hal",
        help="Override HAL node params, key=value (repeatable), e.g. --hal port=/dev/ttyUSB1.",
    ),
    dashboard: bool = typer.Option(
        True,
        "--dashboard/--no-dashboard",
        help="Spawn the live dashboard (default on).",
    ),
    dashboard_port: int = typer.Option(
        4318,
        "--dashboard-port",
        help="Dashboard OTLP port.",
    ),
    foxglove: bool = typer.Option(
        False,
        "--foxglove/--no-foxglove",
        help="Spawn the read-only Foxglove scene view.",
    ),
    foxglove_port: int = typer.Option(
        8765,
        "--foxglove-port",
        help="Foxglove WebSocket port.",
    ),
    initial_task: str | None = typer.Option(
        None,
        "--initial-task",
        help="Operator goal delivered to the reasoner at startup.",
    ),
    enable_reward_monitor: bool | None = typer.Option(
        None,
        "--enable-reward-monitor/--no-enable-reward-monitor",
        help=(
            "Bring up the Robometer reward monitor parallel to the "
            "VLA (same leg `deploy sim` exposes): it scores the robot's first RGB "
            "camera topic and serves /openral/perception/query_task_progress. The "
            "manifest is auto-paired from the VLA palette's reward_rskill_name; "
            "override with --reward-monitor-manifest. Unset = the "
            "scene's runtime.enable_reward_monitor, else off."
        ),
    ),
    reward_monitor_manifest: str | None = typer.Option(
        None,
        "--reward-monitor-manifest",
        help=(
            "Path to a kind:reward rSkill manifest. Empty auto-pairs "
            "from the VLA palette, falling back to rskills/robometer-4b. Ignored "
            "unless --enable-reward-monitor."
        ),
    ),
    dataset_out: str | None = typer.Option(
        None,
        "--dataset-out",
        help=(
            "record the session (proprio + action + camera frames + episode "
            "markers) to this rosbag2 mcap FILE (not a bag directory): its "
            "parent must exist and the file must not. Recording is segmented "
            "by episode markers, which only an EXECUTING rSkill emits — a "
            "session where every dispatch is rejected writes no file. Convert "
            "to a LeRobotDataset v3 offline with `openral dataset from-bag`. "
            "Same flag, same recorder and same launch args as `deploy sim`; "
            "on real hardware it is what leaves a supervised run replayable "
            "(CLAUDE.md §1.8). Empty disables recording."
        ),
    ),
    dataset_repo_id: str | None = typer.Option(
        None,
        "--dataset-repo-id",
        help="repo_id for the recorded dataset (default openral/dataset-<robot>).",
    ),
    dataset_license: str | None = typer.Option(
        None,
        "--dataset-license",
        help="SPDX license carried into `openral dataset from-bag` (default CC-BY-4.0).",
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help="Print the resolved real-mode launch argv and exit without shelling out.",
    ),
) -> None:
    """Run an rSkill on REAL hardware via the production ROS graph.

    Unlike `openral deploy sim`, this drives the **real** hardware HAL: it
    resolves the robot from `--config` (a DeployScene) and shells the SAME
    `deploy_e2e.launch.py` graph with `hal_mode:=real` — the HAL lifecycle node +
    C++ safety kernel + reasoner + world state (+ SLAM/Nav2 when the robot
    declares a lidar). The HAL's `connect()` fails loudly if no hardware is
    attached; a simulation-only robot raises ROSCapabilityMismatch (use
    `openral deploy sim`). Robot HAL defaults come from robot.yaml; `--hal` wins.
    """
    from openral_core import DeployScene  # reason: defer schema import
    from openral_core.exceptions import ROSCapabilityMismatch  # reason: defer
    from pydantic import ValidationError  # reason: defer CLI import

    from openral_cli.deploy_sim import (  # reason: defer heavy CLI import
        _parse_hal_overrides,
        resolve_launch_invocation,
        run_launch_invocation,
    )

    try:
        deploy_scene = DeployScene.from_yaml(str(config))
    except (FileNotFoundError, ROSConfigError, ValidationError) as exc:
        console.print(f"[red]config error:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    overrides = _parse_hal_overrides(hal)

    # A committed, config-relative `calibration_dir` (deploy owns its calibration
    # instead of the ambient HF cache) is resolved against THIS config's
    # directory so it works regardless of the CWD `deploy run` is invoked from.
    cal_dir = overrides.get("calibration_dir")
    if isinstance(cal_dir, str) and cal_dir and not Path(cal_dir).is_absolute():
        overrides["calibration_dir"] = str((config.parent / cal_dir).resolve())

    try:
        invocation = resolve_launch_invocation(
            config=config,
            robot_override=robot or deploy_scene.robot_id,
            dashboard_port=dashboard_port,
            reset_to_pose_service=None,
            dataset_out=dataset_out,
            dataset_repo_id=dataset_repo_id,
            dataset_license=dataset_license,
            deploy_config=config,
            hal_param_overrides=overrides,
            hal_mode="real",
            enable_dashboard=dashboard,
            enable_foxglove=foxglove,
            foxglove_port=foxglove_port,
            initial_task_prompt=initial_task,
            enable_reward_monitor=enable_reward_monitor,
            reward_monitor_manifest=reward_monitor_manifest,
        )
    except (ROSConfigError, ROSCapabilityMismatch) as exc:
        console.print(f"[red]deploy run:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    console.print(
        f"[cyan]deploy run[/cyan] {invocation.robot_id} → real HAL "
        f"(hal_mode=real); the HAL's connect() requires the robot to be attached."
    )
    if dry_run:
        printed = [
            arg.replace("HAL_PARAMS_FILE_PLACEHOLDER", "<hal-params-tmp>")
            for arg in invocation.argv_template
        ]
        console.print(f"  hal_params: {invocation.hal_params}")
        console.print(f"  argv: {shlex.join(printed)}")
        return

    returncode = run_launch_invocation(invocation)
    raise typer.Exit(code=returncode)

deploy_validate(config: Path = typer.Option(..., '--config', '-c', exists=True, readable=True, dir_okay=False, help='DeployScene YAML to check for real-run readiness.'), robot: str | None = typer.Option(None, '--robot', help='Override the robot_id resolved from --config.'), hal: list[str] | None = typer.Option(None, '--hal', help='HAL overrides applied before validation (same precedence as deploy run).')) -> None

Pre-run readiness check for openral deploy run — no hardware, no ROS launch.

Validates the DeployScene + robot manifest resolve, then checks the runtime-required inputs a real run needs are present before the launch — the exact gaps that otherwise fail late at HAL configure / sensor leg:

• HAL transport — a serial port is declared, and its device exists now.
• Calibration — a serial HAL with calibrate_on_connect=false has an id + calibration_dir, and the <calibration_dir>/<id>.json file exists (missing → "has no calibration registered" at every send_action).
• Camera bindings — each deploy sensor (the robot manifest's cameras, whose bindings live in robot.yaml, plus the scene's workcell cameras) has a deploy_binding (else it is never published and a camera VLA gets an empty observation), and any /dev/* device path exists now.

Reports ERROR (missing committed data — exits non-zero) vs WARN (device just not attached right now). HAL param precedence matches deploy run (--hal > scene hal > robot.yaml).

Source code in python/cli/src/openral_cli/main.py

@deploy_app.command("validate")
def deploy_validate(  # noqa: PLR0915  # reason: linear readiness checklist; each check is one append
    config: Path = typer.Option(  # reason: typer Option idiom
        ...,
        "--config",
        "-c",
        exists=True,
        readable=True,
        dir_okay=False,
        help="DeployScene YAML to check for real-run readiness.",
    ),
    robot: str | None = typer.Option(
        None,
        "--robot",
        help="Override the robot_id resolved from --config.",
    ),
    hal: list[str] | None = typer.Option(
        None,
        "--hal",
        help="HAL overrides applied before validation (same precedence as deploy run).",
    ),
) -> None:
    """Pre-run readiness check for `openral deploy run` — no hardware, no ROS launch.

    Validates the DeployScene + robot manifest resolve, then checks the
    runtime-required inputs a real run needs are present *before* the launch —
    the exact gaps that otherwise fail late at HAL configure / sensor leg:

    * **HAL transport** — a serial `port` is declared, and its device exists now.
    * **Calibration** — a serial HAL with `calibrate_on_connect=false` has an
      `id` + `calibration_dir`, and the `<calibration_dir>/<id>.json` file exists
      (missing → "has no calibration registered" at every send_action).
    * **Camera bindings** — each deploy sensor (the robot manifest's cameras, whose
      bindings live in `robot.yaml`, plus the scene's workcell cameras) has a
      `deploy_binding` (else it is never published and a camera VLA gets an empty
      observation), and any `/dev/*` device path exists now.

    Reports ERROR (missing committed data — exits non-zero) vs WARN (device just
    not attached right now). HAL param precedence matches `deploy run`
    (`--hal` > scene `hal` > `robot.yaml`).
    """
    from openral_core import (  # reason: defer schema import
        DeployScene,
        RobotDescription,
        apply_sensor_overlays,
        merge_deploy_sensors,
        resolve_sensor_overlays,
    )
    from openral_core.exceptions import ROSCapabilityMismatch  # reason: defer
    from pydantic import ValidationError  # reason: defer CLI import

    from openral_cli.deploy_sim import (  # reason: defer heavy CLI import
        _parse_hal_overrides,
        resolve_launch_invocation,
    )

    try:
        deploy_scene = DeployScene.from_yaml(str(config))
        SCENES.validate_options(deploy_scene.scene.id, deploy_scene.scene.backend_options)
    except (FileNotFoundError, ROSConfigError, ValidationError) as exc:
        console.print(f"[red]✗ config:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    overrides = _parse_hal_overrides(hal)
    cal_dir_override = overrides.get("calibration_dir")
    if (
        isinstance(cal_dir_override, str)
        and cal_dir_override
        and not Path(cal_dir_override).is_absolute()
    ):
        overrides["calibration_dir"] = str((config.parent / cal_dir_override).resolve())

    # Reuse the deploy-run resolver: raises on sim-only robot, name mismatch,
    # unknown HAL, missing manifest — and produces the merged hal_params
    # (registry → scene hal → --hal) we then inspect for readiness.
    try:
        invocation = resolve_launch_invocation(
            config=config,
            robot_override=robot or deploy_scene.robot_id,
            dashboard_port=4318,
            reset_to_pose_service=None,
            deploy_config=config,
            hal_param_overrides=overrides,
            hal_mode="real",
            enable_dashboard=False,
        )
    except (ROSConfigError, ROSCapabilityMismatch, ValidationError) as exc:
        # ValidationError: the robot manifest failed its real-hardware contract
        # (control rate, explicit safety limits, joint velocity limits) — the
        # message names every missing value.
        console.print(f"[red]✗ resolve:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    errors: list[str] = []
    warns: list[str] = []
    hp = invocation.hal_params

    port = hp.get("port")
    if isinstance(port, str) and port and not Path(port).exists():
        warns.append(f"serial port {port!r} does not exist now (arm not attached?)")

    if isinstance(port, str) and port and not bool(hp.get("calibrate_on_connect", False)):
        cal_id = hp.get("id")
        cal_dir = hp.get("calibration_dir")
        if not cal_id or not cal_dir:
            errors.append(
                "serial HAL with calibrate_on_connect=false but no id/calibration_dir "
                "→ every send_action/read_state raises 'has no calibration registered'. "
                "Add a scene `hal:` binding with id + calibration_dir."
            )
        else:
            cal_file = Path(str(cal_dir)) / f"{cal_id}.json"
            if not cal_file.exists():
                errors.append(f"calibration file {cal_file} does not exist (id={cal_id!r}).")

    # A robot camera's binding lives in its unit overlay (else its manifest), a workcell
    # camera's in the scene. The unit was already resolved (and required) by
    # resolve_launch_invocation above; resolved again here for the sensor list.
    deploy_sensors = merge_deploy_sensors(
        apply_sensor_overlays(
            RobotDescription.from_yaml(str(invocation.robot_yaml)).sensors,
            resolve_sensor_overlays(invocation.robot_yaml, deploy_scene.robot_unit, required=True),
        ),
        deploy_scene.sensors,
    )
    if not deploy_sensors:
        warns.append(
            "robot and scene declare no sensors → a camera VLA will get an empty observation"
        )
    for sensor in deploy_sensors:
        binding = sensor.deploy_binding
        if binding is None:
            warns.append(
                f"sensor {sensor.name!r} has no deploy_binding → not published, VLA won't see it"
            )
            continue
        dev = binding.backend_params.get("device")
        if isinstance(dev, str) and dev.startswith("/dev/") and not Path(dev).exists():
            warns.append(f"sensor {sensor.name!r} device {dev!r} does not exist now")

    console.print(f"[cyan]deploy validate[/cyan] {invocation.robot_id} ← {config}")
    for warn in warns:
        console.print(f"  [yellow]⚠ {warn}[/yellow]")
    for err in errors:
        console.print(f"  [red]✗ {err}[/red]")
    if errors:
        console.print(
            f"[red]{len(errors)} error(s), {len(warns)} warning(s) — "
            "not ready for `deploy run`.[/red]"
        )
        raise typer.Exit(code=1)
    console.print(f"[green]✓ ready for `deploy run` ({len(warns)} warning(s)).[/green]")

detect(output: Path = typer.Option(Path('robot.yaml'), '--output', '-o', help='Output robot.yaml path'), robot_type: str | None = typer.Option(None, '--robot', '--as', help="Force the canonical base manifest (slug e.g. 'so100' or dir name 'so100_follower'), overriding USB/DDS inference. A bare Feetech plug-in defaults to the SO-101; use this to select the SO-100 (the two are indistinguishable over USB)."), report: Path | None = typer.Option(None, '--report', help='Optional path to dump the raw DetectionReport as JSON.'), dds_timeout: float = typer.Option(5.0, '--dds-timeout', help='DDS topic discovery timeout in seconds'), include: str | None = typer.Option(None, '--include', help='Comma-separated probe names to run (default: all). Choices: usb, can, dds, gpu, cameras_v4l2, cameras_realsense, network.'), no_write: bool = typer.Option(False, '--no-write', help='Print summary and skip writing robot.yaml'), deployment: Path | None = typer.Option(None, '--deployment', help='Also scaffold a DeployScene YAML at this path (robot_id + `sensors:` bindings from the camera wizard; safety left to the robot manifest). Paste-able as `openral deploy run --config`.'), yes: bool = typer.Option(False, '--yes', '-y', help='Overwrite existing file without prompting')) -> None

Probe the host and emit a complete RobotDescription robot.yaml.

Runs the auto-provisioning flow from openral_detect:

1. Probe USB / CAN / DDS / GPU / V4L2 / RealSense / network.
2. Identify the rig (DDS topology, SocketCAN interface names, or USB VID/PID match — in that order). If a known robot is detected, load the canonical robots/<name>/robot.yaml directly; otherwise synthesize a minimal scaffold.
3. Reverse-look up each detected sensor in the catalog so its SensorSpec carries real intrinsics, FOV, encoding, rate.
4. Promote detected GPU / Jetson / Apple Silicon caps onto RobotCapabilities so openral rskill check can match RSkillManifest.runtime / quantization.dtype.

Example

openral detect

openral detect --include gpu,network --no-write

Source code in python/cli/src/openral_cli/main.py

@app.command()
def detect(
    output: Path = typer.Option(
        Path("robot.yaml"), "--output", "-o", help="Output robot.yaml path"
    ),
    robot_type: str | None = typer.Option(
        None,
        "--robot",
        "--as",
        help="Force the canonical base manifest (slug e.g. 'so100' or dir name "
        "'so100_follower'), overriding USB/DDS inference. A bare Feetech "
        "plug-in defaults to the SO-101; use this to select the SO-100 (the "
        "two are indistinguishable over USB).",
    ),
    report: Path | None = typer.Option(
        None,
        "--report",
        help="Optional path to dump the raw DetectionReport as JSON.",
    ),
    dds_timeout: float = typer.Option(
        5.0, "--dds-timeout", help="DDS topic discovery timeout in seconds"
    ),
    include: str | None = typer.Option(
        None,
        "--include",
        help="Comma-separated probe names to run (default: all). "
        "Choices: usb, can, dds, gpu, cameras_v4l2, cameras_realsense, network.",
    ),
    no_write: bool = typer.Option(
        False, "--no-write", help="Print summary and skip writing robot.yaml"
    ),
    deployment: Path | None = typer.Option(
        None,
        "--deployment",
        help="Also scaffold a DeployScene YAML at this path (robot_id + "
        "`sensors:` bindings from the camera wizard; safety left to the "
        "robot manifest). Paste-able as `openral deploy run --config`.",
    ),
    yes: bool = typer.Option(
        False, "--yes", "-y", help="Overwrite existing file without prompting"
    ),
) -> None:
    """Probe the host and emit a complete RobotDescription robot.yaml.

    Runs the auto-provisioning flow from ``openral_detect``:

    1. Probe USB / CAN / DDS / GPU / V4L2 / RealSense / network.
    2. Identify the rig (DDS topology, SocketCAN interface names, or USB
       VID/PID match — in that order).  If a
       known robot is detected, load the canonical
       ``robots/<name>/robot.yaml`` directly; otherwise synthesize a
       minimal scaffold.
    3. Reverse-look up each detected sensor in the catalog so its
       ``SensorSpec`` carries **real** intrinsics, FOV, encoding, rate.
    4. Promote detected GPU / Jetson / Apple Silicon caps onto
       ``RobotCapabilities`` so ``openral rskill check`` can match
       ``RSkillManifest.runtime`` / ``quantization.dtype``.

    Example:
        >>> # openral detect
        >>> # openral detect --include gpu,network --no-write
    """
    import yaml as _yaml
    from openral_detect import (
        assemble_robot_description,
        detect_hardware,
    )

    include_set: set[str] | None = (
        {p.strip() for p in include.split(",") if p.strip()} if include else None
    )

    console.print("[bold]openral detect[/bold] — probing host …")
    detection = detect_hardware(dds_timeout_s=dds_timeout, include=include_set)

    _render_detection_summary(detection)

    if report is not None:
        report.write_text(detection.model_dump_json(indent=2), encoding="utf-8")
        console.print(f"[green]Wrote[/green] {report} (raw DetectionReport)")

    # Probe-only inspection short-circuits the builder (keeps CI / --report
    # non-interactive). Building a manifest is always interactive.
    if no_write:
        try:
            canonical = assemble_robot_description(
                detection, force_robot_type=robot_type, enrich_cameras=True
            )
        except ROSConfigError as exc:
            console.print(f"[red]detect:[/red] {exc}")
            raise typer.Exit(code=1) from exc
        console.print("\n[dim]--no-write set — printing yaml to stdout:[/dim]\n")
        console.print(
            _yaml.safe_dump(
                canonical.model_dump(mode="json"), sort_keys=False, default_flow_style=False
            )
        )
        if deployment is not None:
            console.print(
                "[yellow]--deployment ignored under --no-write — no files written.[/yellow]"
            )
        return

    try:
        canonical = assemble_robot_description(
            detection, force_robot_type=robot_type, enrich_cameras=False
        )
    except ROSConfigError as exc:
        console.print(f"[red]detect:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    default_name = str(canonical.name)
    robot_name = _prompt_robot_name(default_name)

    robot_sensors, scene_sensor_specs = _run_camera_binding_wizard(canonical, detection)
    description = canonical.model_copy(
        update={"name": robot_name, "sensors": robot_sensors}, deep=True
    )
    description = _maybe_customize_limits(description)

    if output == Path("robot.yaml"):
        output = Path("robots") / robot_name / "robot.yaml"
        output.parent.mkdir(parents=True, exist_ok=True)

    from openral_detect.registry import canonical_robot_path

    # Mirrors `openral_detect.assemble._infer_bh_robot_type` — DDS, then CAN,
    # then USB — so the asset-relocation base matches the manifest that was
    # actually loaded.
    inferred = (
        robot_type
        or detection.ros2.inferred_robot_type
        or next((m.bh_robot_type for m in detection.can.matches if m.bh_robot_type), None)
        or next((m.bh_robot_type for m in detection.usb.matches if m.bh_robot_type), None)
    )
    canon_path = canonical_robot_path(inferred) if inferred else None
    canonical_dir = canon_path.parent if canon_path is not None else None
    description = _relocate_file_assets(
        description, canonical_dir=canonical_dir, output_path=output
    )

    yaml_text = _yaml.safe_dump(
        description.model_dump(mode="json"),
        sort_keys=False,
        default_flow_style=False,
    )

    if output.exists() and not yes:
        overwrite = typer.confirm(f"{output} already exists. Overwrite?", default=False)
        if not overwrite:
            console.print("[yellow]Aborted.[/yellow]")
            raise typer.Exit(code=0)

    output.write_text(yaml_text, encoding="utf-8")
    console.print(f"\n[green]Wrote[/green] {output} (RobotDescription, {description.name})")
    console.print(f"[dim]Next step:[/dim] openral rskill check --robot {output}")
    _print_safety_fitting_reminders(description, output)

    if deployment is not None:
        _write_deploy_scene_scaffold(
            deployment, description, scene_sensor_specs, detection=detection, assume_yes=yes
        )

doctor(json: bool = typer.Option(False, '--json', help='Output machine-readable JSON')) -> None

Diagnose the host: Python, OS, ROS 2 distro, GPU, USB devices.

Exits 0 when every check is ok, absent, or info; exits 1 if any check has status fail or missing.

Example

openral doctor

openral doctor --json

Source code in python/cli/src/openral_cli/main.py

@app.command()
def doctor(
    json: bool = typer.Option(False, "--json", help="Output machine-readable JSON"),
) -> None:
    """Diagnose the host: Python, OS, ROS 2 distro, GPU, USB devices.

    Exits 0 when every check is ``ok``, ``absent``, or ``info``; exits 1 if
    any check has status ``fail`` or ``missing``.

    Example:
        >>> # openral doctor
        >>> # openral doctor --json
    """
    checks = _gather_checks()

    if json:
        result = [{"check": c.check, "status": c.status, "details": c.details} for c in checks]
        console.print_json(_json.dumps(result))
    else:
        table = Table(title="openral doctor")
        table.add_column("check", style="bold")
        table.add_column("status")
        table.add_column("details")
        for c in checks:
            style = (
                "green" if c.status == "ok" else "yellow" if c.status in _YELLOW_STATUSES else "red"
            )
            table.add_row(c.check, f"[{style}]{c.status}[/{style}]", c.details)
        console.print(table)

    fatal = {"fail", "missing"}
    if any(c.status in fatal for c in checks):
        raise typer.Exit(code=1)

profile_session(action: str = typer.Argument(..., help='One of: start | stop | view.'), output: Path = typer.Option(Path('./lttng-traces'), '--output', '-o', help='LTTng output directory. Used by start (write here) and view (read from here).'), name: str = typer.Option('openral', '--name', '-n', help='LTTng session name.')) -> None

Drive an LTTng session — start, stop, view.

Source code in python/cli/src/openral_cli/main.py

@profile_app.command(
    "session",
    help=(
        "Start, stop, or view an LTTng session for the realtime hot path. "
        "Requires lttng-tools on PATH. Set OPENRAL_ROS2_TRACING=1 on the "
        "agent process to emit tracepoints; the env var is the runtime gate."
    ),
)
def profile_session(
    action: str = typer.Argument(  # reason: typer Argument idiom
        ...,
        help="One of: start | stop | view.",
    ),
    output: Path = typer.Option(  # reason: typer Option idiom
        Path("./lttng-traces"),
        "--output",
        "-o",
        help="LTTng output directory. Used by start (write here) and view (read from here).",
    ),
    name: str = typer.Option(
        "openral",
        "--name",
        "-n",
        help="LTTng session name.",
    ),
) -> None:
    """Drive an LTTng session — start, stop, view."""
    from openral_observability.tracing_lttng import (
        LttngSessionError,
        start_session,
        stop_session,
        view_session,
    )

    try:
        if action == "start":
            session = start_session(name=name, output_dir=output)
            console.print(
                f"[green]openral profile session start:[/green] "
                f"{session.name} → {session.output_dir}"
            )
            console.print(
                "Run your workload with OPENRAL_ROS2_TRACING=1, then "
                "`openral profile session stop` to flush."
            )
        elif action == "stop":
            stop_session(name=name)
            console.print(f"[green]openral profile session stop:[/green] {name}")
        elif action == "view":
            view_session(output_dir=output)
        else:
            console.print(
                f"[red]openral profile session:[/red] unknown action {action!r}; "
                "expected start | stop | view"
            )
            raise typer.Exit(code=2)
    except LttngSessionError as exc:
        console.print(f"[red]openral profile session:[/red] {exc}")
        raise typer.Exit(code=1) from exc

record(out: Path = typer.Option(..., '--out', '-o', help='Output directory passed to `ros2 bag record -o`.'), profile: str = typer.Option('slim', '--profile', help="Recording profile: 'slim' (default) or 'full'."), storage: str = typer.Option('mcap', '--storage', help='rosbag2 storage backend; mcap is the openral default.'), extra_topic: list[str] = typer.Option([], '--extra-topic', help='Additional topic to record verbatim. Repeatable.'), extra_regex: list[str] = typer.Option([], '--extra-regex', help='Additional regex to OR into --regex. Repeatable.'), dry_run: bool = typer.Option(False, '--dry-run', help='Print the composed argv instead of executing.')) -> None

Wrap ros2 bag record with slim/full topic presets.

Source code in python/cli/src/openral_cli/main.py

@app.command(
    "record",
    help=(
        "Spawn `ros2 bag record` for the OpenRAL ROS graph with a slim/full profile. "
        "Requires a sourced ROS 2 install. Use `--dry-run` to print the argv "
        "instead of executing."
    ),
)
def record(
    out: Path = typer.Option(  # reason: typer Option idiom
        ...,
        "--out",
        "-o",
        help="Output directory passed to `ros2 bag record -o`.",
    ),
    profile: str = typer.Option(
        "slim",
        "--profile",
        help="Recording profile: 'slim' (default) or 'full'.",
    ),
    storage: str = typer.Option(
        "mcap",
        "--storage",
        help="rosbag2 storage backend; mcap is the openral default.",
    ),
    extra_topic: list[str] = typer.Option(
        [],
        "--extra-topic",
        help="Additional topic to record verbatim. Repeatable.",
    ),
    extra_regex: list[str] = typer.Option(
        [],
        "--extra-regex",
        help="Additional regex to OR into --regex. Repeatable.",
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help="Print the composed argv instead of executing.",
    ),
) -> None:
    """Wrap `ros2 bag record` with slim/full topic presets."""
    from openral_observability.replay.cli import run_record

    if profile not in {"slim", "full"}:
        console.print(f"[red]openral record:[/red] unknown profile {profile!r}; expected slim|full")
        raise typer.Exit(code=2)
    try:
        argv, completed = run_record(
            profile=profile,  # type: ignore[arg-type] # reason: validated above against the literal set
            output_dir=out,
            storage=storage,
            extra_topics=extra_topic,
            extra_regex=extra_regex,
            dry_run=dry_run,
        )
    except FileNotFoundError as exc:
        console.print(f"[red]openral record:[/red] {exc}")
        raise typer.Exit(code=1) from exc
    if dry_run:
        print(" ".join(argv))
        return
    assert completed is not None
    if completed.returncode != 0:
        raise typer.Exit(code=completed.returncode)

render_banner(version_str: str, *, width: int | None = None) -> RenderableType

Build the REPL welcome box as a rich renderable (Claude-Code style).

Returns a rich.panel.Panel so the same renderable can be printed to the live console and exported to plain text in tests, independent of terminal/TTY/colour state. The white-bordered rounded box carries OPENRAL v<version> inline in the top border and adapts to width:

• Wide (>= _WIDE_MIN): two columns — the logo mark beside the OPENRAL wordmark with the tagline below on the left; the community links above a horizontal divider above the quick-start commands on the right, split by a vertical divider.
• Narrow: a single stacked column keeping every section; the logo sits beside the wordmark while it fits and stacks above it once it does not.

Parameters:

Name	Type	Description	Default
version_str	str	The installed openral-cli version, rendered as vX.Y.Z.	required
width	int | None	Target terminal width in columns; defaults to a wide layout.	None

Example

from io import StringIO from rich.console import Console con = Console(file=StringIO(), width=120) con.print(render_banner("0.1.0", width=120)) "OPENRAL v0.1.0" in con.file.getvalue() True

Source code in python/cli/src/openral_cli/main.py

def render_banner(version_str: str, *, width: int | None = None) -> RenderableType:
    """Build the REPL welcome box as a rich renderable (Claude-Code style).

    Returns a ``rich.panel.Panel`` so the same renderable can be printed to
    the live console *and* exported to plain text in tests, independent of
    terminal/TTY/colour state. The white-bordered rounded box carries ``OPENRAL
    v<version>`` inline in the top border and adapts to ``width``:

    * **Wide** (``>= _WIDE_MIN``): two columns — the logo mark beside the OPENRAL
      wordmark with the tagline below on the left; the community links above a
      horizontal divider above the quick-start commands on the right, split by a
      vertical divider.
    * **Narrow**: a single stacked column keeping every section; the logo sits
      beside the wordmark while it fits and stacks above it once it does not.

    Args:
        version_str: The installed ``openral-cli`` version, rendered as ``vX.Y.Z``.
        width: Target terminal width in columns; defaults to a wide layout.

    Example:
        >>> from io import StringIO
        >>> from rich.console import Console
        >>> con = Console(file=StringIO(), width=120)
        >>> con.print(render_banner("0.1.0", width=120))
        >>> "OPENRAL v0.1.0" in con.file.getvalue()
        True
    """
    cols = width if width is not None else _WIDE_MIN
    links = _kv_grid(_LINKS, "bold white")
    commands = _kv_grid(_COMMANDS, "bold cyan")

    body: RenderableType
    if cols >= _WIDE_MIN:
        columns = Table(
            box=MINIMAL,
            show_header=False,
            show_edge=False,
            show_lines=False,
            pad_edge=False,
            padding=(0, 2),
            border_style="dim",
            expand=False,
        )
        columns.add_column(vertical="top")
        columns.add_column(vertical="top")
        columns.add_row(
            _identity(stacked=False),
            Group(links, Rule(style="dim"), commands),
        )
        body = columns
    else:
        body = Group(
            _identity(stacked=cols < _SIDE_BY_SIDE_MIN),
            Text(""),
            links,
            Rule(style="dim"),
            commands,
        )

    # Size the box to its content (expand=False) rather than stretching it to the
    # terminal: a compact box keeps slack on wide terminals and only breaks if the
    # window is later dragged narrower than the box itself (printed output cannot
    # reflow). The layout above is chosen so the content always fits ``cols``.
    return Panel(
        body,
        box=ROUNDED,
        border_style="white",
        title=f"OPENRAL v{version_str}",
        title_align="left",
        padding=(1, 2),
        expand=False,
    )

replay(bag: Path = typer.Argument(..., exists=True, readable=True, help='Path to a rosbag2 directory or a bare .mcap file.'), trace: str | None = typer.Option(None, '--trace', help='32-hex-char trace_id to filter on. Defaults to the busiest one in the bag.'), frame: str | None = typer.Option(None, '--frame', help="Pivot from a written LeRobotDataset frame: '<repo_id>/<episode>/<frame>' (e.g. 'openral/dataset-pick/0/12'). Reads that frame's trace_id and uses it as the join key. Requires --dataset-root; mutually exclusive with --trace."), dataset_root: Path | None = typer.Option(None, '--dataset-root', help='Root directory of the LeRobotDataset that --frame refers to.'), dashboard: str | None = typer.Option(None, '--dashboard', help='Dashboard base URL (e.g. http://127.0.0.1:8000). Omit for bag-only.'), out: Path | None = typer.Option(None, '--out', '-o', help='Write the timeline JSON to this file; print to stdout when omitted.')) -> None

Read a bag, join it with spans by trace_id, emit a JSON timeline.

Source code in python/cli/src/openral_cli/main.py

@app.command(
    "replay",
    help=(
        "Join a rosbag2/.mcap file with OTel spans from the live dashboard. "
        "Prints a chronological JSON timeline keyed by trace_id; "
        "writes to `--out` when given. `--dashboard` may be omitted for a "
        "bag-only timeline."
    ),
)
def replay(
    bag: Path = typer.Argument(  # reason: typer Argument idiom
        ...,
        exists=True,
        readable=True,
        help="Path to a rosbag2 directory or a bare .mcap file.",
    ),
    trace: str | None = typer.Option(
        None,
        "--trace",
        help="32-hex-char trace_id to filter on. Defaults to the busiest one in the bag.",
    ),
    frame: str | None = typer.Option(
        None,
        "--frame",
        help=(
            "Pivot from a written LeRobotDataset frame: '<repo_id>/<episode>/<frame>' "
            "(e.g. 'openral/dataset-pick/0/12'). Reads that frame's trace_id and uses "
            "it as the join key. Requires --dataset-root; mutually exclusive with --trace."
        ),
    ),
    dataset_root: Path | None = typer.Option(
        None,
        "--dataset-root",
        help="Root directory of the LeRobotDataset that --frame refers to.",
    ),
    dashboard: str | None = typer.Option(
        None,
        "--dashboard",
        help="Dashboard base URL (e.g. http://127.0.0.1:8000). Omit for bag-only.",
    ),
    out: Path | None = typer.Option(
        None,
        "--out",
        "-o",
        help="Write the timeline JSON to this file; print to stdout when omitted.",
    ),
) -> None:
    """Read a bag, join it with spans by trace_id, emit a JSON timeline."""
    from openral_observability.replay.cli import run_replay, write_timeline

    # ISSUE-109 pivot — resolve --frame into a concrete trace_id off the
    # dataset before the join. Done here (not in run_replay) so the
    # openral_observability replay module stays free of the lerobot dep.
    if frame is not None:
        if trace is not None:
            console.print("[red]openral replay:[/red] --frame and --trace are mutually exclusive")
            raise typer.Exit(code=2)
        if dataset_root is None:
            console.print("[red]openral replay:[/red] --frame requires --dataset-root")
            raise typer.Exit(code=2)
        trace = _resolve_frame_trace_id(frame, dataset_root)

    result = run_replay(bag_path=bag, trace_id=trace, dashboard_url=dashboard)
    if out is not None:
        write_timeline(result, out)
        console.print(
            f"[green]openral replay:[/green] wrote {len(result.timeline)} entries to {out}"
        )
        if result.trace_id:
            console.print(f"trace_id: {result.trace_id}")
        return
    print(_json.dumps(result.to_json(), indent=2, sort_keys=False))

robot_vendor_urdf(robot_id: str = typer.Argument(..., help='OpenRAL robot id; names the output file (e.g. ur5e → ur5e.urdf).'), upstream: str = typer.Option(..., '--upstream', help="Upstream source: 'rd:<robot_descriptions module>' (xacro, expanded via xacrodoc) or 'file:<path>' to an already-flat URDF."), out: Path = typer.Option(..., '--out', help="Output directory; '<robot_id>.urdf' is written here."), rename: list[str] | None = typer.Option(None, '--rename', help="Joint-name rename as 'PATTERN=>REPL' (regex re.sub). Repeatable — applied in order (so100/so101 take 6, gr1/h1 take 1). Defaults to the per-robot rule (openarm strips its 'openarm_' prefix)."), raw_text: bool = typer.Option(False, '--raw-text/--no-raw-text', help='Copy an already-flat upstream URDF verbatim and apply --rename to the raw XML (no yourdfpy round-trip), preserving package:// mesh paths byte-for-byte (so100/so101/gr1/h1).')) -> None

Expand an upstream description to a flat, committed URDF.

Source code in python/cli/src/openral_cli/main.py

@robot_app.command("vendor-urdf")
def robot_vendor_urdf(
    robot_id: str = typer.Argument(
        ...,
        help="OpenRAL robot id; names the output file (e.g. ur5e → ur5e.urdf).",
    ),
    upstream: str = typer.Option(
        ...,
        "--upstream",
        help=(
            "Upstream source: 'rd:<robot_descriptions module>' (xacro, expanded "
            "via xacrodoc) or 'file:<path>' to an already-flat URDF."
        ),
    ),
    out: Path = typer.Option(
        ...,
        "--out",
        help="Output directory; '<robot_id>.urdf' is written here.",
    ),
    rename: list[str] | None = typer.Option(
        None,
        "--rename",
        help=(
            "Joint-name rename as 'PATTERN=>REPL' (regex re.sub). Repeatable — "
            "applied in order (so100/so101 take 6, gr1/h1 take 1). Defaults to "
            "the per-robot rule (openarm strips its 'openarm_' prefix)."
        ),
    ),
    raw_text: bool = typer.Option(
        False,
        "--raw-text/--no-raw-text",
        help=(
            "Copy an already-flat upstream URDF verbatim and apply --rename to "
            "the raw XML (no yourdfpy round-trip), preserving package:// mesh "
            "paths byte-for-byte (so100/so101/gr1/h1)."
        ),
    ),
) -> None:
    """Expand an upstream description to a flat, committed URDF."""
    from openral_cli.robot import vendor_urdf

    rename_pairs: list[tuple[str, str]] | None = None
    if rename:
        rename_pairs = []
        for spec in rename:
            if "=>" not in spec:
                raise typer.BadParameter("--rename must be 'PATTERN=>REPL'", param_hint="--rename")
            pat, _, repl = spec.partition("=>")
            rename_pairs.append((pat, repl))
    written = vendor_urdf(
        robot_id, upstream=upstream, out_dir=out, rename=rename_pairs, raw_text=raw_text
    )
    typer.echo(f"Wrote {written}")

rskill_check(rskill_id: str | None = typer.Argument(None, metavar='RSKILL_ID', help='rSkill to check — bare in-tree name, path (rskills/<name>), or HF Hub repo id (e.g. OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16). Omit to walk every installed / in-tree rSkill (walk-all mode).'), robot: Path = typer.Option(Path('robot.yaml'), '--robot', help='Path to a RobotDescription yaml (typically the output of `openral detect`).'), rskills_dir: Path = typer.Option(Path('rskills'), '--rskills-dir', help='(Walk-all mode) in-tree rSkills directory to scan in addition to the installed registry. Skipped if it does not exist.'), json_output: bool = typer.Option(False, '--json', help='Output the CompatibilityReport as JSON.')) -> None

Report whether one (or every) rSkill will run on the current host.

Two modes:

• openral rskill check <rskill_id> resolves the id the same way as openral rskill list / openral benchmark run --rskill <id> (in-tree, installed registry, or HF Hub) and prints a per-section breakdown — embodiment, capability flags, GPU runtime, GPU dtype, sensors, actuators.
• openral rskill check (no arg) walks every installed / in-tree rSkill via openral_detect.check_installed_rskills and prints a one-row-per-rSkill compatibility table.

Exits 1 if any non-informational section fails (single-rSkill mode) or if any installed rSkill is incompatible (walk-all mode); exits 0 otherwise.

Example

openral rskill check smolvla-libero

openral rskill check smolvla-libero --robot /tmp/robot.yaml --json

openral rskill check # walk-all

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("check")
def rskill_check(
    rskill_id: str | None = typer.Argument(
        None,
        metavar="RSKILL_ID",
        help=(
            "rSkill to check — bare in-tree name, path (rskills/<name>), "
            "or HF Hub repo id (e.g. OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16). "
            "Omit to walk every installed / in-tree rSkill (walk-all mode)."
        ),
    ),
    robot: Path = typer.Option(
        Path("robot.yaml"),
        "--robot",
        help="Path to a RobotDescription yaml (typically the output of `openral detect`).",
    ),
    rskills_dir: Path = typer.Option(
        Path("rskills"),
        "--rskills-dir",
        help=(
            "(Walk-all mode) in-tree rSkills directory to scan in addition to "
            "the installed registry. Skipped if it does not exist."
        ),
    ),
    json_output: bool = typer.Option(
        False, "--json", help="Output the CompatibilityReport as JSON."
    ),
) -> None:
    """Report whether one (or every) rSkill will run on the current host.

    Two modes:

    * ``openral rskill check <rskill_id>`` resolves the id the same way as
      ``openral rskill list`` / ``openral benchmark run --rskill <id>``
      (in-tree, installed registry, or HF Hub) and prints a per-section
      breakdown — embodiment, capability flags, GPU runtime, GPU dtype,
      sensors, actuators.
    * ``openral rskill check`` (no arg) walks every installed / in-tree
      rSkill via `openral_detect.check_installed_rskills` and prints a
      one-row-per-rSkill compatibility table.

    Exits 1 if any non-informational section fails (single-rSkill mode)
    or if any installed rSkill is incompatible (walk-all mode); exits 0
    otherwise.

    Example:
        >>> # openral rskill check smolvla-libero
        >>> # openral rskill check smolvla-libero --robot /tmp/robot.yaml --json
        >>> # openral rskill check                                              # walk-all
    """
    import json as _json_mod

    from openral_core.schemas import RobotDescription
    from openral_detect import check_installed_rskills, check_single_rskill

    if not robot.exists():
        console.print(
            f"[red]Robot description not found:[/red] {robot}\n"
            "Run [bold]openral detect[/bold] first."
        )
        raise typer.Exit(code=1)

    description = RobotDescription.from_yaml(str(robot))

    if rskill_id is not None:
        single_report = check_single_rskill(rskill_id, description)
        single_row = single_report.rows[0]
        if json_output:
            console.print_json(_json_mod.dumps(single_report.model_dump(mode="json")))
        else:
            _render_single_rskill_table(single_row, description.name)
        if not single_row.compatible:
            raise typer.Exit(code=1)
        return

    # ── Walk-all (no-arg) ─────────────────────────────────────────────────────
    walk_dir = rskills_dir if rskills_dir.is_dir() else None
    report = check_installed_rskills(description, rskills_dir=walk_dir)
    if json_output:
        console.print_json(_json_mod.dumps(report.model_dump(mode="json")))
    else:
        _render_walk_all_table(report, description.name)

    if report.incompatible:
        raise typer.Exit(code=1)

rskill_install(hub_id: str = typer.Argument(..., metavar='HUB_ID', help='HF Hub repository, e.g. OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16'), revision: str = typer.Option('', '--revision', '-r', help='Git commit SHA or branch to pin (recommended for reproducibility).'), force: bool = typer.Option(False, '--force', '-f', help='Re-download even if cached files already exist.'), non_commercial: bool = typer.Option(False, '--non-commercial', help='Declare non-commercial research intent (relaxes NVIDIA non-commercial guard).'), yes: bool = typer.Option(False, '--yes', '-y', help='Skip confirmation prompt for proprietary or non-commercial licenses.')) -> None

Download an rSkill from the HF Hub, validate it, and register it locally.

Fetches rskill.yaml from the repository, validates the manifest, surfaces the license to the terminal, then downloads the weights snapshot into the local HF Hub cache (~/.cache/openral/rskills/).

The rSkill is registered in ~/.local/share/openral/rskills.json and can be listed with openral rskill list.

Example

openral rskill install OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16

openral rskill install smolvla-libero --revision abc1234

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("install")
def rskill_install(
    hub_id: str = typer.Argument(
        ...,
        metavar="HUB_ID",
        help="HF Hub repository, e.g. OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16",
    ),
    revision: str = typer.Option(
        "",
        "--revision",
        "-r",
        help="Git commit SHA or branch to pin (recommended for reproducibility).",
    ),
    force: bool = typer.Option(
        False,
        "--force",
        "-f",
        help="Re-download even if cached files already exist.",
    ),
    non_commercial: bool = typer.Option(
        False,
        "--non-commercial",
        help="Declare non-commercial research intent (relaxes NVIDIA non-commercial guard).",
    ),
    yes: bool = typer.Option(
        False,
        "--yes",
        "-y",
        help="Skip confirmation prompt for proprietary or non-commercial licenses.",
    ),
) -> None:
    """Download an rSkill from the HF Hub, validate it, and register it locally.

    Fetches ``rskill.yaml`` from the repository, validates the manifest,
    surfaces the license to the terminal, then downloads the weights snapshot
    into the local HF Hub cache (``~/.cache/openral/rskills/``).

    The rSkill is registered in ``~/.local/share/openral/rskills.json`` and
    can be listed with ``openral rskill list``.

    Example:
        >>> # openral rskill install OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16
        >>> # openral rskill install smolvla-libero --revision abc1234
    """
    from openral_rskill.loader import rSkill

    # ── Step 0: a HF repo id needs an `org/name` shape. A bare name (the most
    # common paste mistake) otherwise 404s against a non-existent top-level repo
    # — fail fast with the canonical suggestion instead of a raw Hub stack trace.
    if "/" not in hub_id:
        suggestion = f"{_DEFAULT_RSKILL_ORG}/{hub_id}"
        console.print(
            f"[red]Not a Hub repo id:[/red] '{hub_id}' has no org prefix (expected `org/name`)."
        )
        console.print(f"  Did you mean:  [cyan]openral rskill install {suggestion}[/cyan]")
        console.print(f"  Or find it:    [cyan]openral rskill search {hub_id}[/cyan]")
        raise typer.Exit(code=1)

    console.print(f"[bold]openral rskill install[/bold] — fetching [cyan]{hub_id}[/cyan] …")

    # ── Step 1: fetch manifest only (to surface license before downloading weights)
    try:
        from huggingface_hub import hf_hub_download
        from openral_core.schemas import RSkillManifest
    except ImportError as exc:
        console.print(f"[red]Missing dependency:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    try:
        manifest_path = hf_hub_download(
            repo_id=hub_id,
            filename="rskill.yaml",
            revision=revision or None,
        )
        manifest = RSkillManifest.from_yaml(manifest_path)
    except Exception as exc:  # reason: surface any download/parse error to user
        console.print(f"[red]Failed to fetch manifest:[/red] {exc}")
        if "404" in str(exc) or "Repository Not Found" in str(exc):
            bare = hub_id.rsplit("/", 1)[-1]
            console.print(f"  Browse available skills: [cyan]openral rskill search {bare}[/cyan]")
        raise typer.Exit(code=1)  # noqa: B904

    # ── Step 2: display license + confirm if non-permissive
    _display_license_banner(manifest.name, manifest.license.value, manifest.version, console)

    _permissive = {"apache-2.0", "mit", "bsd"}
    if manifest.license.value not in _permissive and not yes:
        confirmed = typer.confirm("Proceed with installation?", default=False)
        if not confirmed:
            console.print("[yellow]Aborted.[/yellow]")
            raise typer.Exit(code=0)

    # ── Step 3: full install (snapshot download + registry)
    console.print("  Downloading weights snapshot …")
    try:
        pkg = rSkill.from_pretrained(
            hub_id,
            revision=revision or None,
            force_download=force,
            commercial_use=not non_commercial,
        )
    except Exception as exc:  # reason: surface ROSConfigError + network errors
        console.print(f"[red]Install failed:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    console.print(
        f"[green]Installed[/green] [bold]{pkg.manifest.name}[/bold] "
        f"v{pkg.manifest.version} → {pkg.local_dir}"
    )
    if not revision:
        console.print(
            "[yellow]Tip:[/yellow] Pin a revision for reproducibility: "
            f"openral rskill install {hub_id} --revision <sha>"
        )

rskill_list(json: bool = typer.Option(False, '--json', help='Output machine-readable JSON')) -> None

List every available rSkill — in-tree (rskills/) + HF-Hub-installed.

Each row shows the source so users can tell at a glance which entries are paste-able as --rskill <name> (in-tree) versus which need a HF Hub install first. JSON output keeps the same fields.

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("list")
def rskill_list(
    json: bool = typer.Option(False, "--json", help="Output machine-readable JSON"),
) -> None:
    """List every available rSkill — in-tree (``rskills/``) + HF-Hub-installed.

    Each row shows the source so users can tell at a glance which entries
    are paste-able as ``--rskill <name>`` (in-tree) versus which
    need a HF Hub install first. JSON output keeps the same fields.
    """
    import json as _json_mod

    from openral_rskill.loader import discover_intree_rskills, rSkill

    intree = discover_intree_rskills()
    try:
        installed = rSkill.list_installed()
    except Exception as exc:  # reason: surface corrupt registry error
        console.print(f"[red]Failed to read installed registry:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    def _bare_name_from_repo_id(repo_id: str) -> str:
        """Recover the bare rskill name that the loader would resolve to.

        Mirrors `_candidate_local_paths`: strip the org prefix and the
        `rskill-`/`rskill_` prefix so the URI matches the in-tree form.
        """
        tail = repo_id.rsplit("/", 1)[-1]
        return tail.removeprefix("rskill-").removeprefix("rskill_") or repo_id

    if json:
        data = [
            {
                "source": "in-tree",
                "name": name,
                "repo_id": m.name,
                "version": m.version,
                "model_family": m.model_family,
                "role": str(m.role),
                "license": m.license.value,
                "embodiment_tags": list(m.embodiment_tags),
                "uri": name,
            }
            for name, m in intree
        ] + [
            {
                "source": "installed",
                "name": _bare_name_from_repo_id(e.repo_id),
                "repo_id": e.repo_id,
                "version": e.version,
                "model_family": None,
                "role": e.role,
                "license": e.license,
                "embodiment_tags": list(e.embodiment_tags),
                "uri": _bare_name_from_repo_id(e.repo_id),
                "installed_at": e.installed_at,
            }
            for e in installed
        ]
        console.print_json(_json_mod.dumps(data))
        return

    if not intree and not installed:
        console.print(
            "[dim]No rSkills available. Drop one under rskills/<name>/ or run: "
            "openral rskill install <hub-id>[/dim]"
        )
        return

    table = Table(title="Available rSkills")
    table.add_column("source", style="dim")
    table.add_column("name / repo_id", style="cyan bold")
    table.add_column("version")
    table.add_column("family")
    table.add_column("license")
    table.add_column("embodiment_tags")
    table.add_column("paste-able --rskill")

    _permissive = {"apache-2.0", "mit", "bsd"}
    for name, m in intree:
        lic = m.license.value
        lic_color = "green" if lic in _permissive else "yellow"
        table.add_row(
            "in-tree",
            name,
            m.version,
            m.model_family or "—",
            f"[{lic_color}]{lic}[/{lic_color}]",
            ", ".join(m.embodiment_tags) or "—",
            name,
        )
    for entry in installed:
        lic_color = "green" if entry.license in _permissive else "yellow"
        bare = _bare_name_from_repo_id(entry.repo_id)
        table.add_row(
            "installed",
            entry.repo_id,
            entry.version,
            "—",
            f"[{lic_color}]{entry.license}[/{lic_color}]",
            ", ".join(entry.embodiment_tags) or "—",
            bare,
        )
    console.print(table)

rskill_new(rskill_id: str = typer.Argument(..., metavar='ID', help='Local rSkill id, convention <policy>-<task> e.g. pi05-pick-cube.'), out_dir: Path | None = typer.Option(None, '--out-dir', help='Destination directory. Defaults to rskills/<ID>/ under the cwd.'), owner: str | None = typer.Option(None, '--owner', help="HF Hub owner segment for the manifest 'name' field."), license_: str | None = typer.Option(None, '--license', help=('One of: ' + ' | '.join((p.value) for p in RSkillLicensePosture) + '.')), embodiment_tag: str | None = typer.Option(None, '--embodiment-tag', help='An embodiment tag declared by a robots/*/robot.yaml, or any | custom | multi.'), family: str | None = typer.Option(None, '--family', '-f', help="Policy family — one of act | smolvla | pi05 | xvla | diffusion. Sets model_family / chunk_size / dtype / latency budget from the matching in-tree reference manifest so a fresh ACT scaffold doesn't ship pi0.5 numbers. Inferred from --from-hf when set; interactively prompted otherwise."), from_hf: str | None = typer.Option(None, '--from-hf', help="HF Hub repo id (e.g. 'Deepkar/libero-test-act' or 'hf://Deepkar/libero-test-act'). Fetches the checkpoint's config.json, infers the family, and pre-fills chunk_size, sensors_required, state_contract.dim, image_preprocessing aliases, and weights_uri. Eliminates manual rewriting after scaffold."), yes: bool = typer.Option(False, '--yes', '-y', help='Skip interactive prompts and accept the defaults (for scripting / CI).'), overwrite: bool = typer.Option(False, '--overwrite', help='Replace an existing destination directory instead of refusing.')) -> None

Scaffold a new local rSkill from rskills/template/.

Three modes:

• --from-hf <owner/repo> — most intuitive. Fetches the checkpoint's config.json and pre-fills model_family / chunk_size / sensors_required / state_contract / image_preprocessing.aliases / weights_uri from the real Hub-side contract. No more hand-rewriting after scaffold.
• --family <act|smolvla|pi05|xvla|diffusion> — family-aware defaults without Hub introspection. Use when you know the family but the weights live somewhere else (private repo, local mirror).
• Neither — interactive. Prompts for --owner / --license / --embodiment-tag / --family (and offers --from-hf as a one-shot alternative). Pass --yes to skip all prompts and accept the defaults (your-org / apache-2.0 / franka_panda / no-family).

The generated manifest is round-tripped through RSkillManifest.from_yaml and rSkill.from_yaml so a malformed scaffold fails at scaffold-time, not on first load.

Example

openral rskill new act-libero --from-hf Deepkar/libero-test-act

openral rskill new pi05-pick-cube --family pi05 --embodiment-tag franka_panda

openral rskill new act-aloha-insertion --owner foo --embodiment-tag aloha

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("new")
def rskill_new(
    rskill_id: str = typer.Argument(
        ...,
        metavar="ID",
        help="Local rSkill id, convention <policy>-<task> e.g. pi05-pick-cube.",
    ),
    out_dir: Path | None = typer.Option(
        None,
        "--out-dir",
        help="Destination directory. Defaults to rskills/<ID>/ under the cwd.",
    ),
    owner: str | None = typer.Option(
        None,
        "--owner",
        help="HF Hub owner segment for the manifest 'name' field.",
    ),
    license_: str | None = typer.Option(
        None,
        "--license",
        help="One of: " + " | ".join(p.value for p in RSkillLicensePosture) + ".",
    ),
    embodiment_tag: str | None = typer.Option(
        None,
        "--embodiment-tag",
        help="An embodiment tag declared by a robots/*/robot.yaml, or any | custom | multi.",
    ),
    family: str | None = typer.Option(
        None,
        "--family",
        "-f",
        help=(
            "Policy family — one of act | smolvla | pi05 | xvla | diffusion. "
            "Sets model_family / chunk_size / dtype / latency budget from the "
            "matching in-tree reference manifest so a fresh ACT scaffold "
            "doesn't ship pi0.5 numbers. Inferred from --from-hf when set; "
            "interactively prompted otherwise."
        ),
    ),
    from_hf: str | None = typer.Option(
        None,
        "--from-hf",
        help=(
            "HF Hub repo id (e.g. 'Deepkar/libero-test-act' or "
            "'hf://Deepkar/libero-test-act'). Fetches the checkpoint's "
            "config.json, infers the family, and pre-fills chunk_size, "
            "sensors_required, state_contract.dim, image_preprocessing aliases, "
            "and weights_uri. Eliminates manual rewriting after scaffold."
        ),
    ),
    yes: bool = typer.Option(
        False,
        "--yes",
        "-y",
        help="Skip interactive prompts and accept the defaults (for scripting / CI).",
    ),
    overwrite: bool = typer.Option(
        False,
        "--overwrite",
        help="Replace an existing destination directory instead of refusing.",
    ),
) -> None:
    """Scaffold a new local rSkill from ``rskills/template/``.

    Three modes:

    - ``--from-hf <owner/repo>`` — most intuitive. Fetches the
      checkpoint's ``config.json`` and pre-fills ``model_family`` /
      ``chunk_size`` / ``sensors_required`` / ``state_contract`` /
      ``image_preprocessing.aliases`` / ``weights_uri`` from the real
      Hub-side contract. No more hand-rewriting after scaffold.
    - ``--family <act|smolvla|pi05|xvla|diffusion>`` — family-aware
      defaults without Hub introspection. Use when you know the family
      but the weights live somewhere else (private repo, local mirror).
    - Neither — interactive. Prompts for ``--owner`` / ``--license`` /
      ``--embodiment-tag`` / ``--family`` (and offers ``--from-hf`` as
      a one-shot alternative). Pass ``--yes`` to skip all prompts and
      accept the defaults (your-org / apache-2.0 / franka_panda /
      no-family).

    The generated manifest is round-tripped through
    `RSkillManifest.from_yaml` and `rSkill.from_yaml`
    so a malformed scaffold fails at scaffold-time, not on first load.

    Example:
        >>> # openral rskill new act-libero --from-hf Deepkar/libero-test-act
        >>> # openral rskill new pi05-pick-cube --family pi05 --embodiment-tag franka_panda
        >>> # openral rskill new act-aloha-insertion --owner foo --embodiment-tag aloha
    """
    from openral_rskill.loader import intree_embodiment_tags

    from openral_cli._rskill_scaffolder import scaffold_rskill

    valid_tags = sorted(intree_embodiment_tags())
    valid_licenses = [v.value for v in RSkillLicensePosture]

    resolved_owner = _resolve_or_prompt(
        owner,
        prompt=f"HF Hub owner (e.g. your username or org) [{_DEFAULT_OWNER}]",
        default=_DEFAULT_OWNER,
        skip_prompt=yes,
    )
    resolved_license = _resolve_or_prompt(
        license_,
        prompt=f"License posture [{_DEFAULT_LICENSE}]",
        default=_DEFAULT_LICENSE,
        skip_prompt=yes,
    )
    resolved_embodiment = _resolve_or_prompt(
        embodiment_tag,
        prompt=f"Embodiment tag (canonical robot id) [{_DEFAULT_EMBODIMENT}]",
        default=_DEFAULT_EMBODIMENT,
        skip_prompt=yes,
    )

    try:
        license_enum = RSkillLicensePosture(resolved_license)
    except ValueError as exc:
        console.print(
            f"[red]Invalid license:[/red] {resolved_license!r}. "
            f"Valid values: {', '.join(valid_licenses)}"
        )
        raise typer.Exit(code=1) from exc

    if resolved_embodiment not in valid_tags:
        console.print(
            f"[red]Invalid embodiment_tag:[/red] {resolved_embodiment!r}. "
            f"Valid values: {', '.join(valid_tags)}"
        )
        raise typer.Exit(code=1)

    resolved_family, intel_patch = _resolve_family_and_patch(
        family=family, from_hf=from_hf, yes=yes
    )

    resolved_out = out_dir if out_dir is not None else Path("rskills") / rskill_id

    try:
        result = scaffold_rskill(
            rskill_id,
            out_dir=resolved_out,
            owner=resolved_owner,
            license_=license_enum,
            embodiment_tag=resolved_embodiment,
            family=resolved_family,
            patch=intel_patch,
            overwrite=overwrite,
        )
    except ROSConfigError as exc:
        console.print(f"[red]Scaffold failed:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    console.print(f"[green]Scaffolded[/green] [cyan]{rskill_id}[/cyan] → {result}")
    if intel_patch is not None:
        console.print(
            "[dim]Next steps: edit description / README.md, optionally adjust "
            "image_preprocessing.flip_180 for your scene, add eval/<benchmark>.json "
            "results, then publish with tools/rskill_publisher.py.[/dim]"
        )
    else:
        console.print(
            "[dim]Next steps: edit rskill.yaml (weights_uri / chunk_size / "
            "sensors_required / image_preprocessing), update README.md, add "
            "eval/<benchmark>.json results, then publish with "
            "tools/rskill_publisher.py. Tip: pass `--from-hf <owner/repo>` next "
            "time to auto-fill the manifest from a published checkpoint.[/dim]"
        )

rskill_search(query: list[str] | None = typer.Argument(None, metavar='[QUERY]...', help='Free-text query words; every word must match (case-insensitive) the repo id, manifest name/description, family, kind, role, embodiment or Hub tags.'), kind: str = typer.Option('', '--kind', help='Filter by manifest kind (vla, ros_action, detector, …).'), role: str = typer.Option('', '--role', help='Filter by control role (s0, s1, s2).'), embodiment: str = typer.Option('', '--embodiment', help='Filter by embodiment tag (e.g. franka_panda).'), license_: str = typer.Option('', '--license', help='Filter by license posture (e.g. apache-2.0).'), family: str = typer.Option('', '--family', help='Filter by manifest model_family.'), limit: int = typer.Option(0, '--limit', help='Max results shown, after sorting by repo_id. 0 = unlimited.'), json: bool = typer.Option(False, '--json', help='Output machine-readable JSON.')) -> None

Search the OpenRAL HF Hub org for installable rSkills.

Lists every OpenRAL/* repo tagged rskill whose rskill.yaml manifest validates and matches QUERY (every whitespace-split token substring-matched, case-insensitively, against the repo id, manifest name/description/model_family/kind/role/embodiment tags, and Hub tags) plus the optional facet filters — so the printed ids are paste-able into openral rskill install <repo_id>. The HF Hub is the index — there is no bespoke catalog service. Repos without a valid manifest are skipped and the count surfaced.

Example

openral rskill search aloha

openral rskill search --kind detector --license apache-2.0

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("search")
def rskill_search(
    query: list[str] | None = typer.Argument(
        None,
        metavar="[QUERY]...",
        help="Free-text query words; every word must match (case-insensitive) the repo "
        "id, manifest name/description, family, kind, role, embodiment or Hub tags.",
    ),
    kind: str = typer.Option(
        "", "--kind", help="Filter by manifest kind (vla, ros_action, detector, …)."
    ),
    role: str = typer.Option("", "--role", help="Filter by control role (s0, s1, s2)."),
    embodiment: str = typer.Option(
        "", "--embodiment", help="Filter by embodiment tag (e.g. franka_panda)."
    ),
    license_: str = typer.Option(
        "", "--license", help="Filter by license posture (e.g. apache-2.0)."
    ),
    family: str = typer.Option("", "--family", help="Filter by manifest model_family."),
    limit: int = typer.Option(
        0, "--limit", help="Max results shown, after sorting by repo_id. 0 = unlimited."
    ),
    json: bool = typer.Option(False, "--json", help="Output machine-readable JSON."),
) -> None:
    """Search the OpenRAL HF Hub org for installable rSkills.

    Lists every ``OpenRAL/*`` repo tagged ``rskill`` whose ``rskill.yaml``
    manifest validates and matches ``QUERY`` (every whitespace-split token
    substring-matched, case-insensitively, against the repo id, manifest
    name/description/model_family/kind/role/embodiment tags, and Hub tags)
    plus the optional facet filters — so the printed ids are paste-able into
    ``openral rskill install <repo_id>``. The HF Hub is the index — there is
    no bespoke catalog service. Repos without a valid manifest are skipped
    and the count surfaced.

    Example:
        >>> # openral rskill search aloha
        >>> # openral rskill search --kind detector --license apache-2.0
    """
    import json as _json_mod

    from openral_rskill.hub_search import search_hub_rskills

    query_str = " ".join(query or [])

    try:
        result = search_hub_rskills(
            query_str,
            kind=kind,
            role=role,
            embodiment=embodiment,
            license=license_,
            family=family,
            org=_DEFAULT_RSKILL_ORG,
            limit=limit or None,
        )
    except ImportError as exc:
        console.print(f"[red]Missing dependency:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904
    except Exception as exc:  # reason: surface Hub/network errors to the user
        console.print(f"[red]Search failed:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    if json:
        console.print_json(
            _json_mod.dumps(
                [
                    {
                        "repo_id": hit.repo_id,
                        "name": hit.manifest.name,
                        "version": hit.manifest.version,
                        "kind": hit.manifest.kind,
                        "role": hit.manifest.role,
                        "license": hit.manifest.license.value,
                        "model_family": hit.manifest.model_family,
                        "embodiment_tags": list(hit.manifest.embodiment_tags),
                        "description": hit.manifest.description,
                        "local": hit.local,
                    }
                    for hit in result.hits
                ]
            )
        )
        return

    _render_rskill_search_results(result, query_str)

sensor_list(vendor: str = typer.Option('', '--vendor', help='Filter by vendor (lowercase, e.g. intel, orbbec, livox).'), modality: str = typer.Option('', '--modality', help='Filter by sensor modality (e.g. rgb, depth, lidar_2d, point_cloud).'), kind: str = typer.Option('', '--kind', help="Filter by kind: 'sensor' or 'bundle'."), json_output: bool = typer.Option(False, '--json', help='Emit machine-readable JSON instead of a table.')) -> None

List every sensor registered in the openral sensor catalog.

Example

openral sensor list

openral sensor list --vendor intel

openral sensor list --modality lidar_2d --json

Source code in python/cli/src/openral_cli/main.py

@sensor_app.command("list")
def sensor_list(
    vendor: str = typer.Option(
        "",
        "--vendor",
        help="Filter by vendor (lowercase, e.g. intel, orbbec, livox).",
    ),
    modality: str = typer.Option(
        "",
        "--modality",
        help="Filter by sensor modality (e.g. rgb, depth, lidar_2d, point_cloud).",
    ),
    kind: str = typer.Option(
        "",
        "--kind",
        help="Filter by kind: 'sensor' or 'bundle'.",
    ),
    json_output: bool = typer.Option(
        False,
        "--json",
        help="Emit machine-readable JSON instead of a table.",
    ),
) -> None:
    """List every sensor registered in the openral sensor catalog.

    Example:
        >>> # openral sensor list
        >>> # openral sensor list --vendor intel
        >>> # openral sensor list --modality lidar_2d --json
    """
    # Imported lazily so `openral --help` doesn't pay the side-effect import cost.
    from openral_core.schemas import SensorModality
    from openral_sensors import CATALOG

    modality_enum: SensorModality | None = None
    if modality:
        try:
            modality_enum = SensorModality(modality)
        except ValueError:
            valid = ", ".join(m.value for m in SensorModality)
            console.print(f"[red]Unknown --modality {modality!r}.  Valid: {valid}[/red]")
            raise typer.Exit(code=1) from None

    kind_filter: str | None = None
    if kind:
        if kind not in ("sensor", "bundle"):
            console.print(f"[red]--kind must be 'sensor' or 'bundle', got {kind!r}.[/red]")
            raise typer.Exit(code=1)
        kind_filter = kind

    entries = CATALOG.filter(
        vendor=vendor or None,
        modality=modality_enum,
        kind=kind_filter,  # type: ignore[arg-type]  # reason: narrowed above
    )

    if json_output:
        payload = [
            {
                "id": e.id,
                "vendor": e.vendor,
                "model": e.model,
                "kind": e.kind,
                "modalities": [m.value for m in e.modalities],
                "description": e.description,
                "docs_url": e.docs_url,
            }
            for e in entries
        ]
        console.print_json(_json.dumps(payload))
        return

    if not entries:
        console.print("[yellow]No sensors match the requested filters.[/yellow]")
        return

    table = Table(title=f"openral sensor catalog ({len(entries)} entries)")
    table.add_column("id", style="cyan", no_wrap=True)
    table.add_column("kind")
    table.add_column("modalities", style="magenta")
    table.add_column("description")
    for e in entries:
        table.add_row(
            e.id,
            e.kind,
            ",".join(m.value for m in e.modalities),
            e.description,
        )
    console.print(table)

sensor_show(sensor_id: str = typer.Argument(..., metavar='SENSOR_ID', help='Catalog id, e.g. intel/realsense_d435i or slamtec/rplidar_a2'), name: str = typer.Option('sensor', '--name', help='Instance name passed to the factory (used as topic / frame prefix).'), parent_frame: str = typer.Option('base_link', '--parent-frame', help='tf2 parent frame for the static transform.'), json_output: bool = typer.Option(False, '--json', help='Emit the resolved SensorSpec / SensorBundle as JSON.')) -> None

Resolve a catalog entry to a concrete SensorSpec / SensorBundle.

Example

openral sensor show intel/realsense_d435i --name head

openral sensor show slamtec/rplidar_a2 --json

Source code in python/cli/src/openral_cli/main.py

@sensor_app.command("show")
def sensor_show(
    sensor_id: str = typer.Argument(
        ...,
        metavar="SENSOR_ID",
        help="Catalog id, e.g. intel/realsense_d435i or slamtec/rplidar_a2",
    ),
    name: str = typer.Option(
        "sensor",
        "--name",
        help="Instance name passed to the factory (used as topic / frame prefix).",
    ),
    parent_frame: str = typer.Option(
        "base_link",
        "--parent-frame",
        help="tf2 parent frame for the static transform.",
    ),
    json_output: bool = typer.Option(
        False,
        "--json",
        help="Emit the resolved SensorSpec / SensorBundle as JSON.",
    ),
) -> None:
    """Resolve a catalog entry to a concrete ``SensorSpec`` / ``SensorBundle``.

    Example:
        >>> # openral sensor show intel/realsense_d435i --name head
        >>> # openral sensor show slamtec/rplidar_a2 --json
    """
    from openral_sensors import CATALOG

    try:
        entry = CATALOG.get(sensor_id)
    except KeyError as exc:
        console.print(f"[red]{exc.args[0]}[/red]")
        raise typer.Exit(code=1) from None

    resolved = entry.factory(name=name, parent_frame=parent_frame)

    if json_output:
        console.print_json(resolved.model_dump_json(indent=2))
        return

    console.print(f"[bold cyan]{entry.id}[/bold cyan]  ({entry.kind})")
    console.print(f"  vendor      : [magenta]{entry.vendor}[/magenta]")
    console.print(f"  model       : [magenta]{entry.model}[/magenta]")
    console.print(f"  modalities  : {', '.join(m.value for m in entry.modalities)}")
    console.print(f"  description : [dim]{entry.description}[/dim]")
    if entry.docs_url:
        console.print(f"  docs_url    : [dim]{entry.docs_url}[/dim]")
    console.print()
    console.print("[bold]Resolved:[/bold]")
    console.print_json(resolved.model_dump_json(indent=2))

Hardware Abstraction Layer

openral HAL — Hardware Abstraction Layer public API.

Public surface: - HAL: structural Protocol every adapter must satisfy (RFC §8.2). - LifecycleEStopHAL / ResettableLifecycleEStopHAL: typed lifecycle e-stop propagation and recovery contracts. - HALHealthProvider / HALHealthReport: cached-diagnostics contract for the generic lifecycle heartbeat. - RosControlHAL: ros2_control-backed adapter. - SO100FollowerHAL / SO100_DESCRIPTION / so100_with_sensors (catalog-backed sensor loadout, issue #23); SO100DigitalTwin / SO100DigitalTwinConfig in-process sim; SO100MujocoHAL MuJoCo twin, mujoco_menagerie MJCF, same 6-DoF action layout. - UR5eHAL / UR10eHAL (+ *_DESCRIPTION, *_with_sensors); real UR5eRealHAL / UR10eRealHAL via ros2_control + ur_robot_driver (URCap / RTDE). - FrankaPandaHAL / FRANKA_PANDA_DESCRIPTION / franka_panda_with_sensors; real FrankaPandaRealHAL over franka_ros2 / FCI (issue #56). - SawyerRealHAL / SAWYER_DESCRIPTION / SAWYER_REAL_DESCRIPTION over intera_sdk / sawyer_robot (issue #57). - AlohaHAL / ALOHA_DESCRIPTION / ALOHA_REAL_DESCRIPTION over the Trossen Interbotix XS SDK (issue #58); AlohaMujocoHAL MuJoCo twin, gym-aloha's bimanual_viperx_transfer_cube.xml, same 14-DoF action layout. - G1MujocoHAL / G1_DESCRIPTION: Unitree G1, 29-DoF (ADR-0087 glide default, optional ADR-0089 pretrained walking controller in sim); real-HW G1 planned M2 (CLAUDE.md §3 dual-system pattern). - H1MujocoHAL / H1_DESCRIPTION: Unitree H1, 19-DoF (predecessor to G1: 5-DoF/leg, 1-DoF torso, 4-DoF/arm); same validator scope as G1MujocoHAL; real-HW also waits on the M2 S0 cerebellum. - Rizon4MujocoHAL / RIZON4_DESCRIPTION: Flexiv Rizon 4, 7-DoF cobot, whole-body force sensitivity; structurally like the UR/Franka sim HALs. - OpenArmMujocoHAL / OPENARM_DESCRIPTION: Enactic OpenArm v2 bimanual, 2 x (7-DoF arm + 1 gripper) = 16-DoF; fresh HALBase subclass (bimanual doesn't fit MujocoArmHAL); native <position> actuators write target → ctrl directly; MJCF fetched lazily by openral_hal._openarm_v2_assets. - OpenArmRealHAL / OPENARM_REAL_DESCRIPTION: real adapter for the same arm; commands the four openarm_bringup ros2_control controllers (per-side arm + gripper) driving Damiao CAN FD motor buses from C++ at 400 Hz; connect() refuses a bus that is not up. - AnvilOpenArmV2MujocoHAL / ANVIL_OPENARM_V2_DESCRIPTION: MuJoCo twin for the Anvil OpenARM 2.0, same 16-DoF surface; differs in J1 clamp (+/-135 deg) and J6 radial deviation (-45..+70 deg) plus a visual-only wrist bracket; MJCF pinned at a SHA from bensonlee5/anvil-openarm-mujoco via openral_hal._anvil_openarm_v2_assets (openarm:anvil_v2_bimanual). - SimTransport: typed in-memory ros2_control transport for unit tests. - GalaxeaA1HAL / GALAXEA_A1_DESCRIPTION: real Galaxea A1 via an isolated ROS 1 Noetic sidecar (vendor SDK operator-provided).

Where a sim sibling exists, *_REAL_DESCRIPTION constants derive from openral_hal._real_description.make_real_description and share its HAL entrypoints. Real-only platforms (Galaxea A1) publish one description directly until a licensed digital-twin asset is available.

PANDA_MOBILE_BASE_JOINT_NAMES: list[str] = list(PANDA_MOBILE_DESCRIPTION.base_joints or []) module-attribute

Base joints (forward, side, yaw), in order — from robots/panda_mobile/robot.yaml base_joints.

PANDA_MOBILE_JOINT_NAMES: list[str] = [*PANDA_MOBILE_BASE_JOINT_NAMES, *_PANDA_MOBILE_ARM_JOINT_NAMES, _PANDA_MOBILE_GRIPPER_JOINT_NAME] module-attribute

Full 11-DoF joint order: base (3) + arm (7) + gripper (1), matching robots/panda_mobile/robot.yaml.

AlohaHAL(*, left_arm_controller: str = _DEFAULT_LEFT_ARM_CONTROLLER, right_arm_controller: str = _DEFAULT_RIGHT_ARM_CONTROLLER, left_gripper_controller: str = _DEFAULT_LEFT_GRIPPER_CONTROLLER, right_gripper_controller: str = _DEFAULT_RIGHT_GRIPPER_CONTROLLER, joint_state_topic: str = _DEFAULT_ALOHA_JOINT_STATE_TOPIC, arm_namespaces: Sequence[str] = _DEFAULT_ALOHA_ARM_NAMESPACES, publish_fn: _PublishFn | None = None, state_fn: _StateFn | None = None, staleness_limit_s: float | None = None, stop_timeout_s: float = 5.0, description: RobotDescription | None = None)

Bases: HALBase

HAL adapter for a physical Trossen ALOHA bimanual setup.

The 14-DoF action vector is split internally:

• indices 0:6 → left arm joint trajectory (left_arm controller)
• index 6 → left gripper position (left gripper controller)
• indices 7:13 → right arm joint trajectory (right_arm controller)
• index 13 → right gripper position (right gripper controller)

The split keeps each side addressable independently while openral upstream layers see one unified 14-DoF action. It does not currently reach a real ALOHA: see the module docstring and issue #250.

Parameters:

Name	Type	Description	Default
left_arm_controller	str	Topic prefix for the left ViperX. Defaults to "left_arm/arm_controller" — a ros2_control name no real ALOHA exposes (#250).	_DEFAULT_LEFT_ARM_CONTROLLER
right_arm_controller	str	same for the right arm.	_DEFAULT_RIGHT_ARM_CONTROLLER
left_gripper_controller	str	Topic prefix for the left gripper. Defaults to "left_arm/gripper_controller", which matches no ros2_control convention either (#250).	_DEFAULT_LEFT_GRIPPER_CONTROLLER
right_gripper_controller	str	same for the right gripper.	_DEFAULT_RIGHT_GRIPPER_CONTROLLER
joint_state_topic	str	ROS 2 topic publishing aggregated joint state. No real ALOHA publishes one — each arm publishes under its own namespace (#250).	_DEFAULT_ALOHA_JOINT_STATE_TOPIC
arm_namespaces	Sequence[str]	The xs_sdk robot namespaces of the two follower arms; estop() calls /<namespace>/torque_enable on each. Defaults to ("follower_left", "follower_right").	_DEFAULT_ALOHA_ARM_NAMESPACES
stop_timeout_s	float	How long estop waits for each torque-off call.	5.0
publish_fn	_PublishFn | None	Callable forwarding messages to ROS 2 topics. Production use injects the lifecycle node's publisher; tests inject SimTransport.publish.	None
state_fn	_StateFn | None	Callable returning the latest raw joint state as a dict. Production use injects the lifecycle node's subscriber callback; tests inject SimTransport.state.	None
staleness_limit_s	float | None	Maximum age of a read_state() reading before ROSPerceptionStale is raised. None (default) reads the manifest's safety.joint_state_staleness_limit_s.	None
description	RobotDescription | None	The loaded robots/<id>/robot.yaml manifest (threaded by build_hal). None falls back to the in-code ALOHA_REAL_DESCRIPTION mirror.	None

Lifecycle e-stop. estop() calls torque_enable(enable=false) for the all group of every namespace in arm_namespaces through the attached InterbotixStopSeam and records a DownstreamStopReport — stopped only when every arm acknowledged. Torque off means the ViperX arms go limp (Dynamixels have no brakes), so they will settle under gravity; that is the same outcome as the rig's hardware e-stop and is the only stop xs_sdk offers. Recovery is RESTART_REQUIRED: re-torque and re-home the arms with the Interbotix tooling, then restart the HAL lifecycle node.

Example

from openral_hal.aloha import AlohaHAL from openral_hal.sim_transport import SimTransport transport = SimTransport(n_joints=14) hal = AlohaHAL( ... publish_fn=transport.publish, ... state_fn=transport.state, ... ) hal.connect() hal.description.name 'aloha_bimanual' hal.arm_namespaces() ['follower_left', 'follower_right'] hal.disconnect()

Initialise the adapter; no transport is opened until connect().

Source code in python/hal/src/openral_hal/aloha.py

def __init__(
    self,
    *,
    left_arm_controller: str = _DEFAULT_LEFT_ARM_CONTROLLER,
    right_arm_controller: str = _DEFAULT_RIGHT_ARM_CONTROLLER,
    left_gripper_controller: str = _DEFAULT_LEFT_GRIPPER_CONTROLLER,
    right_gripper_controller: str = _DEFAULT_RIGHT_GRIPPER_CONTROLLER,
    joint_state_topic: str = _DEFAULT_ALOHA_JOINT_STATE_TOPIC,
    arm_namespaces: Sequence[str] = _DEFAULT_ALOHA_ARM_NAMESPACES,
    publish_fn: _PublishFn | None = None,
    state_fn: _StateFn | None = None,
    staleness_limit_s: float | None = None,
    stop_timeout_s: float = 5.0,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the adapter; no transport is opened until ``connect()``."""
    if not arm_namespaces:
        raise ROSConfigError("AlohaHAL needs at least one arm namespace to stop.")
    self.description: RobotDescription = description or ALOHA_REAL_DESCRIPTION
    self._left_arm_controller = left_arm_controller
    self._right_arm_controller = right_arm_controller
    self._left_gripper_controller = left_gripper_controller
    self._right_gripper_controller = right_gripper_controller
    self._joint_state_topic = joint_state_topic
    self._arm_namespaces = [str(ns) for ns in arm_namespaces]
    self._publish_fn: _PublishFn = publish_fn or _default_publish
    self._state_fn: _StateFn | None = state_fn
    self._staleness_limit_s = resolve_staleness_limit_s(self.description, staleness_limit_s)
    self._stop_timeout_s = stop_timeout_s
    self._torque_seam: InterbotixStopSeam | None = None
    self._last_stop_report: DownstreamStopReport | None = None

    self._connected: bool = False
    self._last_state_time: float = 0.0
    self._joint_names: list[str] = list(_ALOHA_JOINT_NAMES)

last_stop_report: DownstreamStopReport | None property

Report of the most recent estop(); None before it.

arm_namespaces() -> list[str]

The xs_sdk namespaces estop cuts torque on.

Source code in python/hal/src/openral_hal/aloha.py

def arm_namespaces(self) -> list[str]:
    """The ``xs_sdk`` namespaces ``estop`` cuts torque on."""
    return list(self._arm_namespaces)

attach_torque_stop(seam: InterbotixStopSeam) -> None

Bind the live torque seam (the lifecycle node under hal_mode:=real).

Raises:

Type	Description
ROSConfigError	If seam does not satisfy InterbotixStopSeam.

Source code in python/hal/src/openral_hal/aloha.py

def attach_torque_stop(self, seam: InterbotixStopSeam) -> None:
    """Bind the live torque seam (the lifecycle node under ``hal_mode:=real``).

    Raises:
        ROSConfigError: If ``seam`` does not satisfy ``InterbotixStopSeam``.
    """
    if not isinstance(seam, InterbotixStopSeam):
        raise ROSConfigError(
            f"attach_torque_stop() needs an InterbotixStopSeam (torque_enable); "
            f"got {type(seam).__name__}."
        )
    self._torque_seam = seam

connect() -> None

Open the transport to the four configured command topics (#250).

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/aloha.py

def connect(self) -> None:
    """Open the transport to the four configured command topics (#250).

    Raises:
        ROSRuntimeError: If already connected.
    """
    if self._connected:
        raise ROSRuntimeError("AlohaHAL is already connected.")
    log.info(
        "hal.connect",
        robot=self.description.name,
        left_arm=self._left_arm_controller,
        right_arm=self._right_arm_controller,
    )
    self._connected = True
    self._last_state_time = time.monotonic()

estop() -> None

Trigger an emergency stop: drop the connection, cut torque on every arm, raise.

The connection flag drops first so nothing more leaves this HAL, then torque_enable(enable=false) is called for the all group of every arm namespace. Every arm is attempted even if an earlier one failed, and the report names each acknowledgement. With no seam attached the report says so and stopped is False — a local latch is never reported as a downstream stop.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/aloha.py

def estop(self) -> None:
    """Trigger an emergency stop: drop the connection, cut torque on every arm, raise.

    The connection flag drops first so nothing more leaves this HAL, then
    ``torque_enable(enable=false)`` is called for the ``all`` group of
    every arm namespace. Every arm is attempted even if an earlier one
    failed, and the report names each acknowledgement. With no seam
    attached the report says so and ``stopped`` is ``False`` — a local
    latch is never reported as a downstream stop.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical("hal.estop", robot=self.description.name, arms=self._arm_namespaces)
    self._connected = False
    report = self._cut_torque()
    self._last_stop_report = report
    verdict = "acknowledged" if report.stopped else "NOT acknowledged"
    raise ROSEStopRequested(
        f"Emergency stop triggered on ALOHA bimanual ('{self.description.name}') "
        f"(downstream stop {verdict}: {report.detail})."
    )

read_state() -> JointState

Return the latest joint state for all 14 description joints.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last reading is older than staleness_limit_s.

Source code in python/hal/src/openral_hal/aloha.py

def read_state(self) -> JointState:
    """Return the latest joint state for all 14 description joints.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last reading is older than
            ``staleness_limit_s``.
    """
    self._require_connected("read_state")
    age = time.monotonic() - self._last_state_time
    if age > self._staleness_limit_s:
        raise ROSPerceptionStale(
            f"Joint state is {age:.3f} s old (limit {self._staleness_limit_s} s)."
        )

    n = len(self._joint_names)
    raw: dict[str, object] = {} if self._state_fn is None else self._state_fn()

    return JointState(
        name=list(self._joint_names),
        position=_raw_floats(raw, "position", n),
        velocity=_raw_floats(raw, "velocity", n),
        effort=_raw_floats(raw, "effort", n),
        stamp_ns=int(time.time_ns()),
    )

send_action(action: Action) -> None

Forward an action chunk to the four Interbotix controllers.

The 14-D action is split four ways: each arm gets a 6-D joint trajectory and each gripper gets a 1-D position command.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If action.control_mode is not JOINT_POSITION or the joint count is wrong.

Source code in python/hal/src/openral_hal/aloha.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the four Interbotix controllers.

    The 14-D action is split four ways: each arm gets a 6-D joint
    trajectory and each gripper gets a 1-D position command.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If ``action.control_mode`` is not
            ``JOINT_POSITION`` or the joint count is wrong.
    """
    self._require_connected("send_action")
    if action.control_mode != ControlMode.JOINT_POSITION:
        raise ROSConfigError(
            f"AlohaHAL only supports JOINT_POSITION; got {action.control_mode!r}."
        )
    n = len(self._joint_names)
    if not action.joint_targets:
        raise ROSConfigError("AlohaHAL.send_action requires non-empty joint_targets.")
    for step_idx, step in enumerate(action.joint_targets):
        if len(step) != n:
            raise ROSConfigError(
                f"action.joint_targets[{step_idx}] has {len(step)} values "
                f"but ALOHA exposes {n} joints."
            )

    # Index layout — see class docstring.
    left_arm_chunk = [step[0:6] for step in action.joint_targets]
    right_arm_chunk = [step[7:13] for step in action.joint_targets]
    left_gripper_targets = [step[6] for step in action.joint_targets]
    right_gripper_targets = [step[13] for step in action.joint_targets]

    self._publish_fn(
        f"/{self._left_arm_controller}/joint_trajectory",
        {
            "control_mode": action.control_mode,
            "horizon": action.horizon,
            "joint_targets": left_arm_chunk,
            "stamp_ns": action.stamp_ns,
        },
    )
    self._publish_fn(
        f"/{self._right_arm_controller}/joint_trajectory",
        {
            "control_mode": action.control_mode,
            "horizon": action.horizon,
            "joint_targets": right_arm_chunk,
            "stamp_ns": action.stamp_ns,
        },
    )
    self._publish_fn(
        f"/{self._left_gripper_controller}/command",
        {"position": left_gripper_targets[-1], "stamp_ns": action.stamp_ns},
    )
    self._publish_fn(
        f"/{self._right_gripper_controller}/command",
        {"position": right_gripper_targets[-1], "stamp_ns": action.stamp_ns},
    )
    log.debug(
        "hal.send_action",
        robot=self.description.name,
        horizon=action.horizon,
    )

AlohaMujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Trossen ALOHA bimanual setup (MuJoCo digital twin).

Thin manifest-driven wrapper around MujocoArmHAL; all wiring (MJCF URI, joint→qpos/actuator maps, two NORMALISED grippers with mirror_actuator_index for the antisymmetric finger pair, keyframe seeding) lives in ALOHA_DESCRIPTION.sim.

Public surface mirrors AlohaHAL: a 14-DoF openral_core.Action with the left arm 6 + left gripper 1 + right arm 6 + right gripper 1 layout. Gripper values are normalized [0, 1] (0 closed, 1 open), mapped affinely onto the finger range [0.021, 0.057] m and read back the same way.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file. When None, the file is resolved through the gym_aloha: URI scheme from ALOHA_DESCRIPTION.assets.mjcf.	None
settle_steps	int | None	Number of MuJoCo physics steps per send_action call. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import AlohaMujocoHAL # doctest: +SKIP hal = AlohaMujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 14 joints # doctest: +SKIP 14 hal.disconnect() # doctest: +SKIP

Initialise the adapter; the MJCF is not loaded until connect().

Source code in python/hal/src/openral_hal/aloha.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the adapter; the MJCF is not loaded until ``connect()``."""
    self._init_from_description(
        ALOHA_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

AnvilOpenArmV2MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Anvil OpenARM 2.0 (MuJoCo digital twin).

Thin manifest-driven wrapper around MujocoArmHAL, exactly like the Enactic v2 adapter: the MJCF ships native <position> actuators with per-class PD gains, so all wiring (the openarm:anvil_v2_bimanual MJCF ref, the joint→qpos map that skips the follower fingers, two PASSTHROUGH hinge grippers, seed_ctrl_from_qpos) lives in ANVIL_OPENARM_V2_DESCRIPTION.sim (ADR-0023).

What makes it the Anvil 2.0 rather than the stock v2 arm is entirely in the fetched MJCF: J1 clamped to +/-135 deg, J6 widened to -45..+70 deg of radial deviation, and the wrist support bracket (visual-only) that enables it — see openral_hal._anvil_openarm_v2_assets.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the Anvil 2.0 bimanual MJCF is fetched lazily through ensure_anvil_openarm_v2_mjcf via the openarm:anvil_v2_bimanual URI scheme.	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import AnvilOpenArmV2MujocoHAL # doctest: +SKIP hal = AnvilOpenArmV2MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 + 1 + 7 + 1 # doctest: +SKIP 16 hal.disconnect() # doctest: +SKIP

Initialise the adapter; the MJCF is not loaded until connect().

Source code in python/hal/src/openral_hal/anvil_openarm_v2.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the adapter; the MJCF is not loaded until ``connect()``."""
    self._init_from_description(
        ANVIL_OPENARM_V2_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

ControllerKind

Bases: StrEnum

Which ros2_control controller sits behind one command topic.

A command topic is not self-describing: the topic name says nothing about the message type the controller expects, and publishing the wrong type is silent — DDS simply never delivers it, so the arm does not move and nothing anywhere logs an error. That is the same class of failure as the no-op transport this layer already had to fix once, so the wire format is declared rather than assumed.

A HAL names the kind per topic in command_bindings; RosControlTransport builds the matching publisher and message from it. Adding a robot whose controllers differ therefore means declaring a kind, not writing transport code.

Only kinds this transport can actually build a message for belong here — an unlisted controller must fail loudly at wire-up rather than be approximated by a neighbouring one.

ControllerStopSeam

Bases: Protocol

What a transport must expose for a ros2_control HAL to stop its controllers.

The production implementation is RosControlTransport (real controller_manager services on a live graph); the in-memory SimTransport implements the same seam over a simulated controller table so the unit lane exercises the identical HAL code path. A HAL never sees rclpy: it asks the seam to deactivate / activate controllers by name and to fire the vendor-specific stop (std_srvs/Trigger service or std_msgs/Empty topic) it declared, and gets back an acknowledgement it can put in its DownstreamStopReport.

activate_controllers(names: Sequence[str], *, timeout_s: float) -> ControllerSwitchReport

Activate every named controller and confirm each reads active.

Source code in python/hal/src/openral_hal/ros_control.py

def activate_controllers(
    self, names: Sequence[str], *, timeout_s: float
) -> ControllerSwitchReport:
    """Activate every named controller and confirm each reads ``active``."""
    ...

call_trigger(service: str, *, timeout_s: float) -> TriggerReport

Call one std_srvs/Trigger service the HAL declared in vendor_stop_services.

Source code in python/hal/src/openral_hal/ros_control.py

def call_trigger(self, service: str, *, timeout_s: float) -> TriggerReport:
    """Call one ``std_srvs/Trigger`` service the HAL declared in ``vendor_stop_services``."""
    ...

deactivate_controllers(names: Sequence[str], *, timeout_s: float) -> ControllerSwitchReport

Deactivate every named controller and confirm each reads inactive.

Source code in python/hal/src/openral_hal/ros_control.py

def deactivate_controllers(
    self, names: Sequence[str], *, timeout_s: float
) -> ControllerSwitchReport:
    """Deactivate every named controller and confirm each reads ``inactive``."""
    ...

publish_empty(topic: str) -> None

Publish one std_msgs/Empty on a topic the HAL declared in vendor_stop_topics.

Source code in python/hal/src/openral_hal/ros_control.py

def publish_empty(self, topic: str) -> None:
    """Publish one ``std_msgs/Empty`` on a topic the HAL declared in ``vendor_stop_topics``."""
    ...

ControllerStoppable

Bases: Protocol

The stop-side counterpart of RosControlDrivable.

The lifecycle node attaches a ControllerStopSeam to any real-mode HAL that answers these, and refuses to configure a RosControlDrivable HAL that does not — a robot that can be driven but cannot be stopped from /openral/estop must not come up. RosControlHAL answers all four, so every subclass is covered; the members exist so the fleet conformance test (tests/unit/test_real_hal_estop_fleet_conformance.py) can pin the contract structurally.

attach_controller_stop(seam: ControllerStopSeam) -> None

Bind the live stop seam after construction.

Source code in python/hal/src/openral_hal/ros_control.py

def attach_controller_stop(self, seam: ControllerStopSeam) -> None:
    """Bind the live stop seam after construction."""
    ...

controller_names() -> list[str]

Every controller_manager controller this HAL commands.

Source code in python/hal/src/openral_hal/ros_control.py

def controller_names(self) -> list[str]:
    """Every ``controller_manager`` controller this HAL commands."""
    ...

vendor_stop_services() -> list[str]

std_srvs/Trigger services the vendor stop calls (may be empty).

Source code in python/hal/src/openral_hal/ros_control.py

def vendor_stop_services(self) -> list[str]:
    """``std_srvs/Trigger`` services the vendor stop calls (may be empty)."""
    ...

vendor_stop_topics() -> list[str]

std_msgs/Empty topics the vendor stop publishes on (may be empty).

Source code in python/hal/src/openral_hal/ros_control.py

def vendor_stop_topics(self) -> list[str]:
    """``std_msgs/Empty`` topics the vendor stop publishes on (may be empty)."""
    ...

ControllerSwitchReport(ok: bool, controllers: tuple[str, ...], states: Mapping[str, str] = (lambda: MappingProxyType({}))(), detail: str = '') dataclass

Acknowledged outcome of one controller_manager switch.

ok is the switch service's own verdict and the post-switch confirmation: a transport reports ok=True only when every controller in controllers was listed in the requested state afterwards. states carries what list_controllers (or the in-memory simulator) reported per controller so a refusal names the controller that did not move.

DownstreamStopReport(stopped: bool, controllers: tuple[str, ...], controller_states: Mapping[str, str] = (lambda: MappingProxyType({}))(), vendor_stop: str = '', detail: str = '') dataclass

What a HAL knows about its last downstream stop.

Produced by RosControlHAL.estop (and the Interbotix-backed AlohaHAL.estop) and read by the lifecycle node, which logs FATAL and publishes downstream_stop=unacknowledged on /diagnostics when stopped is False. stopped is only True when every step — controller deactivation and the vendor stop, where one exists — was acknowledged by the thing that executed it; a local latch alone never counts.

fields() -> dict[str, str]

Flatten for the /diagnostics heartbeat (string values only).

Source code in python/hal/src/openral_hal/ros_control.py

def fields(self) -> dict[str, str]:
    """Flatten for the ``/diagnostics`` heartbeat (string values only)."""
    return {
        "downstream_stop": "acknowledged" if self.stopped else "unacknowledged",
        "downstream_controllers": ",".join(self.controllers),
        "downstream_controller_states": ",".join(
            f"{name}={state}" for name, state in self.controller_states.items()
        ),
        "downstream_vendor_stop": self.vendor_stop or "none",
        "downstream_stop_detail": self.detail,
    }

DownstreamStopReporting

Bases: Protocol

HALs that can say whether their last downstream stop was acknowledged.

last_stop_report: DownstreamStopReport | None property

Report of the most recent estop(); None before the first / after reset.

EStopRecovery

Bases: StrEnum

Recovery policy after a lifecycle node forwards an e-stop to a HAL.

FrankaPandaHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Franka Emika Panda (MuJoCo-backed simulation).

The HAL exposes 8 joints to upper layers — the 7 arm joints plus a synthetic panda_gripper channel reported in [0, 1] (0 = closed, 1 = fully open). Internally the gripper command is mapped to the MJCF tendon actuator's [0, 255] range, and the reported gripper position is the sum of the two finger qpos values normalised by 0.08 m.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/franka_emika_panda/panda.xml).	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import FrankaPandaHAL # doctest: +SKIP hal = FrankaPandaHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 arm + 1 gripper # doctest: +SKIP 8 hal.disconnect() # doctest: +SKIP

Initialise the Panda HAL; no MuJoCo state is created until connect().

All wiring (MJCF URI, joint indices, gripper config) lives in FRANKA_PANDA_DESCRIPTION.sim.

Source code in python/hal/src/openral_hal/franka_panda.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the Panda HAL; no MuJoCo state is created until ``connect()``.

    All wiring (MJCF URI, joint indices, gripper config) lives in
    ``FRANKA_PANDA_DESCRIPTION.sim``.
    """
    self._init_from_description(
        FRANKA_PANDA_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

FrankaPandaRealHAL(*, fci_ip: str = '172.16.0.2', controller_name: str = _DEFAULT_FRANKA_CONTROLLER, joint_state_topic: str = _DEFAULT_FRANKA_JOINT_STATE_TOPIC, command_topic: str | None = None, error_recovery_action: str = _DEFAULT_FRANKA_ERROR_RECOVERY, publish_fn: _PublishFn | None = None, state_fn: _StateFn | None = None, staleness_limit_s: float | None = None, description: RobotDescription | None = None)

Bases: RosControlHAL

HAL adapter for a physical Franka Emika Panda over the FCI.

A RosControlHAL subclass (the same shape as the UR adapters) that pins the Franka-specific configuration: the FCI hostname, the franka_ros2 controller name, and the error-recovery action the operator runs after an e-stop. Subclassing rather than wrapping is what makes this adapter structurally RosControlDrivable, so the lifecycle node attaches the production RosControlTransport to it under hal_mode:=real — a composed wrapper exposed none of that surface and was never wired, so a real Panda deploy published into a no-op logger.

Lifecycle e-stop. The stop is the generic RosControlHAL one: deactivating franka_arm_controller through controller_manager makes franka_hardware's on_deactivate call libfranka's stopRobot(), so the FCI control loop ends and the arm holds. There is no separate vendor stop service, and the error-recovery action is deliberately not invoked here — it clears a reflex, which is a recovery step, not a stop. Recovery is RESTART_REQUIRED: the operator releases the Franka user stop, runs error_recovery_action (franka_msgs/action/ErrorRecovery), restarts the HAL lifecycle node and re-aligns.

Parameters:

Name	Type	Description	Default
fci_ip	str	Hostname or IP of the Franka FCI (the robot's "control" port, typically 172.16.0.2 for the default lab subnet). Required; libfranka will refuse to connect without it. Stored as metadata only — the actual TCP connection is opened by franka_hardware inside the lifecycle node.	'172.16.0.2'
controller_name	str	Name of the ros2_control joint trajectory controller exposed by franka_ros2. Defaults to "franka_arm_controller".	_DEFAULT_FRANKA_CONTROLLER
joint_state_topic	str	ROS 2 topic publishing sensor_msgs/JointState. Defaults to "/joint_states".	_DEFAULT_FRANKA_JOINT_STATE_TOPIC
command_topic	str | None	ROS 2 topic for joint trajectory commands. Defaults to "/<controller_name>/joint_trajectory" (set by RosControlHAL).	None
error_recovery_action	str	Name of franka_ros2's error-recovery action the operator runs before re-arming after an e-stop. Recorded for diagnostics and bring-up docs; never called by estop(). Defaults to "/error_recovery".	_DEFAULT_FRANKA_ERROR_RECOVERY
publish_fn	_PublishFn | None	Callable forwarding messages to ROS 2 topics. Production use injects the lifecycle node's publisher; tests inject SimTransport.publish.	None
state_fn	_StateFn | None	Callable returning the latest raw joint state as a dict. Production use injects the lifecycle node's subscriber callback; tests inject SimTransport.state.	None
staleness_limit_s	float | None	Maximum age of a read_state() reading before ROSPerceptionStale is raised. None (default) reads the manifest's safety.joint_state_staleness_limit_s.	None
description	RobotDescription | None	The loaded robots/<id>/robot.yaml manifest (threaded by build_hal). None falls back to the in-code FRANKA_PANDA_REAL_DESCRIPTION mirror.	None

Raises:

Type	Description
ROSConfigError	If fci_ip is empty / whitespace.

Example

from openral_hal.franka_panda_real import FrankaPandaRealHAL from openral_hal.sim_transport import SimTransport transport = SimTransport(n_joints=8) hal = FrankaPandaRealHAL( ... fci_ip="172.16.0.2", ... publish_fn=transport.publish, ... state_fn=transport.state, ... ) hal.connect() hal.description.name 'franka_panda' hal.disconnect()

Initialise the adapter; no TCP connection is opened until connect().

Source code in python/hal/src/openral_hal/franka_panda_real.py

def __init__(
    self,
    *,
    fci_ip: str = "172.16.0.2",
    controller_name: str = _DEFAULT_FRANKA_CONTROLLER,
    joint_state_topic: str = _DEFAULT_FRANKA_JOINT_STATE_TOPIC,
    command_topic: str | None = None,
    error_recovery_action: str = _DEFAULT_FRANKA_ERROR_RECOVERY,
    publish_fn: _PublishFn | None = None,
    state_fn: _StateFn | None = None,
    staleness_limit_s: float | None = None,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the adapter; no TCP connection is opened until ``connect()``."""
    if not fci_ip or not fci_ip.strip():
        raise ROSConfigError(
            "FrankaPandaRealHAL requires a non-empty fci_ip "
            "(the robot's FCI hostname or IP, e.g. '172.16.0.2')."
        )
    super().__init__(
        description or FRANKA_PANDA_REAL_DESCRIPTION,
        controller_name=controller_name,
        joint_state_topic=joint_state_topic,
        command_topic=command_topic,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )
    self._fci_ip = fci_ip
    self._error_recovery_action = error_recovery_action

error_recovery_action: str property

franka_ros2 error-recovery action the operator runs before re-arming.

fci_ip: str property

Hostname / IP of the FCI; consumed by franka_hardware.

connect() -> None

Open the ROS 2 transport to the franka_ros2 controller.

The actual libfranka TCP socket is owned by franka_hardware inside the lifecycle node; this call only marks the adapter live on the injected / attached transport.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/franka_panda_real.py

def connect(self) -> None:
    """Open the ROS 2 transport to the ``franka_ros2`` controller.

    The actual ``libfranka`` TCP socket is owned by ``franka_hardware``
    inside the lifecycle node; this call only marks the adapter live on
    the injected / attached transport.

    Raises:
        ROSRuntimeError: If already connected.
    """
    log.info(
        "hal.connect",
        robot=self.description.name,
        fci_ip=self._fci_ip,
        controller=self._controller_name,
    )
    super().connect()

G1MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5, body_twist_dt_s: float = 0.05, walking_enabled: bool = False, description: RobotDescription | None = None)

Bases: MujocoArmHAL

HAL adapter for the Unitree G1 humanoid (MuJoCo digital twin).

Drives the 29 actuated joints of the menagerie unitree_g1 MJCF through MuJoCo's position-controlled actuators. Exposes a 29-D openral_core.Action matching the joint order in G1_DESCRIPTION (left leg 6 → right leg 6 → waist 3 → left arm 7 → right arm 7).

.. warning::

The default mode does not provide balance; it uses ADR-0087's upright kinematic pin. walking_enabled=True is a sim-only learned controller, not the production C++ S0 controller required for hardware.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/unitree_g1/g1.xml).	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time — required for the contract-validation tests because the floating base falls otherwise.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import G1MujocoHAL # doctest: +SKIP hal = G1MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 29 actuated joints # doctest: +SKIP 29 hal.disconnect() # doctest: +SKIP

Initialise the G1 HAL; no MuJoCo state is created until connect().

All wiring (MJCF URI, floating-base offsets) lives in G1_DESCRIPTION.sim.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path.	None
settle_steps	int | None	MuJoCo physics steps per send_action.	None
gravity_enabled	bool	When False, gravity is zeroed at connect.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5
body_twist_dt_s	float	Logical control timestep one BODY_TWIST action integrates the glide base by (matches SimAttachedHAL / PandaMobileHAL).	0.05
walking_enabled	bool	Use the pinned MuJoCo Playground ONNX walking controller instead of the kinematic glide.	False
description	RobotDescription | None	The loaded robots/g1/robot.yaml manifest (threaded by build_hal); None falls back to the in-code G1_DESCRIPTION mirror.	None

Source code in python/hal/src/openral_hal/g1.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
    body_twist_dt_s: float = 0.05,
    walking_enabled: bool = False,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the G1 HAL; no MuJoCo state is created until ``connect()``.

    All wiring (MJCF URI, floating-base offsets) lives in
    ``G1_DESCRIPTION.sim``.

    Args:
        mjcf_path: Optional override for the MJCF file path.
        settle_steps: MuJoCo physics steps per ``send_action``.
        gravity_enabled: When ``False``, gravity is zeroed at connect.
        staleness_limit_s: Maximum age of a cached state.
        body_twist_dt_s: Logical control timestep one BODY_TWIST
            action integrates the glide base by (matches
            ``SimAttachedHAL`` / ``PandaMobileHAL``).
        walking_enabled: Use the pinned MuJoCo Playground ONNX walking
            controller instead of the kinematic glide.
        description: The loaded ``robots/g1/robot.yaml`` manifest
            (threaded by ``build_hal``); ``None`` falls back to the
            in-code ``G1_DESCRIPTION`` mirror.
    """
    walking_policy_path: str | None = None
    if walking_enabled:
        if not gravity_enabled:
            raise ROSConfigError("G1 walking requires gravity_enabled=True.")
        if mjcf_path is not None:
            raise ROSConfigError(
                "G1 walking uses its pinned policy-tuned MJCF; mjcf_path cannot override it."
            )
        mjcf_path, walking_policy_path = ensure_g1_walking_assets()
    self._init_from_description(
        description or G1_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )
    if body_twist_dt_s <= 0.0:
        raise ROSConfigError("G1MujocoHAL body_twist_dt_s must be > 0.")
    self._body_twist_dt_s = body_twist_dt_s
    self._walking_policy_path = walking_policy_path
    self._walking_controller: G1WalkingController | None = None
    # Kinematic-glide base pin (ADR-0087): ``(x, y, yaw, z0)`` world pose
    # the free joint is re-pinned to every physics step.  Captured lazily
    # from the MJCF default qpos on first use after connect.
    self._base_pin: tuple[float, float, float, float] | None = None
    # Last commanded base twist, ``base_link`` frame; zeroed by any
    # non-BODY_TWIST action (odom publisher contract, like PandaMobileHAL).
    self._last_body_twist: tuple[float, float, float, float, float, float] = (0.0,) * 6

base_pose: tuple[float, float, float] property

Current base (x, y, yaw) for tests / odom publishers.

base_twist: tuple[float, float, float, float, float, float] property

Last commanded base body twist (base_link frame); zeroed on non-twist actions.

connect() -> None

Connect and re-arm the base pin (recaptured from the fresh qpos).

Source code in python/hal/src/openral_hal/g1.py

def connect(self) -> None:
    """Connect and re-arm the base pin (recaptured from the fresh qpos)."""
    super().connect()
    self._base_pin = None
    if self._walking_policy_path is not None:
        try:
            import mujoco as mj  # noqa: PLC0415  # reason: optional sim-only dep

            assert self._model is not None and self._data is not None
            keyframe_id = mj.mj_name2id(self._model, mj.mjtObj.mjOBJ_KEY, "knees_bent")
            if keyframe_id < 0:
                raise ROSConfigError("G1 walking MJCF is missing the 'knees_bent' keyframe.")
            mj.mj_resetDataKeyframe(self._model, self._data, keyframe_id)
            mj.mj_forward(self._model, self._data)
            self._walking_controller = G1WalkingController(self._walking_policy_path)
            self._walking_controller.initialize(self._data)
        except ROSConfigError:
            super().disconnect()
            raise

disconnect() -> None

Release the walking controller and MuJoCo model.

Source code in python/hal/src/openral_hal/g1.py

def disconnect(self) -> None:
    """Release the walking controller and MuJoCo model."""
    self._walking_controller = None
    super().disconnect()

idle_step(wall_dt_s: float | None = None) -> bool

HOLD-step with the glide base pinned (it would free-fall otherwise).

Source code in python/hal/src/openral_hal/g1.py

def idle_step(self, wall_dt_s: float | None = None) -> bool:
    """HOLD-step with the glide base pinned (it would free-fall otherwise)."""
    if self._connected and self._data is not None:
        self._last_body_twist = (0.0,) * 6
        if self._walking_controller is not None:
            # ponytail: pin only while idle; replace with a zero-command
            # standing policy when one is published and validated.
            self._base_pin = None
            self._walking_controller.reset_state()
        self._pin_base()
    return super().idle_step(wall_dt_s=wall_dt_s)

reset_to_pose(pose: list[float]) -> None

Reset the joint pose and walking policy state.

Source code in python/hal/src/openral_hal/g1.py

def reset_to_pose(self, pose: list[float]) -> None:
    """Reset the joint pose and walking policy state."""
    super().reset_to_pose(pose)
    self._base_pin = None
    if self._walking_controller is not None:
        self._walking_controller.reset_state()

send_action(action: Action) -> None

Route BODY_TWIST to the glide base; everything else to the arm path.

A BODY_TWIST action Euler-integrates the planar twist (vx, vy, wz) into the pinned free-joint pose (body frame rotated into world by the current yaw, one body_twist_dt_s per action — the PandaMobileHAL convention), then advances physics settle_steps ticks with the base pinned. The 29 joint actuators keep holding their last targets, so navigation composes with a held upper-body pose.

Source code in python/hal/src/openral_hal/g1.py

def send_action(self, action: Action) -> None:
    """Route BODY_TWIST to the glide base; everything else to the arm path.

    A BODY_TWIST action Euler-integrates the planar twist ``(vx, vy, wz)``
    into the pinned free-joint pose (body frame rotated into world by the
    current yaw, one ``body_twist_dt_s`` per action — the
    ``PandaMobileHAL`` convention), then advances physics ``settle_steps``
    ticks with the base pinned.  The 29 joint actuators keep holding their
    last targets, so navigation composes with a held upper-body pose.
    """
    if action.control_mode != ControlMode.BODY_TWIST.value:
        self._last_body_twist = (0.0,) * 6
        super().send_action(action)
        return

    self._require_connected("send_action")
    self._validate_action(action)
    import mujoco as mj  # noqa: PLC0415  # reason: optional sim-only dep

    assert self._data is not None and self._model is not None
    vx, vy, _vz, _wx, _wy, wz = (float(v) for v in action.body_twist[0])  # type: ignore[index]  # reason: _validate_action guarantees body_twist
    self._last_body_twist = (vx, vy, 0.0, 0.0, 0.0, wz)
    if self._walking_controller is not None:
        self._walk((vx, vy, wz))
        return
    x, y, yaw, z0 = self._pin()
    cy, sy = math.cos(yaw), math.sin(yaw)
    x += (cy * vx - sy * vy) * self._body_twist_dt_s
    y += (sy * vx + cy * vy) * self._body_twist_dt_s
    yaw += wz * self._body_twist_dt_s
    yaw = (yaw + math.pi) % (2.0 * math.pi) - math.pi
    self._base_pin = (x, y, yaw, z0)

    for _ in range(self._settle_steps):
        self._pin_base()
        mj.mj_step(self._model, self._data)
    self._last_state_time = time.monotonic()
    self._last_action_ns = time.monotonic_ns()
    log.debug(
        "hal.send_action",
        robot=self.description.name,
        control_mode=action.control_mode,
        body_twist=self._last_body_twist,
    )

GalaxeaA1HAL(*, host: str = '127.0.0.1', port: int = 46011, state_timeout_s: float = 0.5, status_timeout_s: float = 1.0, feedback_limit_tolerance_rad: float = 0.01, initial_alignment_tolerance_rad: float = 0.05, tracker_alignment_timeout_s: float = 5.0, max_target_step_rad: float = 0.08, command_lease_s: float = 0.5, idle_timeout_error_mask: int = 64, gripper_ignored_error_mask: int = 8, gripper_stroke_min_mm: float = 0.0, gripper_stroke_max_mm: float = 104.0, connect_timeout_s: float = 10.0, transport: _A1Transport | None = None, description: RobotDescription | None = None)

Bases: HALBase

OpenRAL real HAL for the six-axis Galaxea A1 plus parallel gripper.

Example

from openral_hal.galaxea_a1 import GalaxeaA1HAL from openral_core import Action, ControlMode hal = GalaxeaA1HAL(port=46011) # doctest: +SKIP hal.connect() # doctest: +SKIP hal.send_action( # doctest: +SKIP ... Action(control_mode=ControlMode.GRIPPER_POSITION, gripper=[0.5], stamp_ns=0) ... ) hal.disconnect() # doctest: +SKIP

Configure the sidecar endpoint and explicit safety deadlines.

description is the loaded manifest (threaded by build_hal); None keeps the in-code GALAXEA_A1_DESCRIPTION mirror. The sidecar handshake and every command check use the adapter's own arm limits, so a manifest whose arm joints or position limits differ is refused rather than silently overridden.

Raises:

Type	Description
ROSConfigError	description disagrees with the adapter's arm joint names or position limits.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def __init__(
    self,
    *,
    host: str = "127.0.0.1",
    port: int = 46011,
    state_timeout_s: float = 0.5,
    status_timeout_s: float = 1.0,
    feedback_limit_tolerance_rad: float = 0.01,
    initial_alignment_tolerance_rad: float = 0.05,
    tracker_alignment_timeout_s: float = 5.0,
    max_target_step_rad: float = 0.08,
    command_lease_s: float = 0.5,
    idle_timeout_error_mask: int = 64,
    gripper_ignored_error_mask: int = 8,
    gripper_stroke_min_mm: float = 0.0,
    gripper_stroke_max_mm: float = 104.0,
    connect_timeout_s: float = 10.0,
    transport: _A1Transport | None = None,
    description: RobotDescription | None = None,
) -> None:
    """Configure the sidecar endpoint and explicit safety deadlines.

    ``description`` is the loaded manifest (threaded by ``build_hal``);
    ``None`` keeps the in-code ``GALAXEA_A1_DESCRIPTION`` mirror. The sidecar
    handshake and every command check use the adapter's own arm limits, so a
    manifest whose arm joints or position limits differ is refused rather
    than silently overridden.

    Raises:
        ROSConfigError: ``description`` disagrees with the adapter's arm
            joint names or position limits.
    """
    if description is not None:
        declared = {j.name: j.position_limits for j in description.joints}
        for name, (lower, upper) in zip(_JOINT_NAMES, _JOINT_LIMITS, strict=True):
            got = declared.get(name)
            if got is None or not (
                math.isclose(got[0], lower, abs_tol=1e-6)
                and math.isclose(got[1], upper, abs_tol=1e-6)
            ):
                raise ROSConfigError(
                    f"GalaxeaA1HAL enforces {name} limits {(lower, upper)}, but the "
                    f"manifest declares {got}; this adapter cannot honour different "
                    "arm limits — fix the manifest or the adapter."
                )
        self.description = description
    if not isinstance(host, str):
        raise ROSConfigError("GalaxeaA1HAL requires a literal IPv4 loopback address.")
    try:
        sidecar_address = ipaddress.ip_address(host)
    except ValueError as exc:
        raise ROSConfigError("GalaxeaA1HAL requires a literal IPv4 loopback address.") from exc
    if (
        not isinstance(sidecar_address, ipaddress.IPv4Address)
        or not sidecar_address.is_loopback
    ):
        raise ROSConfigError("GalaxeaA1HAL requires a literal IPv4 loopback address.")
    if isinstance(port, bool) or not isinstance(port, int) or not 0 < port < _MAX_TCP_PORT:
        raise ROSConfigError("GalaxeaA1HAL requires a TCP port in [1, 65535].")
    positive = {
        "state_timeout_s": state_timeout_s,
        "status_timeout_s": status_timeout_s,
        "initial_alignment_tolerance_rad": initial_alignment_tolerance_rad,
        "tracker_alignment_timeout_s": tracker_alignment_timeout_s,
        "max_target_step_rad": max_target_step_rad,
        "command_lease_s": command_lease_s,
        "connect_timeout_s": connect_timeout_s,
    }
    if any(not math.isfinite(v) or v <= 0.0 for v in positive.values()):
        raise ROSConfigError(
            f"GalaxeaA1HAL timing and motion limits must be positive: {positive}"
        )
    if not math.isfinite(feedback_limit_tolerance_rad) or feedback_limit_tolerance_rad < 0.0:
        raise ROSConfigError("feedback_limit_tolerance_rad must be finite and non-negative.")
    if (
        not math.isfinite(gripper_stroke_min_mm)
        or not math.isfinite(gripper_stroke_max_mm)
        or gripper_stroke_max_mm <= gripper_stroke_min_mm
    ):
        raise ROSConfigError("gripper_stroke_max_mm must exceed gripper_stroke_min_mm.")
    masks = {
        "idle_timeout_error_mask": idle_timeout_error_mask,
        "gripper_ignored_error_mask": gripper_ignored_error_mask,
    }
    if any(isinstance(v, bool) or not 0 <= int(v) <= _MAX_STATUS_MASK for v in masks.values()):
        raise ROSConfigError(f"GalaxeaA1HAL motor status masks must be uint32 values: {masks}")
    self._host = host
    self._port = port
    self._state_timeout_s = state_timeout_s
    self._status_timeout_s = status_timeout_s
    self._feedback_limit_tolerance = feedback_limit_tolerance_rad
    self._alignment = initial_alignment_tolerance_rad
    self._tracker_alignment_timeout = tracker_alignment_timeout_s
    self._max_step = max_target_step_rad
    self._lease = command_lease_s
    self._idle_mask = int(idle_timeout_error_mask)
    self._gripper_mask = int(gripper_ignored_error_mask)
    self._gripper_min = gripper_stroke_min_mm
    self._gripper_max = gripper_stroke_max_mm
    self._connect_timeout_s = connect_timeout_s
    self._transport: _A1Transport = transport or _SocketTransport(host, self._port)
    self._connected = False
    self._first_joint_command = True
    self._accepted_action_count = 0

connect() -> None

Connect to the sidecar and require one complete fresh snapshot.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def connect(self) -> None:
    """Connect to the sidecar and require one complete fresh snapshot."""
    if self._connected:
        raise ROSRuntimeError("GalaxeaA1HAL is already connected.")
    hello: dict[str, object] = {
        "op": "hello",
        "protocol": _PROTOCOL_VERSION,
        "robot": "galaxea_a1",
        "joint_names": list(_JOINT_NAMES),
        "joint_position_min": [limits[0] for limits in _JOINT_LIMITS],
        "joint_position_max": [limits[1] for limits in _JOINT_LIMITS],
        "initial_alignment_tolerance_rad": self._alignment,
        "tracker_alignment_timeout_s": self._tracker_alignment_timeout,
        "max_target_step_rad": self._max_step,
        "command_lease_s": self._lease,
        "state_timeout_s": self._state_timeout_s,
        "status_timeout_s": self._status_timeout_s,
        "feedback_limit_tolerance_rad": self._feedback_limit_tolerance,
        "idle_timeout_error_mask": self._idle_mask,
        "gripper_ignored_error_mask": self._gripper_mask,
        "gripper_stroke_min_mm": self._gripper_min,
        "gripper_stroke_max_mm": self._gripper_max,
    }
    try:
        self._transport.connect(hello, self._connect_timeout_s)
        self._fresh_snapshot(require_status=True)
    except Exception:
        self._transport.close()
        raise
    self._connected = True
    self._first_joint_command = True
    self._accepted_action_count = 0
    log.info("hal.connect", robot=self.description.name, sidecar=f"{self._host}:{self._port}")

disconnect() -> None

Close the sidecar transport idempotently.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def disconnect(self) -> None:
    """Close the sidecar transport idempotently."""
    self._transport.close()
    self._connected = False

estop() -> None

Stop the sidecar-owned ROS 1 stack and always raise.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def estop(self) -> None:
    """Stop the sidecar-owned ROS 1 stack and always raise."""
    log.critical("hal.estop", robot=self.description.name)
    try:
        # The sidecar acknowledges only after tracker, driver, and roscore
        # have exited. Its bounded shutdown allows up to four seconds
        # (3 s SIGINT grace + 1 s SIGKILL wait).
        self._transport.estop(timeout_s=4.0)
    finally:
        self._connected = False
    raise ROSEStopRequested("Emergency stop triggered on Galaxea A1; ROS 1 sidecar stopped.")

health() -> HALHealthReport

Return cached hardware health for the lifecycle diagnostics heartbeat.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def health(self) -> HALHealthReport:
    """Return cached hardware health for the lifecycle diagnostics heartbeat."""
    state = self._fresh_snapshot(require_status=True)
    now = time.monotonic()
    return HALHealthReport(
        message="A1 feedback and motor status healthy",
        fields={
            "sidecar": f"{self._host}:{self._port}",
            "joint_state_age_s": f"{now - state.received_monotonic:.3f}",
            "motor_status_age_s": f"{now - state.status_received_monotonic:.3f}",
            "motor_status_codes": ",".join(str(code) for code in state.status_codes),
            "command_relay_state": state.relay_state,
            "hal_accepted_action_count": str(self._accepted_action_count),
            "sidecar_accepted_action_count": str(state.sidecar_action_count),
            "gripper_position_normalized": f"{state.gripper_norm:.6f}",
            "staged_position": ",".join(f"{value:.6f}" for value in state.staged_position),
            "forwarded_position": ",".join(
                f"{value:.6f}" for value in state.forwarded_position
            ),
        },
    )

read_state() -> JointState

Return the latest non-stale six-joint feedback snapshot.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def read_state(self) -> JointState:
    """Return the latest non-stale six-joint feedback snapshot."""
    self._require_connected("read_state")
    state = self._fresh_snapshot(require_status=True)
    return JointState(
        name=list(_JOINT_NAMES),
        position=list(state.position),
        velocity=list(state.velocity),
        effort=list(state.effort),
        stamp_ns=state.stamp_ns,
    )

send_action(action: Action) -> None

Queue one validated joint or normalized-gripper target.

Source code in python/hal/src/openral_hal/galaxea_a1.py

def send_action(self, action: Action) -> None:
    """Queue one validated joint or normalized-gripper target."""
    self._require_connected("send_action")
    state = self._fresh_snapshot(require_status=True)
    message: dict[str, object] = {"op": "action", "stamp_ns": action.stamp_ns}
    if action.control_mode == ControlMode.JOINT_POSITION:
        message["joint_targets"] = list(self._validated_joint_target(action, state))
    elif action.control_mode == ControlMode.GRIPPER_POSITION:
        message["gripper"] = self._validated_gripper_target(action)
    else:
        raise ROSConfigError(
            f"GalaxeaA1HAL does not support control mode {action.control_mode.value!r}."
        )
    self._transport.send(message)
    self._accepted_action_count += 1
    if action.control_mode == ControlMode.JOINT_POSITION:
        self._first_joint_command = False

H1MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5, description: RobotDescription | None = None)

Bases: MujocoArmHAL

HAL adapter for the Unitree H1 humanoid (MuJoCo digital twin).

Drives the 19 actuated joints of the menagerie unitree_h1 MJCF through MuJoCo's position-controlled actuators. Exposes a 19-D openral_core.Action matching the joint order in H1_DESCRIPTION (left leg 5 → right leg 5 → torso 1 → left arm 4 → right arm 4).

.. warning::

This HAL does not provide balance. Without an S0 cerebellar controller (CLAUDE.md §3 dual-system pattern, M2 milestone) the robot will fall over under gravity. The closed-loop convergence tests run with gravity_enabled=False for that reason. Use this HAL to validate the action contract / joint indexing / lifecycle, not to roll out a humanoid policy.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/unitree_h1/h1.xml).	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time — required for the contract-validation tests because the floating base falls otherwise.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import H1MujocoHAL # doctest: +SKIP hal = H1MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 19 actuated joints # doctest: +SKIP 19 hal.disconnect() # doctest: +SKIP

Initialise the H1 HAL; no MuJoCo state is created until connect().

All MuJoCo wiring (MJCF URI, floating-base offsets) lives in H1_DESCRIPTION.sim. The software PD gains stay here because they are H1-specific cerebellar substitute behavior, not arm-data. description is the loaded robots/h1/robot.yaml manifest (threaded by build_hal); None falls back to the in-code H1_DESCRIPTION mirror.

Source code in python/hal/src/openral_hal/h1.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the H1 HAL; no MuJoCo state is created until ``connect()``.

    All MuJoCo wiring (MJCF URI, floating-base offsets) lives in
    ``H1_DESCRIPTION.sim``.  The software PD gains
    stay here because they are H1-specific cerebellar substitute
    behavior, not arm-data. ``description`` is the loaded
    ``robots/h1/robot.yaml`` manifest (threaded by ``build_hal``);
    ``None`` falls back to the in-code ``H1_DESCRIPTION`` mirror.
    """
    self._init_from_description(
        description or H1_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )
    self._pd_gains: dict[str, tuple[float, float]] = _h1_pd_gains()

HAL

Bases: Protocol

Structural protocol that every HAL adapter must satisfy.

The hot path (read_state / send_action) is synchronous and must complete within the robot's control cycle budget. Blocking I/O or memory allocation inside these methods is forbidden.

Implementors must raise ROSEStopRequested (a ROSSafetyViolation) from estop() so that the safety supervisor can catch it at the boundary and trigger an incident log entry.

Attributes:

Name	Type	Description
description	RobotDescription	Normative RobotDescription manifest for this robot.

connect() -> None

Open the connection to the robot hardware or simulator.

Raises:

Type	Description
ROSConfigError	If the URDF, controller name, or topic cannot be resolved.
ROSRuntimeError	If the underlying transport fails to initialise.

Source code in python/hal/src/openral_hal/protocol.py

def connect(self) -> None:
    """Open the connection to the robot hardware or simulator.

    Raises:
        ROSConfigError: If the URDF, controller name, or topic cannot be
            resolved.
        ROSRuntimeError: If the underlying transport fails to initialise.
    """
    ...

disconnect() -> None

Close the connection and release all resources gracefully.

Must be idempotent — calling disconnect() on an already-disconnected HAL must not raise.

Source code in python/hal/src/openral_hal/protocol.py

def disconnect(self) -> None:
    """Close the connection and release all resources gracefully.

    Must be idempotent — calling ``disconnect()`` on an already-disconnected
    HAL must not raise.
    """
    ...

estop() -> None

Trigger an emergency stop.

Raises:

Type	Description
ROSEStopRequested	Always. Callers must NOT catch this silently. Only the safety supervisor boundary may catch it.

Source code in python/hal/src/openral_hal/protocol.py

def estop(self) -> None:
    """Trigger an emergency stop.

    Raises:
        ROSEStopRequested: Always.  Callers must NOT catch this silently.
            Only the safety supervisor boundary may catch it.
    """
    ...

read_state() -> JointState

Return the latest joint state snapshot.

Raises:

Type	Description
ROSRuntimeError	If the HAL is not connected.
ROSPerceptionStale	If the most recent reading exceeds the staleness deadline configured in the RobotDescription.

Returns:

Type	Description
JointState	The latest JointState for all joints listed in
JointState	description.joints.

Source code in python/hal/src/openral_hal/protocol.py

def read_state(self) -> JointState:
    """Return the latest joint state snapshot.

    Raises:
        ROSRuntimeError: If the HAL is not connected.
        ROSPerceptionStale: If the most recent reading exceeds the staleness
            deadline configured in the ``RobotDescription``.

    Returns:
        The latest ``JointState`` for all joints listed in
        ``description.joints``.
    """
    ...

send_action(action: Action) -> None

Forward an action chunk to the underlying controller.

The HAL is responsible only for forwarding the typed action; safety clamping happens in the C++ safety kernel before this call is made.

Parameters:

Name	Type	Description	Default
action	Action	The Action produced by a Skill or the safety shaper.	required

Raises:

Type	Description
ROSRuntimeError	If the HAL is not connected.
ROSConfigError	If the action's control_mode is incompatible with the robot's supported_control_modes.

Source code in python/hal/src/openral_hal/protocol.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the underlying controller.

    The HAL is responsible only for forwarding the typed action; safety
    clamping happens in the C++ safety kernel before this call is made.

    Args:
        action: The ``Action`` produced by a Skill or the safety shaper.

    Raises:
        ROSRuntimeError: If the HAL is not connected.
        ROSConfigError: If the action's ``control_mode`` is incompatible
            with the robot's ``supported_control_modes``.
    """
    ...

HALHealthProvider

Bases: Protocol

Optional HAL extension for cached diagnostics.

health() -> HALHealthReport

Return the latest cached health report without performing I/O.

Source code in python/hal/src/openral_hal/protocol.py

def health(self) -> HALHealthReport:
    """Return the latest cached health report without performing I/O."""
    ...

HALHealthReport(message: str, fields: Mapping[str, str]) dataclass

Cached health data exposed through the lifecycle diagnostics heartbeat.

health() implementations must not perform device or network I/O. The lifecycle node calls them from its low-rate diagnostics path.

LifecycleEStopHAL

Bases: Protocol

Optional extension for HALs that propagate the lifecycle e-stop downstream.

HALs that implement this protocol opt into receiving estop() when the generic lifecycle node latches /openral/estop. Existing HALs that do not opt in retain the lifecycle node's local latch-only behavior.

estop() -> None

Stop downstream hardware or owned processes and always raise.

Source code in python/hal/src/openral_hal/protocol.py

def estop(self) -> None:
    """Stop downstream hardware or owned processes and always raise."""
    ...

OpenArmMujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Enactic OpenArm v2 (MuJoCo digital twin).

Thin manifest-driven wrapper around MujocoArmHAL; all wiring (MJCF URI via the openarm_v2: scheme, joint→qpos map that skips the passive follower fingers, two PASSTHROUGH grippers, seed_ctrl_from_qpos to hold the initial pose under the v2 PD actuators) lives in OPENARM_DESCRIPTION.sim.

Public 16-DoF surface (7 arm + 1 gripper per side, left then right) matches what a future OpenArmRealHAL wrapping the LeRobot OpenArm driver will accept. Gripper commands are passthrough radians (left jaw [0, 0.7854], right jaw [-0.7854, 0]).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the v2 bimanual MJCF is fetched lazily through openral_hal._openarm_v2_assets.ensure_openarm_v2_mjcf via the openarm_v2:bimanual URI scheme.	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import OpenArmMujocoHAL # doctest: +SKIP hal = OpenArmMujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 + 1 + 7 + 1 # doctest: +SKIP 16 hal.disconnect() # doctest: +SKIP

Initialise the adapter; the MJCF is not loaded until connect().

OpenArm has no per-robot connect() override: any starting pose a Skill needs is carried by the rSkill manifest's starting_pose: and applied by rskill_runner_node via MujocoArmHAL.reset_to_pose before the first inference tick (bimanual amendment).

Source code in python/hal/src/openral_hal/openarm.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the adapter; the MJCF is not loaded until ``connect()``.

    OpenArm has no per-robot ``connect()`` override: any starting pose
    a Skill needs is carried by the rSkill manifest's
    ``starting_pose:`` and applied by ``rskill_runner_node`` via
    ``MujocoArmHAL.reset_to_pose`` before the first inference
    tick (bimanual amendment).
    """
    self._init_from_description(
        OPENARM_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

OpenArmRealHAL(description: RobotDescription | None = None, *, left_can_interface: str = _LEFT_CAN_INTERFACE, right_can_interface: str = _RIGHT_CAN_INTERFACE, left_arm_controller: str = _LEFT_ARM_CONTROLLER, left_gripper_controller: str = _LEFT_GRIPPER_CONTROLLER, right_arm_controller: str = _RIGHT_ARM_CONTROLLER, right_gripper_controller: str = _RIGHT_GRIPPER_CONTROLLER, joint_state_topic: str = _JOINT_STATE_TOPIC, publish_fn: Callable[[str, dict[str, object]], None] | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float | None = None, require_can_links: bool = True)

Bases: RosControlHAL

ros2_control-backed HAL adapter for the physical OpenArm v2.

Parameters:

Name	Type	Description	Default
description	RobotDescription | None	Manifest to publish. Defaults to OPENARM_REAL_DESCRIPTION; pass a rig-specific manifest (e.g. one written by openral detect) to override sensors or safety limits without changing kinematics.	None
left_can_interface	str	SocketCAN interface for the left arm's motor bus.	_LEFT_CAN_INTERFACE
right_can_interface	str	SocketCAN interface for the right arm's bus.	_RIGHT_CAN_INTERFACE
left_arm_controller	str	ros2_control controller commanding the left arm's 7 joints.	_LEFT_ARM_CONTROLLER
left_gripper_controller	str	Controller commanding the left gripper.	_LEFT_GRIPPER_CONTROLLER
right_arm_controller	str	Controller commanding the right arm's 7 joints.	_RIGHT_ARM_CONTROLLER
right_gripper_controller	str	Controller commanding the right gripper.	_RIGHT_GRIPPER_CONTROLLER
joint_state_topic	str	Topic publishing sensor_msgs/JointState.	_JOINT_STATE_TOPIC
publish_fn	Callable[[str, dict[str, object]], None] | None	Transport callable; defaults to the base adapter's no-op logger. Injected with a real rclpy publisher by the lifecycle node, and by HIL tests.	None
state_fn	Callable[[], dict[str, object]] | None	Callable returning the latest raw joint state as a dict with name / position / velocity / effort keys, in ros2_control joint naming.	None
staleness_limit_s	float | None	Maximum age of a read_state() reading. None (default) reads the manifest's safety.joint_state_staleness_limit_s.	None
require_can_links	bool	When True (the default), connect verifies both CAN interfaces are up and refuses otherwise. Set False only to exercise the ROS wiring against a bringup whose hardware interface is itself simulated.	True

Raises:

Type	Description
ROSConfigError	If the description's joint count is not 16, or if a joint is missing the sim_joint_name that maps it to ros2_control.

Example

from openral_hal.openarm_real import OpenArmRealHAL hal = OpenArmRealHAL(require_can_links=False) hal.ros2_control_joint_names()[0] 'openarm_left_joint1' hal.ros2_control_joint_names()[7] 'openarm_left_finger_joint1'

Initialise the adapter; opens neither ROS nor the CAN bus yet.

Source code in python/hal/src/openral_hal/openarm_real.py

def __init__(
    self,
    description: RobotDescription | None = None,
    *,
    left_can_interface: str = _LEFT_CAN_INTERFACE,
    right_can_interface: str = _RIGHT_CAN_INTERFACE,
    left_arm_controller: str = _LEFT_ARM_CONTROLLER,
    left_gripper_controller: str = _LEFT_GRIPPER_CONTROLLER,
    right_arm_controller: str = _RIGHT_ARM_CONTROLLER,
    right_gripper_controller: str = _RIGHT_GRIPPER_CONTROLLER,
    joint_state_topic: str = _JOINT_STATE_TOPIC,
    publish_fn: Callable[[str, dict[str, object]], None] | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float | None = None,
    require_can_links: bool = True,
) -> None:
    """Initialise the adapter; opens neither ROS nor the CAN bus yet."""
    desc = description or OPENARM_REAL_DESCRIPTION
    if len(desc.joints) != self._EXPECTED_JOINT_COUNT:
        raise ROSConfigError(
            f"OpenArmRealHAL expects a {self._EXPECTED_JOINT_COUNT}-joint "
            f"bimanual manifest (7 arm + 1 gripper per side); "
            f"'{desc.name}' has {len(desc.joints)}."
        )
    super().__init__(
        desc,
        controller_name=left_arm_controller,
        joint_state_topic=joint_state_topic,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )
    self._left_can_interface = left_can_interface
    self._right_can_interface = right_can_interface
    self._require_can_links = require_can_links
    self._can_health: dict[str, str] = {}

    # ADR-0102 — buffers a slot-dispatched tick until every slot has passed
    # safety, so the four controllers are commanded from one reassembled
    # vector instead of from four partial ones.
    self._slot_group = SlotGroupStager()

    # Logical (Skill-facing) name → ros2_control (URDF) name.  Sourced from
    # the manifest so there is exactly one copy of this mapping in the repo.
    missing = [j.name for j in desc.joints if not j.sim_joint_name]
    if missing:
        raise ROSConfigError(
            f"OpenArmRealHAL cannot map {missing} onto ros2_control: these "
            f"joints of '{desc.name}' have no sim_joint_name. The OpenArm "
            "manifest carries the URDF joint name in that field."
        )
    self._ros2_names: list[str] = [str(j.sim_joint_name) for j in desc.joints]

    self._controller_names = [
        left_arm_controller,
        left_gripper_controller,
        right_arm_controller,
        right_gripper_controller,
    ]
    # (command topic, action slice, ros2_control joint names) per controller.
    self._command_groups: tuple[tuple[str, slice, list[str]], ...] = (
        (
            f"/{left_arm_controller}/joint_trajectory",
            _LEFT_ARM_SLICE,
            self._ros2_names[_LEFT_ARM_SLICE],
        ),
        (
            f"/{left_gripper_controller}/joint_trajectory",
            _LEFT_GRIPPER_SLICE,
            self._ros2_names[_LEFT_GRIPPER_SLICE],
        ),
        (
            f"/{right_arm_controller}/joint_trajectory",
            _RIGHT_ARM_SLICE,
            self._ros2_names[_RIGHT_ARM_SLICE],
        ),
        (
            f"/{right_gripper_controller}/joint_trajectory",
            _RIGHT_GRIPPER_SLICE,
            self._ros2_names[_RIGHT_GRIPPER_SLICE],
        ),
    )

last_applied_action: Action | None property

The command the robot was last actually given, as applied (None = none yet).

For a slot group this is the composed full-dof JOINT_POSITION action, not any one slot; set only once the apply succeeded, in step with last_committed_tick. The lifecycle node folds it into the grasp trigger, which must measure "short of the command" against what the jaw was really told.

last_committed_session: int property

runner_session_id of last_committed_tick (0 = none / session-less).

last_committed_tick: int property

Inference tick of the last slot group published (0 = none since disconnect).

Read by the HAL lifecycle node to acknowledge a grouped tick on /openral/action_applied only once the whole group went out, rather than by counting slots that reached send_action.

command_bindings() -> dict[str, ControllerKind]

Return the four command topics this adapter publishes to, with kinds.

All four are JointTrajectoryController instances — openarm_bringup configures each gripper as a 1-DoF one rather than a gripper action server, so the grippers take the same trajectory_msgs/JointTrajectory the arms do.

Example

from openral_hal.openarm_real import OpenArmRealHAL OpenArmRealHAL(require_can_links=False).command_topics()[1] '/left_gripper_controller/joint_trajectory'

Source code in python/hal/src/openral_hal/openarm_real.py

def command_bindings(self) -> dict[str, ControllerKind]:
    """Return the four command topics this adapter publishes to, with kinds.

    All four are `JointTrajectoryController` instances — `openarm_bringup`
    configures each gripper as a 1-DoF one rather than a gripper action
    server, so the grippers take the same `trajectory_msgs/JointTrajectory`
    the arms do.

    Example:
        >>> from openral_hal.openarm_real import OpenArmRealHAL
        >>> OpenArmRealHAL(require_can_links=False).command_topics()[1]
        '/left_gripper_controller/joint_trajectory'
    """
    return {topic: ControllerKind.JOINT_TRAJECTORY for topic, _, _ in self._command_groups}

connect() -> None

Verify both motor buses, then open the ROS connection.

Raises:

Type	Description
ROSConfigError	If require_can_links is set and either CAN interface is missing, is not a CAN link, or is down.
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/openarm_real.py

def connect(self) -> None:
    """Verify both motor buses, then open the ROS connection.

    Raises:
        ROSConfigError: If ``require_can_links`` is set and either CAN
            interface is missing, is not a CAN link, or is down.
        ROSRuntimeError: If already connected.
    """
    if self._require_can_links:
        self._preflight_can_links()
    else:
        self._can_health = {"preflight": "skipped (require_can_links=False)"}
    super().connect()

controller_names() -> list[str]

Return the four controllers estop deactivates, in fan-out order.

Example

from openral_hal.openarm_real import OpenArmRealHAL OpenArmRealHAL(require_can_links=False).controller_names()[1] 'left_gripper_controller'

Source code in python/hal/src/openral_hal/openarm_real.py

def controller_names(self) -> list[str]:
    """Return the four controllers ``estop`` deactivates, in fan-out order.

    Example:
        >>> from openral_hal.openarm_real import OpenArmRealHAL
        >>> OpenArmRealHAL(require_can_links=False).controller_names()[1]
        'left_gripper_controller'
    """
    return list(self._controller_names)

discard_staged_slots() -> int

Drop a half-staged slot group, keeping the committed watermark.

Called by the HAL lifecycle node on every /openral/estop latch and /openral/estop_cleared — for any HAL, opted into the hardware estop or not — so a tick cut short by a stop never outlives it into the next goal. SlotGroupStager.discard semantics: a pre-stop tick replayed afterwards is still refused as stale.

Returns:

Type	Description
int	How many staged slots were dropped.

Source code in python/hal/src/openral_hal/openarm_real.py

def discard_staged_slots(self) -> int:
    """Drop a half-staged slot group, keeping the committed watermark.

    Called by the HAL lifecycle node on every ``/openral/estop`` latch and
    ``/openral/estop_cleared`` — for any HAL, opted into the hardware estop
    or not — so a tick cut short by a stop never outlives it into the next
    goal. ``SlotGroupStager.discard`` semantics: a pre-stop tick replayed
    afterwards is still refused as stale.

    Returns:
        How many staged slots were dropped.
    """
    return self._slot_group.discard()

disconnect() -> None

Close the connection, dropping any half-staged slot group.

Without the reset a slot staged before the disconnect would still be buffered on reconnect, so the first tick after it would raise the incomplete-group error against a tick nobody is waiting for.

Source code in python/hal/src/openral_hal/openarm_real.py

def disconnect(self) -> None:
    """Close the connection, dropping any half-staged slot group.

    Without the reset a slot staged before the disconnect would still be
    buffered on reconnect, so the first tick after it would raise the
    incomplete-group error against a tick nobody is waiting for.
    """
    self._slot_group.reset()
    super().disconnect()

estop() -> None

Trigger an emergency stop, dropping any half-staged slot group.

SlotGroupStager.discard is the estop path (reset, which also clears the watermark, is disconnect-only), and the base estop never routes through either. Without this override a slot staged when the stop landed would survive the stop, and the first tick of the resumed run would be spent raising the incomplete-group error against a tick from before the e-stop. The committed watermark is kept: a pre-stop tick replayed after the stop is still refused as stale. The downstream stop itself — deactivating all four controllers through controller_manager — is the base implementation's.

Raises:

Type	Description
ROSEStopRequested	Always, from the base implementation.

Source code in python/hal/src/openral_hal/openarm_real.py

def estop(self) -> None:
    """Trigger an emergency stop, dropping any half-staged slot group.

    ``SlotGroupStager.discard`` is the estop path (``reset``, which also clears the watermark,
    is disconnect-only), and the base ``estop`` never routes through either. Without this
    override a slot staged when the stop landed would survive the stop, and the first tick of
    the resumed run would be spent raising the incomplete-group error against a tick from before
    the e-stop. The committed watermark is kept: a pre-stop tick replayed after the stop is
    still refused as stale. The downstream stop itself — deactivating all four controllers
    through ``controller_manager`` — is the base implementation's.

    Raises:
        ROSEStopRequested: Always, from the base implementation.
    """
    self._slot_group.discard()
    super().estop()

health() -> HALHealthReport

Return the cached bus health captured at connect().

Performs no I/O — the lifecycle node calls this from its low-rate diagnostics path, and re-reading sysfs on every heartbeat would be both wasteful and, on a busy host, occasionally slow.

Returns:

Type	Description
HALHealthReport	A HALHealthReport naming each arm's
HALHealthReport	CAN interface and whether the preflight found it up.

Example

from openral_hal.openarm_real import OpenArmRealHAL hal = OpenArmRealHAL(require_can_links=False) hal.health().fields["connected"] 'False'

Source code in python/hal/src/openral_hal/openarm_real.py

def health(self) -> HALHealthReport:
    """Return the cached bus health captured at ``connect()``.

    Performs no I/O — the lifecycle node calls this from its low-rate
    diagnostics path, and re-reading sysfs on every heartbeat would be
    both wasteful and, on a busy host, occasionally slow.

    Returns:
        A ``HALHealthReport`` naming each arm's
        CAN interface and whether the preflight found it up.

    Example:
        >>> from openral_hal.openarm_real import OpenArmRealHAL
        >>> hal = OpenArmRealHAL(require_can_links=False)
        >>> hal.health().fields["connected"]
        'False'
    """
    fields = {"connected": str(self._connected), **self._can_health}
    state = "connected" if self._connected else "disconnected"
    return HALHealthReport(
        message=f"OpenArm v2 ({self.description.name}) — {state}",
        fields=fields,
    )

read_state() -> JointState

Return the latest joint state, reordered into manifest order.

sensor_msgs/JointState carries no ordering guarantee, and the four controllers publish into it independently, so positions must be matched by name rather than by index. The returned JointState is always in description.joints order — the order a Skill's action vector uses.

Raises:

Type	Description
ROSRuntimeError	If not connected, or if the incoming state is missing joints the manifest declares.
ROSPerceptionStale	If the last reading exceeds the staleness deadline.

Returns:

Type	Description
JointState	A 16-element JointState in manifest order.

Source code in python/hal/src/openral_hal/openarm_real.py

def read_state(self) -> JointState:
    """Return the latest joint state, reordered into manifest order.

    ``sensor_msgs/JointState`` carries no ordering guarantee, and the
    four controllers publish into it independently, so positions must be
    matched by name rather than by index.  The returned ``JointState`` is
    always in ``description.joints`` order — the order a Skill's action
    vector uses.

    Raises:
        ROSRuntimeError: If not connected, or if the incoming state is
            missing joints the manifest declares.
        ROSPerceptionStale: If the last reading exceeds the staleness
            deadline.

    Returns:
        A 16-element ``JointState`` in manifest order.
    """
    state = super().read_state()
    if self._state_fn is None:
        # No subscriber wired: the base adapter returns a zeroed state,
        # which is already in manifest order.
        return state

    raw = self._state_fn()
    incoming = raw.get("name")
    if not isinstance(incoming, list):
        # A state dict with no names cannot be reordered; the base
        # adapter's positional read is the only available interpretation.
        return state

    index_of = {str(n): i for i, n in enumerate(incoming)}
    absent = [n for n in self._ros2_names if n not in index_of]
    if absent:
        raise ROSRuntimeError(
            f"{self._joint_state_topic} is missing {len(absent)} joint(s) "
            f"declared by '{self.description.name}': {absent}. Check that "
            "every controller is active — each arm and each gripper "
            "publishes its own joints."
        )

    def _reordered(key: str, *, absent: list[float]) -> list[float]:
        values = raw.get(key)
        if not isinstance(values, list) or len(values) != len(incoming):
            return absent
        return [float(values[index_of[n]]) for n in self._ros2_names]

    zeros = [0.0] * len(self._ros2_names)
    return JointState(
        name=list(self._joint_names),
        position=_reordered("position", absent=zeros),
        velocity=_reordered("velocity", absent=zeros),
        # An absent effort channel stays empty, never zero-filled: zeros
        # would fake an effort reading that the driver never published.
        effort=_reordered("effort", absent=[]),
        stamp_ns=state.stamp_ns,
    )

ros2_control_joint_names() -> list[str]

Return the ros2_control joint names in action-vector order.

Returns:

Type	Description
list[str]	16 URDF joint names — left arm 1-7, left finger, right arm 1-7,
list[str]	right finger.

Example

from openral_hal.openarm_real import OpenArmRealHAL names = OpenArmRealHAL(require_can_links=False).ros2_control_joint_names() len(names) 16 names[15] 'openarm_right_finger_joint1'

Source code in python/hal/src/openral_hal/openarm_real.py

def ros2_control_joint_names(self) -> list[str]:
    """Return the ros2_control joint names in action-vector order.

    Returns:
        16 URDF joint names — left arm 1-7, left finger, right arm 1-7,
        right finger.

    Example:
        >>> from openral_hal.openarm_real import OpenArmRealHAL
        >>> names = OpenArmRealHAL(require_can_links=False).ros2_control_joint_names()
        >>> len(names)
        16
        >>> names[15]
        'openarm_right_finger_joint1'
    """
    return list(self._ros2_names)

send_action(action: Action) -> None

Fan one 16-DoF action out to the four ros2_control controllers.

The four messages are built before any is published, so a malformed action cannot leave the arm half-commanded — one side moving while the other holds an older setpoint.

Parameters:

Name	Type	Description	Default
action	Action	A 16-DoF Action in description.joints order.	required

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If the control mode is unsupported or the joint target dimensions do not match the 16-DoF layout.

Source code in python/hal/src/openral_hal/openarm_real.py

def send_action(self, action: Action) -> None:
    """Fan one 16-DoF action out to the four ros2_control controllers.

    The four messages are built before any is published, so a malformed
    action cannot leave the arm half-commanded — one side moving while the
    other holds an older setpoint.

    Args:
        action: A 16-DoF ``Action`` in ``description.joints`` order.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If the control mode is unsupported or the joint
            target dimensions do not match the 16-DoF layout.
    """
    self._require_connected("send_action")

    # ADR-0102 — a slot-dispatched tick arrives as several typed actions
    # (this contract: left arm / left gripper / right arm / right gripper),
    # none of which is a whole-robot command on its own. Reassemble the
    # tick and fall through to the single-vector publish below, so the
    # all-or-nothing guarantee covers the composed group too.
    group: list[Action] | None = None
    if int(action.tick_group_size) > 1:
        group = self._slot_group.stage(action)
        if group is None:
            return
        log.debug(
            "hal.send_action.slot_group",
            robot=self.description.name,
            slots=len(group),
            tick=group[0].tick_index,
            modes=[a.control_mode.value for a in group],
        )
        action = compose_slot_group_action(group, [j.name for j in self.description.joints])
    else:
        # Ungrouped ticked actions (starting-pose ramp, approach) share the
        # watermark, so a restarted runner's renumbering is seen here too.
        self._slot_group.admit(action)

    self._validate_action(action)

    if action.control_mode in GRIPPER_MODES:
        # Reachable only outside a group — a lone gripper action. The four
        # bimanual controllers are commanded from one 16-DoF vector, so
        # there is nowhere to put it. Refuse loudly: the pre-ADR-0102
        # behaviour fell through to the `joint_targets is None` return
        # below and dropped the command silently, which on hardware is
        # arms moving while the grippers never actuate.
        raise ROSConfigError(
            f"{type(self).__name__} cannot execute a standalone "
            f"{action.control_mode.value} action: the bimanual controllers "
            "are commanded as one 16-DoF joint vector, so a gripper command "
            "is only placeable inside a slot group (tick_group_size > 1, "
            "ADR-0102)."
        )

    if action.joint_targets is None:
        log.debug("hal.send_action.empty", robot=self.description.name)
        return

    # Build every message first — publishing is all-or-nothing.
    deadline = self.time_from_start_s(action)
    outbound: list[tuple[str, dict[str, object]]] = []
    for topic, span, names in self._command_groups:
        outbound.append(
            (
                topic,
                {
                    "control_mode": action.control_mode,
                    "horizon": action.horizon,
                    "joint_names": names,
                    "joint_targets": [list(step[span]) for step in action.joint_targets],
                    "stamp_ns": action.stamp_ns,
                    "time_from_start_s": deadline,
                },
            )
        )
    for topic, msg in outbound:
        self._publish_fn(topic, msg)
    if group is not None:
        self._slot_group.commit(group, applied=action)
    else:
        self._slot_group.commit_tick(action)

    log.debug(
        "hal.send_action",
        robot=self.description.name,
        control_mode=action.control_mode,
        horizon=action.horizon,
        controllers=len(outbound),
    )

PandaMobileHAL(*, initial_pose: list[float] | None = None, dt_s: float = 0.05, description: RobotDescription | None = None)

In-process digital-twin HAL for the panda_mobile embodiment.

Maintains 10-DoF qpos state in memory. Routing per Action.control_mode:

• ControlMode.BODY_TWIST — Euler-integrates base pose using the planar components of the twist (linear x, linear y, angular z). Each send_action call advances by dt_s seconds (default 0.05 — 20 Hz nav control rate).
• ControlMode.JOINT_POSITION — sets the seven arm joints directly when the action carries seven targets; sets all ten slots when ten targets are supplied (base position + arm).

Parameters:

Name	Type	Description	Default
initial_pose	list[float] | None	Optional override of the start state. Length must be either 3 (base only) or 10 (base + arm). Defaults to all-zero.	None
dt_s	float	Integration timestep for body-twist commands. Defaults to 0.05 s (20 Hz). The Nav2 controller's default cmd_vel rate.	0.05

Latch initial 10-DoF state. No I/O until connect.

description is the loaded manifest (build_hal threads it, so panda_mobile_vslam reports its own sensors); None falls back to robots/panda_mobile/robot.yaml. The joint layout (base from base_joints, arm/gripper from joint role) is read from it.

Source code in python/hal/src/openral_hal/panda_mobile.py

def __init__(
    self,
    *,
    initial_pose: list[float] | None = None,
    dt_s: float = 0.05,
    description: RobotDescription | None = None,
) -> None:
    """Latch initial 10-DoF state. No I/O until ``connect``.

    ``description`` is the loaded manifest (``build_hal`` threads it, so
    ``panda_mobile_vslam`` reports its own sensors); ``None`` falls back
    to ``robots/panda_mobile/robot.yaml``. The joint layout (base from
    ``base_joints``, arm/gripper from joint ``role``) is read from it.
    """
    # `HALLifecycleNodeBase._publish_joint_state` reads
    # `self._hal.description` for OTel span attributes + per-joint
    # limit population.
    self.description: RobotDescription = description or PANDA_MOBILE_DESCRIPTION
    base = list(self.description.base_joints or [])
    arm = [j.name for j in self.description.joints if j.role == "arm"]
    gripper = next(j.name for j in self.description.joints if j.role == "gripper")
    self._n_base, self._n_arm = len(base), len(arm)
    self._joint_names: list[str] = [*base, *arm, gripper]
    if initial_pose is None:
        self._qpos: list[float] = [0.0] * len(self._joint_names)
    elif len(initial_pose) == self._n_base:
        self._qpos = [*initial_pose, *([0.0] * self._n_arm)]
    elif len(initial_pose) == len(self._joint_names):
        self._qpos = list(initial_pose)
    else:
        raise ROSConfigError(
            f"PandaMobileHAL.initial_pose must have length 3 (base only) "
            f"or 10 (base+arm); got {len(initial_pose)}."
        )
    self._dt_s = float(dt_s)
    self._connected: bool = False
    self._estop_latched: bool = False
    # Last commanded base body twist (vx, vy, vz, wx, wy, wz) in the
    # base_link frame — published as the /odom twist by the lifecycle
    # node. The base integrates this exactly, so it is the velocity.
    # Zeroed on any non-BODY_TWIST action, and read as zero once no
    # BODY_TWIST arrived for ``_TWIST_STALE_PERIODS`` control periods.
    self._last_twist_ns: int = 0
    self._last_body_twist: tuple[float, float, float, float, float, float] = (
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
    )

base_pose: tuple[float, float, float] property

Current base (x, y, yaw) for tests / odom publishers.

base_twist: tuple[float, float, float, float, float, float] property

Last commanded base body twist (vx, vy, vz, wx, wy, wz).

In the base_link frame — published as the /odom twist. Zeroed once a non-BODY_TWIST action is sent, and reads zero once no BODY_TWIST arrived for _TWIST_STALE_PERIODS control periods (the base only moves on a command, so a stopped stream is a still base).

estop_latched: bool property

True while the estop latch is set.

connect() -> None

Idempotent — flips the connected flag.

Source code in python/hal/src/openral_hal/panda_mobile.py

def connect(self) -> None:
    """Idempotent — flips the connected flag."""
    self._connected = True

disconnect() -> None

Idempotent — clears the connected flag.

Source code in python/hal/src/openral_hal/panda_mobile.py

def disconnect(self) -> None:
    """Idempotent — clears the connected flag."""
    self._connected = False

estop() -> None

Latch the estop flag. Subsequent send_action calls no-op.

The latch can only be cleared by calling reset_estop, mirroring the supervisor's recovery contract: estops never auto-clear.

Source code in python/hal/src/openral_hal/panda_mobile.py

def estop(self) -> None:
    """Latch the estop flag. Subsequent ``send_action`` calls no-op.

    The latch can only be cleared by calling ``reset_estop``,
    mirroring the supervisor's recovery contract: estops never
    auto-clear.
    """
    self._estop_latched = True

read_state() -> JointState

Return a fresh 10-DoF JointState snapshot.

Source code in python/hal/src/openral_hal/panda_mobile.py

def read_state(self) -> JointState:
    """Return a fresh 10-DoF ``JointState`` snapshot."""
    if not self._connected:
        raise ROSConfigError("PandaMobileHAL.read_state called before connect().")

    return JointState(
        name=list(self._joint_names),
        position=list(self._qpos),
        velocity=[0.0] * len(self._joint_names),
        effort=[],  # in-memory kinematic stub: no effort is measured
        stamp_ns=time.time_ns(),
    )

reset_estop() -> None

Clear the estop latch. Caller asserts the cause has been resolved.

Source code in python/hal/src/openral_hal/panda_mobile.py

def reset_estop(self) -> None:
    """Clear the estop latch. Caller asserts the cause has been resolved."""
    self._estop_latched = False

send_action(action: Action) -> None

Apply the action to the in-memory state. Branches on control_mode.

Accepts JOINT_POSITION, BODY_TWIST, CARTESIAN_DELTA, GRIPPER_POSITION; each reads its payload from the matching Action field (joint_targets, body_twist, cartesian_delta, gripper).

Raises:

Type	Description
ROSConfigError	If control_mode is not one of the four, or the mode-specific payload field is missing / mis-shaped.

Source code in python/hal/src/openral_hal/panda_mobile.py

def send_action(self, action: Action) -> None:
    """Apply the action to the in-memory state. Branches on control_mode.

    Accepts ``JOINT_POSITION``, ``BODY_TWIST``, ``CARTESIAN_DELTA``,
    ``GRIPPER_POSITION``; each reads its payload from the matching
    ``Action`` field (joint_targets, body_twist, cartesian_delta,
    gripper).

    Raises:
        ROSConfigError: If ``control_mode`` is not one of the four, or
            the mode-specific payload field is missing / mis-shaped.
    """
    if not self._connected:
        raise ROSConfigError("PandaMobileHAL.send_action called before connect().")
    if self._estop_latched:
        # CLAUDE.md §1.1 — don't silently honour actions
        # after an estop; the supervisor reset path must clear the
        # latch explicitly.
        return

    mode = action.control_mode
    # Default: base not velocity-commanded. BODY_TWIST overwrites via
    # _apply_body_twist below; every other mode leaves it zeroed so
    # /odom doesn't report a stale base velocity.
    self._last_body_twist = (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)
    if mode is ControlMode.JOINT_POSITION:
        if not action.joint_targets:
            raise ROSConfigError(
                "PandaMobileHAL.send_action: empty Action.joint_targets for JOINT_POSITION."
            )
        self._apply_joint_position(list(action.joint_targets[0]))
    elif mode is ControlMode.BODY_TWIST:
        if not action.body_twist:
            raise ROSConfigError(
                "PandaMobileHAL.send_action: empty Action.body_twist for BODY_TWIST."
            )
        self._apply_body_twist(list(action.body_twist[0]))
    elif mode is ControlMode.CARTESIAN_DELTA:
        # No Jacobian here: real motion lives in the sim-attached path
        # (robosuite OSC). This just tracks the delta for dashboard
        # observability / test verification; qpos is unchanged.
        if not action.cartesian_delta:
            raise ROSConfigError("PandaMobileHAL.send_action: empty Action.cartesian_delta.")
        self._apply_cartesian_delta(list(action.cartesian_delta[0]))
    elif mode is ControlMode.GRIPPER_POSITION:
        if not action.gripper:
            raise ROSConfigError(
                "PandaMobileHAL.send_action: empty Action.gripper for GRIPPER_POSITION."
            )
        self._apply_gripper_position(float(action.gripper[0]))
    else:
        raise ROSConfigError(
            f"PandaMobileHAL.send_action: unsupported control_mode "
            f"{mode!r}; expected JOINT_POSITION / BODY_TWIST / "
            f"CARTESIAN_DELTA / GRIPPER_POSITION."
        )

ResettableLifecycleEStopHAL

Bases: LifecycleEStopHAL, Protocol

Lifecycle e-stop extension for HALs that support in-process recovery.

reset_estop() -> None

Re-arm the downstream controller after its safety conditions pass.

Source code in python/hal/src/openral_hal/protocol.py

def reset_estop(self) -> None:
    """Re-arm the downstream controller after its safety conditions pass."""
    ...

Rizon4MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Flexiv Rizon 4 (MuJoCo-backed simulation).

Drives the 7 position-controlled actuators of the menagerie flexiv_rizon4 MJCF through MujocoArmHAL. Exposes a 7-D openral_core.Action matching the joint order in RIZON4_DESCRIPTION (joint1 ... joint7).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/flexiv_rizon4/flexiv_rizon4.xml).	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import Rizon4MujocoHAL # doctest: +SKIP hal = Rizon4MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 arm joints # doctest: +SKIP 7 hal.disconnect() # doctest: +SKIP

Initialise the Rizon 4 HAL; no MuJoCo state is created until connect().

All wiring lives in RIZON4_DESCRIPTION.sim.

Source code in python/hal/src/openral_hal/flexiv_rizon4.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the Rizon 4 HAL; no MuJoCo state is created until ``connect()``.

    All wiring lives in ``RIZON4_DESCRIPTION.sim``.
    """
    self._init_from_description(
        RIZON4_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

RosControlHAL(description: RobotDescription, controller_name: str, *, joint_state_topic: str = '/joint_states', command_topic: str | None = None, publish_fn: _PublishFn | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float | None = None, stop_timeout_s: float = _DEFAULT_STOP_TIMEOUT_S)

Bases: HALBase

ros2_control-backed HAL adapter.

The adapter does not import rclpy directly so it can be unit-tested without a live ROS 2 installation. In integration / HIL tests, inject a real publisher via the publish_fn parameter.

Parameters:

Name	Type	Description	Default
description	RobotDescription	Normative RobotDescription for the target robot.	required
controller_name	str	Name of the ros2_control joint trajectory controller, e.g. "joint_trajectory_controller".	required
joint_state_topic	str	ROS 2 topic that publishes sensor_msgs/JointState. Defaults to "/joint_states".	'/joint_states'
command_topic	str | None	ROS 2 topic for joint trajectory commands. Defaults to "/<controller_name>/joint_trajectory".	None
publish_fn	_PublishFn | None	Callable used to send messages to ROS 2 topics. Defaults to a no-op logger; replace with a real publisher in integration tests.	None
state_fn	Callable[[], dict[str, object]] | None	Callable that returns the latest raw joint state as a dict. Defaults to None, in which case the adapter returns a zeroed JointState. Replace with a real subscriber callback in integration tests.	None
staleness_limit_s	float | None	Maximum age (seconds) of a read_state() reading before ROSPerceptionStale is raised. None (default) reads the manifest's safety.joint_state_staleness_limit_s; neither raises ROSConfigError.	None
stop_timeout_s	float	How long estop / reset_estop wait for the controller switch and the vendor stop to be acknowledged.	_DEFAULT_STOP_TIMEOUT_S

The control rate comes from the manifest's action_spec.control_freq_hz — the same field the runner ticks at and the dataset recorder stamps as fps — so there is exactly one place a rig declares it. Every published trajectory point gets time_from_start = horizon / control_freq_hz: the controller is asked to reach each target exactly when the next command is due. A manifest without the field is refused at construction, so build_hal(mode="real") — and with it the lifecycle node's configure — stops and names the missing value. There is no fallback deadline: a fixed 100 ms constant in this gap asks a 30 Hz stream to cover each step in a third of its period (issue #303).

Raises:

Type	Description
ROSConfigError	If description.joints is empty, or the manifest declares no positive action_spec.control_freq_hz.

Lifecycle e-stop. Every RosControlHAL implements LifecycleEStopHAL: /openral/estop reaches estop(), which deactivates every controller in controller_names() through the attached ControllerStopSeam (a controller_manager deactivation holds position on a position interface, zeroes velocity / effort ones, drops late trajectories because the controller's subscriber goes inactive, and writes nothing until re-activated), then runs the vendor-specific _vendor_stop hook, and records a DownstreamStopReport. The default estop_recovery is RESTART_REQUIRED — the conservative policy; a subclass whose controllers can be safely re-armed in place (OpenArm) opts into RESETTABLE and gets reset_estop — re-activation through the same seam, acknowledged before the HAL reconnects.

Initialise the adapter; does not open any connection yet.

Source code in python/hal/src/openral_hal/ros_control.py

def __init__(
    self,
    description: RobotDescription,
    controller_name: str,
    *,
    joint_state_topic: str = "/joint_states",
    command_topic: str | None = None,
    publish_fn: _PublishFn | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float | None = None,
    stop_timeout_s: float = _DEFAULT_STOP_TIMEOUT_S,
) -> None:
    """Initialise the adapter; does not open any connection yet."""
    if not description.joints:
        raise ROSConfigError(
            f"RobotDescription '{description.name}' has no joints; "
            "cannot initialise RosControlHAL."
        )
    spec = description.action_spec
    control_rate_hz = None if spec is None else spec.control_freq_hz
    if control_rate_hz is None or not control_rate_hz > 0.0:
        raise ROSConfigError(
            f"RobotDescription '{description.name}' declares no positive "
            f"action_spec.control_freq_hz (got {control_rate_hz!r}); "
            f"{type(self).__name__} cannot be built without it. It sets every "
            "trajectory point's time_from_start (horizon / rate) and the runner's "
            "tick — add `action_spec: {dim, representation, control_freq_hz}` "
            "to the robot manifest."
        )
    self._control_rate_hz: float = float(control_rate_hz)
    self.description = description
    self._controller_name = controller_name
    self._joint_state_topic = joint_state_topic
    self._command_topic = command_topic or f"/{controller_name}/joint_trajectory"
    self._publish_fn: _PublishFn = publish_fn or _default_publish
    self._state_fn = state_fn
    self._staleness_limit_s = resolve_staleness_limit_s(description, staleness_limit_s)
    self._stop_timeout_s = stop_timeout_s
    self._stop_seam: ControllerStopSeam | None = None
    self._last_stop_report: DownstreamStopReport | None = None

    self._connected: bool = False
    self._last_state_time: float = 0.0
    # Set by `attach_transport` to the transport's real per-message arrival
    # clock. While it is None the HAL has no way to know how old its state
    # is (see `read_state`).
    self._stamp_fn: Callable[[], float] | None = None
    self._joint_names: list[str] = [j.name for j in description.joints]

controller_name: str property

Name of the primary ros2_control controller this HAL commands.

joint_state_topic: str property

The aggregated sensor_msgs/JointState topic this HAL reads.

last_stop_report: DownstreamStopReport | None property

Report of the most recent estop(); None before it / after reset_estop.

attach_controller_stop(seam: ControllerStopSeam) -> None

Bind the live controller stop seam after construction.

Like attach_transport, this exists because build_hal runs before any ROS node exists. The lifecycle node attaches the RosControlTransport (which implements the seam) under hal_mode:=real; unit tests attach a SimTransport.

Parameters:

Name	Type	Description	Default
seam	ControllerStopSeam	Anything satisfying ControllerStopSeam.	required

Raises:

Type	Description
ROSConfigError	If seam does not satisfy the protocol — the e-stop path must never discover a missing method on hardware.

Source code in python/hal/src/openral_hal/ros_control.py

def attach_controller_stop(self, seam: ControllerStopSeam) -> None:
    """Bind the live controller stop seam after construction.

    Like ``attach_transport``, this exists because ``build_hal`` runs
    before any ROS node exists. The lifecycle node attaches the
    ``RosControlTransport`` (which implements the seam) under
    ``hal_mode:=real``; unit tests attach a ``SimTransport``.

    Args:
        seam: Anything satisfying ``ControllerStopSeam``.

    Raises:
        ROSConfigError: If ``seam`` does not satisfy the protocol — the
            e-stop path must never discover a missing method on hardware.
    """
    if not isinstance(seam, ControllerStopSeam):
        raise ROSConfigError(
            f"attach_controller_stop() needs a ControllerStopSeam "
            f"(deactivate_controllers / activate_controllers / call_trigger / "
            f"publish_empty); got {type(seam).__name__}."
        )
    self._stop_seam = seam

attach_transport(publish_fn: _PublishFn, state_fn: Callable[[], dict[str, object]], stamp_fn: Callable[[], float] | None = None) -> None

Bind this HAL to a live transport after construction.

build_hal constructs the HAL from the manifest alone, before any ROS node exists, so a real deployment cannot pass publish_fn/state_fn to __init__. The lifecycle node calls this once it has a node to create publishers on — which is what makes a real ros2_control robot work without any per-robot wiring code.

Parameters:

Name	Type	Description	Default
publish_fn	_PublishFn	Sends one command dict to a controller topic.	required
state_fn	Callable[[], dict[str, object]]	Returns the newest joint state as a raw dict.	required
stamp_fn	Callable[[], float] | None	Returns the time.monotonic() timestamp of the newest joint state. Without it read_state can only measure age from connect(), which is not a freshness check at all.	None

Raises:

Type	Description
ROSConfigError	If stamp_fn is given but is not callable.

Source code in python/hal/src/openral_hal/ros_control.py

def attach_transport(
    self,
    publish_fn: _PublishFn,
    state_fn: Callable[[], dict[str, object]],
    stamp_fn: Callable[[], float] | None = None,
) -> None:
    """Bind this HAL to a live transport after construction.

    ``build_hal`` constructs the HAL from the manifest alone, before any ROS
    node exists, so a real deployment cannot pass ``publish_fn``/``state_fn``
    to ``__init__``. The lifecycle node calls this once it has a node to
    create publishers on — which is what makes a real ros2_control robot
    work without any per-robot wiring code.

    Args:
        publish_fn: Sends one command dict to a controller topic.
        state_fn: Returns the newest joint state as a raw dict.
        stamp_fn: Returns the ``time.monotonic()`` timestamp of the newest
            joint state. Without it ``read_state`` can only measure age from
            ``connect()``, which is not a freshness check at all.

    Raises:
        ROSConfigError: If ``stamp_fn`` is given but is not callable.
    """
    # Checked here rather than at first use: `stamp_fn` feeds the staleness
    # watchdog, so a caller that passes a *value* (a property read once at
    # wire-up, say) instead of a callable would otherwise fail deep in the
    # hot path, on hardware, with the robot already connected.
    if stamp_fn is not None and not callable(stamp_fn):
        raise ROSConfigError(
            f"attach_transport(stamp_fn=...) needs a callable returning the newest "
            f"joint state's time.monotonic() timestamp, got {type(stamp_fn).__name__}. "
            "A property must be wrapped, e.g. `lambda: transport.last_stamp`."
        )
    self._publish_fn = publish_fn
    self._state_fn = state_fn
    self._stamp_fn = stamp_fn
    self._last_state_time = time.monotonic()

command_bindings() -> dict[str, ControllerKind]

Return each controller topic this HAL publishes to, with its kind.

One entry for a single-controller arm; robots whose controllers are split (the bimanual OpenArm's arm+gripper per side) override this. Insertion order is the fan-out order send_action uses, so it is load-bearing.

This is the override point for a new robot: declaring the topic and the message type its controller speaks is what lets the transport publish without guessing, so adding a robot needs no transport code.

Source code in python/hal/src/openral_hal/ros_control.py

def command_bindings(self) -> dict[str, ControllerKind]:
    """Return each controller topic this HAL publishes to, with its kind.

    One entry for a single-controller arm; robots whose controllers are
    split (the bimanual OpenArm's arm+gripper per side) override this.
    Insertion order is the fan-out order `send_action` uses, so it is
    load-bearing.

    This is the override point for a new robot: declaring the topic *and*
    the message type its controller speaks is what lets the transport
    publish without guessing, so adding a robot needs no transport code.
    """
    return {self._command_topic: ControllerKind.JOINT_TRAJECTORY}

command_topics() -> list[str]

Return every controller topic this HAL publishes to, in fan-out order.

Derived from command_bindings so the topic list and the declared wire formats cannot drift apart. Override command_bindings, not this.

Source code in python/hal/src/openral_hal/ros_control.py

def command_topics(self) -> list[str]:
    """Return every controller topic this HAL publishes to, in fan-out order.

    Derived from `command_bindings` so the topic list and the declared wire
    formats cannot drift apart. Override `command_bindings`, not this.
    """
    return list(self.command_bindings())

connect() -> None

Open the connection to the robot hardware or simulator.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/ros_control.py

def connect(self) -> None:
    """Open the connection to the robot hardware or simulator.

    Raises:
        ROSRuntimeError: If already connected.
    """
    if self._connected:
        raise ROSRuntimeError(f"RosControlHAL('{self.description.name}') is already connected.")
    log.info(
        "hal.connect",
        robot=self.description.name,
        controller=self._controller_name,
        joint_state_topic=self._joint_state_topic,
        command_topic=self._command_topic,
    )
    self._connected = True
    self._last_state_time = time.monotonic()

controller_names() -> list[str]

Return every controller_manager controller estop deactivates.

One entry for a single-controller arm; robots whose controllers are split (the bimanual OpenArm) override this alongside command_bindings so the stop covers every topic they publish to.

Source code in python/hal/src/openral_hal/ros_control.py

def controller_names(self) -> list[str]:
    """Return every ``controller_manager`` controller ``estop`` deactivates.

    One entry for a single-controller arm; robots whose controllers are
    split (the bimanual OpenArm) override this alongside
    ``command_bindings`` so the stop covers every topic they publish to.
    """
    return [self._controller_name]

estop() -> None

Trigger an emergency stop: drop the connection, stop downstream, raise.

Ordering is deliberate. The connection flag drops first, so no further send_action from this HAL can reach the wire even if the downstream stop below takes its full timeout. Then every controller in controller_names() is deactivated through the attached ControllerStopSeam (acknowledged by controller_manager and confirmed via its controller list), then the vendor-specific _vendor_stop runs. Both are attempted even if the first fails, so the report names everything that did and did not acknowledge.

The outcome is recorded in last_stop_report rather than changing the exception type: the HAL Protocol promises ROSEStopRequested from every estop(), and the lifecycle node reads the report to log FATAL and publish downstream_stop=unacknowledged when the stop was not proven. With no seam attached (a unit test driving the HAL with bare callables) the report says so and stopped is False — a local latch is never reported as a downstream stop.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/ros_control.py

def estop(self) -> None:
    """Trigger an emergency stop: drop the connection, stop downstream, raise.

    Ordering is deliberate. The connection flag drops **first**, so no
    further ``send_action`` from this HAL can reach the wire even if the
    downstream stop below takes its full timeout. Then every controller in
    ``controller_names()`` is deactivated through the attached
    ``ControllerStopSeam`` (acknowledged by ``controller_manager`` and
    confirmed via its controller list), then the vendor-specific
    ``_vendor_stop`` runs. Both are attempted even if the first fails, so
    the report names everything that did and did not acknowledge.

    The outcome is recorded in ``last_stop_report`` rather than changing
    the exception type: the HAL Protocol promises ``ROSEStopRequested``
    from every ``estop()``, and the lifecycle node reads the report to log
    FATAL and publish ``downstream_stop=unacknowledged`` when the stop was
    not proven. With no seam attached (a unit test driving the HAL with
    bare callables) the report says so and ``stopped`` is ``False`` — a
    local latch is never reported as a downstream stop.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical("hal.estop", robot=self.description.name, controllers=self.controller_names())
    self._connected = False
    report = self._stop_downstream()
    self._last_stop_report = report
    verdict = "acknowledged" if report.stopped else "NOT acknowledged"
    raise ROSEStopRequested(
        f"Emergency stop triggered on robot '{self.description.name}' "
        f"(downstream stop {verdict}: {report.detail})."
    )

read_state() -> JointState

Return the latest joint state snapshot.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last reading is older than staleness_limit_s.

Returns:

Type	Description
JointState	Latest JointState for all joints in description.joints.

Source code in python/hal/src/openral_hal/ros_control.py

def read_state(self) -> JointState:
    """Return the latest joint state snapshot.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last reading is older than
            ``staleness_limit_s``.

    Returns:
        Latest ``JointState`` for all joints in ``description.joints``.
    """
    self._require_connected("read_state")
    # With a transport attached, age is measured from the arrival of the
    # newest joint state — the only thing that actually says whether the
    # robot is still talking to us. Without one, the best available
    # reference is `connect()`; that is a liveness floor, not a freshness
    # check, and it is why a transport must supply `stamp_fn` on real
    # hardware. A transport that has received nothing yet reports 0.0,
    # which must read as "infinitely stale", never as "fresh at epoch".
    if self._stamp_fn is not None:
        last = self._stamp_fn()
        age = float("inf") if last <= 0.0 else time.monotonic() - last
    else:
        age = time.monotonic() - self._last_state_time
    sample_age_s = age if self._stamp_fn is not None else 0.0
    if age > self._staleness_limit_s:
        raise ROSPerceptionStale(
            f"Joint state is {age:.3f} s old (limit {self._staleness_limit_s} s)."
        )

    n = len(self._joint_names)
    raw: dict[str, object] = {}
    if self._state_fn is not None:
        raw = self._state_fn()

    return JointState(
        name=self._joint_names,
        position=_raw_floats(raw, "position", n),
        velocity=_raw_floats(raw, "velocity", n),
        effort=_raw_floats(raw, "effort", n),
        # When the sample arrived, not when it was read: a consumer that pairs
        # joint states with other sensors by stamp (the robot self-filter's
        # skew check) must see how old the sample really is.
        stamp_ns=int(time.time_ns()) - int(max(sample_age_s, 0.0) * 1e9),
    )

reset_estop() -> None

Re-arm after estop() — only for RESETTABLE adapters.

Runs the vendor reset, re-activates every controller through the seam and, only once controller_manager confirms them active, marks the HAL connected again. The lifecycle node calls this from /openral/estop_cleared and keeps its latch when it raises, so no command can flow before the downstream re-arm succeeded.

Raises:

Type	Description
ROSRuntimeError	If this adapter's estop_recovery is not RESETTABLE (a RESTART_REQUIRED robot must never be re-armed in process), if no seam is attached, or if the re-activation was not acknowledged.

Source code in python/hal/src/openral_hal/ros_control.py

def reset_estop(self) -> None:
    """Re-arm after ``estop()`` — only for ``RESETTABLE`` adapters.

    Runs the vendor reset, re-activates every controller through the seam
    and, only once ``controller_manager`` confirms them ``active``, marks
    the HAL connected again. The lifecycle node calls this from
    ``/openral/estop_cleared`` and keeps its latch when it raises, so no
    command can flow before the downstream re-arm succeeded.

    Raises:
        ROSRuntimeError: If this adapter's ``estop_recovery`` is not
            ``RESETTABLE`` (a ``RESTART_REQUIRED`` robot must never be
            re-armed in process), if no seam is attached, or if the
            re-activation was not acknowledged.
    """
    if self.estop_recovery is not EStopRecovery.RESETTABLE:
        raise ROSRuntimeError(
            f"{type(self).__name__} declares estop_recovery="
            f"{self.estop_recovery.value}; in-process reset is forbidden — restart the "
            "lifecycle node and re-align."
        )
    seam = self._stop_seam
    if seam is None:
        raise ROSRuntimeError(
            f"{type(self).__name__}.reset_estop(): no ControllerStopSeam attached, so the "
            "controllers cannot be re-activated."
        )
    names = tuple(self.controller_names())
    self._vendor_reset(seam)
    switch = seam.activate_controllers(names, timeout_s=self._stop_timeout_s)
    if not switch.ok:
        raise ROSRuntimeError(
            f"{type(self).__name__}.reset_estop(): controller re-activation not "
            f"acknowledged: {switch.detail}"
        )
    log.warning(
        "hal.estop.reset", robot=self.description.name, controllers=names, states=switch.states
    )
    self._connected = True
    self._last_state_time = time.monotonic()
    self._last_stop_report = None

ros2_control_joint_names() -> list[str]

Return the joint names in ros2_control's namespace, in action order.

Defaults to the manifest's own names; robots whose URDF names differ from the manifest's (again, OpenArm) override it. /joint_states is keyed by these, so a transport matches on them.

Source code in python/hal/src/openral_hal/ros_control.py

def ros2_control_joint_names(self) -> list[str]:
    """Return the joint names in ros2_control's namespace, in action order.

    Defaults to the manifest's own names; robots whose URDF names differ
    from the manifest's (again, OpenArm) override it. ``/joint_states`` is
    keyed by these, so a transport matches on them.
    """
    return list(self._joint_names)

send_action(action: Action) -> None

Forward an action chunk to the ros2_control joint trajectory controller.

Parameters:

Name	Type	Description	Default
action	Action	The Action produced by a Skill.	required

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If action.control_mode is not in the robot's supported_control_modes, or if the joint target dimensions do not match the robot's joint count.

Source code in python/hal/src/openral_hal/ros_control.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the ros2_control joint trajectory controller.

    Args:
        action: The ``Action`` produced by a Skill.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If ``action.control_mode`` is not in the robot's
            ``supported_control_modes``, or if the joint target dimensions
            do not match the robot's joint count.
    """
    self._require_connected("send_action")
    self._validate_action(action)

    # `joint_names` travels with the command so the transport forwards what
    # the HAL chose rather than keeping a second copy of the mapping that
    # can drift from it (the rule ADR-0102 set for the OpenArm fan-out).
    msg: dict[str, object] = {
        "control_mode": action.control_mode,
        "horizon": action.horizon,
        "joint_names": self.ros2_control_joint_names(),
        "joint_targets": action.joint_targets,
        "stamp_ns": action.stamp_ns,
    }
    msg["time_from_start_s"] = self.time_from_start_s(action)
    self._publish_fn(self._command_topic, msg)
    log.debug(
        "hal.send_action",
        robot=self.description.name,
        control_mode=action.control_mode,
        horizon=action.horizon,
    )

time_from_start_s(action: Action) -> float

Trajectory deadline for action: horizon / control_freq_hz.

One command arrives per control period, so a chunk of horizon steps should be reached just as its replacement is due; the transport spreads the chunk's rows evenly up to this deadline.

Example

from openral_core.schemas import Action, ControlMode from openral_hal.openarm_real import OpenArmRealHAL hal = OpenArmRealHAL(require_can_links=False) # manifest: 30 Hz a = Action( ... control_mode=ControlMode.JOINT_POSITION, ... horizon=3, ... joint_targets=[[0.0] * 16] * 3, ... ) round(hal.time_from_start_s(a), 3) 0.1

Source code in python/hal/src/openral_hal/ros_control.py

def time_from_start_s(self, action: Action) -> float:
    """Trajectory deadline for ``action``: ``horizon / control_freq_hz``.

    One command arrives per control period, so a chunk of ``horizon``
    steps should be reached just as its replacement is due; the transport
    spreads the chunk's rows evenly up to this deadline.

    Example:
        >>> from openral_core.schemas import Action, ControlMode
        >>> from openral_hal.openarm_real import OpenArmRealHAL
        >>> hal = OpenArmRealHAL(require_can_links=False)  # manifest: 30 Hz
        >>> a = Action(
        ...     control_mode=ControlMode.JOINT_POSITION,
        ...     horizon=3,
        ...     joint_targets=[[0.0] * 16] * 3,
        ... )
        >>> round(hal.time_from_start_s(a), 3)
        0.1
    """
    return max(int(action.horizon), 1) / self._control_rate_hz

vendor_stop_services() -> list[str]

std_srvs/Trigger services _vendor_stop calls; none by default.

Declared up front so the transport creates the client at wire-up and the seam can refuse an undeclared service loudly instead of the stop path discovering a missing client on hardware.

Source code in python/hal/src/openral_hal/ros_control.py

def vendor_stop_services(self) -> list[str]:
    """``std_srvs/Trigger`` services ``_vendor_stop`` calls; none by default.

    Declared up front so the transport creates the client at wire-up and
    the seam can refuse an undeclared service loudly instead of the stop
    path discovering a missing client on hardware.
    """
    return []

vendor_stop_topics() -> list[str]

std_msgs/Empty topics _vendor_stop publishes on; none by default.

Source code in python/hal/src/openral_hal/ros_control.py

def vendor_stop_topics(self) -> list[str]:
    """``std_msgs/Empty`` topics ``_vendor_stop`` publishes on; none by default."""
    return []

SO100DigitalTwin(config: SO100DigitalTwinConfig)

Bases: Robot

In-process digital twin for the SO-100 follower arm.

Implements the full lerobot Robot interface without any serial port. Joint state is initialised from config.initial_positions and updated each time send_action() is called, so observation sequences reflect sent commands.

Parameters:

Name	Type	Description	Default
config	SO100DigitalTwinConfig	SO100DigitalTwinConfig with optional initial positions.	required

Example

cfg = SO100DigitalTwinConfig() twin = SO100DigitalTwin(cfg) twin.connect(calibrate=False) twin.is_connected True twin.get_observation()["gripper.pos"] 50.0 twin.disconnect()

Initialise the digital twin; no connection is opened.

Source code in python/hal/src/openral_hal/so100_sim.py

def __init__(self, config: SO100DigitalTwinConfig) -> None:
    """Initialise the digital twin; no connection is opened."""
    super().__init__(config)
    self.config = config
    self._is_connected: bool = False
    self._is_calibrated: bool = True  # pre-calibrated; no wizard needed
    # Internal state: positions in lerobot's native units
    self._positions: dict[str, float] = {
        name: config.initial_positions.get(name, _DEFAULT_POSITIONS[name])
        for name in _JOINT_NAMES
    }

action_features: dict[str, type] cached property

Action feature schema — one float per joint position target.

Returns:

Type	Description
dict[str, type]	Dict mapping "<joint>.pos" keys to float type.

is_calibrated: bool property

Always True — the digital twin requires no calibration.

is_connected: bool property

True if connect() has been called and disconnect() has not.

observation_features: dict[str, type] cached property

Observation feature schema — one float per joint position.

Returns:

Type	Description
dict[str, type]	Dict mapping "<joint>.pos" keys to float type.

calibrate() -> None

No-op — the digital twin requires no interactive calibration.

Source code in python/hal/src/openral_hal/so100_sim.py

def calibrate(self) -> None:
    """No-op — the digital twin requires no interactive calibration."""
    self._is_calibrated = True

configure() -> None

No-op — no bus configuration needed.

Source code in python/hal/src/openral_hal/so100_sim.py

def configure(self) -> None:
    """No-op — no bus configuration needed."""

connect(calibrate: bool = True) -> None

Activate the twin. No serial port is opened.

Parameters:

Name	Type	Description	Default
calibrate	bool	Ignored — the twin is always pre-calibrated.	True

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_already_connected  # type: ignore[untyped-decorator]
def connect(self, calibrate: bool = True) -> None:
    """Activate the twin.  No serial port is opened.

    Args:
        calibrate: Ignored — the twin is always pre-calibrated.
    """
    self._is_connected = True

disconnect() -> None

Deactivate the twin. Idempotent if called via the decorator.

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_not_connected  # type: ignore[untyped-decorator]
def disconnect(self) -> None:
    """Deactivate the twin.  Idempotent if called via the decorator."""
    self._is_connected = False

get_observation() -> RobotObservation

Return current joint positions in lerobot's native units.

Returns:

Type	Description
RobotObservation	Dict {"<joint>.pos": float} with revolute joints in degrees
RobotObservation	and gripper in [0, 100].

Raises:

Type	Description
RuntimeError	If not connected (enforced by decorator).

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_not_connected  # type: ignore[untyped-decorator]
def get_observation(self) -> RobotObservation:
    """Return current joint positions in lerobot's native units.

    Returns:
        Dict ``{"<joint>.pos": float}`` with revolute joints in degrees
        and gripper in [0, 100].

    Raises:
        RuntimeError: If not connected (enforced by decorator).
    """
    return {f"{name}.pos": self._positions[name] for name in _JOINT_NAMES}

send_action(action: RobotAction) -> RobotAction

Apply a position command and update internal state.

Parameters:

Name	Type	Description	Default
action	RobotAction	Dict {"<joint>.pos": float} in lerobot's native units.	required

Returns:

Type	Description
RobotAction	The same action dict (lerobot convention).

Raises:

Type	Description
RuntimeError	If not connected (enforced by decorator).

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_not_connected  # type: ignore[untyped-decorator]
def send_action(self, action: RobotAction) -> RobotAction:
    """Apply a position command and update internal state.

    Args:
        action: Dict ``{"<joint>.pos": float}`` in lerobot's native units.

    Returns:
        The same action dict (lerobot convention).

    Raises:
        RuntimeError: If not connected (enforced by decorator).
    """
    for name in _JOINT_NAMES:
        key = f"{name}.pos"
        if key in action:
            self._positions[name] = float(action[key])
    return action

SO100DigitalTwinConfig(*, initial_positions: dict[str, float] = (lambda: dict(_DEFAULT_POSITIONS))()) dataclass

Bases: RobotConfig

Configuration for the SO-100 digital twin.

Parameters:

Name	Type	Description	Default
initial_positions	dict[str, float]	Initial joint positions in lerobot's native units (revolute joints in degrees, gripper in [0, 100]). Defaults to all-zero revolute joints and gripper at 50 (half-open).	(lambda: dict(_DEFAULT_POSITIONS))()

Example

cfg = SO100DigitalTwinConfig(initial_positions={"shoulder_pan": 90.0}) cfg.initial_positions["shoulder_pan"] 90.0

SO100FollowerHAL(port: str = '/dev/ttyUSB0', *, calibrate_on_connect: bool = False, id: str | None = None, calibration_dir: str | None = None, max_relative_target: float | dict[str, float] | None = None, staleness_limit_s: float | None = None, robot: _LeRobotRobot | None = None, description: RobotDescription | None = None)

Bases: HALBase

HAL adapter wrapping lerobot's SO-100 follower arm.

The adapter is instantiated without a live robot connection; call connect() to open the USB serial port. No lerobot import happens at module load time so the class can be used in environments where lerobot is not installed (e.g., CI type-checking runs).

For testing without hardware, pass an already-constructed lerobot Robot instance via the robot keyword argument. The injected robot is used directly and no serial port is opened:

>>> from openral_hal.so100_sim import SO100DigitalTwin, SO100DigitalTwinConfig
>>> twin = SO100DigitalTwin(SO100DigitalTwinConfig())
>>> hal = SO100FollowerHAL(robot=twin)
>>> hal.connect()
>>> hal.description.name
'so100_follower'

Parameters:

Name	Type	Description	Default
port	str	USB serial port, e.g. "/dev/ttyUSB0". Ignored when robot is provided. Defaults to "/dev/ttyUSB0" on Linux / "/dev/cu.usbserial-0001" on macOS.	'/dev/ttyUSB0'
calibrate_on_connect	bool	If True (default False), run lerobot's interactive calibration wizard at connect() time. Set to False for automated / HIL use; the stored calibration file is used instead.	False
max_relative_target	float | dict[str, float] | None	Per-joint or scalar cap on goal-position deltas forwarded to the motor bus. None means no capping. Ignored when robot is provided.	None
staleness_limit_s	float | None	How old (seconds) a read_state() timestamp may be before ROSPerceptionStale is raised. None (default) reads the manifest's safety.joint_state_staleness_limit_s.	None
robot	Robot | None	Optional pre-constructed lerobot Robot instance. When provided, connect() calls robot.connect() directly instead of constructing a SO100Follower and opening the serial port. Intended for testing via SO100DigitalTwin.	None
description	RobotDescription | None	The loaded robots/<id>/robot.yaml manifest (threaded by build_hal). None falls back to the in-code SO100_DESCRIPTION mirror.	None

Raises:

Type	Description
ROSConfigError	At connect() time if lerobot is not installed and no robot was injected.

Initialise the adapter; does not open any connection yet.

id is lerobot's calibration identity: the stored calibration file resolves to <calibration_dir>/<id>.json. calibration_dir defaults (in lerobot) to ~/.cache/huggingface/lerobot/calibration/ robots/so_follower/; pass it to load a calibration committed alongside the deploy config instead — so the deploy owns its own calibration and never silently picks up a stale/other cached file (two so_follower/*.json for one arm is a real footgun). Without an id (and with calibrate_on_connect=False) the bus connects but every get_observation() raises has no calibration registered — pass the same id the arm was calibrated with (lerobot-calibrate --robot.id=<id>), e.g. via the deploy config's hal.params.id + hal.params.calibration_dir.

Source code in python/hal/src/openral_hal/so100_follower.py

def __init__(
    self,
    port: str = "/dev/ttyUSB0",
    *,
    calibrate_on_connect: bool = False,
    id: str | None = None,  # reason: mirrors lerobot RobotConfig.id verbatim
    calibration_dir: str | None = None,  # reason: mirrors lerobot RobotConfig.calibration_dir
    max_relative_target: float | dict[str, float] | None = None,
    staleness_limit_s: float | None = None,
    robot: _LeRobotRobot | None = None,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the adapter; does not open any connection yet.

    ``id`` is lerobot's calibration identity: the stored calibration file
    resolves to ``<calibration_dir>/<id>.json``. ``calibration_dir``
    defaults (in lerobot) to ``~/.cache/huggingface/lerobot/calibration/
    robots/so_follower/``; pass it to load a calibration committed
    alongside the deploy config instead — so the deploy owns its own
    calibration and never silently picks up a stale/other cached file
    (two ``so_follower/*.json`` for one arm is a real footgun). Without an
    ``id`` (and with ``calibrate_on_connect=False``) the bus connects but
    every ``get_observation()`` raises ``has no calibration registered`` —
    pass the same ``id`` the arm was calibrated with
    (``lerobot-calibrate --robot.id=<id>``), e.g. via the deploy config's
    ``hal.params.id`` + ``hal.params.calibration_dir``.
    """
    # The loaded manifest (build_hal threads it) wins — so101_follower
    # reuses this class and must report its own name/limits.
    self.description: RobotDescription = description or SO100_DESCRIPTION
    self._port = port
    self._calibrate_on_connect = calibrate_on_connect
    self._id = id
    self._calibration_dir = calibration_dir
    self._max_relative_target = max_relative_target
    self._staleness_limit_s = resolve_staleness_limit_s(self.description, staleness_limit_s)
    self._injected_robot: _LeRobotRobot | None = robot

    self._robot: _LeRobotRobot | None = None
    self._connected: bool = False
    self._last_obs_time: float = 0.0
    self._last_obs: dict[str, float] = {}

connect() -> None

Open the USB serial connection to the SO-100 arm.

If a robot was passed at construction time, that instance is used directly (no serial port is opened). Otherwise, a SO100Follower is constructed from port and the lerobot package is imported lazily.

Raises:

Type	Description
ROSConfigError	If lerobot is not installed and no robot was injected, or if already connected.
ROSRuntimeError	If the underlying serial open fails.

Source code in python/hal/src/openral_hal/so100_follower.py

def connect(self) -> None:
    """Open the USB serial connection to the SO-100 arm.

    If a ``robot`` was passed at construction time, that instance is used
    directly (no serial port is opened).  Otherwise, a ``SO100Follower``
    is constructed from ``port`` and the lerobot package is imported lazily.

    Raises:
        ROSConfigError: If ``lerobot`` is not installed and no robot was
            injected, or if already connected.
        ROSRuntimeError: If the underlying serial open fails.
    """
    if self._connected:
        raise ROSRuntimeError(f"SO100FollowerHAL(port={self._port!r}) is already connected.")

    if self._injected_robot is not None:
        # Use the pre-constructed robot (e.g. SO100DigitalTwin for testing).
        try:
            self._injected_robot.connect(calibrate=self._calibrate_on_connect)
        except Exception as exc:
            raise ROSRuntimeError(f"Injected robot connect() failed: {exc}") from exc
        self._robot = self._injected_robot
    else:
        try:
            from lerobot.robots.so_follower import (  # noqa: PLC0415
                SO100Follower,
                SOFollowerRobotConfig,
            )
        except ModuleNotFoundError as exc:
            raise ROSConfigError(
                "lerobot is not installed. Install it with: "
                "uv add lerobot --package openral-hal"
            ) from exc

        from pathlib import Path  # noqa: PLC0415

        cfg: _SOFollowerRobotConfig = SOFollowerRobotConfig(
            port=self._port,
            id=self._id,  # calibration identity → <calibration_dir>/<id>.json
            # None → lerobot's default HF cache dir; a path → the deploy's
            # committed calibration (see __init__).
            calibration_dir=Path(self._calibration_dir) if self._calibration_dir else None,
            max_relative_target=self._max_relative_target,
            use_degrees=True,  # adapter converts degrees ↔ radians
        )
        try:
            robot = SO100Follower(cfg)
        except ModuleNotFoundError as exc:
            # lerobot's FeetechMotorsBus pulls in `scservo_sdk` at __init__
            # time; surface a typed config error instead of leaking the raw
            # ImportError traceback.
            raise ROSConfigError(
                f"SO-100 driver dependency missing ({exc.name!r}). "
                "Install with: uv add scservo_sdk --package openral-hal"
            ) from exc
        if not os.path.exists(self._port):
            raise ROSConfigError(
                f"SO-100 serial port {self._port!r} does not exist. "
                "Connect the arm via USB, or pass --port /dev/ttyUSBn."
            )
        try:
            robot.connect(calibrate=self._calibrate_on_connect)
        except Exception as exc:
            raise ROSRuntimeError(
                f"Failed to connect to SO-100 on {self._port!r}: {exc}"
            ) from exc
        self._robot = robot

    try:
        self._preflight_servo_ping(self._robot)
    except Exception:
        with contextlib.suppress(Exception):
            self._robot.disconnect()
        self._robot = None
        raise
    self._connected = True
    self._last_obs_time = time.monotonic()
    log.info("hal.connect", robot="so100_follower", port=self._port)

disconnect() -> None

Close the USB connection, disabling motor torque. Idempotent.

Source code in python/hal/src/openral_hal/so100_follower.py

def disconnect(self) -> None:
    """Close the USB connection, disabling motor torque.  Idempotent."""
    if not self._connected:
        return
    if self._robot is not None:
        with contextlib.suppress(Exception):
            self._robot.disconnect()
    self._robot = None
    self._connected = False
    log.info("hal.disconnect", robot="so100_follower", port=self._port)

estop() -> None

Trigger an emergency stop: disconnect motors then raise.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/so100_follower.py

def estop(self) -> None:
    """Trigger an emergency stop: disconnect motors then raise.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical("hal.estop", robot="so100_follower", port=self._port)
    if self._robot is not None and self._connected:
        with contextlib.suppress(Exception):
            self._robot.disconnect()
    self._robot = None
    self._connected = False
    raise ROSEStopRequested(f"Emergency stop triggered on SO-100 at port '{self._port}'.")

read_state() -> JointState

Return the latest joint state in radians.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last observation is too old.

Returns:

Type	Description
JointState	JointState with positions in radians (gripper in [0, 1]).

Source code in python/hal/src/openral_hal/so100_follower.py

def read_state(self) -> JointState:
    """Return the latest joint state in radians.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last observation is too old.

    Returns:
        ``JointState`` with positions in radians (gripper in [0, 1]).
    """
    self._require_connected("read_state")

    assert self._robot is not None  # guaranteed by _require_connected
    try:
        obs: dict[str, object] = self._robot.get_observation()
    except Exception as exc:
        raise ROSRuntimeError(f"get_observation() failed: {exc}") from exc

    self._last_obs_time = time.monotonic()
    self._last_obs = {k: float(v) for k, v in obs.items() if isinstance(v, (int, float))}

    positions = self._obs_to_positions(self._last_obs)
    return JointState(
        name=_SO100_JOINT_NAMES,
        position=positions,
        stamp_ns=time.time_ns(),
    )

reset_to_pose(pose: list[float]) -> None

Ramp the real arm to pose with a slow linear interpolation.

The real-hardware counterpart of the sim arms' qpos snap (_mujoco_arm.reset_to_pose): the rskill_runner_node calls the HAL's /openral/<robot>/reset_to_pose service before the first inference tick so the VLA starts from its manifest starting_pose instead of wherever the arm was left. A snap is fine for a simulator; on real Feetech servos an instantaneous position command is a violent full-speed move, so this interpolates current → target at _RESET_STEP_HZ with the per-joint speed capped by _RESET_MAX_RAD_S (duration clamped to [_RESET_MIN_S, _RESET_MAX_S]).

Parameters:

Name	Type	Description	Default
pose	list[float]	Target joint positions in manifest order — radians for the five arm joints, [0, 1] for the gripper channel (the same contract as read_state / send_action).	required

Raises:

Type	Description
ROSRuntimeError	If not connected (or a bus write fails mid-ramp).
ROSConfigError	If pose length doesn't match the joint count.

Example

hal.connect()

hal.reset_to_pose([-0.115, -1.79, 1.57, 1.03, 0.10, 0.02])

Source code in python/hal/src/openral_hal/so100_follower.py

def reset_to_pose(self, pose: list[float]) -> None:
    """Ramp the real arm to ``pose`` with a slow linear interpolation.

    The real-hardware counterpart of the sim arms' ``qpos`` snap
    (``_mujoco_arm.reset_to_pose``): the ``rskill_runner_node`` calls the
    HAL's ``/openral/<robot>/reset_to_pose`` service before the first
    inference tick so the VLA starts from its manifest ``starting_pose``
    instead of wherever the arm was left. A snap is fine for a simulator;
    on real Feetech servos an instantaneous position command is a violent
    full-speed move, so this interpolates current → target at
    ``_RESET_STEP_HZ`` with the per-joint speed capped by
    ``_RESET_MAX_RAD_S`` (duration clamped to
    [``_RESET_MIN_S``, ``_RESET_MAX_S``]).

    Args:
        pose: Target joint positions in manifest order — radians for the
            five arm joints, ``[0, 1]`` for the gripper channel (the same
            contract as ``read_state`` / ``send_action``).

    Raises:
        ROSRuntimeError: If not connected (or a bus write fails mid-ramp).
        ROSConfigError: If ``pose`` length doesn't match the joint count.

    Example:
        >>> # hal.connect()
        >>> # hal.reset_to_pose([-0.115, -1.79, 1.57, 1.03, 0.10, 0.02])
    """
    self._require_connected("reset_to_pose")
    n = len(_SO100_JOINT_NAMES)
    if len(pose) != n:
        raise ROSConfigError(
            f"reset_to_pose expects {n} values (SO-100 joint order); got {len(pose)}."
        )

    current = self.read_state().position
    deltas = [t - c for t, c in zip(pose, current, strict=True)]
    max_delta = max(abs(d) for d in deltas)
    if max_delta < _RESET_DEADBAND_RAD:
        return  # already there — no motion

    # ponytail: fixed-rate linear ramp; per-joint trapezoids if anyone
    # ever needs smoother-than-gentle. The gripper channel is [0, 1] not
    # radians, so the shared speed cap treats a full gripper stroke like
    # a 1-rad joint move — conservative, never fast.
    duration_s = min(max(max_delta / _RESET_MAX_RAD_S, _RESET_MIN_S), _RESET_MAX_S)
    steps = max(int(duration_s * _RESET_STEP_HZ), 1)
    log.info(
        "hal.reset_to_pose.ramp",
        robot="so100_follower",
        max_delta_rad=round(max_delta, 3),
        duration_s=round(duration_s, 2),
        steps=steps,
    )
    assert self._robot is not None  # guaranteed by _require_connected
    for i in range(1, steps + 1):
        alpha = i / steps
        waypoint = [c + d * alpha for c, d in zip(current, deltas, strict=True)]
        try:
            self._robot.send_action(_joint_values_to_lerobot(waypoint))
        except Exception as exc:
            raise ROSRuntimeError(f"reset_to_pose send_action failed: {exc}") from exc
        time.sleep(1.0 / _RESET_STEP_HZ)

send_action(action: Action) -> None

Forward one action step to the SO-100 motor bus.

Only the first step of the action chunk is sent; lerobot handles single-step commands. Action chunks are executed by calling send_action repeatedly in the skill executor's async loop.

Parameters:

Name	Type	Description	Default
action	Action	The Action produced by a Skill. Must use ControlMode.JOINT_POSITION with joint_targets set.	required

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If the action cannot be converted.

Source code in python/hal/src/openral_hal/so100_follower.py

def send_action(self, action: Action) -> None:
    """Forward one action step to the SO-100 motor bus.

    Only the **first step** of the action chunk is sent; lerobot handles
    single-step commands.  Action chunks are executed by calling
    ``send_action`` repeatedly in the skill executor's async loop.

    Args:
        action: The ``Action`` produced by a Skill.  Must use
            ``ControlMode.JOINT_POSITION`` with ``joint_targets`` set.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If the action cannot be converted.
    """
    self._require_connected("send_action")
    lerobot_action = self._action_to_lerobot(action)
    assert self._robot is not None
    try:
        self._robot.send_action(lerobot_action)
    except Exception as exc:
        raise ROSRuntimeError(f"send_action() failed: {exc}") from exc

SO100MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the SO-100 follower arm (MuJoCo-backed simulation).

The HAL exposes the canonical 6 SO-100 joints (5 arm + 1 gripper) on its public surface, using the lerobot-style names from openral_hal.SO100_DESCRIPTION. Internally it drives the Menagerie MJCF's 6 position actuators (Rotation, Pitch, Elbow, Wrist_Pitch, Wrist_Roll, Jaw) and maps the revolute Jaw range [-0.174, 1.75] rad to a normalised [0, 1] gripper channel — so the same 6-DoF action chunk drives the sim twin and the real hardware HAL (openral_hal.SO100FollowerHAL) identically.

The MJCF position limits are tighter than the conservative [-π, π] declared on openral_hal.SO100_DESCRIPTION; commands outside the MJCF range are clipped by MuJoCo's own position controllers. Tests should command targets inside the menagerie range (e.g. Rotation is [-1.92, 1.92]).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/trs_so_arm100/so_arm100.xml).	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import SO100MujocoHAL # doctest: +SKIP hal = SO100MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 5 arm + 1 gripper # doctest: +SKIP 6 hal.disconnect() # doctest: +SKIP

Initialise the SO-100 MuJoCo HAL; no MuJoCo state is created until connect().

All wiring (MJCF URI, joint indices, Jaw affine_low_high gripper read mode) lives in SO100_DESCRIPTION.sim.

Source code in python/hal/src/openral_hal/so100_mujoco.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the SO-100 MuJoCo HAL; no MuJoCo state is created until ``connect()``.

    All wiring (MJCF URI, joint indices, Jaw ``affine_low_high`` gripper
    read mode) lives in ``SO100_DESCRIPTION.sim``.
    """
    self._init_from_description(
        SO100_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

SawyerRealHAL(*, hostname: str = 'sawyer.local', controller_name: str = _DEFAULT_SAWYER_CONTROLLER, joint_state_topic: str = _DEFAULT_SAWYER_JOINT_STATE_TOPIC, command_topic: str | None = None, estop_topic: str = _DEFAULT_SAWYER_ESTOP_TOPIC, publish_fn: _PublishFn | None = None, state_fn: _StateFn | None = None, staleness_limit_s: float | None = None, description: RobotDescription | None = None)

Bases: RosControlHAL

HAL adapter for a physical Rethink Sawyer over intera_sdk / ROS 2.

A RosControlHAL subclass (the same shape as the UR adapters) pinning the sawyer_robot controller name, the legacy intera joint-state topic and the vendor halt topic. Subclassing rather than wrapping is what makes this adapter structurally RosControlDrivable, so the lifecycle node attaches the production RosControlTransport to it under hal_mode:=real — a composed wrapper exposed none of that surface and was never wired, so a real Sawyer deploy published into a no-op logger.

Lifecycle e-stop. estop() first deactivates sawyer_arm_controller through controller_manager (the generic RosControlHAL stop), then publishes std_msgs/Empty on estop_topic — intera's super stop, the software equivalent of the pendant e-stop — on a publisher the transport created at wire-up. The topic publish carries no service acknowledgement; the acknowledged part of the report is the controller deactivation, and the HIL gate reads /robot/state.stopped for the vendor half. Recovery is RESTART_REQUIRED: the operator publishes /robot/set_super_reset, re-enables the robot, restarts the HAL lifecycle node and re-aligns.

Parameters:

Name	Type	Description	Default
hostname	str	Hostname of the Sawyer's onboard PC, typically "sawyer.local" on a lab subnet. Required; the upstream intera_sdk (and sawyer_robot fork) refuses to connect without it. Stored as metadata only — the actual TCP connection is opened by the lifecycle node.	'sawyer.local'
controller_name	str	Name of the ros2_control joint trajectory controller exported by sawyer_robot. Defaults to "sawyer_arm_controller".	_DEFAULT_SAWYER_CONTROLLER
joint_state_topic	str	ROS 2 topic publishing sensor_msgs/JointState. Defaults to "/robot/joint_states" (the legacy intera_sdk topic name).	_DEFAULT_SAWYER_JOINT_STATE_TOPIC
command_topic	str | None	ROS 2 topic for joint trajectory commands. Defaults to "/<controller_name>/joint_trajectory".	None
estop_topic	str	std_msgs/Empty topic estop() publishes on after deactivating the controller. Defaults to "/robot/set_super_stop" (intera's super stop).	_DEFAULT_SAWYER_ESTOP_TOPIC
publish_fn	_PublishFn | None	Callable forwarding messages to ROS 2 topics. Production use injects the lifecycle node's publisher; tests inject SimTransport.publish.	None
state_fn	_StateFn | None	Callable returning the latest raw joint state as a dict. Production use injects the lifecycle node's subscriber callback; tests inject SimTransport.state.	None
staleness_limit_s	float | None	Maximum age of a read_state() reading before ROSPerceptionStale is raised. None (default) reads the manifest's safety.joint_state_staleness_limit_s.	None
description	RobotDescription | None	The loaded robots/<id>/robot.yaml manifest (threaded by build_hal). None falls back to the in-code SAWYER_REAL_DESCRIPTION mirror.	None

Raises:

Type	Description
ROSConfigError	If hostname is empty / whitespace.

Example

from openral_hal.sawyer_real import SawyerRealHAL from openral_hal.sim_transport import SimTransport transport = SimTransport(n_joints=8) hal = SawyerRealHAL( ... hostname="sawyer.local", ... publish_fn=transport.publish, ... state_fn=transport.state, ... ) hal.connect() hal.description.name 'sawyer' hal.disconnect()

Initialise the adapter; no TCP connection is opened until connect().

Source code in python/hal/src/openral_hal/sawyer_real.py

def __init__(
    self,
    *,
    hostname: str = "sawyer.local",
    controller_name: str = _DEFAULT_SAWYER_CONTROLLER,
    joint_state_topic: str = _DEFAULT_SAWYER_JOINT_STATE_TOPIC,
    command_topic: str | None = None,
    estop_topic: str = _DEFAULT_SAWYER_ESTOP_TOPIC,
    publish_fn: _PublishFn | None = None,
    state_fn: _StateFn | None = None,
    staleness_limit_s: float | None = None,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the adapter; no TCP connection is opened until ``connect()``."""
    if not hostname or not hostname.strip():
        raise ROSConfigError(
            "SawyerRealHAL requires a non-empty hostname "
            "(e.g. 'sawyer.local' or the robot's IP)."
        )
    super().__init__(
        description or SAWYER_REAL_DESCRIPTION,
        controller_name=controller_name,
        joint_state_topic=joint_state_topic,
        command_topic=command_topic,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )
    self._hostname = hostname
    self._estop_topic = estop_topic

estop_topic: str property

The std_msgs/Empty super-stop topic the vendor stop publishes on.

hostname: str property

Hostname / IP of the Sawyer's onboard PC.

connect() -> None

Open the ROS 2 transport to the sawyer_robot controller.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/sawyer_real.py

def connect(self) -> None:
    """Open the ROS 2 transport to the ``sawyer_robot`` controller.

    Raises:
        ROSRuntimeError: If already connected.
    """
    log.info(
        "hal.connect",
        robot=self.description.name,
        hostname=self._hostname,
        controller=self._controller_name,
    )
    super().connect()

vendor_stop_topics() -> list[str]

The intera super-stop topic, so the transport creates its publisher at wire-up.

Source code in python/hal/src/openral_hal/sawyer_real.py

def vendor_stop_topics(self) -> list[str]:
    """The intera super-stop topic, so the transport creates its publisher at wire-up."""
    return [self._estop_topic]

SimTransport(n_joints: int, *, controllers: Sequence[str] | None = None, trigger_responses: Mapping[str, tuple[bool, str]] | None = None, switch_faults: Mapping[str, Exception] | None = None)

In-memory transport simulating a ros2_control joint trajectory controller.

Published joint trajectory commands update internal joint positions, which are then returned by state. All published messages are recorded for assertion in tests.

Parameters:

Name	Type	Description	Default
n_joints	int	Number of joints to simulate. Initial positions, velocities, and efforts are all 0.0.	required
controllers	Sequence[str] | None	The controller_manager table. When given, every switch is checked against it the way a STRICT switch is — a name not in the table is refused, exactly like an unloaded controller. When omitted, controllers are registered active the first time a topic or switch names them.	None
trigger_responses	Mapping[str, tuple[bool, str]] | None	Per-service (success, message) a call_trigger returns, to simulate e.g. a UR dashboard whose stop refuses. Unlisted services succeed.	None
switch_faults	Mapping[str, Exception] | None	Per-operation ("deactivate" / "activate") exception the switch raises instead of answering — the rclpy-shaped fault (executor or context shut down, a dead client handle) a real controller_manager call can die with. The HAL must contain it in its stop report.	None

Example

transport = SimTransport(n_joints=2) transport.publish( ... "/ctrl/traj", ... { ... "joint_targets": [[0.5, -0.5]], ... "control_mode": "joint_position", ... "horizon": 1, ... "stamp_ns": 0, ... }, ... ) tuple(transport.state()["position"]) (0.5, -0.5) transport.call_count 1 topic, msg = transport.last_call # type: ignore[misc] topic '/ctrl/traj'

Initialise zeroed joint state for n_joints joints.

Source code in python/hal/src/openral_hal/sim_transport.py

def __init__(
    self,
    n_joints: int,
    *,
    controllers: Sequence[str] | None = None,
    trigger_responses: Mapping[str, tuple[bool, str]] | None = None,
    switch_faults: Mapping[str, Exception] | None = None,
) -> None:
    """Initialise zeroed joint state for *n_joints* joints."""
    self._n_joints = n_joints
    self._positions: list[float] = [0.0] * n_joints
    self._velocities: list[float] = [0.0] * n_joints
    self._efforts: list[float] = [0.0] * n_joints
    self._published: list[tuple[str, dict[str, object]]] = []
    self._dropped: list[tuple[str, dict[str, object]]] = []
    self._strict = controllers is not None
    self._controller_states: dict[str, str] = {name: "active" for name in controllers or ()}
    self._switches: list[tuple[str, tuple[str, ...]]] = []
    self._trigger_responses = dict(trigger_responses or {})
    self._switch_faults = dict(switch_faults or {})
    self._triggers: list[str] = []
    self._empties: list[str] = []

call_count: int property

Number of times publish delivered a command to an active controller.

calls: list[tuple[str, dict[str, object]]] property

All delivered (topic, msg) pairs in chronological order.

dropped_calls: list[tuple[str, dict[str, object]]] property

Every command that reached an inactive controller and was dropped.

empty_publishes: list[str] property

Every std_msgs/Empty topic published on, in order.

last_call: tuple[str, dict[str, object]] | None property

The most recent delivered (topic, msg) pair, or None.

switch_calls: list[tuple[str, tuple[str, ...]]] property

Every ("deactivate" | "activate", names) switch, in order.

trigger_calls: list[str] property

Every std_srvs/Trigger service called, in order.

activate_controllers(names: Sequence[str], *, timeout_s: float) -> ControllerSwitchReport

Flip every named controller to active (STRICT: unknown names refuse).

Source code in python/hal/src/openral_hal/sim_transport.py

def activate_controllers(
    self, names: Sequence[str], *, timeout_s: float
) -> ControllerSwitchReport:
    """Flip every named controller to ``active`` (STRICT: unknown names refuse)."""
    return self._switch("activate", names, "active")

call_trigger(service: str, *, timeout_s: float) -> TriggerReport

Record the call and answer from trigger_responses (default: success).

Source code in python/hal/src/openral_hal/sim_transport.py

def call_trigger(self, service: str, *, timeout_s: float) -> TriggerReport:
    """Record the call and answer from ``trigger_responses`` (default: success)."""
    self._triggers.append(service)
    success, message = self._trigger_responses.get(service, (True, "ok"))
    return TriggerReport(success=success, message=message)

controller_state(name: str) -> str

active / inactive / unloaded for one controller.

Source code in python/hal/src/openral_hal/sim_transport.py

def controller_state(self, name: str) -> str:
    """``active`` / ``inactive`` / ``unloaded`` for one controller."""
    return self._controller_states.get(name, "unloaded" if self._strict else "active")

deactivate_controllers(names: Sequence[str], *, timeout_s: float) -> ControllerSwitchReport

Flip every named controller to inactive (STRICT: unknown names refuse).

Source code in python/hal/src/openral_hal/sim_transport.py

def deactivate_controllers(
    self, names: Sequence[str], *, timeout_s: float
) -> ControllerSwitchReport:
    """Flip every named controller to ``inactive`` (STRICT: unknown names refuse)."""
    return self._switch("deactivate", names, "inactive")

publish(topic: str, msg: dict[str, object]) -> None

Record msg and apply joint_targets to internal state.

If msg["joint_targets"] is a list of trajectory steps, the last step is applied to positions -- mirroring how a real joint trajectory controller would reach the final waypoint. A command on a topic whose controller is inactive is recorded in dropped_calls and not applied — what a deactivated JointTrajectoryController does with a late trajectory.

Parameters:

Name	Type	Description	Default
topic	str	The ROS 2 topic name.	required
msg	dict[str, object]	The published message dict.	required

Source code in python/hal/src/openral_hal/sim_transport.py

def publish(self, topic: str, msg: dict[str, object]) -> None:
    """Record *msg* and apply ``joint_targets`` to internal state.

    If ``msg["joint_targets"]`` is a list of trajectory steps, the **last
    step** is applied to positions -- mirroring how a real joint trajectory
    controller would reach the final waypoint. A command on a topic whose
    controller is ``inactive`` is recorded in ``dropped_calls`` and **not**
    applied — what a deactivated ``JointTrajectoryController`` does with a
    late trajectory.

    Args:
        topic: The ROS 2 topic name.
        msg: The published message dict.
    """
    if self._state_of(_controller_of(topic)) != "active":
        self._dropped.append((topic, msg))
        return
    self._published.append((topic, msg))
    targets = msg.get("joint_targets")
    if isinstance(targets, list) and targets:
        last_step = targets[-1]
        if isinstance(last_step, list):
            self._positions = [float(v) for v in last_step]

publish_empty(topic: str) -> None

Record one std_msgs/Empty publish.

Source code in python/hal/src/openral_hal/sim_transport.py

def publish_empty(self, topic: str) -> None:
    """Record one ``std_msgs/Empty`` publish."""
    self._empties.append(topic)

state() -> dict[str, object]

Return the current simulated joint state.

Returns:

Type	Description
dict[str, object]	Dict with "position", "velocity", and "effort" keys,
dict[str, object]	each containing a list[float] of length n_joints.

Source code in python/hal/src/openral_hal/sim_transport.py

def state(self) -> dict[str, object]:
    """Return the current simulated joint state.

    Returns:
        Dict with ``"position"``, ``"velocity"``, and ``"effort"`` keys,
        each containing a ``list[float]`` of length ``n_joints``.
    """
    return {
        "position": list(self._positions),
        "velocity": list(self._velocities),
        "effort": list(self._efforts),
    }

TriggerReport(success: bool, message: str = '') dataclass

Acknowledged outcome of one std_srvs/Trigger-shaped vendor call.

UR10eHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Universal Robots UR10e (MuJoCo-backed simulation).

Args mirror UR5eHAL; the only difference is the underlying MJCF and RobotDescription (different velocity / effort envelopes), both of which now live in UR10e_DESCRIPTION.

Example

from openral_hal import UR10eHAL # doctest: +SKIP hal = UR10eHAL(gravity_enabled=False) # doctest: +SKIP

Initialise the UR10e HAL; no MuJoCo state is created until connect().

Source code in python/hal/src/openral_hal/ur.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the UR10e HAL; no MuJoCo state is created until ``connect()``."""
    self._init_from_description(
        UR10e_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

UR10eRealHAL(*, robot_ip: str | None = None, publish_fn: Callable[[str, dict[str, object]], None] | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float | None = None, description: RobotDescription | None = None)

Bases: _URRealHAL

Real-hardware HAL adapter for the Universal Robots UR10e.

Identical to UR5eRealHAL except for the wrapped RobotDescription (12.5 kg payload, 1.30 m reach, larger torques) — ur_robot_driver itself is the same binary for both arms, differentiated by the URDF and per-joint envelope.

Args mirror UR5eRealHAL.

Example::

>>> from openral_hal.sim_transport import SimTransport
>>> from openral_hal.ur_real import UR10eRealHAL
>>> transport = SimTransport(n_joints=6)
>>> hal = UR10eRealHAL(
...     publish_fn=transport.publish,
...     state_fn=transport.state,
... )
>>> hal.description.name
'ur10e'
>>> hal.description.hal.real
'openral_hal.ur_real:UR10eRealHAL'

Initialise the UR10e real-HW HAL; transport defaults match ur_robot_driver.

description is the loaded manifest (threaded by build_hal); None falls back to UR10e_REAL_DESCRIPTION.

Source code in python/hal/src/openral_hal/ur_real.py

def __init__(
    self,
    *,
    robot_ip: str | None = None,
    publish_fn: Callable[[str, dict[str, object]], None] | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float | None = None,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the UR10e real-HW HAL; transport defaults match ``ur_robot_driver``.

    ``description`` is the loaded manifest (threaded by ``build_hal``);
    ``None`` falls back to ``UR10e_REAL_DESCRIPTION``.
    """
    super().__init__(
        description or UR10e_REAL_DESCRIPTION,
        robot_ip=robot_ip,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )

UR5eHAL(*, mjcf_path: str | None = None, settle_steps: int | None = None, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Universal Robots UR5e (MuJoCo-backed simulation).

Thin manifest-driven wrapper — every wiring constant lives in UR5e_DESCRIPTION.sim.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, UR5e_DESCRIPTION.assets.mjcf is resolved at construction time (robot_descriptions:ur5e_mj_description).	None
settle_steps	int | None	Number of MuJoCo physics steps performed in send_action. None (default) derives one control period of description.control_rate_hz at connect (issue #358); an explicit count is a pinned, logged override.	None
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import UR5eHAL # doctest: +SKIP hal = UR5eHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP hal.disconnect() # doctest: +SKIP

Initialise the UR5e HAL; no MuJoCo state is created until connect().

Source code in python/hal/src/openral_hal/ur.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int | None = None,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the UR5e HAL; no MuJoCo state is created until ``connect()``."""
    self._init_from_description(
        UR5e_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

UR5eRealHAL(*, robot_ip: str | None = None, publish_fn: Callable[[str, dict[str, object]], None] | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float | None = None, description: RobotDescription | None = None)

Bases: _URRealHAL

Real-hardware HAL adapter for the Universal Robots UR5e.

Drives a real UR5e via ros2_control + ur_robot_driver (URCap + RTDE). The Python adapter itself contains no rclpy import — the transport is injected, so the same class drives the real arm in production and a SimTransport in unit tests.

Parameters:

Name	Type	Description	Default
robot_ip	str | None	Static IP of the UR5e controller, e.g. "192.168.1.42". Recorded for observability; the driver itself must have been launched against this IP separately.	None
publish_fn	Callable[[str, dict[str, object]], None] | None	Optional injected publish callable for tests.	None
state_fn	Callable[[], dict[str, object]] | None	Optional injected state-read callable for tests.	None
staleness_limit_s	float | None	Maximum age of a cached state.	None

Example::

>>> from openral_hal.sim_transport import SimTransport
>>> from openral_hal.ur_real import UR5eRealHAL
>>> transport = SimTransport(n_joints=6)
>>> hal = UR5eRealHAL(
...     publish_fn=transport.publish,
...     state_fn=transport.state,
... )
>>> hal.description.name
'ur5e'
>>> hal.description.sdk_kind
'closed'
>>> hal.description.hal.real
'openral_hal.ur_real:UR5eRealHAL'

Initialise the UR5e real-HW HAL; transport defaults match ur_robot_driver.

description is the loaded manifest (threaded by build_hal); None falls back to UR5e_REAL_DESCRIPTION.

Source code in python/hal/src/openral_hal/ur_real.py

def __init__(
    self,
    *,
    robot_ip: str | None = None,
    publish_fn: Callable[[str, dict[str, object]], None] | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float | None = None,
    description: RobotDescription | None = None,
) -> None:
    """Initialise the UR5e real-HW HAL; transport defaults match ``ur_robot_driver``.

    ``description`` is the loaded manifest (threaded by ``build_hal``);
    ``None`` falls back to ``UR5e_REAL_DESCRIPTION``.
    """
    super().__init__(
        description or UR5e_REAL_DESCRIPTION,
        robot_ip=robot_ip,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )

build_hal(description: RobotDescription, *, mode: HalMode, transport: dict[str, object] | None = None, sim_env_yaml: str | None = None) -> HAL

Construct the simulation or real-hardware HAL for description.

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot manifest (typically loaded via RobotDescription.from_yaml).	required
mode	HalMode	"sim" for the simulation HAL (deploy sim / sim run harness), "real" for the real-hardware HAL (deploy run).	required
transport	dict[str, object] | None	Constructor kwargs for the real HAL (serial port, robot_ip, fci_ip, …). Keys the target constructor does not accept are dropped. Ignored by the derived sim path. Merged over the manifest's hal.parameters.defaults, so an explicit deploy run transport override wins.	None
sim_env_yaml	str | None	Path to a SimScene YAML. When provided with mode="sim", returns a SimAttachedHAL wrapping the scene's SimRollout; bypasses the bare-twin / hal.sim class. Mutually exclusive with mode="real" — raises ROSConfigError if both are supplied.	None

Returns:

Type	Description
HAL	An un-connected HAL instance.

Raises:

Type	Description
ROSCapabilityMismatch	The robot has no HAL for mode (sim-only robot asked for "real", or real-only robot asked for "sim" with no sim: block to derive from).
ROSConfigError	mode is invalid, sim_env_yaml is supplied with mode="real", or a declared entrypoint is malformed / unresolvable.

Example

from openral_core import RobotDescription desc = RobotDescription.from_yaml("robots/so100_follower/robot.yaml") # doctest: +SKIP hal = build_hal(desc, mode="sim") # doctest: +SKIP

Source code in python/hal/src/openral_hal/resolver.py

def build_hal(
    description: RobotDescription,
    *,
    mode: HalMode,
    transport: dict[str, object] | None = None,
    sim_env_yaml: str | None = None,
) -> HAL:
    """Construct the simulation or real-hardware HAL for ``description``.

    Args:
        description: The robot manifest (typically loaded via
            ``RobotDescription.from_yaml``).
        mode: ``"sim"`` for the simulation HAL (``deploy sim`` / ``sim run``
            harness), ``"real"`` for the real-hardware HAL (``deploy run``).
        transport: Constructor kwargs for the real HAL (serial ``port``,
            ``robot_ip``, ``fci_ip``, …). Keys the target constructor does not
            accept are dropped. Ignored by the derived sim path. Merged
            **over** the manifest's ``hal.parameters.defaults``, so
            an explicit ``deploy run`` transport override wins.
        sim_env_yaml: Path to a SimScene YAML. When
            provided with ``mode="sim"``, returns a
            ``SimAttachedHAL`` wrapping the
            scene's ``SimRollout``; bypasses the
            bare-twin / ``hal.sim`` class. Mutually exclusive with
            ``mode="real"`` — raises ``ROSConfigError``
            if both are supplied.

    Returns:
        An un-connected HAL instance.

    Raises:
        ROSCapabilityMismatch: The robot has no HAL for ``mode`` (sim-only
            robot asked for ``"real"``, or real-only robot asked for
            ``"sim"`` with no ``sim:`` block to derive from).
        ROSConfigError: ``mode`` is invalid, ``sim_env_yaml`` is supplied
            with ``mode="real"``, or a declared entrypoint is malformed /
            unresolvable.

    Example:
        >>> from openral_core import RobotDescription
        >>> desc = RobotDescription.from_yaml("robots/so100_follower/robot.yaml")  # doctest: +SKIP
        >>> hal = build_hal(desc, mode="sim")  # doctest: +SKIP
    """
    # The manifest's hal.parameters.defaults supply the HAL's
    # construction kwargs (serial port, robot_ip, …) so a parameterised robot
    # needs no bespoke lifecycle subclass. Explicit ``transport`` overrides
    # them; _construct() then drops any key the constructor does not accept.
    # The joint-state window is a typed safety field, not a free-form default
    # (the schema refuses both); it reaches every HAL that takes
    # `staleness_limit_s`, sim-derived included, as before.
    staleness = description.safety.joint_state_staleness_limit_s
    resolved = {
        **description.hal.parameters.defaults,
        **({} if staleness is None else {"staleness_limit_s": staleness}),
        **(transport or {}),
    }
    if sim_env_yaml is not None and mode != "sim":
        raise ROSConfigError(
            "build_hal: sim_env_yaml is only valid with mode='sim' "
            f"(got mode={mode!r}); a real-hardware HAL never attaches a sim scene."
        )
    if mode == "sim":
        if sim_env_yaml is not None:
            # Deploy sim attaches the scene's SimRollout behind a
            # SimAttachedHAL (the scene owns physics + pixels). Bypasses the
            # bare twin / hal.sim class. openral_sim imported lazily so
            # openral_hal stays import-safe without the sim group.
            from openral_hal.sim_attached import SimAttachedHAL
            from openral_hal.sim_bringup import build_sim_env_from_yaml

            env, seed = build_sim_env_from_yaml(sim_env_yaml, robot_id_fallback=description.name)
            return SimAttachedHAL(env, description, env_reset_seed=seed)
        entry = description.hal.sim
        if entry is None:
            if description.sim is None:
                raise ROSCapabilityMismatch(
                    f"robot {description.name!r} has no simulation HAL: hal.sim is "
                    "null and there is no `sim:` block to derive MujocoArmHAL from. "
                    "It is real-hardware-only — use `deploy run`."
                )
            # Same transport as the class-based HALs, under one renamed key:
            # `deploy sim` composes a scene MJCF via
            # openral_hal.lifecycle._compose_scene_mjcf as `mjcf_path`, but
            # from_description spells it `mjcf_path_override`. Must be
            # renamed, not dropped — so100/so101 (`hal.sim: null`) need the
            # composed scene.
            derived = dict(resolved)
            if "mjcf_path" in derived:
                derived["mjcf_path_override"] = derived.pop("mjcf_path")
            return _construct(MujocoArmHAL.from_description, description, derived)
        return _construct(_import_object(entry), description, resolved)
    if mode == "real":
        entry = description.hal.real
        if entry is None:
            raise ROSCapabilityMismatch(
                f"robot {description.name!r} has no real-hardware HAL (hal.real is "
                "null); it is simulation-only — use `deploy sim` / `sim run`."
            )
        return _construct(_import_object(entry), description, resolved)
    raise ROSConfigError(f"build_hal: unknown mode {mode!r}; expected 'sim' or 'real'.")

franka_panda_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of FRANKA_PANDA_DESCRIPTION with catalog sensors attached.

The Franka Panda research community reference setup uses a wrist-mounted RealSense D435i; pass None to get that default, or override.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the Franka reference loadout (["intel/realsense_d435i"]).	None

Returns:

Type	Description
RobotDescription	A new RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = franka_panda_with_sensors() desc.sensor_bundles[0].sensors[0].vendor 'Intel'

Source code in python/hal/src/openral_hal/franka_panda.py

def franka_panda_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of ``FRANKA_PANDA_DESCRIPTION`` with catalog sensors attached.

    The Franka Panda research community reference setup uses a wrist-mounted
    RealSense D435i; pass ``None`` to get that default, or override.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the Franka
            reference loadout (``["intel/realsense_d435i"]``).

    Returns:
        A new ``RobotDescription`` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = franka_panda_with_sensors()
        >>> desc.sensor_bundles[0].sensors[0].vendor
        'Intel'
    """
    if catalog_ids is None:
        catalog_ids = ["intel/realsense_d435i"]
    return with_sensors(FRANKA_PANDA_DESCRIPTION, catalog_ids)

hal_joint_states_topic(description: RobotDescription, *, mode: HalMode, hal_node_name: str, override: str | None = None) -> str | None

The JointState topic the deploy runtime's Python nodes should read.

A real ros2_control arm's global /joint_states is the joint_state_broadcaster's full-rate stream (0.5-1 kHz); every message wakes the runtime's Python executor and starves an in-process VLA of the GIL. Every HAL lifecycle node already republishes its state rate-limited on ~/joint_states (at action_spec.control_freq_hz), so a real HAL that HALLifecycleNodeBase wires to a RosControlTransport points the runtime there. Every other HAL (serial SO-100, interbotix ALOHA, any sim HAL) publishes /joint_states itself at its own rate and is left alone.

Membership mirrors _attach_ros_control_transport: mode == "real" and the manifest's hal.real class exposes every RosControlDrivable member, checked on the class without constructing it (construction may open a bus).

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot manifest.	required
mode	HalMode	"sim" or "real", as passed to build_hal.	required
hal_node_name	str	The HAL lifecycle node's name (openral_hal_<robot_id>).	required
override	str | None	DeployRuntime.joint_states_topic; wins when set.	None

Returns:

Type	Description
str | None	The topic, or None for the default /joint_states.

Raises:

Type	Description
ROSConfigError	hal.real is a malformed or unimportable entrypoint.

Example

from openral_core import RobotDescription desc = RobotDescription.from_yaml("robots/ur5e/robot.yaml") # doctest: +SKIP node = "openral_hal_ur5e" hal_joint_states_topic(desc, mode="real", hal_node_name=node) # doctest: +SKIP '/openral_hal_ur5e/joint_states'

Source code in python/hal/src/openral_hal/resolver.py

def hal_joint_states_topic(
    description: RobotDescription,
    *,
    mode: HalMode,
    hal_node_name: str,
    override: str | None = None,
) -> str | None:
    """The ``JointState`` topic the deploy runtime's Python nodes should read.

    A real ros2_control arm's global ``/joint_states`` is the
    ``joint_state_broadcaster``'s full-rate stream (0.5-1 kHz); every message
    wakes the runtime's Python executor and starves an in-process VLA of the
    GIL. Every HAL lifecycle node already republishes its state rate-limited on
    ``~/joint_states`` (at ``action_spec.control_freq_hz``), so a real HAL that
    ``HALLifecycleNodeBase`` wires to a ``RosControlTransport`` points the
    runtime there. Every other HAL (serial SO-100, interbotix ALOHA, any sim
    HAL) publishes ``/joint_states`` itself at its own rate and is left alone.

    Membership mirrors ``_attach_ros_control_transport``: ``mode == "real"``
    and the manifest's ``hal.real`` class exposes every
    ``RosControlDrivable`` member, checked on the class without constructing
    it (construction may open a bus).

    Args:
        description: The robot manifest.
        mode: ``"sim"`` or ``"real"``, as passed to ``build_hal``.
        hal_node_name: The HAL lifecycle node's name (``openral_hal_<robot_id>``).
        override: ``DeployRuntime.joint_states_topic``; wins when set.

    Returns:
        The topic, or ``None`` for the default ``/joint_states``.

    Raises:
        ROSConfigError: ``hal.real`` is a malformed or unimportable entrypoint.

    Example:
        >>> from openral_core import RobotDescription
        >>> desc = RobotDescription.from_yaml("robots/ur5e/robot.yaml")  # doctest: +SKIP
        >>> node = "openral_hal_ur5e"
        >>> hal_joint_states_topic(desc, mode="real", hal_node_name=node)  # doctest: +SKIP
        '/openral_hal_ur5e/joint_states'
    """
    if override:
        return override
    if mode != "real" or description.hal.real is None:
        return None
    from openral_hal.ros_control_transport import RosControlDrivable

    cls = _import_object(description.hal.real)
    members: frozenset[str] = RosControlDrivable.__protocol_attrs__  # type: ignore[attr-defined]  # reason: typing's 3.12 protocol member set, no public accessor before 3.13
    if all(hasattr(cls, m) for m in members):
        return f"/{hal_node_name}/joint_states"
    return None

so100_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of SO100_DESCRIPTION with catalog sensors attached.

The reference LeRobot SO-100 setup uses a single Logitech C920 scene camera; pass None to get that default, or pass an explicit list of catalog ids ("logitech/c920", "intel/realsense_d405", …) to override. Pass [] to get an empty sensor loadout.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the LeRobot reference loadout (["logitech/c920"]).	None

Returns:

Type	Description
RobotDescription	A new RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = so100_with_sensors() desc.sensors[0].vendor 'Logitech'

Source code in python/hal/src/openral_hal/so100_follower.py

def so100_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of ``SO100_DESCRIPTION`` with catalog sensors attached.

    The reference LeRobot SO-100 setup uses a single Logitech C920 scene
    camera; pass ``None`` to get that default, or pass an explicit list of
    catalog ids (``"logitech/c920"``, ``"intel/realsense_d405"``, …) to
    override.  Pass ``[]`` to get an empty sensor loadout.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the LeRobot
            reference loadout (``["logitech/c920"]``).

    Returns:
        A new ``RobotDescription`` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = so100_with_sensors()
        >>> desc.sensors[0].vendor
        'Logitech'
    """
    if catalog_ids is None:
        catalog_ids = ["logitech/c920"]
    return with_sensors(SO100_DESCRIPTION, catalog_ids)

ur10e_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of UR10e_DESCRIPTION with catalog sensors attached.

Default loadout matches ur5e_with_sensors (D435 + FT-300S); the UR10e shares the same flange interface and is usually paired with the higher-range D435 instead of the D415.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the UR10e reference loadout (["intel/realsense_d435", "robotiq/ft_300s"]).	None

Returns:

Type	Description
RobotDescription	A new RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = ur10e_with_sensors() desc.sensor_bundles[0].sensors[0].vendor 'Intel'

Source code in python/hal/src/openral_hal/ur.py

def ur10e_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of ``UR10e_DESCRIPTION`` with catalog sensors attached.

    Default loadout matches ``ur5e_with_sensors`` (D435 + FT-300S); the
    UR10e shares the same flange interface and is usually paired with the
    higher-range D435 instead of the D415.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the UR10e
            reference loadout (``["intel/realsense_d435", "robotiq/ft_300s"]``).

    Returns:
        A new ``RobotDescription`` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = ur10e_with_sensors()
        >>> desc.sensor_bundles[0].sensors[0].vendor
        'Intel'
    """
    if catalog_ids is None:
        catalog_ids = ["intel/realsense_d435", "robotiq/ft_300s"]
    return with_sensors(UR10e_DESCRIPTION, catalog_ids)

ur5e_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of UR5e_DESCRIPTION with catalog sensors attached.

The reference UR5e research setup uses a flange-mounted RealSense D415 plus a Robotiq FT-300S wrist sensor; pass None to get that default.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the UR5e reference loadout (["intel/realsense_d415", "robotiq/ft_300s"]).	None

Returns:

Type	Description
RobotDescription	A new RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = ur5e_with_sensors() desc.sensors[0].vendor 'Robotiq'

Source code in python/hal/src/openral_hal/ur.py

def ur5e_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of ``UR5e_DESCRIPTION`` with catalog sensors attached.

    The reference UR5e research setup uses a flange-mounted RealSense D415
    plus a Robotiq FT-300S wrist sensor; pass ``None`` to get that default.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the UR5e
            reference loadout (``["intel/realsense_d415", "robotiq/ft_300s"]``).

    Returns:
        A new ``RobotDescription`` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = ur5e_with_sensors()
        >>> desc.sensors[0].vendor
        'Robotiq'
    """
    if catalog_ids is None:
        catalog_ids = ["intel/realsense_d415", "robotiq/ft_300s"]
    return with_sensors(UR5e_DESCRIPTION, catalog_ids)

urdf_joint_names(description: RobotDescription, urdf_xml: str) -> list[str]

The URDF joint each manifest joint's state drives, or [] when the names agree.

A sim HAL publishes JointState under the manifest's logical joint names (left_joint1) — what the kernel, the runner and every Skill read. The manifest's URDF may name the same joints differently (the OpenArm's vendored URDF says openarm_left_joint1, which JointSpec.sim_joint_name carries), and robot_state_publisher only moves links whose URDF joint names it is sent: under the logical names every moving link is missing from /tf, so nothing that looks a link up through tf2 (the vision attachment leg's TCP and attach link, the octomap bridge's payload clearing) can work. On real hardware the vendor's joint_state_broadcaster already publishes the URDF names.

Each joint maps to its own name when the URDF declares it, else to its sim_joint_name when the URDF declares that, else to "" (no URDF joint; left out of the renamed stream).

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot manifest.	required
urdf_xml	str	The URDF robot_state_publisher is given.	required

Returns:

Type	Description
list[str]	URDF names parallel to description.joints, or [] when every joint is
list[str]	already named as in the URDF (no renamed stream is needed).

Raises:

Type	Description
ROSConfigError	urdf_xml is not parseable XML.

Example

from openral_core import RobotDescription desc = RobotDescription.from_yaml("robots/openarm/robot.yaml") # doctest: +SKIP urdf = open("robots/openarm/openarm.urdf").read() # doctest: +SKIP urdf_joint_names(desc, urdf)[:2] # doctest: +SKIP ['openarm_left_joint1', 'openarm_left_joint2']

Source code in python/hal/src/openral_hal/resolver.py

def urdf_joint_names(description: RobotDescription, urdf_xml: str) -> list[str]:
    """The URDF joint each manifest joint's state drives, or ``[]`` when the names agree.

    A sim HAL publishes ``JointState`` under the manifest's logical joint names
    (``left_joint1``) — what the kernel, the runner and every Skill read. The
    manifest's URDF may name the same joints differently (the OpenArm's vendored
    URDF says ``openarm_left_joint1``, which ``JointSpec.sim_joint_name`` carries),
    and ``robot_state_publisher`` only moves links whose URDF joint names it is
    sent: under the logical names every moving link is missing from ``/tf``, so
    nothing that looks a link up through tf2 (the vision attachment leg's TCP and
    attach link, the octomap bridge's payload clearing) can work. On real hardware
    the vendor's ``joint_state_broadcaster`` already publishes the URDF names.

    Each joint maps to its own ``name`` when the URDF declares it, else to its
    ``sim_joint_name`` when the URDF declares that, else to ``""`` (no URDF joint;
    left out of the renamed stream).

    Args:
        description: The robot manifest.
        urdf_xml: The URDF ``robot_state_publisher`` is given.

    Returns:
        URDF names parallel to ``description.joints``, or ``[]`` when every joint is
        already named as in the URDF (no renamed stream is needed).

    Raises:
        ROSConfigError: ``urdf_xml`` is not parseable XML.

    Example:
        >>> from openral_core import RobotDescription
        >>> desc = RobotDescription.from_yaml("robots/openarm/robot.yaml")  # doctest: +SKIP
        >>> urdf = open("robots/openarm/openarm.urdf").read()  # doctest: +SKIP
        >>> urdf_joint_names(desc, urdf)[:2]  # doctest: +SKIP
        ['openarm_left_joint1', 'openarm_left_joint2']
    """
    import xml.etree.ElementTree as ET  # reason: stdlib, only on this launch-time path

    try:
        root = ET.fromstring(urdf_xml)  # the manifest's own URDF, not untrusted input
    except ET.ParseError as exc:
        raise ROSConfigError(f"urdf_joint_names: the URDF does not parse: {exc}") from exc
    declared = {j.get("name") for j in root.iter("joint")}
    names = []
    for joint in description.joints:
        if joint.name in declared:
            names.append(joint.name)
        elif joint.sim_joint_name and joint.sim_joint_name in declared:
            names.append(joint.sim_joint_name)
        else:
            names.append("")
    if names == [joint.name for joint in description.joints]:
        return []
    return names

Sensors

openral Sensors — sensor adapters, catalog and launch generators.

Public surface: - CATALOG: global SensorCatalog populated by every vendor module on import. Used by openral sensor list / show and SensorSpec.catalog_id provenance on robot-mounted sensors. - SensorCatalog, SensorCatalogEntry: the registry types. - Per-vendor factories — see each <vendor>.py module: * realsense: D415 / D435 / D435i bundles. * luxonis: OAK-D Pro bundle. * stereolabs: ZED Mini bundle. * arducam: B0495 USB3 global-shutter RGB. * usb_uvc: generic USB UVC RGB, Logitech C920. * force_torque: Robotiq FT 300-S 6-axis F/T sensor. - Launch helpers (RealSense): bundle_to_node_params, generate_launch_py, calibrate_camera_cmd.

SensorCatalog(_entries: dict[str, SensorCatalogEntry] = dict()) dataclass

In-memory registry of SensorCatalogEntry rows.

The default global instance is CATALOG; tests should create a private SensorCatalog to avoid polluting the global registry.

Example

cat = SensorCatalog() def make() -> SensorSpec: ... return SensorSpec(name="x", modality=SensorModality.RGB, frame_id="x", rate_hz=30.0) _ = cat.register( ... SensorCatalogEntry( ... id="acme/cam", ... vendor="acme", ... model="cam", ... kind="sensor", ... factory=make, ... modalities=(SensorModality.RGB,), ... ) ... ) cat.get("acme/cam").vendor 'acme' cat.list_ids() ['acme/cam']

__contains__(sensor_id: object) -> bool

Membership check by sensor id.

Source code in python/sensors/src/openral_sensors/catalog.py

def __contains__(self, sensor_id: object) -> bool:
    """Membership check by sensor id."""
    return isinstance(sensor_id, str) and sensor_id in self._entries

__iter__() -> object

Iterate over registered ids.

Source code in python/sensors/src/openral_sensors/catalog.py

def __iter__(self) -> object:
    """Iterate over registered ids."""
    return iter(self._entries)

__len__() -> int

Number of registered entries.

Source code in python/sensors/src/openral_sensors/catalog.py

def __len__(self) -> int:
    """Number of registered entries."""
    return len(self._entries)

build(sensor_id: str, **kwargs: object) -> SensorSpec | SensorBundle

Resolve sensor_id and call its factory with **kwargs.

This is the entry point used by HAL helpers, openral detect, and authors who want to seed a fully materialized RobotDescription sensor before adding per-robot calibration and placement.

Source code in python/sensors/src/openral_sensors/catalog.py

def build(self, sensor_id: str, **kwargs: object) -> SensorSpec | SensorBundle:
    """Resolve ``sensor_id`` and call its factory with ``**kwargs``.

    This is the entry point used by HAL helpers, ``openral detect``, and
    authors who want to seed a fully materialized RobotDescription sensor
    before adding per-robot calibration and placement.
    """
    entry = self.get(sensor_id)
    return entry.factory(**kwargs)

entries() -> list[SensorCatalogEntry]

All entries, sorted alphabetically by id.

Source code in python/sensors/src/openral_sensors/catalog.py

def entries(self) -> list[SensorCatalogEntry]:
    """All entries, sorted alphabetically by id."""
    return [self._entries[k] for k in sorted(self._entries)]

filter(*, vendor: str | None = None, modality: SensorModality | None = None, kind: Literal['sensor', 'bundle'] | None = None) -> list[SensorCatalogEntry]

Return entries that match all provided predicates.

Source code in python/sensors/src/openral_sensors/catalog.py

def filter(
    self,
    *,
    vendor: str | None = None,
    modality: SensorModality | None = None,
    kind: Literal["sensor", "bundle"] | None = None,
) -> list[SensorCatalogEntry]:
    """Return entries that match all provided predicates."""
    out: list[SensorCatalogEntry] = []
    for entry in self.entries():
        if vendor is not None and entry.vendor != vendor:
            continue
        if modality is not None and modality not in entry.modalities:
            continue
        if kind is not None and entry.kind != kind:
            continue
        out.append(entry)
    return out

find_by_signature(signature: SensorSignature) -> SensorCatalogEntry | None

Look up an entry by probe-side signature.

Returns None when no registered entry advertises the signature. Callers (the openral detect assembler) decide between catalog-backed materialisation and a fallback generic SensorSpec.

Example

from openral_sensors import CATALOG from openral_sensors.catalog import SensorSignature entry = CATALOG.find_by_signature(SensorSignature(kind="realsense", value="D435I")) entry is not None and entry.id == "intel/realsense_d435i" True

Source code in python/sensors/src/openral_sensors/catalog.py

def find_by_signature(self, signature: SensorSignature) -> SensorCatalogEntry | None:
    """Look up an entry by probe-side signature.

    Returns ``None`` when no registered entry advertises the signature.
    Callers (the ``openral detect`` assembler) decide between catalog-backed
    materialisation and a fallback generic ``SensorSpec``.

    Example:
        >>> from openral_sensors import CATALOG
        >>> from openral_sensors.catalog import SensorSignature
        >>> entry = CATALOG.find_by_signature(SensorSignature(kind="realsense", value="D435I"))
        >>> entry is not None and entry.id == "intel/realsense_d435i"
        True
    """
    for entry in self._entries.values():
        if signature in entry.signatures:
            return entry
    return None

get(sensor_id: str) -> SensorCatalogEntry

Look up an entry by id. Raises KeyError if absent.

Source code in python/sensors/src/openral_sensors/catalog.py

def get(self, sensor_id: str) -> SensorCatalogEntry:
    """Look up an entry by id.  Raises ``KeyError`` if absent."""
    try:
        return self._entries[sensor_id]
    except KeyError as exc:
        raise KeyError(
            f"Unknown sensor id {sensor_id!r}.  Known ids: {sorted(self._entries)!r}"
        ) from exc

list_ids() -> list[str]

All registered ids, sorted alphabetically.

Source code in python/sensors/src/openral_sensors/catalog.py

def list_ids(self) -> list[str]:
    """All registered ids, sorted alphabetically."""
    return sorted(self._entries)

register(entry: SensorCatalogEntry, *, replace: bool = False) -> SensorCatalogEntry

Register an entry.

Raises KeyError on duplicate id unless replace=True.

Source code in python/sensors/src/openral_sensors/catalog.py

def register(self, entry: SensorCatalogEntry, *, replace: bool = False) -> SensorCatalogEntry:
    """Register an entry.

    Raises ``KeyError`` on duplicate id unless ``replace=True``.
    """
    if not replace and entry.id in self._entries:
        raise KeyError(
            f"Sensor catalog already has an entry for id={entry.id!r}; "
            "pass replace=True to overwrite."
        )
    self._entries[entry.id] = entry
    return entry

register_many(entries: Iterable[SensorCatalogEntry], *, replace: bool = True) -> None

Bulk-register catalog entries.

Used by vendor modules at import time to populate the global CATALOG. Defaults to replace=True because side-effect imports may run more than once in some test setups.

Source code in python/sensors/src/openral_sensors/catalog.py

def register_many(self, entries: Iterable[SensorCatalogEntry], *, replace: bool = True) -> None:
    """Bulk-register catalog entries.

    Used by vendor modules at import time to populate the global CATALOG.
    Defaults to ``replace=True`` because side-effect imports may run more
    than once in some test setups.
    """
    for entry in entries:
        self.register(entry, replace=replace)

unregister(sensor_id: str) -> None

Remove an entry. Idempotent.

Source code in python/sensors/src/openral_sensors/catalog.py

def unregister(self, sensor_id: str) -> None:
    """Remove an entry.  Idempotent."""
    self._entries.pop(sensor_id, None)

SensorCatalogEntry(id: str, vendor: str, model: str, kind: Literal['sensor', 'bundle'], factory: SensorFactory, modalities: tuple[SensorModality, ...], description: str = '', docs_url: str | None = None, signatures: tuple[SensorSignature, ...] = ()) dataclass

One row in the sensor catalog.

Attributes:

Name	Type	Description
id	str	Stable identifier in the form "<vendor>/<model>" — lowercase, slugified. Used as the lookup key and as SensorSpec.catalog_id provenance in robot.yaml.
vendor	str	Vendor or maintainer name (free-form, lowercase).
model	str	Vendor model identifier (free-form, lowercase / slugified).
kind	Literal['sensor', 'bundle']	Whether the factory returns a single SensorSpec or a multi-sensor SensorBundle.
factory	SensorFactory	Callable that constructs the spec/bundle. Must accept at least the same keyword arguments as the underlying factory; the common convention is name, parent_frame, rate_hz style.
modalities	tuple[SensorModality, ...]	SensorModality values that this sensor exposes.
description	str	One-line human-readable summary.
docs_url	str | None	Optional link to the vendor data sheet or product page.
signatures	tuple[SensorSignature, ...]	Probe-side identifiers that resolve to this entry. Used by openral detect to map a discovered device (RealSense model_id, USB VID/PID, V4L2 product string) to the canonical catalog ID.

SensorSignature(kind: SensorSignatureKind, value: str) dataclass

A probe-side identifier that resolves to one catalog entry.

SensorSignature is the bridge between a live device discovered by openral detect and the canonical SensorCatalogEntry whose factory materialises a fully-populated SensorSpec (with real intrinsics, FOV, encoding, rate, …). Each catalog entry may publish multiple signatures (e.g. a USB UVC camera with several VID/PID rebrands).

Encoding by kind: - "realsense" — pyrealsense2 model_id (e.g. "D435I"). - "orbbec" — Orbbec SDK product name. - "usb_uvc" — "0xVVVV:0xPPPP" lowercase hex. - "v4l2_name" — V4L2 product string (substring match acceptable).

Attributes:

Name	Type	Description
kind	SensorSignatureKind	Probe family that produced this signature.
value	str	Vendor-specific canonical key.

Example

SensorSignature(kind="realsense", value="D435I") SensorSignature(kind='realsense', value='D435I')

arducam_b0495_spec(name: str = 'arducam', parent_frame: str = 'base_link', rate_hz: float = 30.0, width: int = _B0495_NATIVE_WIDTH, height: int = _B0495_NATIVE_HEIGHT, hfov_deg: float | None = None, serial: str = '') -> SensorSpec

Build a SensorSpec for an Arducam B0495 USB3 global-shutter camera.

Parameters:

Name	Type	Description	Default
name	str	Sensor name — also the driver topic prefix and frame-id stem. SensorSpec.ros2_topic carries that driver-native topic (ADR-0108); OpenRAL consumers use openral_core.camera_topic(name).	'arducam'
parent_frame	str	tf2 parent frame for the static transform.	'base_link'
rate_hz	float	Publish rate. The device supports up to 50 Hz at the native 1920×1200 and up to 80 Hz at 960×600; values above the mode's maximum are recorded as-is and will be clamped by the driver.	30.0
width	int	Stream width. Native is 1920; 960 is the other supported mode.	_B0495_NATIVE_WIDTH
height	int	Stream height. Native is 1200; 600 is the other supported mode.	_B0495_NATIVE_HEIGHT
hfov_deg	float | None	Horizontal field of view of the mounted M12 lens, in degrees. Supply it to materialise nominal pinhole intrinsics; leave it None (the default) to leave intrinsics unset rather than record a guess. Either way, replace with values from openral calibrate camera before deployment.	None
serial	str	Optional USB serial string, recorded in metadata so a rig with several identical B0495s can address a specific one.	''

Returns:

Type	Description
SensorSpec	A fully-materialised SensorSpec.

Example

spec = arducam_b0495_spec(name="wrist", rate_hz=50.0) spec.rate_hz 50.0 spec.metadata["shutter"] 'global'

Source code in python/sensors/src/openral_sensors/arducam.py

def arducam_b0495_spec(
    name: str = "arducam",
    parent_frame: str = "base_link",
    rate_hz: float = 30.0,
    width: int = _B0495_NATIVE_WIDTH,
    height: int = _B0495_NATIVE_HEIGHT,
    hfov_deg: float | None = None,
    serial: str = "",
) -> SensorSpec:
    """Build a ``SensorSpec`` for an Arducam B0495 USB3 global-shutter camera.

    Args:
        name: Sensor name — also the driver topic prefix and frame-id stem.
            ``SensorSpec.ros2_topic`` carries that *driver-native* topic (ADR-0108);
            OpenRAL consumers use ``openral_core.camera_topic(name)``.
        parent_frame: tf2 parent frame for the static transform.
        rate_hz: Publish rate.  The device supports up to 50 Hz at the native
            1920×1200 and up to 80 Hz at 960×600; values above the mode's
            maximum are recorded as-is and will be clamped by the driver.
        width: Stream width.  Native is 1920; 960 is the other supported mode.
        height: Stream height.  Native is 1200; 600 is the other supported mode.
        hfov_deg: Horizontal field of view of the **mounted M12 lens**, in
            degrees.  Supply it to materialise nominal pinhole intrinsics;
            leave it ``None`` (the default) to leave ``intrinsics`` unset
            rather than record a guess.  Either way, replace with values from
            ``openral calibrate camera`` before deployment.
        serial: Optional USB serial string, recorded in ``metadata`` so a rig
            with several identical B0495s can address a specific one.

    Returns:
        A fully-materialised ``SensorSpec``.

    Example:
        >>> spec = arducam_b0495_spec(name="wrist", rate_hz=50.0)
        >>> spec.rate_hz
        50.0
        >>> spec.metadata["shutter"]
        'global'
    """
    intrinsics: IntrinsicsPinhole | None = None
    fov_v_deg: float | None = None
    if hfov_deg is not None:
        fx = width / (2.0 * math.tan(math.radians(hfov_deg / 2.0)))
        intrinsics = IntrinsicsPinhole(
            width=width,
            height=height,
            fx=fx,
            fy=fx,
            cx=width / 2.0,
            cy=height / 2.0,
            distortion_model="plumb_bob",
            distortion_coeffs=[0.0, 0.0, 0.0, 0.0, 0.0],
        )
        fov_v_deg = math.degrees(2.0 * math.atan(height / (2.0 * fx)))

    metadata: dict[str, object] = {
        "shutter": "global",
        "image_sensor": "AR0234",
        "usb_bridge": "Cypress FX3 (USB 3.0)",
        "native_resolution": [_B0495_NATIVE_WIDTH, _B0495_NATIVE_HEIGHT],
        "max_rate_hz": _B0495_MAX_RATE_HZ,
        "lens_mount": "M12",
        "needs_calibration": intrinsics is None,
    }
    if serial:
        metadata["serial_no"] = serial

    return SensorSpec(
        name=name,
        modality=SensorModality.RGB,
        frame_id=f"{name}_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rate_hz,
        encoding="yuyv",
        intrinsics=intrinsics,
        fov_h_deg=hfov_deg,
        fov_v_deg=fov_v_deg,
        ros2_topic=f"/{name}/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        catalog_id="arducam/b0495",
        vendor="Arducam",
        model="B0495",
        driver_pkg="usb_cam",
        metadata=metadata,
    )

bundle_to_node_params(bundle: SensorBundle, serial_no: str = '') -> NodeParams

Map a RealSense SensorBundle to realsense2_camera node parameters.

Only bundles whose sensors carry driver_pkg="realsense2_camera" are supported. Parameters are derived from the first RGB and depth SensorSpec found in the bundle.

Parameters:

Name	Type	Description	Default
bundle	SensorBundle	A SensorBundle produced by realsense_d435_bundle, realsense_d435i_bundle, or realsense_d415_bundle.	required
serial_no	str	Device serial number. Overrides any value stored in sensor metadata.	''

Returns:

Type	Description
NodeParams	A flat parameter dict suitable for the realsense2_camera
NodeParams	composable node.

Raises:

Type	Description
ValueError	If no RGB sensor is found in the bundle.

Example

bundle = realsense_d435_bundle(name="head") params = bundle_to_node_params(bundle) params["camera_name"] 'head' params["rgb_camera.color_profile"] '640x480x30'

Source code in python/sensors/src/openral_sensors/realsense.py

def bundle_to_node_params(bundle: SensorBundle, serial_no: str = "") -> NodeParams:
    """Map a RealSense ``SensorBundle`` to ``realsense2_camera`` node parameters.

    Only bundles whose sensors carry ``driver_pkg="realsense2_camera"`` are
    supported.  Parameters are derived from the first RGB and depth
    ``SensorSpec`` found in the bundle.

    Args:
        bundle: A ``SensorBundle`` produced by ``realsense_d435_bundle``,
            ``realsense_d435i_bundle``, or ``realsense_d415_bundle``.
        serial_no: Device serial number.  Overrides any value stored in
            sensor metadata.

    Returns:
        A flat parameter dict suitable for the ``realsense2_camera``
        composable node.

    Raises:
        ValueError: If no RGB sensor is found in the bundle.

    Example:
        >>> bundle = realsense_d435_bundle(name="head")
        >>> params = bundle_to_node_params(bundle)
        >>> params["camera_name"]
        'head'
        >>> params["rgb_camera.color_profile"]
        '640x480x30'
    """
    rgb = next(
        (s for s in bundle.sensors if s.modality == SensorModality.RGB),
        None,
    )
    if rgb is None:
        raise ValueError(f"SensorBundle '{bundle.bundle_name}' has no RGB sensor.")

    depth = next(
        (s for s in bundle.sensors if s.modality == SensorModality.DEPTH),
        None,
    )
    imu = next(
        (s for s in bundle.sensors if s.modality == SensorModality.IMU),
        None,
    )

    # Resolve serial number: explicit arg > metadata > empty
    sn = serial_no or str(rgb.metadata.get("serial_no", ""))

    rgb_w = rgb.intrinsics.width if rgb.intrinsics else 640
    rgb_h = rgb.intrinsics.height if rgb.intrinsics else 480
    rgb_fps = int(rgb.rate_hz)

    params: NodeParams = {
        "camera_name": bundle.bundle_name,
        "camera_namespace": f"/{bundle.bundle_name}",
        "serial_no": sn,
        "enable_color": True,
        "rgb_camera.color_profile": f"{rgb_w}x{rgb_h}x{rgb_fps}",
        "enable_depth": depth is not None,
        "enable_gyro": imu is not None,
        "enable_accel": imu is not None,
        "unite_imu_method": "1",  # linear interpolation
        "align_depth.enable": True,
        "pointcloud.enable": False,
    }

    if depth is not None:
        d_w = depth.intrinsics.width if depth.intrinsics else 640
        d_h = depth.intrinsics.height if depth.intrinsics else 480
        d_fps = int(depth.rate_hz)
        params["depth_module.depth_profile"] = f"{d_w}x{d_h}x{d_fps}"

    return params

calibrate_camera_cmd(sensor: SensorSpec, chessboard_cols: int = 8, chessboard_rows: int = 6, square_size_m: float = 0.025) -> list[str]

Build a ros2 run camera_calibration cameracalibrator command.

The command can be passed to subprocess.run or printed with --dry-run so the user can inspect it before execution.

Parameters:

Name	Type	Description	Default
sensor	SensorSpec	The RGB SensorSpec to calibrate.	required
chessboard_cols	int	Number of internal corners along the long axis.	8
chessboard_rows	int	Number of internal corners along the short axis.	6
square_size_m	float	Physical size of one chessboard square in metres.	0.025

Returns:

Type	Description
list[str]	A list of command tokens suitable for subprocess.run.

Raises:

Type	Description
ValueError	If sensor.modality is not RGB.

Example

from openral_sensors.realsense import realsense_d435_bundle bundle = realsense_d435_bundle(name="head") rgb_spec = bundle.sensors[0] cmd = calibrate_camera_cmd(rgb_spec, chessboard_cols=8, chessboard_rows=6) cmd[0] 'ros2' "--size" in cmd True "8x6" in cmd True

Source code in python/sensors/src/openral_sensors/realsense.py

def calibrate_camera_cmd(
    sensor: SensorSpec,
    chessboard_cols: int = 8,
    chessboard_rows: int = 6,
    square_size_m: float = 0.025,
) -> list[str]:
    """Build a ``ros2 run camera_calibration cameracalibrator`` command.

    The command can be passed to ``subprocess.run`` or printed with
    ``--dry-run`` so the user can inspect it before execution.

    Args:
        sensor: The RGB ``SensorSpec`` to calibrate.
        chessboard_cols: Number of internal corners along the long axis.
        chessboard_rows: Number of internal corners along the short axis.
        square_size_m: Physical size of one chessboard square in metres.

    Returns:
        A list of command tokens suitable for ``subprocess.run``.

    Raises:
        ValueError: If ``sensor.modality`` is not ``RGB``.

    Example:
        >>> from openral_sensors.realsense import realsense_d435_bundle
        >>> bundle = realsense_d435_bundle(name="head")
        >>> rgb_spec = bundle.sensors[0]
        >>> cmd = calibrate_camera_cmd(rgb_spec, chessboard_cols=8, chessboard_rows=6)
        >>> cmd[0]
        'ros2'
        >>> "--size" in cmd
        True
        >>> "8x6" in cmd
        True
    """
    if sensor.modality != SensorModality.RGB:
        raise ValueError(
            f"Sensor '{sensor.name}' has modality '{sensor.modality}', "
            "expected 'rgb' for camera calibration."
        )

    size_arg = f"{chessboard_cols}x{chessboard_rows}"
    square_arg = str(square_size_m)

    # ``ros2_topic`` is the *driver's* image topic (ADR-0108); its CameraInfo is the
    # image_pipeline sibling in the same namespace (``camera_info_topic_for``, the one rule
    # every publisher here uses), e.g. /head/color/image_raw → /head/color/camera_info.
    topic = sensor.ros2_topic
    if topic is None:
        raise ValueError(
            f"Sensor '{sensor.name}' has no ros2_topic; cannot derive camera_info remap."
        )
    from openral_sensors.ros_publisher import camera_info_topic_for  # noqa: PLC0415

    info_topic = camera_info_topic_for(topic)

    return [
        "ros2",
        "run",
        "camera_calibration",
        "cameracalibrator",
        "--size",
        size_arg,
        "--square",
        square_arg,
        "--ros-args",
        "-r",
        f"image:={topic}",
        "-r",
        f"camera_info:={info_topic}",
    ]

generate_launch_py(bundle: SensorBundle, serial_no: str = '') -> str

Generate a ROS 2 Python launch file string for a RealSense bundle.

The generated file uses the realsense2_camera composable node and does not depend on launch at generation time — this function is fully testable without a ROS 2 installation.

Parameters:

Name	Type	Description	Default
bundle	SensorBundle	A SensorBundle for the target camera.	required
serial_no	str	Optional device serial number forwarded to the node.	''

Returns:

Type	Description
str	A Python string that, when written to <name>.launch.py and
str	executed with ros2 launch, starts the camera driver with the
str	correct parameters.

Example

bundle = realsense_d435_bundle(name="head") src = generate_launch_py(bundle) "realsense2_camera" in src True "head" in src True src.startswith("# Generated by OpenRAL") True

Source code in python/sensors/src/openral_sensors/realsense.py

def generate_launch_py(bundle: SensorBundle, serial_no: str = "") -> str:
    """Generate a ROS 2 Python launch file string for a RealSense bundle.

    The generated file uses the ``realsense2_camera`` composable node and
    does **not** depend on ``launch`` at generation time — this function is
    fully testable without a ROS 2 installation.

    Args:
        bundle: A ``SensorBundle`` for the target camera.
        serial_no: Optional device serial number forwarded to the node.

    Returns:
        A Python string that, when written to ``<name>.launch.py`` and
        executed with ``ros2 launch``, starts the camera driver with the
        correct parameters.

    Example:
        >>> bundle = realsense_d435_bundle(name="head")
        >>> src = generate_launch_py(bundle)
        >>> "realsense2_camera" in src
        True
        >>> "head" in src
        True
        >>> src.startswith("# Generated by OpenRAL")
        True
    """
    params = bundle_to_node_params(bundle, serial_no=serial_no)

    # Format the parameter dict as Python literal lines for the launch file.
    param_lines = "\n".join(f"            {k!r}: {v!r}," for k, v in params.items())

    return textwrap.dedent(
        f"""\
        # Generated by OpenRAL ral sensors — do not edit by hand.
        # Re-generate with: ral launch sensor --bundle {bundle.bundle_name}
        from launch import LaunchDescription
        from launch_ros.actions import ComposableNodeContainer
        from launch_ros.descriptions import ComposableNode


        def generate_launch_description() -> LaunchDescription:
            container = ComposableNodeContainer(
                name="{bundle.bundle_name}_container",
                namespace="/{bundle.bundle_name}",
                package="rclcpp_components",
                executable="component_container",
                composable_node_descriptions=[
                    ComposableNode(
                        package="realsense2_camera",
                        plugin="realsense2_camera::RealSenseNodeFactory",
                        name="{bundle.bundle_name}",
                        parameters=[
                            {{
        {param_lines}
                            }}
                        ],
                        extra_arguments=[{{"use_intra_process_comms": True}}],
                    ),
                ],
                output="screen",
            )
            return LaunchDescription([container])
        """
    )

generic_uvc_rgb_spec(name: str = 'usb_cam', parent_frame: str = 'base_link', rate_hz: float = 30.0, width: int = 640, height: int = 480, hfov_deg: float = 70.0) -> SensorSpec

Build a generic USB UVC RGB camera spec for calibrated per-robot overrides.

Use this catalog entry when the robot manifest knows "USB UVC wrist camera" but not a stable vendor model. The manifest should override intrinsics with calibrated values and may override vendor / model when known.

Source code in python/sensors/src/openral_sensors/usb_uvc.py

def generic_uvc_rgb_spec(
    name: str = "usb_cam",
    parent_frame: str = "base_link",
    rate_hz: float = 30.0,
    width: int = 640,
    height: int = 480,
    hfov_deg: float = 70.0,
) -> SensorSpec:
    """Build a generic USB UVC RGB camera spec for calibrated per-robot overrides.

    Use this catalog entry when the robot manifest knows "USB UVC wrist camera"
    but not a stable vendor model. The manifest should override intrinsics with
    calibrated values and may override ``vendor`` / ``model`` when known.
    """
    return SensorSpec(
        name=name,
        modality=SensorModality.RGB,
        frame_id=f"{name}_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rate_hz,
        encoding="rgb8",
        intrinsics=_uvc_intrinsics(width, height, hfov_deg),
        fov_h_deg=hfov_deg,
        ros2_topic=f"/{name}/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        catalog_id="generic/usb_uvc_rgb",
        vendor="generic_usb",
        model="usb_uvc_rgb",
        driver_pkg="usb_cam",
    )

oak_d_pro_bundle(name: str = 'oak', parent_frame: str = 'base_link', mxid: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0, rgb_width: int = 1920, rgb_height: int = 1080, depth_width: int = 1280, depth_height: int = 800) -> SensorBundle

Build a SensorBundle for a Luxonis OAK-D Pro.

The bundle exposes three streams: RGB (IMX378), depth (OV9282 global-shutter stereo) and the on-device BNO086 IMU. Nominal intrinsics are from the Luxonis datasheet at the default resolutions; replace with calibrated values before deploying to hardware.

Parameters:

Name	Type	Description	Default
name	str	Bundle name — used as a prefix for topic names and frame ids (e.g. "oak_top" → /oak_top/rgb/image_raw). SensorSpec.ros2_topic carries that driver-native topic (ADR-0108); OpenRAL consumers use openral_core.camera_topic(name).	'oak'
parent_frame	str	tf2 parent frame for the static transform.	'base_link'
mxid	str	Optional Luxonis MXID for device targeting (multi-device hosts). Stored in metadata.mxid and forwarded to the ROS driver's i_mxid parameter.	''
rgb_rate_hz	float	RGB stream publish rate.	30.0
depth_rate_hz	float	Depth stream publish rate.	30.0
imu_rate_hz	float	IMU publish rate (BNO086).	400.0
rgb_width	int	RGB stream width. The factory rescales the nominal intrinsics linearly when this differs from the default 1920.	1920
rgb_height	int	RGB stream height.	1080
depth_width	int	Depth stream width.	1280
depth_height	int	Depth stream height.	800

Returns:

Type	Description
SensorBundle	A SensorBundle with sync="hardware" and 5 ms
SensorBundle	tolerance (RVC2 syncs RGB + depth on-device).

Example

bundle = oak_d_pro_bundle(name="oak_top", parent_frame="box_ceiling") bundle.sensors[1].modality 'depth' bundle.sensors[1].range_max_m 19.0

Source code in python/sensors/src/openral_sensors/luxonis.py

def oak_d_pro_bundle(
    name: str = "oak",
    parent_frame: str = "base_link",
    mxid: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
    rgb_width: int = 1920,
    rgb_height: int = 1080,
    depth_width: int = 1280,
    depth_height: int = 800,
) -> SensorBundle:
    """Build a ``SensorBundle`` for a Luxonis OAK-D Pro.

    The bundle exposes three streams: RGB (IMX378), depth (OV9282
    global-shutter stereo) and the on-device BNO086 IMU. Nominal
    intrinsics are from the Luxonis datasheet at the default
    resolutions; replace with calibrated values before deploying to
    hardware.

    Args:
        name: Bundle name — used as a prefix for topic names and frame
            ids (e.g. ``"oak_top"`` → ``/oak_top/rgb/image_raw``).
            ``SensorSpec.ros2_topic`` carries that *driver-native* topic (ADR-0108);
            OpenRAL consumers use ``openral_core.camera_topic(name)``.
        parent_frame: tf2 parent frame for the static transform.
        mxid: Optional Luxonis MXID for device targeting (multi-device
            hosts). Stored in ``metadata.mxid`` and forwarded to the
            ROS driver's ``i_mxid`` parameter.
        rgb_rate_hz: RGB stream publish rate.
        depth_rate_hz: Depth stream publish rate.
        imu_rate_hz: IMU publish rate (BNO086).
        rgb_width: RGB stream width. The factory rescales the nominal
            intrinsics linearly when this differs from the default
            1920.
        rgb_height: RGB stream height.
        depth_width: Depth stream width.
        depth_height: Depth stream height.

    Returns:
        A ``SensorBundle`` with ``sync="hardware"`` and 5 ms
        tolerance (RVC2 syncs RGB + depth on-device).

    Example:
        >>> bundle = oak_d_pro_bundle(name="oak_top", parent_frame="box_ceiling")
        >>> bundle.sensors[1].modality
        'depth'
        >>> bundle.sensors[1].range_max_m
        19.0
    """
    meta: dict[str, object] = {"mxid": mxid} if mxid else {}

    rgb_intr = _scale_intrinsics(_OAK_D_PRO_RGB_INTRINSICS, rgb_width, rgb_height)
    depth_intr = _scale_intrinsics(_OAK_D_PRO_DEPTH_INTRINSICS, depth_width, depth_height)

    rgb = SensorSpec(
        name=f"{name}_color",
        modality=SensorModality.RGB,
        frame_id=f"{name}_rgb_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rgb_rate_hz,
        encoding="rgb8",
        intrinsics=rgb_intr,
        fov_h_deg=95.0,
        fov_v_deg=70.0,
        ros2_topic=f"/{name}/rgb/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Luxonis",
        model="OAK-D Pro",
        driver_pkg="depthai_ros_driver",
        metadata=meta,
    )
    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_depth_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="16UC1",
        intrinsics=depth_intr,
        fov_h_deg=71.86,
        fov_v_deg=56.0,
        range_min_m=0.20,
        range_max_m=19.0,
        ros2_topic=f"/{name}/stereo/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Luxonis",
        model="OAK-D Pro",
        driver_pkg="depthai_ros_driver",
        metadata=meta,
    )
    imu = SensorSpec(
        name=f"{name}_imu",
        modality=SensorModality.IMU,
        frame_id=f"{name}_imu_frame",
        parent_frame=parent_frame,
        rate_hz=imu_rate_hz,
        accel_noise_density=3.0e-3,
        gyro_noise_density=3.5e-3,
        ros2_topic=f"/{name}/imu/data",
        ros2_msg_type="sensor_msgs/Imu",
        vendor="Luxonis",
        model="OAK-D Pro (BNO086 IMU)",
        driver_pkg="depthai_ros_driver",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[rgb, depth, imu],
        sync="hardware",
        sync_tolerance_ms=5.0,
    )

realsense_d415_bundle(name: str = 'realsense', parent_frame: str = 'base_link', serial_no: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0) -> SensorBundle

Build a SensorBundle for an Intel RealSense D415.

Rolling-shutter IR-stereo camera, 65°×40° FOV, 0.16 – 10 m range; no IMU.

Example

bundle = realsense_d415_bundle(name="head") bundle.sensors[1].range_min_m 0.16

Source code in python/sensors/src/openral_sensors/realsense.py

def realsense_d415_bundle(
    name: str = "realsense",
    parent_frame: str = "base_link",
    serial_no: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
) -> SensorBundle:
    """Build a ``SensorBundle`` for an Intel RealSense D415.

    Rolling-shutter IR-stereo camera, 65°×40° FOV, 0.16 – 10 m range; no IMU.

    Example:
        >>> bundle = realsense_d415_bundle(name="head")
        >>> bundle.sensors[1].range_min_m
        0.16
    """
    meta: dict[str, object] = {"serial_no": serial_no} if serial_no else {}
    rgb = SensorSpec(
        name=f"{name}_color",
        modality=SensorModality.RGB,
        frame_id=f"{name}_color_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rgb_rate_hz,
        encoding="rgb8",
        intrinsics=_D415_RGB_INTRINSICS,
        fov_h_deg=65.0,
        fov_v_deg=40.0,
        ros2_topic=f"/{name}/color/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D415",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_depth_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="16UC1",
        intrinsics=_D415_DEPTH_INTRINSICS,
        fov_h_deg=65.0,
        fov_v_deg=40.0,
        range_min_m=0.16,
        range_max_m=10.0,
        ros2_topic=f"/{name}/depth/image_rect_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D415",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[rgb, depth],
        sync="hardware",
        sync_tolerance_ms=5.0,
    )

realsense_d435_bundle(name: str = 'realsense', parent_frame: str = 'base_link', serial_no: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0) -> SensorBundle

Build a SensorBundle for an Intel RealSense D435.

The bundle contains three sensors: RGB, depth, and IMU. Nominal intrinsics are from the Intel D400 Series data sheet at 640x480; replace them with calibrated values before deploying to hardware.

Parameters:

Name	Type	Description	Default
name	str	Sensor bundle name, used as a prefix for topic names and frame IDs (e.g. "head" → /head/color/image_raw). SensorSpec.ros2_topic carries that driver-native topic (ADR-0108); OpenRAL consumers use openral_core.camera_topic(name).	'realsense'
parent_frame	str	tf2 parent frame for the static transform.	'base_link'
serial_no	str	Optional serial number for the device. Stored in metadata and forwarded to realsense2_camera as serial_no.	''
rgb_rate_hz	float	RGB stream publish rate.	30.0
depth_rate_hz	float	Depth stream publish rate.	30.0
imu_rate_hz	float	IMU publish rate.	400.0

Returns:

Type	Description
SensorBundle	A SensorBundle with sync="hardware" and 5 ms tolerance.

Example

bundle = realsense_d435_bundle(name="wrist", parent_frame="ee_link") bundle.sensors[0].frame_id 'wrist_color_optical_frame' bundle.sensors[1].modality 'depth' bundle.sensors[2].rate_hz 400.0

Source code in python/sensors/src/openral_sensors/realsense.py

def realsense_d435_bundle(
    name: str = "realsense",
    parent_frame: str = "base_link",
    serial_no: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
) -> SensorBundle:
    """Build a ``SensorBundle`` for an Intel RealSense D435.

    The bundle contains three sensors: RGB, depth, and IMU.  Nominal
    intrinsics are from the Intel D400 Series data sheet at 640x480; replace
    them with calibrated values before deploying to hardware.

    Args:
        name: Sensor bundle name, used as a prefix for topic names and frame
            IDs (e.g. ``"head"`` → ``/head/color/image_raw``).
            ``SensorSpec.ros2_topic`` carries that *driver-native* topic (ADR-0108);
            OpenRAL consumers use ``openral_core.camera_topic(name)``.
        parent_frame: tf2 parent frame for the static transform.
        serial_no: Optional serial number for the device.  Stored in metadata
            and forwarded to ``realsense2_camera`` as ``serial_no``.
        rgb_rate_hz: RGB stream publish rate.
        depth_rate_hz: Depth stream publish rate.
        imu_rate_hz: IMU publish rate.

    Returns:
        A ``SensorBundle`` with ``sync="hardware"`` and 5 ms tolerance.

    Example:
        >>> bundle = realsense_d435_bundle(name="wrist", parent_frame="ee_link")
        >>> bundle.sensors[0].frame_id
        'wrist_color_optical_frame'
        >>> bundle.sensors[1].modality
        'depth'
        >>> bundle.sensors[2].rate_hz
        400.0
    """
    meta: dict[str, object] = {"serial_no": serial_no} if serial_no else {}
    rgb = SensorSpec(
        name=f"{name}_color",
        modality=SensorModality.RGB,
        frame_id=f"{name}_color_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rgb_rate_hz,
        encoding="rgb8",
        intrinsics=_D435_RGB_INTRINSICS,
        fov_h_deg=69.4,
        fov_v_deg=42.5,
        ros2_topic=f"/{name}/color/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D435",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_depth_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="16UC1",
        intrinsics=_D435_DEPTH_INTRINSICS,
        fov_h_deg=87.0,
        fov_v_deg=58.0,
        range_min_m=0.1,
        range_max_m=10.0,
        ros2_topic=f"/{name}/depth/image_rect_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D435",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    imu = SensorSpec(
        name=f"{name}_imu",
        modality=SensorModality.IMU,
        frame_id=f"{name}_imu_optical_frame",
        parent_frame=parent_frame,
        rate_hz=imu_rate_hz,
        accel_noise_density=2.0e-3,
        gyro_noise_density=5.0e-3,
        ros2_topic=f"/{name}/imu",
        ros2_msg_type="sensor_msgs/Imu",
        vendor="Intel",
        model="RealSense D435",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[rgb, depth, imu],
        sync="hardware",
        sync_tolerance_ms=5.0,
    )

realsense_d435i_bundle(name: str = 'realsense', parent_frame: str = 'base_link', serial_no: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0) -> SensorBundle

Build a SensorBundle for an Intel RealSense D435i.

The D435i is a D435 with an integrated Bosch BMI085 IMU; the existing realsense_d435_bundle already includes the IMU sensor, so this factory delegates to it and stamps model="RealSense D435i" on each constituent SensorSpec.

Example

bundle = realsense_d435i_bundle(name="wrist") bundle.sensors[0].model 'RealSense D435i'

Source code in python/sensors/src/openral_sensors/realsense.py

def realsense_d435i_bundle(
    name: str = "realsense",
    parent_frame: str = "base_link",
    serial_no: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
) -> SensorBundle:
    """Build a ``SensorBundle`` for an Intel RealSense D435i.

    The D435i is a D435 with an integrated Bosch BMI085 IMU; the existing
    ``realsense_d435_bundle`` already includes the IMU sensor, so this
    factory delegates to it and stamps ``model="RealSense D435i"`` on each
    constituent ``SensorSpec``.

    Example:
        >>> bundle = realsense_d435i_bundle(name="wrist")
        >>> bundle.sensors[0].model
        'RealSense D435i'
    """
    bundle = realsense_d435_bundle(
        name=name,
        parent_frame=parent_frame,
        serial_no=serial_no,
        rgb_rate_hz=rgb_rate_hz,
        depth_rate_hz=depth_rate_hz,
        imu_rate_hz=imu_rate_hz,
    )
    return bundle.model_copy(
        update={
            "sensors": [s.model_copy(update={"model": "RealSense D435i"}) for s in bundle.sensors],
        }
    )

zed_mini_bundle(name: str = 'zed', parent_frame: str = 'base_link', serial: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0, width: int = 1280, height: int = 720) -> SensorBundle

Build a SensorBundle for a StereoLabs ZED Mini.

The bundle exposes four streams: left and right rectified RGB, host-computed stereo depth, and the on-device 6-DoF IMU. Nominal intrinsics are the datasheet values at HD720; replace with calibrated values before deploying.

Parameters:

Name	Type	Description	Default
name	str	Bundle name — used as a prefix for topic names and frame ids (e.g. "zed_head" → /zed_head/left/image_rect_color). SensorSpec.ros2_topic carries that driver-native topic (ADR-0108); OpenRAL consumers use openral_core.camera_topic(name).	'zed'
parent_frame	str	tf2 parent frame for the static transform.	'base_link'
serial	str	Optional ZED serial number for multi-camera hosts. Stored in metadata.serial_no and forwarded to the ROS driver's serial_number parameter.	''
rgb_rate_hz	float	Per-eye RGB publish rate.	30.0
depth_rate_hz	float	Depth publish rate.	30.0
imu_rate_hz	float	IMU publish rate.	400.0
width	int	Per-eye stream width. The factory rescales the nominal intrinsics linearly when this differs from the HD720 default.	1280
height	int	Per-eye stream height.	720

Returns:

Type	Description
SensorBundle	A SensorBundle with sync="hardware" — the two eyes come
SensorBundle	out of one sensor pair in a single UVC frame, so they are exposed
SensorBundle	simultaneously by construction.

Example

bundle = zed_mini_bundle(name="head") bundle.sensors[2].range_max_m 15.0 bundle.sensors[0].metadata["baseline_m"] 0.063

Source code in python/sensors/src/openral_sensors/stereolabs.py

def zed_mini_bundle(
    name: str = "zed",
    parent_frame: str = "base_link",
    serial: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
    width: int = 1280,
    height: int = 720,
) -> SensorBundle:
    """Build a ``SensorBundle`` for a StereoLabs ZED Mini.

    The bundle exposes four streams: left and right rectified RGB, host-computed
    stereo depth, and the on-device 6-DoF IMU.  Nominal intrinsics are the
    datasheet values at HD720; replace with calibrated values before deploying.

    Args:
        name: Bundle name — used as a prefix for topic names and frame ids
            (e.g. ``"zed_head"`` → ``/zed_head/left/image_rect_color``).
            ``SensorSpec.ros2_topic`` carries that *driver-native* topic (ADR-0108);
            OpenRAL consumers use ``openral_core.camera_topic(name)``.
        parent_frame: tf2 parent frame for the static transform.
        serial: Optional ZED serial number for multi-camera hosts.  Stored in
            ``metadata.serial_no`` and forwarded to the ROS driver's
            ``serial_number`` parameter.
        rgb_rate_hz: Per-eye RGB publish rate.
        depth_rate_hz: Depth publish rate.
        imu_rate_hz: IMU publish rate.
        width: Per-eye stream width.  The factory rescales the nominal
            intrinsics linearly when this differs from the HD720 default.
        height: Per-eye stream height.

    Returns:
        A ``SensorBundle`` with ``sync="hardware"`` — the two eyes come
        out of one sensor pair in a single UVC frame, so they are exposed
        simultaneously by construction.

    Example:
        >>> bundle = zed_mini_bundle(name="head")
        >>> bundle.sensors[2].range_max_m
        15.0
        >>> bundle.sensors[0].metadata["baseline_m"]
        0.063
    """
    intrinsics = scale_intrinsics_to(_ZED_MINI_EYE_INTRINSICS, width, height)
    meta: dict[str, object] = {
        "baseline_m": _ZED_MINI_BASELINE_M,
        "stereo": "passive",
        # Depth and rectification run on the host GPU via the ZED SDK; a bare
        # UVC host gets one side-by-side RGB stream and nothing else.
        "sdk_required": "zed-sdk",
        "uvc_side_by_side": [width * 2, height],
    }
    if serial:
        meta["serial_no"] = serial

    def _eye(side: str) -> SensorSpec:
        return SensorSpec(
            name=f"{name}_{side}",
            modality=SensorModality.RGB,
            frame_id=f"{name}_{side}_camera_optical_frame",
            parent_frame=parent_frame,
            rate_hz=rgb_rate_hz,
            encoding="rgb8",
            intrinsics=intrinsics,
            fov_h_deg=_ZED_MINI_FOV_H_DEG,
            fov_v_deg=_ZED_MINI_FOV_V_DEG,
            ros2_topic=f"/{name}/{side}/image_rect_color",
            ros2_msg_type="sensor_msgs/Image",
            vendor="StereoLabs",
            model="ZED Mini",
            driver_pkg="zed_wrapper",
            metadata=meta,
        )

    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_left_camera_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="32FC1",
        intrinsics=intrinsics,
        fov_h_deg=_ZED_MINI_FOV_H_DEG,
        fov_v_deg=_ZED_MINI_FOV_V_DEG,
        range_min_m=_ZED_MINI_DEPTH_MIN_M,
        range_max_m=_ZED_MINI_DEPTH_MAX_M,
        ros2_topic=f"/{name}/depth/depth_registered",
        ros2_msg_type="sensor_msgs/Image",
        vendor="StereoLabs",
        model="ZED Mini",
        driver_pkg="zed_wrapper",
        metadata=meta,
    )
    imu = SensorSpec(
        name=f"{name}_imu",
        modality=SensorModality.IMU,
        frame_id=f"{name}_imu_link",
        parent_frame=parent_frame,
        rate_hz=imu_rate_hz,
        ros2_topic=f"/{name}/imu/data",
        ros2_msg_type="sensor_msgs/Imu",
        vendor="StereoLabs",
        model="ZED Mini (integrated IMU)",
        driver_pkg="zed_wrapper",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[_eye("left"), _eye("right"), depth, imu],
        sync="hardware",
        sync_tolerance_ms=1.0,
    )

World State

openral World State — snapshot aggregator + persistent spatial memory public API.

Public surface: - WorldStateAggregator: tf2-aware, injectable aggregator that produces WorldState Pydantic snapshots at up to 30 Hz. - DEFAULT_RATE_HZ: Advertised default snapshot rate. - DEFAULT_STALENESS_S: Default staleness threshold in seconds. - SpatialMemory: persistent object-centric scene-graph memory — accumulates WorldState.detected_objects and answers RecallObjectQuery / ResolvePlaceQuery. - compute_approach_viewpoint: camera-facing standoff-pose helper. - look_at_quat_wxyz / compute_gaze_pose: shared gaze geometry (look-at rotations per camera convention; the rskill-moveit-multi-look_at goal pose). - OccupancyGridIndex / refine_approach_pose: occupancy-grid queries + approach-pose snapping (free cell + line-of-sight; planning-layer, not a safety surface). - OpenClipEmbedder / TextEmbedder: open-vocab text embedder (OpenCLIP ViT-B/32, MIT) — optional, uv sync --group clip. - emit_scene_objects_span / scene_objects_payload: publish the remembered object nodes as a world.scene_objects OTel span for the dashboard. - VoxelFrustumLifter / ObjectMemory / lift helpers: lift 2D detections to 3D object centres and remember them with IoU gating.

DEFAULT_ASSOC_DISTANCE_M = 0.3 module-attribute

Default radius (m) for label-based instance association when track_id is absent.

DEFAULT_CAMERA_FRAME = 'gripper_camera' module-attribute

Default tf2 frame the approach viewpoint orients toward.

DEFAULT_CLIP_MODEL = 'ViT-B-32-quickgelu' module-attribute

OpenCLIP model name (default) — the quickgelu variant matches the openai pretrained weights' activation (avoids the QuickGELU mismatch warning).

DEFAULT_CLIP_PRETRAINED = 'openai' module-attribute

OpenCLIP pretrained tag — the original CLIP weights (MIT).

DEFAULT_MAP_FRAME = 'map' module-attribute

Default durable frame object poses are anchored in.

DEFAULT_MIN_TEXT_SIMILARITY = 0.85 module-attribute

Min CLIP cosine for an embedding-only object match (calibrated for ViT-B/32 openai).

OpenCLIP text-text cosines cluster high (~0.76-0.96), so this floor gates embedding-only hits; an exact/substring label match always qualifies regardless.

DEFAULT_STANDOFF_M = 0.6 module-attribute

Default standoff distance (m) from an object for the camera-facing approach viewpoint.

FREE_MAX = 25 module-attribute

Highest nav_msgs/OccupancyGrid value still treated as free (map_server's free_thresh is ~20/100; everything above, plus -1 unknown, blocks).

ObjectMemory(*, iou_threshold: float = 0.3, max_misses: int = 1)

Stateful, IoU-gated spatial memory of detected objects.

Parameters:

Name	Type	Description	Default
iou_threshold	float	Minimum 3D AABB IoU (same label) to associate a new detection with an existing track.	0.3
max_misses	int	Evict an in-view, unmatched track after this many consecutive misses. 1 ⇒ remove if not re-seen the next run.	1

Example

from openral_core.schemas import DetectedObject, Pose6D mem = ObjectMemory(iou_threshold=0.3, max_misses=1) cand = DetectedObject( ... label="cup", ... confidence=0.9, ... pose=Pose6D(xyz=(0.5, 0.5, 0.5), quat_xyzw=(0, 0, 0, 1), frame_id="map"), ... bbox_3d=(0.0, 0.0, 0.0, 1.0, 1.0, 1.0), ... ) out = mem.tick([cand], stamp_ns=1, in_fov=lambda o: True) out[0].track_id 0

Validate the IoU threshold and miss budget.

Source code in python/world_state/src/openral_world_state/object_memory.py

def __init__(self, *, iou_threshold: float = 0.3, max_misses: int = 1) -> None:
    """Validate the IoU threshold and miss budget."""
    if not 0.0 <= iou_threshold <= 1.0:
        raise ValueError(f"iou_threshold must be in [0,1]; got {iou_threshold}")
    if max_misses < 1:
        raise ValueError(f"max_misses must be >= 1; got {max_misses}")
    self._iou = iou_threshold
    self._max_misses = max_misses
    self._tracks: list[_Tracked] = []
    self._next_id = 0

tick(candidates: list[DetectedObject], *, stamp_ns: int, in_fov: Callable[[DetectedObject], bool]) -> list[DetectedObject]

Associate candidates, freeze matches, create new, evict in-view misses.

Parameters:

Name	Type	Description	Default
candidates	list[DetectedObject]	Newly lifted DetectedObjects for this run (map frame).	required
stamp_ns	int	Run timestamp in nanoseconds (stored as last-seen).	required
in_fov	Callable[[DetectedObject], bool]	Predicate — was this remembered object in the camera's view this run? Unmatched in-view objects accrue a miss; out-of-view objects are retained.	required

Returns:

Type	Description
list[DetectedObject]	The current remembered object set (stable track_ids).

Source code in python/world_state/src/openral_world_state/object_memory.py

def tick(
    self,
    candidates: list[DetectedObject],
    *,
    stamp_ns: int,
    in_fov: Callable[[DetectedObject], bool],
) -> list[DetectedObject]:
    """Associate candidates, freeze matches, create new, evict in-view misses.

    Args:
        candidates: Newly lifted ``DetectedObject``s for this run (map frame).
        stamp_ns: Run timestamp in nanoseconds (stored as last-seen).
        in_fov: Predicate — was this remembered object in the camera's view
            this run? Unmatched in-view objects accrue a miss; out-of-view
            objects are retained.

    Returns:
        The current remembered object set (stable ``track_id``s).
    """
    matched: set[int] = set()
    for cand in sorted(candidates, key=lambda d: d.confidence, reverse=True):
        best_i, best_iou = -1, self._iou
        for i, tr in enumerate(self._tracks):
            if tr.obj.label != cand.label:
                continue
            if cand.bbox_3d is None or tr.obj.bbox_3d is None:
                continue
            iou = aabb_iou_3d(cand.bbox_3d, tr.obj.bbox_3d)
            if iou >= best_iou:
                best_iou, best_i = iou, i
        if best_i >= 0:
            tr = self._tracks[best_i]
            tr.last_seen_ns = stamp_ns
            tr.miss_count = 0
            if cand.confidence > tr.obj.confidence:
                tr.obj = tr.obj.model_copy(update={"confidence": cand.confidence})
            matched.add(best_i)
        else:
            # A new track adopts the detection-time id propagated
            # from the 2D detector (DetectedObject.track_id, from
            # ObjectDetection2D.det_id) when present, so a physical object
            # carries one id across the 2D in_view line and the 3D
            # scene_objects line. Fall back to minting for legacy detectors
            # (track_id None / < 0). Matched tracks keep their stored id (3D
            # association is more stable than 2D re-id).
            adopted = cand.track_id
            new_id = adopted if adopted is not None and adopted >= 0 else self._next_id
            self._tracks.append(
                _Tracked(
                    obj=cand.model_copy(update={"track_id": new_id}),
                    last_seen_ns=stamp_ns,
                    miss_count=0,
                ),
            )
            matched.add(len(self._tracks) - 1)
            self._next_id = max(self._next_id, new_id) + 1

    survivors: list[_Tracked] = []
    for i, tr in enumerate(self._tracks):
        if i in matched:
            survivors.append(tr)
            continue
        if in_fov(tr.obj):
            tr.miss_count += 1
            if tr.miss_count >= self._max_misses:
                continue
        survivors.append(tr)
    self._tracks = survivors
    return [tr.obj for tr in self._tracks]

OccupancyGridIndex(data: NDArray[np.int8], *, resolution_m: float, origin_xy: tuple[float, float], origin_yaw: float = 0.0)

Queryable view over one nav_msgs/OccupancyGrid snapshot.

Parameters:

Name	Type	Description	Default
data	NDArray[int8]	(height, width) int8 occupancy array (row-major, as on the wire: row 0 is the cell at the grid origin).	required
resolution_m	float	Cell edge length in metres.	required
origin_xy	tuple[float, float]	World x, y of the corner of cell (0, 0).	required
origin_yaw	float	Grid rotation about +Z (slam_toolbox maps are normally axis-aligned, but the origin pose may carry yaw).	0.0

Example

import numpy as np idx = OccupancyGridIndex( ... np.zeros((4, 4), dtype=np.int8), resolution_m=0.5, origin_xy=(0.0, 0.0) ... ) idx.is_free(1.0, 1.0) True

See the class docstring for argument semantics.

Source code in python/world_state/src/openral_world_state/grid.py

def __init__(
    self,
    data: NDArray[np.int8],
    *,
    resolution_m: float,
    origin_xy: tuple[float, float],
    origin_yaw: float = 0.0,
) -> None:
    """See the class docstring for argument semantics."""
    if data.ndim != _GRID_NDIM:
        raise ValueError(f"occupancy data must be 2-D (height, width); got {data.shape}")
    if resolution_m <= 0.0:
        raise ValueError(f"resolution_m must be > 0; got {resolution_m}")
    self._data = data
    self._res = float(resolution_m)
    self._ox, self._oy = float(origin_xy[0]), float(origin_xy[1])
    self._cos = math.cos(origin_yaw)
    self._sin = math.sin(origin_yaw)

resolution_m: float property

Cell edge length in metres.

from_msg(msg: Any) -> OccupancyGridIndex classmethod

Build from a (duck-typed) nav_msgs/OccupancyGrid message.

Source code in python/world_state/src/openral_world_state/grid.py

@classmethod
def from_msg(cls, msg: Any) -> OccupancyGridIndex:  # noqa: ANN401  # reason: duck-typed nav_msgs/OccupancyGrid keeps world_state ROS-free
    """Build from a (duck-typed) ``nav_msgs/OccupancyGrid`` message."""
    info = msg.info
    data = np.asarray(msg.data, dtype=np.int8).reshape(info.height, info.width)
    origin = info.origin
    yaw = quat_xyzw_to_yaw(
        origin.orientation.x,
        origin.orientation.y,
        origin.orientation.z,
        origin.orientation.w,
    )
    return cls(
        data,
        resolution_m=float(info.resolution),
        origin_xy=(float(origin.position.x), float(origin.position.y)),
        origin_yaw=yaw,
    )

is_free(x: float, y: float, *, inflation_m: float = 0.0) -> bool

True iff the point — and every cell within inflation_m of it — is free.

Conservative: the disc is measured in world space from the query point to each cell's nearest face (not centre-cell to centre-cell, which under-inflates near cell edges). Off-grid points, and any part of the disc extending off-grid, count as blocked — the robot footprint must sit inside known-free space.

Source code in python/world_state/src/openral_world_state/grid.py

def is_free(self, x: float, y: float, *, inflation_m: float = 0.0) -> bool:
    """True iff the point — and every cell within ``inflation_m`` of it — is free.

    Conservative: the disc is measured in **world space** from the query
    point to each cell's nearest face (not centre-cell to centre-cell,
    which under-inflates near cell edges). Off-grid points, and any part
    of the disc extending off-grid, count as blocked — the robot footprint
    must sit inside known-free space.
    """
    centre = self.world_to_cell(x, y)
    if centre is None:
        return False
    row0, col0 = centre
    h, w = self._data.shape
    # Work in (rotated) grid coordinates so cell rectangles are axis-aligned.
    dx, dy = x - self._ox, y - self._oy
    gx = self._cos * dx + self._sin * dy
    gy = -self._sin * dx + self._cos * dy
    steps = max(0, math.ceil(inflation_m / self._res) + 1)
    for dr in range(-steps, steps + 1):
        for dc in range(-steps, steps + 1):
            r, c = row0 + dr, col0 + dc
            # Nearest point of cell (r, c) to the query point, grid coords.
            cx = min(max(gx, c * self._res), (c + 1) * self._res)
            cy = min(max(gy, r * self._res), (r + 1) * self._res)
            if math.hypot(cx - gx, cy - gy) > inflation_m + 1e-12:
                continue
            if not (0 <= r < h and 0 <= c < w):
                return False
            value = int(self._data[r, c])
            if value < 0 or value > FREE_MAX:
                return False
    return True

line_of_sight(a_xy: tuple[float, float], b_xy: tuple[float, float]) -> bool

True iff every cell strictly before b on the segment is free.

Bresenham over the grid; unknown cells block sight and either endpoint off-grid is no sight. The final cell is exempt: the target is typically an object whose own footprint is occupied in the 2-D grid (a mug shares its cell with the counter it sits on) — sight means an unobstructed ray up to it, not through it.

Source code in python/world_state/src/openral_world_state/grid.py

def line_of_sight(self, a_xy: tuple[float, float], b_xy: tuple[float, float]) -> bool:
    """True iff every cell strictly before ``b`` on the segment is free.

    Bresenham over the grid; unknown cells block sight and either endpoint
    off-grid is no sight. The **final cell is exempt**: the target is
    typically an object whose own footprint is occupied in the 2-D grid
    (a mug shares its cell with the counter it sits on) — sight means an
    unobstructed ray *up to* it, not through it.
    """
    a = self.world_to_cell(*a_xy)
    b = self.world_to_cell(*b_xy)
    if a is None or b is None:
        return False
    r0, c0 = a
    r1, c1 = b
    dr, dc = abs(r1 - r0), abs(c1 - c0)
    sr = 1 if r1 >= r0 else -1
    sc = 1 if c1 >= c0 else -1
    err = dc - dr
    r, c = r0, c0
    while True:
        if (r, c) == (r1, c1):
            return True
        value = int(self._data[r, c])
        if value < 0 or value > FREE_MAX:
            return False
        e2 = 2 * err
        if e2 > -dr:
            err -= dr
            c += sc
        if e2 < dc:
            err += dc
            r += sr

world_to_cell(x: float, y: float) -> tuple[int, int] | None

(row, col) containing the world point, or None off-grid.

Source code in python/world_state/src/openral_world_state/grid.py

def world_to_cell(self, x: float, y: float) -> tuple[int, int] | None:
    """``(row, col)`` containing the world point, or ``None`` off-grid."""
    dx, dy = x - self._ox, y - self._oy
    gx = self._cos * dx + self._sin * dy
    gy = -self._sin * dx + self._cos * dy
    col = math.floor(gx / self._res)
    row = math.floor(gy / self._res)
    h, w = self._data.shape
    if 0 <= row < h and 0 <= col < w:
        return (row, col)
    return None

OpenClipEmbedder(*, model_name: str = DEFAULT_CLIP_MODEL, pretrained: str = DEFAULT_CLIP_PRETRAINED, device: str | None = None)

OpenCLIP text embedder (ViT-B/32, MIT) for open-vocab matching.

Parameters:

Name	Type	Description	Default
model_name	str	OpenCLIP architecture (default "ViT-B-32").	DEFAULT_CLIP_MODEL
pretrained	str	OpenCLIP pretrained tag (default "openai" — MIT weights).	DEFAULT_CLIP_PRETRAINED
device	str | None	Torch device; defaults to "cuda" when available else "cpu".	None

Raises:

Type	Description
ROSConfigError	When open_clip / torch are not installed (uv sync --group clip) or the model cannot be created.

Create the embedder, loading the OpenCLIP model onto device.

Source code in python/world_state/src/openral_world_state/embedder.py

def __init__(
    self,
    *,
    model_name: str = DEFAULT_CLIP_MODEL,
    pretrained: str = DEFAULT_CLIP_PRETRAINED,
    device: str | None = None,
) -> None:
    """Create the embedder, loading the OpenCLIP model onto ``device``."""
    try:
        import open_clip  # noqa: PLC0415  # reason: optional dep; mypy via mypy.ini ignore_missing_imports
        import torch  # noqa: PLC0415  # reason: optional/compute-gated import (clip group)
    except ImportError as exc:  # pragma: no cover - exercised via the clip group
        raise ROSConfigError(
            "OpenClipEmbedder requires open-clip-torch + torch (`uv sync --group clip`)."
        ) from exc

    self._torch = torch
    self._device = device or ("cuda" if torch.cuda.is_available() else "cpu")
    try:
        model, _, _ = open_clip.create_model_and_transforms(
            model_name, pretrained=pretrained, device=self._device
        )
        self._tokenizer = open_clip.get_tokenizer(model_name)
    except Exception as exc:  # reason: open_clip raises bare exceptions on bad model/download
        raise ROSConfigError(
            f"OpenClipEmbedder failed to load {model_name!r}/{pretrained!r}: {exc}"
        ) from exc
    model.eval()
    self._model = model
    self._dim = int(model.text_projection.shape[-1])
    self.model_name = model_name
    self.pretrained = pretrained

dim: int property

Embedding dimensionality (512 for ViT-B/32).

embed_text(texts: Sequence[str]) -> NDArray[np.float32]

Embed texts into an (N, dim) L2-normalized float32 array.

Source code in python/world_state/src/openral_world_state/embedder.py

def embed_text(self, texts: Sequence[str]) -> NDArray[np.float32]:
    """Embed ``texts`` into an ``(N, dim)`` L2-normalized float32 array."""
    if not texts:
        return np.zeros((0, self._dim), dtype=np.float32)
    torch = self._torch
    tokens = self._tokenizer(list(texts)).to(self._device)
    with torch.no_grad():
        feats = self._model.encode_text(tokens)
        feats = feats / feats.norm(dim=-1, keepdim=True)
    return feats.cpu().numpy().astype(np.float32)  # type: ignore[no-any-return]  # reason: torch→numpy is untyped

SpatialMemory(*, assoc_distance_m: float = DEFAULT_ASSOC_DISTANCE_M, default_standoff_m: float = DEFAULT_STANDOFF_M, camera_frame_id: str = DEFAULT_CAMERA_FRAME, map_frame: str = DEFAULT_MAP_FRAME, embedder: TextEmbedder | None = None, min_text_similarity: float = DEFAULT_MIN_TEXT_SIMILARITY)

Accumulating, queryable scene-graph spatial memory.

Parameters:

Name	Type	Description	Default
assoc_distance_m	float	Radius for label-based instance association when a detection carries no track_id.	DEFAULT_ASSOC_DISTANCE_M
default_standoff_m	float	Standoff used for approach viewpoints.	DEFAULT_STANDOFF_M
camera_frame_id	str	tf2 frame approach viewpoints orient toward.	DEFAULT_CAMERA_FRAME
map_frame	str	Durable frame object poses are anchored in.	DEFAULT_MAP_FRAME

Initialize an empty memory.

embedder (optional) enables open-vocabulary matching: object labels are embedded on creation and free-text queries match by CLIP cosine similarity (>= min_text_similarity) in addition to label substring. Without it, matching is label + pose + recency only.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def __init__(
    self,
    *,
    assoc_distance_m: float = DEFAULT_ASSOC_DISTANCE_M,
    default_standoff_m: float = DEFAULT_STANDOFF_M,
    camera_frame_id: str = DEFAULT_CAMERA_FRAME,
    map_frame: str = DEFAULT_MAP_FRAME,
    embedder: TextEmbedder | None = None,
    min_text_similarity: float = DEFAULT_MIN_TEXT_SIMILARITY,
) -> None:
    """Initialize an empty memory.

    ``embedder`` (optional) enables open-vocabulary matching:
    object labels are embedded on creation and free-text queries match by
    CLIP cosine similarity (>= ``min_text_similarity``) in addition to
    label substring. Without it, matching is label + pose + recency only.
    """
    self._assoc_distance_m = assoc_distance_m
    self._default_standoff_m = default_standoff_m
    self._camera_frame_id = camera_frame_id
    self._map_frame = map_frame
    self._embedder = embedder
    self._min_text_similarity = min_text_similarity
    self._nodes: dict[str, SpatialNode] = {}
    self._edges: dict[tuple[str, str, SpatialRelationKind], SpatialEdge] = {}
    self._embeddings: dict[str, NDArray[np.float32]] = {}
    self._auto_counter = 0

add_edge(src: str, dst: str, kind: SpatialRelationKind) -> None

Add a directed relation; src/dst must already exist (idempotent).

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def add_edge(self, src: str, dst: str, kind: SpatialRelationKind) -> None:
    """Add a directed relation; src/dst must already exist (idempotent)."""
    if src not in self._nodes:
        raise KeyError(f"unknown src node: {src!r}")
    if dst not in self._nodes:
        raise KeyError(f"unknown dst node: {dst!r}")
    self._edges[(src, dst, kind)] = SpatialEdge(src=src, dst=dst, kind=kind)

from_scene_graph(graph: SceneGraph, *, embedder: TextEmbedder | None = None) -> SpatialMemory classmethod

Build a memory from a persisted SceneGraph.

Embeddings are not serialized; when an embedder is supplied, object labels are re-embedded here (cheap, deterministic) so open-vocab queries work against a loaded graph.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

@classmethod
def from_scene_graph(
    cls, graph: SceneGraph, *, embedder: TextEmbedder | None = None
) -> SpatialMemory:
    """Build a memory from a persisted ``SceneGraph``.

    Embeddings are not serialized; when an ``embedder`` is supplied, object
    labels are re-embedded here (cheap, deterministic) so open-vocab queries
    work against a loaded graph.
    """
    mem = cls(embedder=embedder)
    for node in graph.nodes:
        mem._nodes[node.node_id] = node
        mem._embed_node(node)
    for edge in graph.edges:
        mem._edges[(edge.src, edge.dst, edge.kind)] = edge
    return mem

ingest_detected_objects(objects: Sequence[DetectedObject], *, now_ns: int) -> list[str]

Fold a snapshot's detected objects into the graph.

Instance association: a detection is matched to an existing object node by track_id when present, else by identical label within assoc_distance_m. A match updates the pose, bumps last_seen_ns and observation_count and keeps the higher confidence; otherwise a new node is created. Object poses are expected in the map frame.

Parameters:

Name	Type	Description	Default
objects	Sequence[DetectedObject]	Detections from a WorldState.	required
now_ns	int	Observation timestamp in nanoseconds.	required

Returns:

Type	Description
list[str]	The node ids touched (created or updated), in input order.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def ingest_detected_objects(
    self, objects: Sequence[DetectedObject], *, now_ns: int
) -> list[str]:
    """Fold a snapshot's detected objects into the graph.

    Instance association: a detection is matched to an existing ``object``
    node by ``track_id`` when present, else by identical label within
    ``assoc_distance_m``. A match updates the pose, bumps ``last_seen_ns``
    and ``observation_count`` and keeps the higher confidence; otherwise a
    new node is created. Object poses are expected in the map frame.

    Args:
        objects: Detections from a ``WorldState``.
        now_ns: Observation timestamp in nanoseconds.

    Returns:
        The node ids touched (created or updated), in input order.
    """
    touched: list[str] = []
    for obj in objects:
        match_id = self._associate(obj)
        if match_id is None:
            node_id = self._new_node_id(obj)
            self._nodes[node_id] = SpatialNode(
                node_id=node_id,
                kind=SpatialNodeKind.OBJECT,
                pose=obj.pose,
                label=obj.label,
                confidence=obj.confidence,
                bbox_3d=obj.bbox_3d,
                first_seen_ns=now_ns,
                last_seen_ns=now_ns,
                observation_count=1,
            )
            self._embed_node(self._nodes[node_id])
            touched.append(node_id)
        else:
            prev = self._nodes[match_id]
            self._nodes[match_id] = prev.model_copy(
                update={
                    "pose": obj.pose,
                    "bbox_3d": obj.bbox_3d if obj.bbox_3d is not None else prev.bbox_3d,
                    "confidence": max(prev.confidence, obj.confidence),
                    "last_seen_ns": max(prev.last_seen_ns, now_ns),
                    "observation_count": prev.observation_count + 1,
                }
            )
            touched.append(match_id)
    return touched

load(path: str | Path, *, embedder: TextEmbedder | None = None) -> SpatialMemory classmethod

Load a memory from a JSON scene graph written by save.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

@classmethod
def load(cls, path: str | Path, *, embedder: TextEmbedder | None = None) -> SpatialMemory:
    """Load a memory from a JSON scene graph written by ``save``."""
    graph = SceneGraph.model_validate_json(Path(path).read_text())
    return cls.from_scene_graph(graph, embedder=embedder)

recall_object(query: RecallObjectQuery, *, now_ns: int) -> RecallObjectResult

Recall remembered objects matching query.

Matches object nodes by exact (case-insensitive) or substring label against query.label / query.text, applies the optional recency filter, ranks by match quality x confidence (proximity to query.near as a tiebreaker), and attaches a camera-facing approach viewpoint and any occluding container. Returns an empty result when nothing matches — the caller decides whether to raise ROSObjectNotInMemory or search.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def recall_object(self, query: RecallObjectQuery, *, now_ns: int) -> RecallObjectResult:
    """Recall remembered objects matching ``query``.

    Matches ``object`` nodes by exact (case-insensitive) or substring label
    against ``query.label`` / ``query.text``, applies the optional recency
    filter, ranks by match quality x confidence (proximity to
    ``query.near`` as a tiebreaker), and attaches a camera-facing approach
    viewpoint and any occluding container. Returns an empty result when
    nothing matches — the caller decides whether to raise
    ``ROSObjectNotInMemory`` or search.
    """
    term = (query.label or query.text).strip().lower()
    query_text = (query.text or query.label).strip()
    query_emb = (
        self._embedder.embed_text([query_text])[0]
        if self._embedder is not None and query_text
        else None
    )
    scored: list[tuple[float, float, SpatialNode]] = []
    for node in self._nodes.values():
        if node.kind is not SpatialNodeKind.OBJECT:
            continue
        if query.max_age_ns is not None and now_ns - node.last_seen_ns > query.max_age_ns:
            continue
        label_q = _label_match_quality(node.label, term)
        emb_sim = 0.0
        if query_emb is not None and node.node_id in self._embeddings:
            emb_sim = float(np.dot(query_emb, self._embeddings[node.node_id]))
        # Include on a label hit, or an embedding hit above the floor.
        if label_q <= 0.0 and emb_sim < self._min_text_similarity:
            continue
        score = max(label_q, emb_sim) * node.confidence
        proximity = _xyz_distance(node.pose, query.near) if query.near is not None else 0.0
        scored.append((score, proximity, node))

    # Rank: score desc, then nearer first, then most-recently seen.
    scored.sort(key=lambda t: (-t[0], t[1], -t[2].last_seen_ns))
    matches = [
        RecallObjectMatch(
            node_id=node.node_id,
            label=node.label,
            pose=node.pose,
            score=min(1.0, max(0.0, score)),
            last_seen_ns=node.last_seen_ns,
            approach=self._approach_for(node),
            inside_container_id=self._occluding_container_of(node.node_id),
        )
        for score, _prox, node in scored[: query.limit]
    ]
    return RecallObjectResult(matches=matches)

resolve_place(query: ResolvePlaceQuery, *, from_node_id: str | None = None) -> ResolvePlaceResult

Resolve a place/room/agent reference to a goal pose + path.

Raises:

Type	Description
ROSObjectNotInMemory	When the reference matches no node (the caller degrades to "unknown" / active search — it never fabricates a pose).

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def resolve_place(
    self, query: ResolvePlaceQuery, *, from_node_id: str | None = None
) -> ResolvePlaceResult:
    """Resolve a place/room/agent reference to a goal pose + path.

    Raises:
        ROSObjectNotInMemory: When the reference matches no node (the caller
            degrades to "unknown" / active search — it never fabricates a pose).
    """
    target = self._resolve_reference(query.reference, query.kind)
    if target is None:
        raise ROSObjectNotInMemory(
            f"no scene-graph node resolves reference {query.reference!r}"
        )
    goal_node = target
    if target.kind is not SpatialNodeKind.PLACE:
        place_id = self._at_place_of(target.node_id)
        if place_id is not None:
            goal_node = self._nodes[place_id]
    path = (
        self._traversable_path(from_node_id, goal_node.node_id)
        if from_node_id is not None
        else []
    )
    return ResolvePlaceResult(
        node_id=goal_node.node_id, goal=goal_node.pose, path_node_ids=path
    )

save(path: str | Path) -> None

Persist the scene graph to path as JSON (the SceneGraph contract).

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def save(self, path: str | Path) -> None:
    """Persist the scene graph to ``path`` as JSON (the SceneGraph contract)."""
    Path(path).write_text(self.to_scene_graph().model_dump_json(indent=2))

to_scene_graph() -> SceneGraph

Snapshot the memory as an immutable SceneGraph.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def to_scene_graph(self) -> SceneGraph:
    """Snapshot the memory as an immutable ``SceneGraph``."""
    return SceneGraph(nodes=list(self._nodes.values()), edges=list(self._edges.values()))

upsert_node(node: SpatialNode) -> None

Insert or replace a node by node_id.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def upsert_node(self, node: SpatialNode) -> None:
    """Insert or replace a node by ``node_id``."""
    self._nodes[node.node_id] = node

TextEmbedder

Bases: Protocol

A text → unit-vector embedder for open-vocabulary matching.

dim: int property

Embedding dimensionality.

embed_text(texts: Sequence[str]) -> NDArray[np.float32]

Embed texts into an (N, dim) L2-normalized float32 array.

Source code in python/world_state/src/openral_world_state/embedder.py

def embed_text(self, texts: Sequence[str]) -> NDArray[np.float32]:
    """Embed ``texts`` into an ``(N, dim)`` L2-normalized float32 array."""
    ...

VoxelFrustumLifter(*, k_nearest: int = 25, min_voxels: int = 3)

Lift 2D detections to 3D object centres via voxel-frustum K-nearest.

For each detection: project the occupied voxels into the camera image, keep those inside the (resolution-scaled) box, take the k_nearest whose projection is closest to the box centre, and return their mean position + axis-aligned bbox in the map frame. Detections with fewer than min_voxels in-box voxels are skipped (insufficient 3D evidence).

Validate K and the min-voxel floor.

Source code in python/world_state/src/openral_world_state/object_lift.py

def __init__(self, *, k_nearest: int = 25, min_voxels: int = 3) -> None:
    """Validate K and the min-voxel floor."""
    if k_nearest < 1:
        raise ValueError(f"k_nearest must be >= 1; got {k_nearest}")
    if min_voxels < 1:
        raise ValueError(f"min_voxels must be >= 1; got {min_voxels}")
    self._k = k_nearest
    self._min_voxels = min_voxels

lift(*, detections: Sequence[ObjectDetection2D], occupied_centers_base: NDArray[np.float64], intrinsics: IntrinsicsPinhole, frame_size: tuple[int, int], t_cam_from_base: NDArray[np.float64], t_map_from_base: NDArray[np.float64], map_frame: str = 'map') -> list[DetectedObject]

Lift each 2D detection to a DetectedObject in the map frame.

Parameters:

Name	Type	Description	Default
detections	Sequence[ObjectDetection2D]	Per-camera 2D detections (pixel bbox_xyxy).	required
occupied_centers_base	NDArray[float64]	(N, 3) occupied voxel centres in the robot base frame.	required
intrinsics	IntrinsicsPinhole	Pinhole intrinsics of the detection camera.	required
frame_size	tuple[int, int]	(width, height) in pixels of the frame the detector ran on; the box is scaled to the intrinsics resolution.	required
t_cam_from_base	NDArray[float64]	(4, 4) transform mapping base-frame points into the camera optical frame.	required
t_map_from_base	NDArray[float64]	(4, 4) transform mapping base-frame points into the map frame.	required
map_frame	str	tf2 frame id stamped on the returned poses.	'map'

Returns:

Type	Description
list[DetectedObject]	One DetectedObject per detection with enough in-box voxels;
list[DetectedObject]	detections below min_voxels are omitted.

Source code in python/world_state/src/openral_world_state/object_lift.py

def lift(
    self,
    *,
    detections: Sequence[ObjectDetection2D],
    occupied_centers_base: NDArray[np.float64],
    intrinsics: IntrinsicsPinhole,
    frame_size: tuple[int, int],
    t_cam_from_base: NDArray[np.float64],
    t_map_from_base: NDArray[np.float64],
    map_frame: str = "map",
) -> list[DetectedObject]:
    """Lift each 2D detection to a ``DetectedObject`` in the map frame.

    Args:
        detections: Per-camera 2D detections (pixel ``bbox_xyxy``).
        occupied_centers_base: ``(N, 3)`` occupied voxel centres in the
            robot base frame.
        intrinsics: Pinhole intrinsics of the detection camera.
        frame_size: ``(width, height)`` in pixels of the frame the detector
            ran on; the box is scaled to the intrinsics resolution.
        t_cam_from_base: ``(4, 4)`` transform mapping base-frame points into
            the camera optical frame.
        t_map_from_base: ``(4, 4)`` transform mapping base-frame points into
            the map frame.
        map_frame: tf2 frame id stamped on the returned poses.

    Returns:
        One ``DetectedObject`` per detection with enough in-box voxels;
        detections below ``min_voxels`` are omitted.
    """
    out: list[DetectedObject] = []
    n = int(occupied_centers_base.shape[0])
    if n == 0 or not detections:
        return out

    homo = np.concatenate(
        [occupied_centers_base, np.ones((n, 1), dtype=np.float64)],
        axis=1,
    )  # (N,4)
    cam = (t_cam_from_base @ homo.T).T[:, :3]  # (N,3) camera-optical
    z = cam[:, 2]
    in_front = z > _DEPTH_EPS
    u = np.full(n, -1.0, dtype=np.float64)
    v = np.full(n, -1.0, dtype=np.float64)
    u[in_front] = intrinsics.fx * cam[in_front, 0] / z[in_front] + intrinsics.cx
    v[in_front] = intrinsics.fy * cam[in_front, 1] / z[in_front] + intrinsics.cy

    map_pts = (t_map_from_base @ homo.T).T[:, :3]  # (N,3) map frame

    fw, fh = frame_size
    scale_x = intrinsics.width / fw if fw else 1.0
    scale_y = intrinsics.height / fh if fh else 1.0

    for det in detections:
        x0, y0, x1, y1 = det.bbox_xyxy
        bx0, bx1 = x0 * scale_x, x1 * scale_x
        by0, by1 = y0 * scale_y, y1 * scale_y
        ucen, vcen = (bx0 + bx1) / 2.0, (by0 + by1) / 2.0
        inside = in_front & (u >= bx0) & (u <= bx1) & (v >= by0) & (v <= by1)
        idx = np.nonzero(inside)[0]
        if idx.size < self._min_voxels:
            continue
        d2 = (u[idx] - ucen) ** 2 + (v[idx] - vcen) ** 2
        nearest = idx[np.argpartition(d2, self._k)[: self._k]] if idx.size > self._k else idx
        pts = map_pts[nearest]
        center = pts.mean(axis=0)
        mn = pts.min(axis=0)
        mx = pts.max(axis=0)
        out.append(
            DetectedObject(
                label=det.label,
                confidence=det.confidence,
                pose=Pose6D(
                    xyz=(float(center[0]), float(center[1]), float(center[2])),
                    quat_xyzw=(0.0, 0.0, 0.0, 1.0),
                    frame_id=map_frame,
                ),
                bbox_3d=(
                    float(mn[0]),
                    float(mn[1]),
                    float(mn[2]),
                    float(mx[0]),
                    float(mx[1]),
                    float(mx[2]),
                ),
                # Carry the detection-time id through the lift so a
                # physical object keeps one id across 2D (in_view) and 3D
                # (scene_objects). >= 0 → propagate; -1 (untracked) → None,
                # leaving ObjectMemory to mint as before.
                track_id=det.det_id if det.det_id >= 0 else None,
            ),
        )
    return out

WorldStateAggregator(description: RobotDescription, *, staleness_limit_s: float = DEFAULT_STALENESS_S, image_staleness_limit_s: float | None = None, policy_state_staleness_limit_s: float = DEFAULT_POLICY_STATE_STALENESS_S, clock_fn: Callable[[], int] | None = None)

Aggregates sensor data and produces WorldState snapshots.

The aggregator holds no ROS 2 imports. All data arrives via typed update methods called from subscriber callbacks in the enclosing ROS 2 lifecycle node. snapshot can be called from any thread; internal state is protected by a reentrant lock.

Parameters:

Name	Type	Description	Default
description	RobotDescription	Normative RobotDescription for the target robot. Used to initialise diagnostic keys for all declared sensor bundles and end-effectors.	required
staleness_limit_s	float	Maximum age (seconds) for a component reading before it is classified as "stale". Default 0.5 s.	DEFAULT_STALENESS_S
image_staleness_limit_s	float | None	Camera-specific age threshold. Defaults to staleness_limit_s; deploy sim raises it for slow rendered frames.	None
policy_state_staleness_limit_s	float	Separate window for the step-locked policy_state component, published once per simulator step rather than at a fixed rate. Default 5.0 s — see DEFAULT_POLICY_STATE_STALENESS_S.	DEFAULT_POLICY_STATE_STALENESS_S
clock_fn	Callable[[], int] | None	Callable returning the current time in nanoseconds; defaults to time.time_ns. Override in tests. Stamps arrivals/staleness only — place-declaration liveness uses the attachment stream's own clock (see update_attached_objects), never this one.	None

Example

from openral_core.schemas import JointState from openral_world_state.aggregator import WorldStateAggregator import time from openral_core import ( ... RobotDescription, ... EmbodimentKind, ... JointSpec, ... JointType, ... RobotCapabilities, ... SafetyEnvelope, ... ControlMode, ... ) desc = RobotDescription( ... name="t", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j0", ... joint_type=JointType.REVOLUTE, ... parent_link="base", ... child_link="l0", ... ) ... ], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION] ... ), ... safety=SafetyEnvelope(), ... ) agg = WorldStateAggregator(desc) agg.update_joint_state(JointState(name=["j0"], position=[0.5], stamp_ns=time.time_ns())) agg.snapshot().joint_state.position [0.5]

Initialise the aggregator; does not open any connection.

Source code in python/world_state/src/openral_world_state/aggregator.py

def __init__(
    self,
    description: RobotDescription,
    *,
    staleness_limit_s: float = DEFAULT_STALENESS_S,
    image_staleness_limit_s: float | None = None,
    policy_state_staleness_limit_s: float = DEFAULT_POLICY_STATE_STALENESS_S,
    clock_fn: Callable[[], int] | None = None,
) -> None:
    """Initialise the aggregator; does not open any connection."""
    self.description = description
    self._staleness_limit_ns: int = int(staleness_limit_s * 1e9)
    self._image_staleness_limit_ns: int = int(
        (staleness_limit_s if image_staleness_limit_s is None else image_staleness_limit_s)
        * 1e9
    )
    self._policy_state_staleness_limit_ns: int = int(policy_state_staleness_limit_s * 1e9)
    self._clock_fn: Callable[[], int] = clock_fn or time.time_ns
    self._lock = threading.RLock()

    # ── Tracked state (all protected by _lock) ────────────────────────
    self._joint_state: JointState | None = None
    self._joint_state_stamp_ns: int = 0

    # sensor_name → (topic_ref, stamp_ns)
    self._images: dict[str, tuple[str, int]] = {}
    # sensor_name → (SensorFrame, stamp_ns) — actual bytes for the
    # consumer that wants pixels in-process (skill_runner →
    # rSkill). Populated by `update_image_frame`; surfaced through
    # `WorldState.image_frames` so a Skill can read pixels without
    # opening its own ROS subscription.
    self._image_frames: dict[str, tuple[SensorFrame, int]] = {}
    # ee_name → (Pose6D, stamp_ns)
    self._ee_poses: dict[str, tuple[Pose6D, int]] = {}
    # base pose + stamp
    self._base_pose: Pose6D | None = None
    self._base_pose_stamp_ns: int = 0
    self._base_twist: tuple[float, float, float, float, float, float] | None = None
    self._policy_state: list[float] | None = None
    self._policy_state_stamp_ns: int = 0
    # battery
    self._battery_pct: float | None = None
    # latest object-memory snapshot (already deduped/evicted by
    # ObjectMemory in the enclosing node). Stored verbatim; no staleness
    # logic here (the memory owns lifecycle and refreshes every tick).
    self._detected_objects: list[DetectedObject] = []
    # Atomic attached-payload snapshot keyed by stable world object id.
    self._attached_objects: dict[str, AttachedCollisionObject] = {}
    self._attachment_revision: int = 0
    self._attachment_stamp_ns: int = 0
    # ADR-0097 (+2026-08-14 amendment): place-phase declaration for the
    # carried payload, replaced atomically with the attachment set it
    # scopes to; re-evaluated per attachment message (HZ-0097-3) so a
    # dead dispatcher can't leave a region armed.
    #
    # Liveness uses the ATTACHMENT STREAM'S own clock (`_attachment_stamp_ns`
    # — the rSkill runner's ROS clock, sim time under `use_sim_time`), never
    # `_clock_fn` (wall `time.time_ns`): comparing against `_clock_fn` once
    # put every sim-stamped declaration ~1.79e18 ns past its backstop and
    # published `place_declaration=None` forever. See `snapshot`.
    #
    # HZ-0097-3/-4 coverage: (1) the producer heartbeat re-runs the backstop
    # each beat, so an outlived declaration drops on the next message; (2) a
    # stopped stream can't expire it here, but also starves the kernel's
    # `attached_collision_deadline_s` freshness gate, which then refuses
    # every candidate action, not just the allowance; (3) the kernel
    # re-checks `place_declaration_live()` per action on its own ROS clock —
    # the declaration's actual domain.
    self._place_declaration: PlaceDeclaration | None = None
    # Grasp-phase declaration (real pick-and-place design §2.1): same
    # envelope, same atomic replacement, same stream-clock liveness gate as
    # the place declaration above.
    self._grasp_declaration: GraspDeclaration | None = None
    # latched diagnostics for explicitly set errors
    self._forced_errors: dict[str, DiagStatus] = {}
    # Stale components from the previous snapshot — used by snapshot() to
    # emit a ``staleness_latched`` span event only on the tick where a
    # component first goes stale (not every tick it stays stale).
    self._prev_stale_components: set[str] = set()
    # Same for latched errors so we don't re-emit on every snapshot.
    self._prev_latched_errors: set[str] = set()

    # Initialise diagnostic keys from description
    self._sensor_names: set[str] = {
        s.name for bundle in description.sensor_bundles for s in bundle.sensors
    }
    # End-effector poses are lazily registered (like cameras, see
    # update_ee_pose / update_image_frame): a declared EE only appears in
    # the diagnostics once it has received at least one pose. Pre-populating
    # from description.end_effectors meant every robot reported its
    # gripper(s) permanently STALE whenever no pose source was wired (the
    # tf2/forward-kinematics EE feed is not active on the sim deploy path),
    # which is pure noise. Track what the description *declares* separately
    # for reference/logging without forcing it into the stale ledger.
    self._declared_ee_names: frozenset[str] = (
        frozenset(ee.name for ee in description.end_effectors)
        if description.end_effectors
        else frozenset()
    )
    self._ee_names: set[str] = set()

    log.info(
        "world_state.aggregator.init",
        robot=description.name,
        staleness_limit_s=staleness_limit_s,
        image_staleness_limit_s=self._image_staleness_limit_ns / 1e9,
        sensor_count=len(self._sensor_names),
        ee_count=len(self._declared_ee_names),
    )

clear_error(component: str) -> None

Remove a forced diagnostic entry for a named component.

Parameters:

Name	Type	Description	Default
component	str	Diagnostic key to clear.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def clear_error(self, component: str) -> None:
    """Remove a forced diagnostic entry for a named component.

    Args:
        component: Diagnostic key to clear.
    """
    with self._lock:
        self._forced_errors.pop(component, None)

set_error(component: str, status: DiagStatus = 'error') -> None

Latch an explicit diagnostic status for a named component.

Use to surface hardware faults or driver errors that go beyond mere staleness. The forced status persists until clear_error is called.

Parameters:

Name	Type	Description	Default
component	str	Diagnostic key (e.g. "joint_state", sensor name).	required
status	DiagStatus	Diagnostic level to latch; typically "error" or "warn".	'error'

Source code in python/world_state/src/openral_world_state/aggregator.py

def set_error(self, component: str, status: DiagStatus = "error") -> None:
    """Latch an explicit diagnostic status for a named component.

    Use to surface hardware faults or driver errors that go beyond
    mere staleness.  The forced status persists until ``clear_error``
    is called.

    Args:
        component: Diagnostic key (e.g. ``"joint_state"``, sensor name).
        status: Diagnostic level to latch; typically ``"error"`` or
            ``"warn"``.
    """
    with self._lock:
        self._forced_errors[component] = status
    log.warning("world_state.error.latched", component=component, status=status)

snapshot() -> WorldState

Produce a typed WorldState snapshot from current aggregated data.

Staleness is evaluated at call time. Components older than staleness_limit_s appear as "stale" in WorldState.diagnostics. Latched errors override staleness.

Emits a world_state.snapshot OTel span recording openral.world_state.components_stale and openral.world_state.has_latched_error, plus a transition-only openral.event.staleness_latched / ..._error_latched event and a per-component openral.world_state.staleness_ms histogram entry.

Returns:

Type	Description
WorldState	An immutable WorldState snapshot.

Source code in python/world_state/src/openral_world_state/aggregator.py

def snapshot(self) -> WorldState:
    """Produce a typed ``WorldState`` snapshot from current aggregated data.

    Staleness is evaluated at call time. Components older than
    ``staleness_limit_s`` appear as ``"stale"`` in
    ``WorldState.diagnostics``. Latched errors override staleness.

    Emits a ``world_state.snapshot`` OTel span recording
    ``openral.world_state.components_stale`` and
    ``openral.world_state.has_latched_error``, plus a transition-only
    ``openral.event.staleness_latched`` / ``..._error_latched`` event and a
    per-component ``openral.world_state.staleness_ms`` histogram entry.

    Returns:
        An immutable ``WorldState`` snapshot.
    """
    with (
        self._lock,
        _tracer().start_as_current_span(semconv.SPAN_WORLD_STATE_SNAPSHOT) as span,
    ):
        now_ns = self._clock_fn()
        diag: dict[str, str] = {}
        ages_ms: dict[str, float] = {}

        # Joint state
        if self._joint_state is None:
            # Create a zeroed state so Skills always get a valid object;
            # mark diagnostic as stale.
            joint_count = len(self.description.joints)
            joint_names = [j.name for j in self.description.joints]
            js = JointState(
                name=joint_names,
                position=[0.0] * joint_count,
                velocity=[0.0] * joint_count,
                effort=[],  # not measured: zeros would read as an unloaded arm
                stamp_ns=now_ns,
            )
            diag["joint_state"] = "stale"
        else:
            js = self._joint_state
            age_ns = now_ns - self._joint_state_stamp_ns
            diag["joint_state"] = "ok" if age_ns <= self._staleness_limit_ns else "stale"
            ages_ms["joint_state"] = age_ns / 1e6

        if self._policy_state is not None:
            policy_age_ns = now_ns - self._policy_state_stamp_ns
            diag["policy_state"] = (
                "ok" if policy_age_ns <= self._policy_state_staleness_limit_ns else "stale"
            )
            ages_ms["policy_state"] = policy_age_ns / 1e6

        # Images — topic refs from last received frames
        images: dict[str, str] = {}
        for sensor_name in self._sensor_names:
            if sensor_name in self._images:
                topic, stamp = self._images[sensor_name]
                age_ns = now_ns - stamp
                diag[sensor_name] = (
                    "ok" if age_ns <= self._image_staleness_limit_ns else "stale"
                )
                images[sensor_name] = topic
                ages_ms[sensor_name] = age_ns / 1e6
            else:
                diag[sensor_name] = "stale"

        # EE poses
        ee_poses: dict[str, Pose6D] = {}
        for ee_name in self._ee_names:
            if ee_name in self._ee_poses:
                pose, stamp = self._ee_poses[ee_name]
                age_ns = now_ns - stamp
                diag[ee_name] = "ok" if age_ns <= self._staleness_limit_ns else "stale"
                ee_poses[ee_name] = pose
                ages_ms[ee_name] = age_ns / 1e6
            else:
                diag[ee_name] = "stale"

        # Forced errors override staleness classification
        diag.update(self._forced_errors)

        self._emit_snapshot_telemetry(span, diag, ages_ms)
        # Surface the embodiment view on the same span the dashboard
        # consumes — ee poses (named) + battery + joint stamp + the
        # per-component diagnostics list. Stale data still flows so
        # the dashboard can render a warn pill on the stale row.
        _producer.record_ee_poses(span, ee_poses)
        if self._battery_pct is not None:
            span.set_attribute("openral.world_state.battery_pct", float(self._battery_pct))
        span.set_attribute("openral.world_state.diagnostics_keys", sorted(diag.keys()))
        span.set_attribute(
            "openral.world_state.diagnostics_values",
            [diag[k] for k in sorted(diag.keys())],
        )
        span.set_attribute(semconv.HAL_JOINT_NAMES, list(js.name))
        span.set_attribute(
            semconv.HAL_JOINT_POSITIONS,
            [round(float(v), 3) for v in js.position],
        )

        # Inline pixel payloads: surface every camera that fed
        # `update_image_frame` since the last reset. The Pydantic
        # field is Optional; pass None when nothing has arrived so
        # the snapshot stays cheap when no cameras are wired.
        image_frames: dict[str, SensorFrame] | None = None
        if self._image_frames:
            image_frames = {n: f for n, (f, _) in self._image_frames.items()}

        return WorldState(
            stamp_ns=now_ns,
            joint_state=js,
            base_pose=self._base_pose,
            base_twist=self._base_twist,
            policy_state=list(self._policy_state) if self._policy_state is not None else None,
            ee_poses=ee_poses,
            images=images,
            image_frames=image_frames,
            battery_pct=self._battery_pct,
            attached_objects=[
                self._attached_objects[object_id]
                for object_id in sorted(self._attached_objects)
            ],
            attachment_revision=self._attachment_revision,
            attachment_stamp_ns=self._attachment_stamp_ns,
            # Published verbatim: liveness was decided in
            # `update_attached_objects`, against this stream's own clock.
            # `now_ns` is `_clock_fn` — a DIFFERENT clock — and re-checking
            # here against it is what silently killed every sim-time
            # declaration. Between attachment messages World State holds no
            # reading of the stream's clock, so it cannot expire a stored
            # declaration on its own and does not pretend to; see the
            # HZ-0097-3 note in `__init__` for what does.
            place_declaration=self._place_declaration,
            grasp_declaration=self._grasp_declaration,
            diagnostics=diag,
            detected_objects=list(self._detected_objects),
        )

update_attached_objects(objects: list[AttachedCollisionObject], *, revision: int = 0, stamp_ns: int | None = None, place_declaration: PlaceDeclaration | None = None, grasp_declaration: GraspDeclaration | None = None) -> None

Atomically replace the attached-payload set.

Parameters:

Name	Type	Description	Default
objects	list[AttachedCollisionObject]	Complete current attachment set. Object ids must be unique.	required
revision	int	Monotonic producer revision.	0
stamp_ns	int | None	Producer confirmation time; arrival time when omitted.	None
place_declaration	PlaceDeclaration | None	The place-phase declaration the evidence producer resolved for this payload, with its measured region (ADR-0097's 2026-08-14 amendment), or None for no place phase. Replaced atomically with the attachment set it scopes to, so the kernel can never apply a region and an attachment snapshot that disagree. A declaration already retracted or expired is stored as None. Liveness is evaluated against stamp_ns (this message's own stream-clock stamp) — the only clock it is comparable with. This is the single evaluation point; snapshot publishes what is stored here.	None
grasp_declaration	GraspDeclaration | None	The grasp-phase declaration the evidence producer resolved, with its measured region, or None. Stored atomically with the set and gated on is_live against the same stamp_ns as place_declaration; a set that carries none clears it.	None

Raises:

Type	Description
ValueError	If ids duplicate or the revision moves backwards.

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_attached_objects(
    self,
    objects: list[AttachedCollisionObject],
    *,
    revision: int = 0,
    stamp_ns: int | None = None,
    place_declaration: PlaceDeclaration | None = None,
    grasp_declaration: GraspDeclaration | None = None,
) -> None:
    """Atomically replace the attached-payload set.

    Args:
        objects: Complete current attachment set. Object ids must be unique.
        revision: Monotonic producer revision.
        stamp_ns: Producer confirmation time; arrival time when omitted.
        place_declaration: The place-phase declaration the evidence producer
            resolved for this payload, with its measured region (ADR-0097's
            2026-08-14 amendment), or ``None`` for no place phase. Replaced
            atomically with the attachment set it scopes to, so the kernel
            can never apply a region and an attachment snapshot that
            disagree. A declaration already retracted or expired is stored
            as ``None``.

            Liveness is evaluated against ``stamp_ns`` (this message's own
            stream-clock stamp) — the only clock it is comparable with. This
            is the single evaluation point; ``snapshot`` publishes what
            is stored here.
        grasp_declaration: The grasp-phase declaration the evidence
            producer resolved, with its measured region, or ``None``.
            Stored atomically with the set and gated on ``is_live`` against
            the same ``stamp_ns`` as ``place_declaration``; a set that
            carries none clears it.

    Raises:
        ValueError: If ids duplicate or the revision moves backwards.
    """
    by_id = {obj.object_id: obj for obj in objects}
    if len(by_id) != len(objects):
        raise ValueError("Attached collision object ids must be unique.")
    # With no producer stamp the arrival moment IS the stream's clock reading,
    # so `_clock_fn` is the right "now" for that case and only that case.
    applied_stamp_ns = self._clock_fn() if stamp_ns is None else stamp_ns
    live_declaration = (
        place_declaration
        if place_declaration is not None and place_declaration.is_live(now_ns=applied_stamp_ns)
        else None
    )
    live_grasp = (
        grasp_declaration
        if grasp_declaration is not None and grasp_declaration.is_live(now_ns=applied_stamp_ns)
        else None
    )
    with self._lock:
        if revision < self._attachment_revision:
            raise ValueError(
                f"Attachment revision moved backwards: "
                f"{revision} < {self._attachment_revision}."
            )
        # The producer republishes its latched set on a heartbeat; log the
        # set changing, not every confirmation of it.
        changed = (
            by_id != self._attached_objects
            or live_declaration != self._place_declaration
            or live_grasp != self._grasp_declaration
        )
        self._attached_objects = by_id
        self._attachment_revision = revision
        self._attachment_stamp_ns = applied_stamp_ns
        self._place_declaration = live_declaration
        self._grasp_declaration = live_grasp
    if changed:
        log.info(
            "world_state.attached_objects.updated",
            count=len(objects),
            place_target=live_declaration.target_id if live_declaration is not None else None,
            place_region=live_declaration is not None and live_declaration.region is not None,
            grasp_target=live_grasp.target_id if live_grasp is not None else None,
            grasp_region=live_grasp is not None and live_grasp.region is not None,
        )

update_base_pose(pose: Pose6D, twist: tuple[float, float, float, float, float, float] | None = None) -> None

Record a fresh base link pose (and optional twist) from tf2.

Parameters:

Name	Type	Description	Default
pose	Pose6D	Base link 6D pose in the world/map frame.	required
twist	tuple[float, float, float, float, float, float] | None	Optional (vx, vy, vz, wx, wy, wz) twist.	None

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_base_pose(
    self,
    pose: Pose6D,
    twist: tuple[float, float, float, float, float, float] | None = None,
) -> None:
    """Record a fresh base link pose (and optional twist) from tf2.

    Args:
        pose: Base link 6D pose in the world/map frame.
        twist: Optional (vx, vy, vz, wx, wy, wz) twist.
    """
    with self._lock:
        self._base_pose = pose
        self._base_pose_stamp_ns = self._clock_fn()
        self._base_twist = twist
    log.debug("world_state.base_pose.updated")

update_battery(pct: float) -> None

Record the latest battery percentage.

Parameters:

Name	Type	Description	Default
pct	float	Battery percentage in [0, 100].	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_battery(self, pct: float) -> None:
    """Record the latest battery percentage.

    Args:
        pct: Battery percentage in [0, 100].
    """
    with self._lock:
        self._battery_pct = pct

update_detected_objects(objects: list[DetectedObject]) -> None

Replace the remembered detected-object set.

Called from the world-state node's memory tick with the current ObjectMemory output (already associated, frozen, and evicted). A copy is stored so later external mutation of objects cannot alter the snapshot.

Parameters:

Name	Type	Description	Default
objects	list[DetectedObject]	Current spatial-memory objects, anchored in the map frame.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_detected_objects(self, objects: list[DetectedObject]) -> None:
    """Replace the remembered detected-object set.

    Called from the world-state node's memory tick with the current
    ``ObjectMemory`` output (already associated, frozen, and evicted). A
    copy is stored so later external mutation of ``objects`` cannot alter
    the snapshot.

    Args:
        objects: Current spatial-memory objects, anchored in the map frame.
    """
    with self._lock:
        self._detected_objects = list(objects)
    log.debug("world_state.detected_objects.updated", count=len(objects))

update_ee_pose(ee_name: str, pose: Pose6D) -> None

Record a fresh end-effector pose from a tf2 lookup.

Parameters:

Name	Type	Description	Default
ee_name	str	Matches an EndEffectorSpec.name in the description.	required
pose	Pose6D	Current 6D pose in the world frame.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_ee_pose(self, ee_name: str, pose: Pose6D) -> None:
    """Record a fresh end-effector pose from a tf2 lookup.

    Args:
        ee_name: Matches an ``EndEffectorSpec.name`` in the description.
        pose: Current 6D pose in the world frame.
    """
    with self._lock:
        self._ee_poses[ee_name] = (pose, self._clock_fn())
        # Lazy registration: only EEs that have actually been observed enter
        # the diagnostics ledger (see __init__ for why pre-population is noise).
        self._ee_names.add(ee_name)
    log.debug("world_state.ee_pose.updated", ee=ee_name)

update_image(sensor_name: str, topic: str, stamp_ns: int) -> None

Record a fresh image frame arrival for a named sensor.

Parameters:

Name	Type	Description	Default
sensor_name	str	Matches a SensorSpec.name in the description.	required
topic	str	ROS 2 topic the image was published on.	required
stamp_ns	int	Frame timestamp in nanoseconds.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_image(self, sensor_name: str, topic: str, stamp_ns: int) -> None:
    """Record a fresh image frame arrival for a named sensor.

    Args:
        sensor_name: Matches a ``SensorSpec.name`` in the description.
        topic: ROS 2 topic the image was published on.
        stamp_ns: Frame timestamp in nanoseconds.
    """
    with self._lock:
        self._images[sensor_name] = (topic, self._clock_fn())
    log.debug("world_state.image.updated", sensor=sensor_name)

update_image_frame(sensor_name: str, frame: SensorFrame) -> None

Record an inline pixel payload for a named sensor.

Unlike update_image (topic ref only), stores the actual SensorFrame so snapshot carries pixels inline via WorldState.image_frames — the path a Skill uses without its own ROS subscription (CLAUDE.md §3, Layer 1→2→3). Accepts sensor names not declared in RobotDescription.sensor_bundles: synthetic digital-twin cameras live in the active MJCF, not the robot manifest.

Parameters:

Name	Type	Description	Default
sensor_name	str	Camera id; key in WorldState.image_frames.	required
frame	SensorFrame	Validated SensorFrame (data/topic/handle).	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_image_frame(self, sensor_name: str, frame: SensorFrame) -> None:
    """Record an inline pixel payload for a named sensor.

    Unlike ``update_image`` (topic ref only), stores the actual
    ``SensorFrame`` so ``snapshot`` carries pixels inline via
    ``WorldState.image_frames`` — the path a Skill uses without its
    own ROS subscription (CLAUDE.md §3, Layer 1→2→3). Accepts sensor
    names not declared in ``RobotDescription.sensor_bundles``: synthetic
    digital-twin cameras live in the active MJCF, not the robot manifest.

    Args:
        sensor_name: Camera id; key in ``WorldState.image_frames``.
        frame: Validated ``SensorFrame`` (``data``/``topic``/``handle``).
    """
    stamp = self._clock_fn()
    with self._lock:
        self._image_frames[sensor_name] = (frame, stamp)
        # Mirror into `_images` so the diagnostics + the
        # `WorldState.images` topic map stay populated even for
        # sensors that are not in the robot.yaml. Use the frame's
        # `topic` when set, otherwise a placeholder.
        self._images[sensor_name] = (
            frame.topic if frame.topic is not None else f"<inline:{sensor_name}>",
            stamp,
        )
        self._sensor_names.add(sensor_name)
    log.debug(
        "world_state.image_frame.updated",
        sensor=sensor_name,
        encoding=str(frame.encoding),
        width=frame.width,
        height=frame.height,
    )

update_joint_state(state: JointState) -> None

Record a fresh joint state reading.

Parameters:

Name	Type	Description	Default
state	JointState	Latest JointState from the /joint_states topic.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_joint_state(self, state: JointState) -> None:
    """Record a fresh joint state reading.

    Args:
        state: Latest ``JointState`` from the ``/joint_states`` topic.
    """
    with self._lock:
        self._joint_state = state
        self._joint_state_stamp_ns = self._clock_fn()
    log.debug("world_state.joint_state.updated", robot=self.description.name)

update_policy_state(values: list[float]) -> None

Record the simulator-native policy proprioception vector.

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_policy_state(self, values: list[float]) -> None:
    """Record the simulator-native policy proprioception vector."""
    with self._lock:
        self._policy_state = [float(value) for value in values]
        self._policy_state_stamp_ns = self._clock_fn()
    log.debug(
        "world_state.policy_state.updated",
        robot=self.description.name,
        dim=len(values),
    )

aabb_iou_3d(a: tuple[float, float, float, float, float, float], b: tuple[float, float, float, float, float, float]) -> float

3D axis-aligned bbox IoU.

Parameters:

Name	Type	Description	Default
a	tuple[float, float, float, float, float, float]	First box as (xmin, ymin, zmin, xmax, ymax, zmax).	required
b	tuple[float, float, float, float, float, float]	Second box as (xmin, ymin, zmin, xmax, ymax, zmax).	required

Returns:

Type	Description
float	Intersection-over-union in [0, 1]; 0.0 for disjoint or
float	degenerate (zero-volume) boxes.

Source code in python/world_state/src/openral_world_state/object_lift.py

def aabb_iou_3d(
    a: tuple[float, float, float, float, float, float],
    b: tuple[float, float, float, float, float, float],
) -> float:
    """3D axis-aligned bbox IoU.

    Args:
        a: First box as ``(xmin, ymin, zmin, xmax, ymax, zmax)``.
        b: Second box as ``(xmin, ymin, zmin, xmax, ymax, zmax)``.

    Returns:
        Intersection-over-union in ``[0, 1]``; ``0.0`` for disjoint or
        degenerate (zero-volume) boxes.
    """
    ix = max(0.0, min(a[3], b[3]) - max(a[0], b[0]))
    iy = max(0.0, min(a[4], b[4]) - max(a[1], b[1]))
    iz = max(0.0, min(a[5], b[5]) - max(a[2], b[2]))
    inter = ix * iy * iz
    if inter <= 0.0:
        return 0.0
    vol_a = max(0.0, a[3] - a[0]) * max(0.0, a[4] - a[1]) * max(0.0, a[5] - a[2])
    vol_b = max(0.0, b[3] - b[0]) * max(0.0, b[4] - b[1]) * max(0.0, b[5] - b[2])
    union = vol_a + vol_b - inter
    return inter / union if union > 0.0 else 0.0

build_in_fov_predicate(*, intrinsics: IntrinsicsPinhole, t_cam_from_map: NDArray[np.float64]) -> Callable[[DetectedObject], bool]

Return a predicate: does a map-frame object project into the camera image?

Used by ObjectMemory to decide whether an unmatched object should have been re-seen (in view => a real miss) or is merely out of view (retain). Objects whose centre is behind the camera or outside the image bounds return False. Image bounds are inclusive of the far edge (0 <= u <= width), using continuous pixel coordinates.

Parameters:

Name	Type	Description	Default
intrinsics	IntrinsicsPinhole	Pinhole intrinsics of the camera; width/height bound the image rectangle.	required
t_cam_from_map	NDArray[float64]	(4, 4) transform mapping map-frame points into the camera optical frame.	required

Returns:

Type	Description
Callable[[DetectedObject], bool]	A callable in_fov(obj) -> bool.

Source code in python/world_state/src/openral_world_state/object_lift.py

def build_in_fov_predicate(
    *,
    intrinsics: IntrinsicsPinhole,
    t_cam_from_map: NDArray[np.float64],
) -> Callable[[DetectedObject], bool]:
    """Return a predicate: does a map-frame object project into the camera image?

    Used by ``ObjectMemory`` to decide whether an unmatched object *should* have
    been re-seen (in view => a real miss) or is merely out of view (retain).
    Objects whose centre is behind the camera or outside the image bounds return
    ``False``. Image bounds are inclusive of the far edge (``0 <= u <= width``),
    using continuous pixel coordinates.

    Args:
        intrinsics: Pinhole intrinsics of the camera; ``width``/``height`` bound
            the image rectangle.
        t_cam_from_map: ``(4, 4)`` transform mapping map-frame points into the
            camera optical frame.

    Returns:
        A callable ``in_fov(obj) -> bool``.
    """
    w, h = float(intrinsics.width), float(intrinsics.height)

    def in_fov(obj: DetectedObject) -> bool:
        x, y, z = obj.pose.xyz
        p = t_cam_from_map @ np.array([x, y, z, 1.0], dtype=np.float64)
        if p[2] <= _DEPTH_EPS:
            return False
        u = intrinsics.fx * p[0] / p[2] + intrinsics.cx
        v = intrinsics.fy * p[1] / p[2] + intrinsics.cy
        return bool(0.0 <= u <= w and 0.0 <= v <= h)

    return in_fov

compute_approach_viewpoint(target: Pose6D, *, standoff_m: float = DEFAULT_STANDOFF_M, camera_frame_id: str = DEFAULT_CAMERA_FRAME, approach_from: Pose6D | None = None) -> ApproachViewpoint

Compute a standoff pose whose camera faces target.

The viewpoint is placed standoff_m away from the target in the horizontal (x, y) plane and yawed to look at it. When approach_from is given (e.g. the place a robot would stand at), the viewpoint sits on that side of the target; otherwise it defaults to approaching from -X.

Parameters:

Name	Type	Description	Default
target	Pose6D	Object pose to view, in the map frame.	required
standoff_m	float	Standoff distance in metres (> 0).	DEFAULT_STANDOFF_M
camera_frame_id	str	tf2 frame of the camera the viewpoint orients toward.	DEFAULT_CAMERA_FRAME
approach_from	Pose6D | None	Optional pose giving the side to approach from.	None

Returns:

Type	Description
ApproachViewpoint	An ApproachViewpoint in the target's frame.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def compute_approach_viewpoint(
    target: Pose6D,
    *,
    standoff_m: float = DEFAULT_STANDOFF_M,
    camera_frame_id: str = DEFAULT_CAMERA_FRAME,
    approach_from: Pose6D | None = None,
) -> ApproachViewpoint:
    """Compute a standoff pose whose camera faces ``target``.

    The viewpoint is placed ``standoff_m`` away from the target in the
    horizontal (x, y) plane and yawed to look at it. When ``approach_from`` is
    given (e.g. the place a robot would stand at), the viewpoint sits on that
    side of the target; otherwise it defaults to approaching from -X.

    Args:
        target: Object pose to view, in the map frame.
        standoff_m: Standoff distance in metres (``> 0``).
        camera_frame_id: tf2 frame of the camera the viewpoint orients toward.
        approach_from: Optional pose giving the side to approach from.

    Returns:
        An ``ApproachViewpoint`` in the target's frame.
    """
    tx, ty, tz = target.xyz
    if approach_from is not None:
        ax, ay, _az = approach_from.xyz
        dx, dy = ax - tx, ay - ty
        norm = math.hypot(dx, dy)
        ux, uy = (dx / norm, dy / norm) if norm > _MIN_DIR_NORM else (-1.0, 0.0)
    else:
        ux, uy = -1.0, 0.0
    vp_x, vp_y = tx + ux * standoff_m, ty + uy * standoff_m
    # Yaw so the camera (looking down its +X) points from the viewpoint at the target.
    yaw = math.atan2(ty - vp_y, tx - vp_x)
    return ApproachViewpoint(
        pose=Pose6D(
            xyz=(vp_x, vp_y, tz), quat_xyzw=yaw_to_quat_xyzw(yaw), frame_id=target.frame_id
        ),
        standoff_m=standoff_m,
        camera_frame_id=camera_frame_id,
    )

compute_gaze_pose(camera_xyz: tuple[float, float, float], target_xyz: tuple[float, float, float], *, frame_id: str = 'map', up: tuple[float, float, float] = (0.0, 0.0, 1.0), view_axis: ViewAxis = '+z') -> Pose6D

Full 6-DOF camera pose at camera_xyz whose view axis hits target_xyz.

The pose the rskill-moveit-multi-look_at rSkill plans the camera frame to: position fixed at camera_xyz, orientation from look_at_quat_wxyz. Defaults to the ROS optical-frame convention ("+z" forward) since real camera frame_ids are optical frames.

Parameters:

Name	Type	Description	Default
camera_xyz	tuple[float, float, float]	Where the camera sits, in frame_id.	required
target_xyz	tuple[float, float, float]	The point to aim at, in frame_id.	required
frame_id	str	tf2 frame both points are expressed in.	'map'
up	tuple[float, float, float]	World up used to level the camera roll.	(0.0, 0.0, 1.0)
view_axis	ViewAxis	Camera forward-axis convention.	'+z'

Returns:

Type	Description
Pose6D	A Pose6D (quat_xyzw order).

Example

pose = compute_gaze_pose((0.0, 0.0, 1.0), (1.0, 0.0, 1.0)) pose.frame_id 'map' pose.xyz (0.0, 0.0, 1.0)

Source code in python/core/src/openral_core/geometry.py

def compute_gaze_pose(
    camera_xyz: tuple[float, float, float],
    target_xyz: tuple[float, float, float],
    *,
    frame_id: str = "map",
    up: tuple[float, float, float] = (0.0, 0.0, 1.0),
    view_axis: ViewAxis = "+z",
) -> Pose6D:
    """Full 6-DOF camera pose at ``camera_xyz`` whose view axis hits ``target_xyz``.

    The pose the ``rskill-moveit-multi-look_at`` rSkill plans the
    camera frame to: position fixed at ``camera_xyz``, orientation from
    ``look_at_quat_wxyz``. Defaults to the ROS optical-frame convention
    (``"+z"`` forward) since real camera ``frame_id``s are optical frames.

    Args:
        camera_xyz: Where the camera sits, in ``frame_id``.
        target_xyz: The point to aim at, in ``frame_id``.
        frame_id: tf2 frame both points are expressed in.
        up: World up used to level the camera roll.
        view_axis: Camera forward-axis convention.

    Returns:
        A ``Pose6D`` (``quat_xyzw`` order).

    Example:
        >>> pose = compute_gaze_pose((0.0, 0.0, 1.0), (1.0, 0.0, 1.0))
        >>> pose.frame_id
        'map'
        >>> pose.xyz
        (0.0, 0.0, 1.0)
    """
    w, x, y, z = look_at_quat_wxyz(camera_xyz, target_xyz, up=up, view_axis=view_axis)
    return Pose6D(xyz=camera_xyz, quat_xyzw=(x, y, z, w), frame_id=frame_id)

decode_occupied_centers(*, origin: tuple[float, float, float], resolution: float, size_xyz: tuple[int, int, int], occupancy: bytes, orientation_xyzw: tuple[float, float, float, float] = (0.0, 0.0, 0.0, 1.0)) -> NDArray[np.float64]

Occupied-voxel centers (N, 3) in the grid's reference frame.

Cell (ix, iy, iz)'s center is origin + R * ((index + 0.5) * resolution), where R is orientation_xyzw. A cell is occupied when its byte is non-zero.

openral_msgs/OccupancyVoxels is an ORIENTED lattice: its cell axes are the source map's, not header.frame_id's, so the rotation is part of placing a cell and not decoration. Dropping it puts every returned center in the wrong place whenever the robot is not aligned with the map.

Parameters:

Name	Type	Description	Default
origin	tuple[float, float, float]	(x, y, z) position of cell (0, 0, 0)'s minimum corner, in header.frame_id, in metres.	required
resolution	float	Edge length of a cubic voxel in metres.	required
size_xyz	tuple[int, int, int]	Grid dimensions (size_x, size_y, size_z) in voxels.	required
occupancy	bytes	Flat occupancy buffer, one byte per voxel, row-major with x fastest: idx = ix + size_x * (iy + size_y * iz). Non-zero byte means occupied.	required
orientation_xyzw	tuple[float, float, float, float]	Grid orientation as (x, y, z, w). Defaults to identity for callers building an axis-aligned grid themselves; a caller decoding a wire message must pass the message's own, and an all-zero quaternion (the unset default) is rejected rather than read as identity.	(0.0, 0.0, 0.0, 1.0)

Returns:

Type	Description
NDArray[float64]	A (N, 3) float64 array of occupied-voxel centers in the grid's
NDArray[float64]	reference frame, or shape (0, 3) when none are occupied.

Raises:

Type	Description
ObjectsLiftError	If len(occupancy) does not equal size_x * size_y * size_z, or orientation_xyzw is not a unit quaternion.

Example

centers = decode_occupied_centers( ... origin=(0.0, 0.0, 0.0), ... resolution=0.1, ... size_xyz=(2, 1, 1), ... occupancy=bytes([0, 1]), ... ) tuple(round(float(v), 3) for v in centers[0]) (0.15, 0.05, 0.05)

Source code in python/world_state/src/openral_world_state/object_lift.py

def decode_occupied_centers(
    *,
    origin: tuple[float, float, float],
    resolution: float,
    size_xyz: tuple[int, int, int],
    occupancy: bytes,
    orientation_xyzw: tuple[float, float, float, float] = (0.0, 0.0, 0.0, 1.0),
) -> NDArray[np.float64]:
    """Occupied-voxel centers ``(N, 3)`` in the grid's reference frame.

    Cell ``(ix, iy, iz)``'s center is ``origin + R * ((index + 0.5) *
    resolution)``, where ``R`` is ``orientation_xyzw``. A cell is occupied when
    its byte is non-zero.

    ``openral_msgs/OccupancyVoxels`` is an ORIENTED lattice: its cell axes are
    the source map's, not ``header.frame_id``'s, so the rotation is part of
    placing a cell and not decoration. Dropping it puts every returned center in
    the wrong place whenever the robot is not aligned with the map.

    Args:
        origin: ``(x, y, z)`` position of cell ``(0, 0, 0)``'s minimum corner,
            in ``header.frame_id``, in metres.
        resolution: Edge length of a cubic voxel in metres.
        size_xyz: Grid dimensions ``(size_x, size_y, size_z)`` in voxels.
        occupancy: Flat occupancy buffer, one byte per voxel, row-major with x
            fastest: ``idx = ix + size_x * (iy + size_y * iz)``. Non-zero byte
            means occupied.
        orientation_xyzw: Grid orientation as ``(x, y, z, w)``. Defaults to
            identity for callers building an axis-aligned grid themselves;
            a caller decoding a wire message must pass the message's own, and
            an all-zero quaternion (the unset default) is rejected rather than
            read as identity.

    Returns:
        A ``(N, 3)`` float64 array of occupied-voxel centers in the grid's
        reference frame, or shape ``(0, 3)`` when none are occupied.

    Raises:
        ObjectsLiftError: If ``len(occupancy)`` does not equal
            ``size_x * size_y * size_z``, or ``orientation_xyzw`` is not a unit
            quaternion.

    Example:
        >>> centers = decode_occupied_centers(
        ...     origin=(0.0, 0.0, 0.0),
        ...     resolution=0.1,
        ...     size_xyz=(2, 1, 1),
        ...     occupancy=bytes([0, 1]),
        ... )
        >>> tuple(round(float(v), 3) for v in centers[0])
        (0.15, 0.05, 0.05)
    """
    sx, sy, sz = size_xyz
    expected = sx * sy * sz
    arr = np.frombuffer(occupancy, dtype=np.uint8)
    if arr.size != expected:
        raise ObjectsLiftError(
            f"occupancy length {arr.size} != size_x*size_y*size_z {expected}",
        )
    quat = np.asarray(orientation_xyzw, dtype=np.float64)
    norm2 = float(quat @ quat)
    if not np.isfinite(norm2) or abs(norm2 - 1.0) > _QUAT_NORM_TOL:
        raise ObjectsLiftError(
            f"orientation {orientation_xyzw!r} is not a unit quaternion "
            "(an unset OccupancyVoxels.orientation is all zeros, not identity)",
        )
    occ = np.nonzero(arr)[0]
    if occ.size == 0:
        return np.empty((0, 3), dtype=np.float64)
    ix = occ % sx
    iy = (occ // sx) % sy
    iz = occ // (sx * sy)
    idx = np.stack([ix, iy, iz], axis=1).astype(np.float64)
    local = (idx + 0.5) * float(resolution)
    x, y, z, w = quat
    rot = np.array(
        [
            [1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w)],
            [2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w)],
            [2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y)],
        ],
        dtype=np.float64,
    )
    return np.asarray(origin, dtype=np.float64) + local @ rot.T

depth_cloud_to_centers_base(points_cloud: NDArray[np.float64], t_base_from_cloud: NDArray[np.float64], *, max_points: int = 0) -> NDArray[np.float64]

Depth-cloud points → occupied centers (M, 3) in the base frame (#11).

The octomap-free fallback depth source for VoxelFrustumLifter. Drops non-finite returns (depth holes), uniformly subsamples to max_points so a dense cloud can't stall the per-detection projection, and maps the cloud from its optical frame into the robot base frame. The result is interchangeable with decode_occupied_centers output as the lifter's occupied_centers_base argument.

Parameters:

Name	Type	Description	Default
points_cloud	NDArray[float64]	(N, 3) raw cloud points in the camera optical frame.	required
t_base_from_cloud	NDArray[float64]	(4, 4) transform mapping cloud-frame points into the robot base frame.	required
max_points	int	Cap on returned points (uniform stride). 0 = no cap.	0

Returns:

Type	Description
NDArray[float64]	An (M, 3) float64 array of finite points in the base frame, or shape
NDArray[float64]	(0, 3) when the cloud has no finite points.

Source code in python/world_state/src/openral_world_state/object_lift.py

def depth_cloud_to_centers_base(
    points_cloud: NDArray[np.float64],
    t_base_from_cloud: NDArray[np.float64],
    *,
    max_points: int = 0,
) -> NDArray[np.float64]:
    """Depth-cloud points → occupied centers ``(M, 3)`` in the base frame (#11).

    The octomap-free fallback depth source for ``VoxelFrustumLifter``. Drops
    non-finite returns (depth holes), uniformly subsamples to ``max_points`` so
    a dense cloud can't stall the per-detection projection, and maps the cloud
    from its optical frame into the robot base frame. The result is
    interchangeable with ``decode_occupied_centers`` output as the lifter's
    ``occupied_centers_base`` argument.

    Args:
        points_cloud: ``(N, 3)`` raw cloud points in the camera optical frame.
        t_base_from_cloud: ``(4, 4)`` transform mapping cloud-frame points into
            the robot base frame.
        max_points: Cap on returned points (uniform stride). ``0`` = no cap.

    Returns:
        An ``(M, 3)`` float64 array of finite points in the base frame, or shape
        ``(0, 3)`` when the cloud has no finite points.
    """
    pts: NDArray[np.float64] = np.asarray(points_cloud, dtype=np.float64).reshape(-1, 3)
    pts = pts[np.isfinite(pts).all(axis=1)]
    if pts.shape[0] == 0:
        return np.empty((0, 3), dtype=np.float64)
    if max_points and pts.shape[0] > max_points:
        stride = pts.shape[0] // max_points + 1
        pts = pts[::stride]
    homog = np.hstack([pts, np.ones((pts.shape[0], 1), dtype=np.float64)])
    return np.ascontiguousarray((homog @ np.asarray(t_base_from_cloud, dtype=np.float64).T)[:, :3])

emit_scene_objects_span(graph: SceneGraph, *, source_node: str) -> int

Emit one world.scene_objects span carrying graph's object nodes.

Parameters:

Name	Type	Description	Default
graph	SceneGraph	A scene-graph snapshot to publish.	required
source_node	str	Name of the node emitting the span (shown on the card so an operator knows which process produced the map).	required

Returns:

Type	Description
int	The number of object nodes published.

Example

import time from openral_core import DetectedObject, Pose6D from openral_world_state import SpatialMemory mem = SpatialMemory() mug = DetectedObject( ... label="mug", ... confidence=0.9, ... pose=Pose6D(xyz=(1.0, 2.0, 0.8), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"), ... ) _ = mem.ingest_detected_objects([mug], now_ns=time.time_ns()) emit_scene_objects_span(mem.to_scene_graph(), source_node="demo") 1

Source code in python/world_state/src/openral_world_state/scene_objects_span.py

def emit_scene_objects_span(graph: SceneGraph, *, source_node: str) -> int:
    """Emit one ``world.scene_objects`` span carrying ``graph``'s object nodes.

    Args:
        graph: A scene-graph snapshot to publish.
        source_node: Name of the node emitting the span (shown on the card so
            an operator knows which process produced the map).

    Returns:
        The number of object nodes published.

    Example:
        >>> import time
        >>> from openral_core import DetectedObject, Pose6D
        >>> from openral_world_state import SpatialMemory
        >>> mem = SpatialMemory()
        >>> mug = DetectedObject(
        ...     label="mug",
        ...     confidence=0.9,
        ...     pose=Pose6D(xyz=(1.0, 2.0, 0.8), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"),
        ... )
        >>> _ = mem.ingest_detected_objects([mug], now_ns=time.time_ns())
        >>> emit_scene_objects_span(mem.to_scene_graph(), source_node="demo")
        1
    """
    objects = scene_objects_payload(graph)
    frame = str(objects[0]["frame_id"]) if objects else _DEFAULT_FRAME
    tracer = trace.get_tracer("openral")
    with tracer.start_as_current_span(semconv.SPAN_WORLD_SCENE_OBJECTS) as span:
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_COUNT, len(objects))
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_SOURCE_NODE, source_node)
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_FRAME, frame)
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_LIST, json.dumps(objects))
    return len(objects)

homogeneous_from_quat_xyz(translation: tuple[float, float, float], quat_xyzw: tuple[float, float, float, float]) -> NDArray[np.float64]

Build a 4x4 homogeneous transform from a translation + xyzw quaternion.

Parameters:

Name	Type	Description	Default
translation	tuple[float, float, float]	(x, y, z) translation in metres.	required
quat_xyzw	tuple[float, float, float, float]	Rotation quaternion as (x, y, z, w); need not be unit length (it is normalised internally).	required

Returns:

Type	Description
NDArray[float64]	A (4, 4) float64 homogeneous transform matrix.

Raises:

Type	Description
ObjectsLiftError	If the quaternion norm is effectively zero.

Note

The math now lives in openral_core.geometry.homogeneous_from_quat_xyz — the layer-0 HAL needs the same TF→matrix step for its vision attachment bridge and must not reach up to layer 2 for it. This wrapper keeps the world-state-local ObjectsLiftError contract that every caller here already handles.

Source code in python/world_state/src/openral_world_state/object_lift.py

def homogeneous_from_quat_xyz(
    translation: tuple[float, float, float],
    quat_xyzw: tuple[float, float, float, float],
) -> NDArray[np.float64]:
    """Build a 4x4 homogeneous transform from a translation + xyzw quaternion.

    Args:
        translation: ``(x, y, z)`` translation in metres.
        quat_xyzw: Rotation quaternion as ``(x, y, z, w)``; need not be unit
            length (it is normalised internally).

    Returns:
        A ``(4, 4)`` float64 homogeneous transform matrix.

    Raises:
        ObjectsLiftError: If the quaternion norm is effectively zero.

    Note:
        The math now lives in ``openral_core.geometry.homogeneous_from_quat_xyz``
        — the layer-0 HAL needs the same TF→matrix step for its vision
        attachment bridge and must not reach up to layer 2 for it. This wrapper
        keeps the world-state-local ``ObjectsLiftError`` contract that every
        caller here already handles.
    """
    try:
        return _core_homogeneous_from_quat_xyz(translation, quat_xyzw)
    except ROSConfigError as exc:
        raise ObjectsLiftError(str(exc)) from exc

look_at_quat_wxyz(eye: tuple[float, float, float], target: tuple[float, float, float], *, up: tuple[float, float, float] = (0.0, 0.0, 1.0), view_axis: ViewAxis = '-z') -> tuple[float, float, float, float]

Quaternion (w, x, y, z) orienting a camera at eye to face target.

The rotated frame's view_axis points from eye toward target. Degenerate inputs fall back instead of raising: target == eye returns the MuJoCo straight-down quat for "-z" (preserving the sim composers' documented behaviour) and identity otherwise; a gaze nearly parallel to up swaps in a +Y up vector.

Parameters:

Name	Type	Description	Default
eye	tuple[float, float, float]	Camera position.	required
target	tuple[float, float, float]	Point to look at, same frame as eye.	required
up	tuple[float, float, float]	World up used to level the camera roll.	(0.0, 0.0, 1.0)
view_axis	ViewAxis	Which camera axis must point at the target (see module docstring for the conventions).	'-z'

Returns:

Type	Description
tuple[float, float, float, float]	Unit quaternion in (w, x, y, z) order (MuJoCo convention).

Example

look_at_quat_wxyz((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), view_axis="+x") (1.0, 0.0, 0.0, 0.0)

Source code in python/core/src/openral_core/geometry.py

def look_at_quat_wxyz(
    eye: tuple[float, float, float],
    target: tuple[float, float, float],
    *,
    up: tuple[float, float, float] = (0.0, 0.0, 1.0),
    view_axis: ViewAxis = "-z",
) -> tuple[float, float, float, float]:
    """Quaternion (``w, x, y, z``) orienting a camera at ``eye`` to face ``target``.

    The rotated frame's ``view_axis`` points from ``eye`` toward ``target``.
    Degenerate inputs fall back instead of raising: ``target == eye`` returns
    the MuJoCo straight-down quat for ``"-z"`` (preserving the sim composers'
    documented behaviour) and identity otherwise; a gaze nearly parallel to
    ``up`` swaps in a +Y up vector.

    Args:
        eye: Camera position.
        target: Point to look at, same frame as ``eye``.
        up: World up used to level the camera roll.
        view_axis: Which camera axis must point at the target (see module
            docstring for the conventions).

    Returns:
        Unit quaternion in ``(w, x, y, z)`` order (MuJoCo convention).

    Example:
        >>> look_at_quat_wxyz((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), view_axis="+x")
        (1.0, 0.0, 0.0, 0.0)
    """
    eye_v = np.asarray(eye, dtype=np.float64)
    target_v = np.asarray(target, dtype=np.float64)
    forward = target_v - eye_v
    norm = float(np.linalg.norm(forward))
    if norm < _ZERO_NORM:
        # Sim composers' documented fallback: 180° flip about X looks straight
        # down for a -Z camera. Other conventions get identity.
        return (0.0, 1.0, 0.0, 0.0) if view_axis == "-z" else (1.0, 0.0, 0.0, 0.0)
    forward /= norm
    up_v = np.asarray(up, dtype=np.float64)
    if abs(float(np.dot(up_v, forward))) > _PARALLEL:
        # Looking nearly straight up/down the up vector: pick an alternate up.
        up_v = np.asarray((0.0, 1.0, 0.0), dtype=np.float64)

    if view_axis == "-z":
        z_axis = -forward
        x_axis = np.cross(up_v, z_axis)
        x_axis /= float(np.linalg.norm(x_axis))
        y_axis = np.cross(z_axis, x_axis)
    elif view_axis == "+z":
        z_axis = forward
        x_axis = np.cross(forward, up_v)
        x_axis /= float(np.linalg.norm(x_axis))
        y_axis = np.cross(z_axis, x_axis)
    else:  # "+x"
        x_axis = forward
        y_axis = np.cross(up_v, forward)
        y_axis /= float(np.linalg.norm(y_axis))
        z_axis = np.cross(x_axis, y_axis)

    rot = np.column_stack([x_axis, y_axis, z_axis])
    return _basis_to_quat_wxyz(rot)

refine_approach_pose(grid: OccupancyGridIndex, viewpoint: ApproachViewpoint, target_xyz: tuple[float, float, float], *, inflation_m: float = 0.25, max_radius_m: float = 2.0, min_standoff_m: float | None = None, max_standoff_m: float | None = None) -> ApproachViewpoint | None

Snap an ApproachViewpoint to the occupancy grid.

Returns the viewpoint unchanged when its pose already sits on a free cell (under inflation_m) with line-of-sight to the target. Otherwise ring- searches outward (at grid resolution) for the nearest point that is free, keeps the standoff within [min_standoff_m, max_standoff_m] (default 0.5x / 2.0x the ideal standoff), and still sees the target — and re-aims the yaw from there via compute_approach_viewpoint. Returns None when nothing qualifies inside max_radius_m: the caller reports "no reachable viewpoint" rather than fabricating one.

Parameters:

Name	Type	Description	Default
grid	OccupancyGridIndex	Decoded occupancy snapshot.	required
viewpoint	ApproachViewpoint	The geometric (grid-blind) viewpoint to validate/refine.	required
target_xyz	tuple[float, float, float]	The object the camera must face, same frame as the grid.	required
inflation_m	float	Robot-footprint radius every candidate must clear.	0.25
max_radius_m	float	Search radius around the ideal viewpoint.	2.0
min_standoff_m	float | None	Closest admissible standoff (default 0.5 * viewpoint.standoff_m).	None
max_standoff_m	float | None	Farthest admissible standoff (default 2.0 * viewpoint.standoff_m).	None

Source code in python/world_state/src/openral_world_state/grid.py

def refine_approach_pose(
    grid: OccupancyGridIndex,
    viewpoint: ApproachViewpoint,
    target_xyz: tuple[float, float, float],
    *,
    inflation_m: float = 0.25,
    max_radius_m: float = 2.0,
    min_standoff_m: float | None = None,
    max_standoff_m: float | None = None,
) -> ApproachViewpoint | None:
    """Snap an ``ApproachViewpoint`` to the occupancy grid.

    Returns the viewpoint unchanged when its pose already sits on a free cell
    (under ``inflation_m``) with line-of-sight to the target. Otherwise ring-
    searches outward (at grid resolution) for the nearest point that is free,
    keeps the standoff within ``[min_standoff_m, max_standoff_m]`` (default
    0.5x / 2.0x the ideal standoff), and still sees the target — and re-aims
    the yaw from there via ``compute_approach_viewpoint``. Returns ``None``
    when nothing qualifies inside ``max_radius_m``: the caller reports "no
    reachable viewpoint" rather than fabricating one.

    Args:
        grid: Decoded occupancy snapshot.
        viewpoint: The geometric (grid-blind) viewpoint to validate/refine.
        target_xyz: The object the camera must face, same frame as the grid.
        inflation_m: Robot-footprint radius every candidate must clear.
        max_radius_m: Search radius around the ideal viewpoint.
        min_standoff_m: Closest admissible standoff (default ``0.5 *
            viewpoint.standoff_m``).
        max_standoff_m: Farthest admissible standoff (default ``2.0 *
            viewpoint.standoff_m``).
    """
    lo = 0.5 * viewpoint.standoff_m if min_standoff_m is None else min_standoff_m
    hi = 2.0 * viewpoint.standoff_m if max_standoff_m is None else max_standoff_m
    ideal_x, ideal_y, _ideal_z = viewpoint.pose.xyz
    tx, ty = target_xyz[0], target_xyz[1]

    def _admissible(x: float, y: float) -> bool:
        standoff = math.hypot(x - tx, y - ty)
        return (
            lo <= standoff <= hi
            and grid.is_free(x, y, inflation_m=inflation_m)
            and grid.line_of_sight((x, y), (tx, ty))
        )

    if _admissible(ideal_x, ideal_y):
        return viewpoint

    target_pose = Pose6D(
        xyz=target_xyz, quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id=viewpoint.pose.frame_id
    )
    step = grid.resolution_m
    radius = step
    while radius <= max_radius_m + 1e-9:
        # Enough samples that consecutive points on the ring are ~one cell apart.
        n_samples = max(8, math.ceil(2.0 * math.pi * radius / step))
        best: tuple[float, float, float] | None = None  # (standoff_error, x, y)
        for k in range(n_samples):
            theta = 2.0 * math.pi * k / n_samples
            x = ideal_x + radius * math.cos(theta)
            y = ideal_y + radius * math.sin(theta)
            if not _admissible(x, y):
                continue
            err = abs(math.hypot(x - tx, y - ty) - viewpoint.standoff_m)
            if best is None or err < best[0]:
                best = (err, x, y)
        if best is not None:
            _err, x, y = best
            candidate = Pose6D(
                xyz=(x, y, viewpoint.pose.xyz[2]),
                quat_xyzw=(0.0, 0.0, 0.0, 1.0),
                frame_id=viewpoint.pose.frame_id,
            )
            return compute_approach_viewpoint(
                target_pose,
                standoff_m=math.hypot(x - tx, y - ty),
                camera_frame_id=viewpoint.camera_frame_id,
                approach_from=candidate,
            )
        radius += step
    return None

rotation_to_quat_wxyz(rot: NDArray[np.float64]) -> tuple[float, float, float, float]

Public matrix→quaternion conversion (w, x, y, z), Shepperd's method.

Used by the rskill-moveit-multi-look_at skill to re-express a camera gaze pose for the camera's mount link after a homogeneous-matrix composition.

Example

import numpy as np rotation_to_quat_wxyz(np.eye(3)) (1.0, 0.0, 0.0, 0.0)

Source code in python/core/src/openral_core/geometry.py

def rotation_to_quat_wxyz(rot: NDArray[np.float64]) -> tuple[float, float, float, float]:
    """Public matrix→quaternion conversion (``w, x, y, z``), Shepperd's method.

    Used by the ``rskill-moveit-multi-look_at`` skill to re-express a camera gaze pose for
    the camera's mount link after a homogeneous-matrix composition.

    Example:
        >>> import numpy as np
        >>> rotation_to_quat_wxyz(np.eye(3))
        (1.0, 0.0, 0.0, 0.0)
    """
    return _basis_to_quat_wxyz(rot)

scene_objects_payload(graph: SceneGraph) -> list[dict[str, object]]

Project the object-kind nodes to JSON-friendly dicts.

Each dict carries the map-frame position, the semantic label, and the recency/confidence the dashboard renders. Non-object nodes (places, rooms, agents) are skipped — the dashboard surface is "what objects do we remember, and where".

Parameters:

Name	Type	Description	Default
graph	SceneGraph	A scene-graph snapshot (e.g. SpatialMemory.to_scene_graph()).	required

Returns:

Type	Description
list[dict[str, object]]	One dict per object node, in graph order.

Source code in python/world_state/src/openral_world_state/scene_objects_span.py

def scene_objects_payload(graph: SceneGraph) -> list[dict[str, object]]:
    """Project the ``object``-kind nodes to JSON-friendly dicts.

    Each dict carries the map-frame position, the semantic label, and the
    recency/confidence the dashboard renders. Non-object nodes (places, rooms,
    agents) are skipped — the dashboard surface is "what objects do we
    remember, and where".

    Args:
        graph: A scene-graph snapshot (e.g. ``SpatialMemory.to_scene_graph()``).

    Returns:
        One dict per object node, in graph order.
    """
    objects: list[dict[str, object]] = []
    for node in graph.nodes:
        if node.kind is not SpatialNodeKind.OBJECT:
            continue
        x, y, z = node.pose.xyz
        objects.append(
            {
                "id": node.node_id,
                "label": node.label,
                "x": x,
                "y": y,
                "z": z,
                "frame_id": node.pose.frame_id,
                "confidence": node.confidence,
                "last_seen_ns": node.last_seen_ns,
                "observation_count": node.observation_count,
                "is_container": node.is_container,
            }
        )
    return objects

rSkill (S1)

OpenRAL rSkill — lifecycle base class, runtime protocol, and rSkill loader.

Public surface

• rSkillBase: Abstract base class with the ROS 2 lifecycle state machine.
• Runtime: Structural protocol for inference backends.
• NullRuntime: No-op runtime for tests and development.
• rSkill: HF Hub rSkill loader (manifest + weights + license guard).
• InstalledRSkillEntry: Local registry entry schema.
• search_hub_rskills: Free-text + facet search over an HF Hub org's rSkills (HubRSkillHit, HubRSkillSearchResult).
• resolve_runtime_backend: Name -> Runtime class, built-in or via the openral.runtime_backends entry-point group (the OpenRAL Pro extraction seam).
• maybe_attach_pro_hooks: Generic OpenRAL Pro policy-attach-hook lookup via the openral.policy_attach_hooks entry-point group.
• hf_download_cached_first: Cache-first HF Hub file resolution shared by the VLA adapters and openral_sim's quantization helpers.
• local_snapshot_dir: The resolved checkpoint as a local directory — the directory itself for an installed rSkill snapshot, a Hub snapshot otherwise.
• gpu_allocated_mb: Current CUDA allocator usage in MB, the single GPU-memory probe shared by the phase-timer heartbeats and the rskill runner node's eviction accounting.
• find_repo_root_from: Locate the OpenRAL repo root by walking up from a start path.
• validate_skill_ref: Validate a bare rSkill reference string.

Heavy-dependency backends (PyTorchRuntime, ONNXRuntime) are not imported here. Import them explicitly when their dependencies are installed:

from openral_rskill.runtime_pytorch import PyTorchRuntime
from openral_rskill.runtime_onnx import ONNXRuntime

A TensorRT Runtime is not shipped in this package (the TensorRT/NVMM fast path is a private OpenRAL Pro plugin); resolve it via resolve_runtime_backend("tensorrt") when openral-pro-trt is installed.

DEFAULT_REGISTRY_PATH: Path = _DATA_HOME / 'rskills.json' module-attribute

Default JSON registry file written by rSkill.from_pretrained.

HubRSkillHit

Bases: BaseModel

One Hub repo whose rskill.yaml validated and matched the query.

Attributes:

Name	Type	Description
repo_id	str	HF Hub repository identifier, e.g. "OpenRAL/rskill-act-aloha-aloha_transfer_cube-fp32".
manifest	RSkillManifest	The validated RSkillManifest fetched from the repo.
local	Literal['in-tree', 'installed'] | None	"in-tree" when the Hub repo_id matches the name declared by a rskills/<dir>/rskill.yaml in this checkout (compared against the local manifest's name, never the remote one — a renamed Hub repo with a stale manifest name must not read as in-tree); "installed" when the local HF-install registry (rSkill.list_installed()) has a matching repo_id; None when neither.
hub_tags	tuple[str, ...]	The repo's model-card tags as reported by the Hub (includes "OpenRAL" and "rskill", plus embodiment/license tags set by the publisher).

Example

from openral_rskill.loader import load_rskill_manifest m = load_rskill_manifest("act-aloha") hit = HubRSkillHit( ... repo_id=m.name, ... manifest=m, ... local="in-tree", ... hub_tags=("OpenRAL", "rskill"), ... ) hit.local 'in-tree'

HubRSkillSearchResult

Bases: BaseModel

Result of search_hub_rskills — matching hits plus bookkeeping.

Attributes:

Name	Type	Description
hits	list[HubRSkillHit]	Matching repos, sorted by repo_id and capped by limit.
skipped	int	Count of inspected org repos with no valid rskill.yaml (excluded from hits regardless of query/filters). Always equal to len(skipped_repo_ids).
skipped_repo_ids	tuple[str, ...]	The repo ids counted in skipped, sorted. This module never logs per-repo skips (the CLI never calls structlog.configure(), so structlog's default PrintLogger would write straight to stdout and corrupt --json piping, e.g. into jq) — this field is the explicit record instead (CLAUDE.md §1.4: explicit beats implicit, nothing silently dropped).
inspected	int	Total org repos inspected (before query/facet filtering and before limit truncation) — i.e. len(hits) + skipped plus any repos that had a valid manifest but did not match.

Example

HubRSkillSearchResult(hits=[], skipped=0, skipped_repo_ids=(), inspected=0) HubRSkillSearchResult(hits=[], skipped=0, skipped_repo_ids=(), inspected=0)

InstalledRSkillEntry

Bases: BaseModel

One row in the local rSkill registry (~/.local/share/openral/rskills.json).

Attributes:

Name	Type	Description
repo_id	str	HF Hub repository identifier, e.g. "openral/rskill-pick-cube-so100".
version	str	SemVer version string from the manifest.
revision	str | None	HF Hub commit SHA (None when installed from a local path).
local_dir	str	Absolute path to the snapshot directory in the HF Hub cache.
manifest_path	str	Absolute path to the rskill.yaml file on disk.
license	str	License posture value string from RSkillLicensePosture.
role	str	Skill slot — "s0", "s1", or "s2".
embodiment_tags	list[str]	Embodiment tags declared in the manifest.
installed_at	str	ISO 8601 timestamp of installation.

Example

entry = InstalledRSkillEntry( ... repo_id="example/skill", ... version="0.1.0", ... revision=None, ... local_dir="/tmp/skills/example", ... manifest_path="/tmp/skills/example/rskill.yaml", ... license="apache-2.0", ... role="s1", ... embodiment_tags=["so100_follower"], ... installed_at="2026-01-01T00:00:00+00:00", ... ) entry.repo_id 'example/skill'

NullRuntime(device: str = 'cpu')

No-op inference backend for testing and development.

Satisfies the Runtime Protocol without requiring any ML framework. infer() always returns an empty dict. All state transitions are tracked faithfully so lifecycle tests can assert on is_loaded.

Parameters:

Name	Type	Description	Default
device	str	Device string to report (default "cpu").	'cpu'

Example

rt = NullRuntime() rt.is_loaded False rt.load("any/path.pt") rt.is_loaded True rt.unload() rt.is_loaded False

Initialize with the given device string.

Parameters:

Name	Type	Description	Default
device	str	PyTorch-style device string (e.g. "cpu", "cuda:0").	'cpu'

Source code in python/rskill/src/openral_rskill/runtime.py

def __init__(self, device: str = "cpu") -> None:
    """Initialize with the given device string.

    Args:
        device: PyTorch-style device string (e.g. ``"cpu"``, ``"cuda:0"``).
    """
    self._device = device
    self._loaded = False

device: str property

PyTorch-style device string.

is_loaded: bool property

True after load() has been called and before unload().

infer(inputs: dict[str, Any]) -> dict[str, Any]

Return an empty dict without computation.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Ignored; accepted for protocol compatibility.	required

Returns:

Type	Description
dict[str, Any]	Always {}.

Source code in python/rskill/src/openral_rskill/runtime.py

def infer(self, inputs: dict[str, Any]) -> dict[str, Any]:
    """Return an empty dict without computation.

    Args:
        inputs: Ignored; accepted for protocol compatibility.

    Returns:
        Always ``{}``.
    """
    return {}

load(path: Path | str) -> None

Mark weights as loaded (no I/O performed).

Parameters:

Name	Type	Description	Default
path	Path | str	Ignored; accepted for protocol compatibility.	required

Source code in python/rskill/src/openral_rskill/runtime.py

def load(self, path: Path | str) -> None:
    """Mark weights as loaded (no I/O performed).

    Args:
        path: Ignored; accepted for protocol compatibility.
    """
    self._loaded = True

quantize(config: QuantizationConfig) -> None

Accept any quantization config silently (no-op).

Parameters:

Name	Type	Description	Default
config	QuantizationConfig	Ignored; accepted for protocol compatibility.	required

Source code in python/rskill/src/openral_rskill/runtime.py

def quantize(self, config: QuantizationConfig) -> None:
    """Accept any quantization config silently (no-op).

    Args:
        config: Ignored; accepted for protocol compatibility.
    """

unload() -> None

Mark weights as unloaded (no memory freed).

Source code in python/rskill/src/openral_rskill/runtime.py

def unload(self) -> None:
    """Mark weights as unloaded (no memory freed)."""
    self._loaded = False

warmup(inputs: dict[str, Any]) -> None

No-op warmup.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Ignored; accepted for protocol compatibility.	required

Source code in python/rskill/src/openral_rskill/runtime.py

def warmup(self, inputs: dict[str, Any]) -> None:
    """No-op warmup.

    Args:
        inputs: Ignored; accepted for protocol compatibility.
    """

Runtime

Bases: Protocol

Structural protocol for skill inference backends.

All methods are called from the Skill lifecycle hooks. Implementations must be thread-safe if step() is called concurrently (e.g. async executor pattern as in SmolVLA). Instances are created by backend-specific constructors (e.g. PyTorchRuntime(...), OnnxRuntime(...), NullRuntime()); the protocol itself is not instantiable.

Properties

is_loaded: True after load completes successfully. device: PyTorch-style device string, e.g. "cpu" or "cuda:0".

device: str property

PyTorch-style device string.

is_loaded: bool property

True after load() completes successfully.

infer(inputs: dict[str, Any]) -> dict[str, Any]

Run one forward pass and return named outputs.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Named input tensors / arrays. Keys and shapes depend on the model contract declared in rskill.yaml.	required

Returns:

Type	Description
dict[str, Any]	Named output tensors / arrays.

Raises:

Type	Description
ROSRuntimeError	If no model is loaded or the forward pass fails.

Source code in python/rskill/src/openral_rskill/runtime.py

def infer(self, inputs: dict[str, Any]) -> dict[str, Any]:
    """Run one forward pass and return named outputs.

    Args:
        inputs: Named input tensors / arrays.  Keys and shapes depend on
            the model contract declared in ``rskill.yaml``.

    Returns:
        Named output tensors / arrays.

    Raises:
        ROSRuntimeError: If no model is loaded or the forward pass fails.
    """
    ...

load(path: Path | str) -> None

Load model weights from path into memory.

Parameters:

Name	Type	Description	Default
path	Path | str	Local filesystem path or URL to weights file (*.safetensors, *.onnx, *.pt, etc.).	required

Raises:

Type	Description
ROSRuntimeError	If the file is missing, corrupt, or incompatible.

Source code in python/rskill/src/openral_rskill/runtime.py

def load(self, path: Path | str) -> None:
    """Load model weights from *path* into memory.

    Args:
        path: Local filesystem path or URL to weights file
            (``*.safetensors``, ``*.onnx``, ``*.pt``, etc.).

    Raises:
        ROSRuntimeError: If the file is missing, corrupt, or incompatible.
    """
    ...

quantize(config: QuantizationConfig) -> None

Apply post-load quantization.

Parameters:

Name	Type	Description	Default
config	QuantizationConfig	Target dtype, backend, and optional calibration dataset.	required

Raises:

Type	Description
ROSRuntimeError	If the requested dtype/backend combination is unsupported, or if no model has been loaded yet.

Source code in python/rskill/src/openral_rskill/runtime.py

def quantize(self, config: QuantizationConfig) -> None:
    """Apply post-load quantization.

    Args:
        config: Target dtype, backend, and optional calibration dataset.

    Raises:
        ROSRuntimeError: If the requested dtype/backend combination is
            unsupported, or if no model has been loaded yet.
    """
    ...

unload() -> None

Release model weights and free device memory.

Source code in python/rskill/src/openral_rskill/runtime.py

def unload(self) -> None:
    """Release model weights and free device memory."""
    ...

warmup(inputs: dict[str, Any]) -> None

Run a dummy forward pass to amortize JIT/kernel-launch overhead.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Dummy inputs with the correct shapes (values are ignored).	required

Source code in python/rskill/src/openral_rskill/runtime.py

def warmup(self, inputs: dict[str, Any]) -> None:
    """Run a dummy forward pass to amortize JIT/kernel-launch overhead.

    Args:
        inputs: Dummy inputs with the correct shapes (values are ignored).
    """
    ...

rSkill(manifest: RSkillManifest, local_dir: Path)

Packaged, capability-tagged robot skill from the HF Hub.

.. warning:: Signature verification (sigstore) is not yet implemented; the loader validates the manifest and license but does not verify the provenance of downloaded weights. See module docstring.

An rSkill represents the distribution artefact: the rskill.yaml manifest plus the associated weight files stored in the HF Hub cache (or locally). It is distinct from the runtime Skill ABC, which is the in-process lifecycle node.

Typical workflow::

pkg = rSkill.from_pretrained("openral/rskill-pick-cube-so100")
# pkg.manifest  → RSkillManifest (license, embodiment_tags, latency_budget …)
# pkg.local_dir → Path to weights on disk

Attributes:

Name	Type	Description
manifest		Parsed and validated RSkillManifest.
local_dir		Filesystem path to the directory containing all skill files.

Store the manifest and local directory path.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	Parsed and validated rSkill manifest.	required
local_dir	Path	Filesystem path to the directory containing all skill files.	required

Source code in python/rskill/src/openral_rskill/loader.py

def __init__(self, manifest: RSkillManifest, local_dir: Path) -> None:
    """Store the manifest and local directory path.

    Args:
        manifest: Parsed and validated rSkill manifest.
        local_dir: Filesystem path to the directory containing all skill files.
    """
    self.manifest = manifest
    self.local_dir = local_dir

__repr__() -> str

Return a concise developer representation.

Source code in python/rskill/src/openral_rskill/loader.py

def __repr__(self) -> str:
    """Return a concise developer representation."""
    return (
        f"rSkill(name={self.manifest.name!r}, "
        f"version={self.manifest.version!r}, "
        f"license={self.manifest.license.value!r})"
    )

check_capabilities(manifest: RSkillManifest, robot_capabilities: RobotCapabilities, *, compute: ComputeSpec | None = None) -> None staticmethod

Verify that the robot satisfies the rSkill's capability requirements.

Composition of check_embodiment_tags, check_capability_flags, check_runtime, and check_quantization_dtype — raises on the first mismatch. Pass compute=robot.compute_edge or robot.compute_local to enable runtime / dtype checks against the robot's ComputeSpec.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest to check.	required
robot_capabilities	RobotCapabilities	The target robot's declared capabilities.	required
compute	ComputeSpec | None	Optional compute spec resolved from the deployment tier (RobotDescription.compute_edge falling back to RobotDescription.compute_local). When None, runtime and dtype checks are skipped if the legacy capability fields are also absent.	None

Raises:

Type	Description
ROSCapabilityMismatch	If any required capability is not satisfied or if embodiment tags do not intersect.

Example

from openral_core.schemas import RobotCapabilities caps = RobotCapabilities(embodiment_tags=["so100_follower"])

rSkill.check_capabilities(manifest, caps) # raises if mismatch

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_capabilities(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
    *,
    compute: ComputeSpec | None = None,
) -> None:
    """Verify that the robot satisfies the rSkill's capability requirements.

    Composition of ``check_embodiment_tags``,
    ``check_capability_flags``, ``check_runtime``, and
    ``check_quantization_dtype`` — raises on the first mismatch.
    Pass ``compute=robot.compute_edge or robot.compute_local`` to enable
    runtime / dtype checks against the robot's ``ComputeSpec``.

    Args:
        manifest: The rSkill manifest to check.
        robot_capabilities: The target robot's declared capabilities.
        compute: Optional compute spec resolved from the deployment tier
            (``RobotDescription.compute_edge`` falling back to
            ``RobotDescription.compute_local``).  When
            ``None``, runtime and dtype checks are skipped if the legacy
            capability fields are also absent.

    Raises:
        ROSCapabilityMismatch: If any required capability is not satisfied
            or if embodiment tags do not intersect.

    Example:
        >>> from openral_core.schemas import RobotCapabilities
        >>> caps = RobotCapabilities(embodiment_tags=["so100_follower"])
        >>> # rSkill.check_capabilities(manifest, caps)  # raises if mismatch
    """
    rSkill.check_embodiment_tags(manifest, robot_capabilities)
    rSkill.check_capability_flags(manifest, robot_capabilities)
    rSkill.check_runtime(manifest, robot_capabilities, compute=compute)
    rSkill.check_quantization_dtype(manifest, robot_capabilities, compute=compute)

check_capability_flags(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify every manifest.capabilities_required flag.

Raises:

Type	Description
ROSCapabilityMismatch	On the first unsatisfied flag or unknown field name.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_capability_flags(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify every ``manifest.capabilities_required`` flag.

    Raises:
        ROSCapabilityMismatch: On the first unsatisfied flag or
            unknown field name.
    """
    for flag, required_value in manifest.capabilities_required.items():
        robot_value = getattr(robot_capabilities, flag, None)
        if robot_value is None:
            raise ROSCapabilityMismatch(
                f"rSkill '{manifest.name}' requires capability flag '{flag}', "
                "which is not a known field of RobotCapabilities."
            )
        if isinstance(required_value, bool):
            if not robot_value:
                raise ROSCapabilityMismatch(
                    f"rSkill '{manifest.name}' requires '{flag}=True', "
                    f"but robot reports '{flag}={robot_value}'."
                )
        elif isinstance(required_value, (int, float)) and float(robot_value) < float(
            required_value
        ):
            raise ROSCapabilityMismatch(
                f"rSkill '{manifest.name}' requires '{flag}>={required_value}', "
                f"but robot reports '{flag}={robot_value}'."
            )

check_compatibility(manifest: RSkillManifest, robot: RobotDescription, *, scene_gripper_convention: str | None = None) -> None staticmethod

Run every rSkill ↔ robot compatibility check in one call.

This is the umbrella entry point — combines check_capabilities (embodiment tags + boolean / numeric capability flags) and check_sensors (sensor requirements against robot.sensors). Use this from runtime code that has the full RobotDescription; the two narrower methods remain for cases where only one half is available.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest to check.	required
robot	RobotDescription	The target robot's full description.	required
scene_gripper_convention	str | None	The gripper encoding the simulated scene's environment consumes (SCENES.meta(id) ["gripper_convention"]), which overrides the end effectors' own; None for a real robot or a bare twin.	None

Raises:

Type	Description
ROSCapabilityMismatch	If embodiment tags mismatch, a capability flag is unsatisfied, any sensor requirement fails, or the gripper encodings differ.

Example

rSkill.check_compatibility(manifest, robot)

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_compatibility(
    manifest: RSkillManifest,
    robot: RobotDescription,
    *,
    scene_gripper_convention: str | None = None,
) -> None:
    """Run every rSkill ↔ robot compatibility check in one call.

    This is the umbrella entry point — combines
    ``check_capabilities`` (embodiment tags + boolean / numeric
    capability flags) and ``check_sensors`` (sensor requirements
    against ``robot.sensors``). Use this from runtime code that has the
    full ``RobotDescription``; the two narrower methods remain for
    cases where only one half is available.

    Args:
        manifest: The rSkill manifest to check.
        robot: The target robot's full description.
        scene_gripper_convention: The gripper encoding the simulated
            scene's environment consumes (``SCENES.meta(id)
            ["gripper_convention"]``), which overrides the end effectors'
            own; ``None`` for a real robot or a bare twin.

    Raises:
        ROSCapabilityMismatch: If embodiment tags mismatch, a capability
            flag is unsatisfied, any sensor requirement fails, or the
            gripper encodings differ.

    Example:
        >>> # rSkill.check_compatibility(manifest, robot)
    """
    rSkill.check_capabilities(
        manifest, robot.capabilities, compute=robot.compute_edge or robot.compute_local
    )
    rSkill.check_sensors(manifest, robot.sensors)
    rSkill.check_gripper_conventions(
        manifest, robot, scene_gripper_convention=scene_gripper_convention
    )

check_embodiment_tags(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify the manifest's embodiment tags intersect the robot's.

Embodiment-agnostic rSkills declare the explicit wildcard embodiment_tags: ["any"] — perception kinds (detector / vlm / reward) and playbook decision procedures — and run on any robot. Agnosticism is declared, not derived: the manifest validator rejects an empty tag list, so emptiness never silently means match-any. Used by check_capabilities and by the per-section presenter in openral_detect.check_single_rskill.

Raises:

Type	Description
ROSCapabilityMismatch	If the tag sets are disjoint.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_embodiment_tags(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify the manifest's embodiment tags intersect the robot's.

    Embodiment-agnostic rSkills declare the explicit wildcard
    ``embodiment_tags: ["any"]`` — perception kinds (detector /
    vlm / reward) and ``playbook`` decision procedures — and run on any
    robot. Agnosticism is declared, not derived: the manifest validator
    rejects an empty tag list, so emptiness never silently means match-any.
    Used by ``check_capabilities`` and by the per-section presenter in
    ``openral_detect.check_single_rskill``.

    Raises:
        ROSCapabilityMismatch: If the tag sets are disjoint.
    """
    skill_tags = set(manifest.embodiment_tags)
    if _EMBODIMENT_ANY_TAG in skill_tags:
        return
    robot_tags = set(robot_capabilities.embodiment_tags)
    if not skill_tags.intersection(robot_tags):
        raise ROSCapabilityMismatch(
            f"rSkill '{manifest.name}' requires embodiment tag(s) "
            f"{sorted(skill_tags)}, but robot only has {sorted(robot_tags)}."
        )

check_gripper_conventions(manifest: RSkillManifest, robot: RobotDescription, *, scene_gripper_convention: str | None = None) -> None staticmethod

Refuse a skill whose gripper encoding differs from the end effector it drives.

A GRIPPER_* value reaches the HAL unconverted, so the skill's gripper_convention (on its GRIPPER_* actuator) must equal the target end effector's command_convention — or, in a simulated scene, the encoding the scene's environment consumes. The targets are the gripper action slots, explicit or expanded from action_contract.representation; a skill with neither must match every actuated gripper end effector it could address. Both sides must be declared — an undeclared encoding is a mismatch, never a guess.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest to check.	required
robot	RobotDescription	The target robot's full description.	required
scene_gripper_convention	str | None	Overrides every end effector's command_convention (see check_compatibility).	None

Raises:

Type	Description
ROSCapabilityMismatch	When a gripper actuator or slot has no declared convention, a slot names an unknown end effector, the end effector declares no command_convention, or the two conventions differ.

Example

from openral_core import RobotDescription, RSkillManifest skill = RSkillManifest.from_yaml("rskills/xr1-robocasa365/rskill.yaml") robot = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml") rSkill.check_gripper_conventions(skill, robot) # same encoding: no raise

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_gripper_conventions(
    manifest: RSkillManifest,
    robot: RobotDescription,
    *,
    scene_gripper_convention: str | None = None,
) -> None:
    """Refuse a skill whose gripper encoding differs from the end effector it drives.

    A ``GRIPPER_*`` value reaches the HAL unconverted, so the skill's
    ``gripper_convention`` (on its ``GRIPPER_*`` actuator) must equal the
    target end effector's ``command_convention`` — or, in a simulated
    scene, the encoding the scene's environment consumes. The targets are
    the gripper action slots, explicit or expanded from
    ``action_contract.representation``; a skill with neither must match
    every actuated gripper end effector it could address. Both sides must
    be declared — an undeclared encoding is a mismatch, never a guess.

    Args:
        manifest: The rSkill manifest to check.
        robot: The target robot's full description.
        scene_gripper_convention: Overrides every end effector's
            ``command_convention`` (see ``check_compatibility``).

    Raises:
        ROSCapabilityMismatch: When a gripper actuator or slot has no
            declared convention, a slot names an unknown end effector,
            the end effector declares no ``command_convention``, or the
            two conventions differ.

    Example:
        >>> from openral_core import RobotDescription, RSkillManifest
        >>> skill = RSkillManifest.from_yaml("rskills/xr1-robocasa365/rskill.yaml")
        >>> robot = RobotDescription.from_yaml("robots/panda_mobile/robot.yaml")
        >>> rSkill.check_gripper_conventions(skill, robot)  # same encoding: no raise
    """
    declared, targets = rSkill._gripper_targets(manifest, robot)
    if not targets:
        return

    ees = {ee.name: ee for ee in robot.end_effectors}
    grippers = [ee for ee in robot.end_effectors if ee.actuated and ee.kind != "tool"]
    for mode, ee_name in targets:
        conventions = declared.get(mode, set())
        if len(conventions) != 1:
            raise ROSCapabilityMismatch(
                f"rSkill '{manifest.name}' emits {mode.value} but declares "
                + (
                    "no gripper_convention for it"
                    if not conventions
                    else f"conflicting gripper_conventions {sorted(conventions)}"
                )
                + f"; add one actuators_required entry of kind {mode.value!r} with "
                "control_mode_semantics.gripper_convention."
            )
        (convention,) = conventions
        if ee_name is not None:
            if ee_name not in ees:
                raise ROSCapabilityMismatch(
                    f"rSkill '{manifest.name}' addresses gripper end effector "
                    f"{ee_name!r}; robot '{robot.name}' declares {sorted(ees) or '<none>'}."
                )
            candidates = [ees[ee_name]]
        else:
            candidates = grippers
        if not candidates:
            raise ROSCapabilityMismatch(
                f"rSkill '{manifest.name}' emits {mode.value} but robot '{robot.name}' "
                "has no actuated gripper end effector."
            )
        for ee in candidates:
            consumed = scene_gripper_convention or ee.command_convention
            if consumed != convention:
                where = (
                    "this scene's environment"
                    if scene_gripper_convention
                    else f"robot '{robot.name}' end effector {ee.name!r}"
                )
                raise ROSCapabilityMismatch(
                    f"rSkill '{manifest.name}' emits {mode.value} as {convention!r} but "
                    f"{where} consumes {consumed or '<undeclared>'!r}; the value reaches "
                    "the HAL unconverted, so the encodings must match."
                )

check_quantization_dtype(manifest: RSkillManifest, robot_capabilities: RobotCapabilities, *, compute: ComputeSpec | None = None) -> None staticmethod

Verify the manifest's quantization dtype is in the robot's set.

Reads gpu_supported_dtypes from compute when provided; falls back to robot_capabilities for legacy callers. Skipped when the resolved list is empty.

Raises:

Type	Description
ROSCapabilityMismatch	If the dtype is not supported.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_quantization_dtype(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
    *,
    compute: ComputeSpec | None = None,
) -> None:
    """Verify the manifest's quantization dtype is in the robot's set.

    Reads ``gpu_supported_dtypes`` from *compute* when provided; falls
    back to *robot_capabilities* for legacy callers.  Skipped when the
    resolved list is empty.

    Raises:
        ROSCapabilityMismatch: If the dtype is not supported.
    """
    supported_dtypes = (
        compute.gpu_supported_dtypes
        if compute is not None
        else getattr(robot_capabilities, "gpu_supported_dtypes", [])
    )
    if supported_dtypes and manifest.quantization.dtype not in supported_dtypes:
        raise ROSCapabilityMismatch(
            f"rSkill '{manifest.name}' requires quantization dtype "
            f"'{manifest.quantization.dtype.value}', but robot only supports "
            f"{[d.value for d in supported_dtypes]}."
        )

check_runtime(manifest: RSkillManifest, robot_capabilities: RobotCapabilities, *, compute: ComputeSpec | None = None) -> None staticmethod

Verify the manifest's runtime is in the robot's supported set.

Reads gpu_supported_runtimes from compute when provided (the preferred path after the ComputeSpec split); falls back to the same field on robot_capabilities for legacy callers. Skipped when the resolved list is empty (host not yet probed — treat as unknown).

Raises:

Type	Description
ROSCapabilityMismatch	If the runtime is not supported.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_runtime(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
    *,
    compute: ComputeSpec | None = None,
) -> None:
    """Verify the manifest's runtime is in the robot's supported set.

    Reads ``gpu_supported_runtimes`` from *compute* when provided (the
    preferred path after the ``ComputeSpec`` split); falls back to the
    same field on *robot_capabilities* for legacy callers.  Skipped when
    the resolved list is empty (host not yet probed — treat as unknown).

    Raises:
        ROSCapabilityMismatch: If the runtime is not supported.
    """
    supported_runtimes = (
        compute.gpu_supported_runtimes
        if compute is not None
        else getattr(robot_capabilities, "gpu_supported_runtimes", [])
    )
    if supported_runtimes and manifest.runtime not in supported_runtimes:
        raise ROSCapabilityMismatch(
            f"rSkill '{manifest.name}' requires runtime "
            f"'{manifest.runtime.value}', but robot only supports "
            f"{[r.value for r in supported_runtimes]}."
        )

check_sensors(manifest: RSkillManifest, robot_sensors: list[SensorSpec]) -> None staticmethod

Verify the robot exposes every sensor the rSkill requires.

Resolves each SensorRequirement against robot_sensors per the rules documented on SensorRequirement:

• If a requirement carries a vla_feature_key, exactly one robot sensor must expose that key, with matching modality, and meeting any specified min_width / min_height minimum.
• Otherwise, the robot must expose at least count sensors of the requested modality, each meeting any resolution minimum.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest whose sensors_required to check.	required
robot_sensors	list[SensorSpec]	The robot's declared SensorSpec list (typically RobotDescription.sensors).	required

Raises:

Type	Description
ROSCapabilityMismatch	If any requirement is not satisfied.

Example

from openral_core import SensorSpec, SensorModality robot = [ ... SensorSpec( ... name="cam", modality=SensorModality.RGB, frame_id="world", rate_hz=20.0 ... ) ... ]

rSkill.check_sensors(manifest, robot) # raises if mismatch

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_sensors(
    manifest: RSkillManifest,
    robot_sensors: list[SensorSpec],
) -> None:
    """Verify the robot exposes every sensor the rSkill requires.

    Resolves each ``SensorRequirement`` against
    ``robot_sensors`` per the rules documented on
    ``SensorRequirement``:

    * If a requirement carries a ``vla_feature_key``, exactly one robot
      sensor must expose that key, with matching modality, and meeting
      any specified ``min_width`` / ``min_height`` minimum.
    * Otherwise, the robot must expose at least ``count`` sensors of
      the requested modality, each meeting any resolution minimum.

    Args:
        manifest: The rSkill manifest whose ``sensors_required`` to check.
        robot_sensors: The robot's declared ``SensorSpec`` list
            (typically ``RobotDescription.sensors``).

    Raises:
        ROSCapabilityMismatch: If any requirement is not satisfied.

    Example:
        >>> from openral_core import SensorSpec, SensorModality
        >>> robot = [
        ...     SensorSpec(
        ...         name="cam", modality=SensorModality.RGB, frame_id="world", rate_hz=20.0
        ...     )
        ... ]
        >>> # rSkill.check_sensors(manifest, robot)  # raises if mismatch
    """
    for req in manifest.sensors_required:
        cls = rSkill  # for the raised-error name; statically rSkill here
        if req.vla_feature_key is not None:
            cls._match_keyed_sensor(manifest.name, req, robot_sensors)
        else:
            cls._match_modality_count(manifest.name, req, robot_sensors)

from_pretrained(repo_id: str, *, revision: str | None = None, cache_dir: Path | None = None, force_download: bool = False, commercial_use: bool = True, registry_path: Path | None = None) -> rSkill classmethod

Download an rSkill from the HF Hub, validate it, and register it locally.

Steps performed in order:

1. Download rskill.yaml via huggingface_hub.hf_hub_download.
2. Parse and validate against RSkillManifest.
3. Run the license guard (hard-block for NVIDIA non-commercial without env).
4. Run the provenance guard (warn that signatures are unverified; fail closed if OPENRAL_REQUIRE_SIGNED_SKILLS=1).
5. Download the full snapshot via huggingface_hub.snapshot_download.
6. Register the entry in the local JSON registry.

.. warning:: No cryptographic signature verification is performed. The weights are trusted on the basis of HF Hub transport security only. Pin revision to a commit SHA for reproducibility, and treat any *.pt weights as untrusted code (see PyTorchRuntime).

Parameters:

Name	Type	Description	Default
repo_id	str	HF Hub repository, e.g. "openral/rskill-pick-cube-so100".	required
revision	str | None	Git commit SHA or branch to pin. None uses the latest commit on the default branch (not reproducible — pin in production).	None
cache_dir	Path | None	Override the default HF Hub cache directory.	None
force_download	bool	Re-download even if cached files exist.	False
commercial_use	bool	Set to False if the calling deployment is non-commercial research. Relaxes the NVIDIA non-commercial guard.	True
registry_path	Path | None	Override the default registry JSON path.	None

Returns:

Type	Description
rSkill	A validated rSkill instance ready for use.

Raises:

Type	Description
ROSConfigError	If the manifest is invalid, the license blocks the deployment, or huggingface_hub is not installed.
ImportError	Propagated if huggingface_hub is not importable (caught and re-raised as ROSConfigError).

Example

rSkill.from_pretrained("openral/rskill-pick-cube-so100")

rSkill.from_pretrained("openral/rskill-pick-cube-so100", revision="abc123")

Source code in python/rskill/src/openral_rskill/loader.py

@classmethod
def from_pretrained(
    cls,
    repo_id: str,
    *,
    revision: str | None = None,
    cache_dir: Path | None = None,
    force_download: bool = False,
    commercial_use: bool = True,
    registry_path: Path | None = None,
) -> rSkill:
    """Download an rSkill from the HF Hub, validate it, and register it locally.

    Steps performed in order:

    1. Download ``rskill.yaml`` via ``huggingface_hub.hf_hub_download``.
    2. Parse and validate against ``RSkillManifest``.
    3. Run the license guard (hard-block for NVIDIA non-commercial without env).
    4. Run the provenance guard (warn that signatures are unverified; fail
       closed if ``OPENRAL_REQUIRE_SIGNED_SKILLS=1``).
    5. Download the full snapshot via ``huggingface_hub.snapshot_download``.
    6. Register the entry in the local JSON registry.

    .. warning::
       No cryptographic signature verification is performed.  The
       weights are trusted on the basis of HF Hub transport security only.
       Pin ``revision`` to a commit SHA for reproducibility, and treat any
       ``*.pt`` weights as untrusted code (see ``PyTorchRuntime``).

    Args:
        repo_id: HF Hub repository, e.g. ``"openral/rskill-pick-cube-so100"``.
        revision: Git commit SHA or branch to pin.  ``None`` uses the latest
            commit on the default branch (not reproducible — pin in production).
        cache_dir: Override the default HF Hub cache directory.
        force_download: Re-download even if cached files exist.
        commercial_use: Set to ``False`` if the calling deployment is
            non-commercial research.  Relaxes the NVIDIA non-commercial guard.
        registry_path: Override the default registry JSON path.

    Returns:
        A validated ``rSkill`` instance ready for use.

    Raises:
        ROSConfigError: If the manifest is invalid, the license blocks the
            deployment, or ``huggingface_hub`` is not installed.
        ImportError: Propagated if ``huggingface_hub`` is not importable (caught
            and re-raised as ``ROSConfigError``).

    Example:
        >>> # rSkill.from_pretrained("openral/rskill-pick-cube-so100")
        >>> # rSkill.from_pretrained("openral/rskill-pick-cube-so100", revision="abc123")
    """
    try:
        from huggingface_hub import hf_hub_download, snapshot_download  # noqa: PLC0415
    except ImportError as exc:
        raise ROSConfigError(
            "rSkill.from_pretrained requires 'huggingface_hub'. "
            "Install it: uv add huggingface_hub --package openral-rskill"
        ) from exc

    _cache = str(cache_dir or _CACHE_HOME)

    # ── 1. Fetch manifest ──────────────────────────────────────────────────
    log.info("rskill.fetch_manifest", repo_id=repo_id, revision=revision)
    try:
        manifest_path_str = hf_hub_download(
            repo_id=repo_id,
            filename="rskill.yaml",
            revision=revision,
            cache_dir=_cache,
            force_download=force_download,
        )
    except Exception as exc:
        raise ROSConfigError(f"Failed to download rskill.yaml for '{repo_id}': {exc}") from exc

    # ── 2. Parse manifest ──────────────────────────────────────────────────
    manifest = RSkillManifest.from_yaml(manifest_path_str)

    # ── 3. License guard ───────────────────────────────────────────────────
    cls._check_license(manifest, commercial_use=commercial_use)
    _warn_unknown_benchmarks(manifest, source=repo_id)

    # ── 4. Provenance guard (signatures not yet verified) ───────────────────
    cls._check_provenance(manifest, source=repo_id)

    # ── 5. Download weights ────────────────────────────────────────────────
    log.info("rskill.download_snapshot", repo_id=repo_id, revision=revision)
    try:
        local_dir_str = snapshot_download(
            repo_id=repo_id,
            revision=revision,
            cache_dir=_cache,
            force_download=force_download,
        )
    except Exception as exc:
        raise ROSConfigError(f"Failed to download snapshot for '{repo_id}': {exc}") from exc

    # ── 6. Register ────────────────────────────────────────────────────────
    entry = InstalledRSkillEntry(
        repo_id=repo_id,
        version=manifest.version,
        revision=revision,
        local_dir=local_dir_str,
        manifest_path=manifest_path_str,
        license=manifest.license.value,
        role=manifest.role,
        embodiment_tags=manifest.embodiment_tags,
        installed_at=datetime.now(timezone.utc).isoformat(),
    )
    cls._register(entry, registry_path or DEFAULT_REGISTRY_PATH)

    log.info(
        "rskill.installed",
        repo_id=repo_id,
        version=manifest.version,
        license=manifest.license.value,
    )
    return cls(manifest=manifest, local_dir=Path(local_dir_str))

from_yaml(path: str | Path, *, local_dir: Path | None = None) -> rSkill classmethod

Load an rSkill from a local rskill.yaml without network access.

Does NOT register in the local registry. Useful for offline development and testing.

Parameters:

Name	Type	Description	Default
path	str | Path	Filesystem path to rskill.yaml.	required
local_dir	Path | None	Directory containing the skill files (defaults to the directory that contains rskill.yaml).	None

Returns:

Type	Description
rSkill	A validated rSkill instance.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.
ROSConfigError	If the license blocks the deployment, or OPENRAL_REQUIRE_SIGNED_SKILLS=1 and verification is unavailable (see _check_provenance).

Example

rSkill.from_yaml("/path/to/my-skill/rskill.yaml")

Source code in python/rskill/src/openral_rskill/loader.py

@classmethod
def from_yaml(cls, path: str | Path, *, local_dir: Path | None = None) -> rSkill:
    """Load an rSkill from a local ``rskill.yaml`` without network access.

    Does NOT register in the local registry.  Useful for offline development
    and testing.

    Args:
        path: Filesystem path to ``rskill.yaml``.
        local_dir: Directory containing the skill files (defaults to the
            directory that contains ``rskill.yaml``).

    Returns:
        A validated ``rSkill`` instance.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
        ROSConfigError: If the license blocks the deployment, or
            ``OPENRAL_REQUIRE_SIGNED_SKILLS=1`` and verification is
            unavailable (see ``_check_provenance``).

    Example:
        >>> # rSkill.from_yaml("/path/to/my-skill/rskill.yaml")
    """
    manifest = RSkillManifest.from_yaml(str(path))
    resolved = Path(path).resolve()
    cls._check_license(manifest, commercial_use=True)
    cls._check_provenance(manifest, source=str(resolved))
    skill_dir = local_dir or resolved.parent
    cls._validate_eval_jsons(skill_dir)
    return cls(manifest=manifest, local_dir=skill_dir)

list_installed(registry_path: Path | None = None) -> list[InstalledRSkillEntry] staticmethod

Return all entries in the local skill registry.

Parameters:

Name	Type	Description	Default
registry_path	Path | None	Override the default registry JSON path.	None

Returns:

Type	Description
list[InstalledRSkillEntry]	List of InstalledRSkillEntry, newest-first.

Raises:

Type	Description
ROSConfigError	If the registry file is corrupt (invalid JSON or schema).

Example

import tempfile, pathlib empty = pathlib.Path(tempfile.mkdtemp()) / "registry.json" result = rSkill.list_installed(registry_path=empty) isinstance(result, list) and len(result) == 0 True

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def list_installed(registry_path: Path | None = None) -> list[InstalledRSkillEntry]:
    """Return all entries in the local skill registry.

    Args:
        registry_path: Override the default registry JSON path.

    Returns:
        List of ``InstalledRSkillEntry``, newest-first.

    Raises:
        ROSConfigError: If the registry file is corrupt (invalid JSON or schema).

    Example:
        >>> import tempfile, pathlib
        >>> empty = pathlib.Path(tempfile.mkdtemp()) / "registry.json"
        >>> result = rSkill.list_installed(registry_path=empty)
        >>> isinstance(result, list) and len(result) == 0
        True
    """
    reg = registry_path or DEFAULT_REGISTRY_PATH
    if not reg.exists():
        return []
    try:
        raw: list[dict[str, Any]] = json.loads(reg.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        raise ROSConfigError(f"Corrupt skill registry at {reg}: {exc}") from exc
    entries = [InstalledRSkillEntry.model_validate(r) for r in raw]
    # newest-first
    return sorted(entries, key=lambda e: e.installed_at, reverse=True)

uninstall(repo_id: str, registry_path: Path | None = None) -> bool staticmethod

Remove a skill from the local registry (does NOT delete cached weights).

Parameters:

Name	Type	Description	Default
repo_id	str	The HF Hub repository identifier to remove.	required
registry_path	Path | None	Override the default registry JSON path.	None

Returns:

Type	Description
bool	True if the entry was found and removed, False if not found.

Example

rSkill.uninstall("openral/rskill-pick-cube-so100") False

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def uninstall(repo_id: str, registry_path: Path | None = None) -> bool:
    """Remove a skill from the local registry (does NOT delete cached weights).

    Args:
        repo_id: The HF Hub repository identifier to remove.
        registry_path: Override the default registry JSON path.

    Returns:
        ``True`` if the entry was found and removed, ``False`` if not found.

    Example:
        >>> rSkill.uninstall("openral/rskill-pick-cube-so100")
        False
    """
    reg = registry_path or DEFAULT_REGISTRY_PATH
    if not reg.exists():
        return False
    try:
        raw: list[dict[str, Any]] = json.loads(reg.read_text(encoding="utf-8"))
    except json.JSONDecodeError:
        return False
    before = len(raw)
    raw = [r for r in raw if r.get("repo_id") != repo_id]
    if len(raw) == before:
        return False
    reg.write_text(json.dumps(raw, indent=2), encoding="utf-8")
    log.info("rskill.uninstalled", repo_id=repo_id)
    return True

rSkillBase(name: str, *, version: str = '0.1.0', role: str = 's1', embodiment_tags: list[str] | None = None, latency_budget_ms: float | None = None)

Bases: ABC

Abstract base class for all OpenRAL skills.

Subclasses must implement:

• _configure_impl — load config, parse manifest, etc.
• _activate_impl — final pre-execution setup (e.g. warm-up).
• _deactivate_impl — pause execution without unloading weights.
• _shutdown_impl — release all resources.
• _step_impl — one inference step; returns an Action.

Subclasses may optionally override:

• on_load_weights — called during configure to load model weights. Sets info.weights_loaded = True on return.
• on_quantize — called after weight loading to quantize. Sets info.quantized = True on return.
• on_warmup — called during activate to run a dummy inference. Sets info.warmed_up = True on return.

Parameters:

Name	Type	Description	Default
name	str	Skill name, used as the ROS 2 node name and in /skill/<name>/info.	required
version	str	SemVer string, forwarded to RSkillInfo.	'0.1.0'
role	str	Skill slot — "s0", "s1", or "s2".	's1'
embodiment_tags	list[str] | None	Embodiment tags for capability matching.	None
latency_budget_ms	float | None	Maximum allowed inference latency. Exceeded latency is logged as a warning but does not raise.	None

Initialise the skill; does not configure or load weights.

Source code in python/rskill/src/openral_rskill/base.py

def __init__(
    self,
    name: str,
    *,
    version: str = "0.1.0",
    role: str = "s1",
    embodiment_tags: list[str] | None = None,
    latency_budget_ms: float | None = None,
) -> None:
    """Initialise the skill; does not configure or load weights."""
    self._info = RSkillInfo(
        name=name,
        version=version,
        state=RSkillState.UNCONFIGURED,
        role=role,
        embodiment_tags=embodiment_tags or [],
        latency_budget_ms=latency_budget_ms,
        stamp_ns=time.time_ns(),
    )
    log.info("skill.created", name=name, role=role, version=version)

info: RSkillInfo property

Current RSkillInfo snapshot.

Returns a copy — mutating the returned object has no effect.

name: str property

Skill name.

state: RSkillState property

Current primary lifecycle state.

activate() -> None

Transition inactive → active.

Calls on_warmup, then _activate_impl.

Raises:

Type	Description
ROSRuntimeError	If the current state does not allow this transition.

Source code in python/rskill/src/openral_rskill/base.py

@final
def activate(self) -> None:
    """Transition ``inactive → active``.

    Calls ``on_warmup``, then ``_activate_impl``.

    Raises:
        ROSRuntimeError: If the current state does not allow this
            transition.
    """
    self._require_transition(RSkillState.ACTIVE)
    log.info("rskill.activate", name=self.name)
    try:
        with rskill_span("rskill.activate", rskill_id=self.name, role=self._info.role):
            self.on_warmup()
            self._update(warmed_up=True)
            self._activate_impl()
            self._transition(RSkillState.ACTIVE)
    except Exception as exc:
        self._enter_error(str(exc))
        raise

configure() -> None

Transition unconfigured → inactive.

Calls on_load_weights, then on_quantize, then _configure_impl in order.

Raises:

Type	Description
ROSRuntimeError	If the current state does not allow this transition.

Source code in python/rskill/src/openral_rskill/base.py

@final
def configure(self) -> None:
    """Transition ``unconfigured → inactive``.

    Calls ``on_load_weights``, then ``on_quantize``, then
    ``_configure_impl`` in order.

    Raises:
        ROSRuntimeError: If the current state does not allow this
            transition.
    """
    self._require_transition(RSkillState.INACTIVE)
    log.info("rskill.configure", name=self.name)
    try:
        with rskill_span("rskill.configure", rskill_id=self.name, role=self._info.role):
            self.on_load_weights()
            self._update(weights_loaded=True)
            self.on_quantize()
            self._update(quantized=True)
            self._configure_impl()
            self._transition(RSkillState.INACTIVE)
    except Exception as exc:
        self._enter_error(str(exc))
        raise

deactivate() -> None

Transition active → inactive.

Raises:

Type	Description
ROSRuntimeError	If the current state is not active.

Source code in python/rskill/src/openral_rskill/base.py

@final
def deactivate(self) -> None:
    """Transition ``active → inactive``.

    Raises:
        ROSRuntimeError: If the current state is not ``active``.
    """
    if self._info.state is not RSkillState.ACTIVE:
        raise ROSRuntimeError(
            f"Skill '{self.name}': deactivate() requires 'active' state, "
            f"current state is '{self._info.state.value}'."
        )
    log.info("rskill.deactivate", name=self.name)
    try:
        self._deactivate_impl()
        self._transition(RSkillState.INACTIVE)
    except Exception as exc:
        self._enter_error(str(exc))
        raise

on_load_weights() -> None

Load model weights into memory.

Called during configure before on_quantize. Default is a no-op; override to load safetensors / ONNX files.

Source code in python/rskill/src/openral_rskill/base.py

def on_load_weights(self) -> None:
    """Load model weights into memory.

    Called during ``configure`` before ``on_quantize``.
    Default is a no-op; override to load ``safetensors`` / ONNX files.
    """

on_quantize() -> None

Apply quantization to loaded weights.

Called during configure after on_load_weights. Default is a no-op; override to apply INT8 / INT4 / NVFP4.

Source code in python/rskill/src/openral_rskill/base.py

def on_quantize(self) -> None:
    """Apply quantization to loaded weights.

    Called during ``configure`` after ``on_load_weights``.
    Default is a no-op; override to apply INT8 / INT4 / NVFP4.
    """

on_unload_weights() -> None

Release model weights from memory.

Symmetric with on_load_weights. Called by shutdown when weights_loaded is set (before _shutdown_impl), so the skill runner's single-resident eviction frees GPU VRAM before the next skill loads. Default is a no-op; override to drop model references and call torch.cuda.empty_cache() (or terminate an inference sidecar).

Source code in python/rskill/src/openral_rskill/base.py

def on_unload_weights(self) -> None:
    """Release model weights from memory.

    Symmetric with ``on_load_weights``. Called by ``shutdown``
    when ``weights_loaded`` is set (before ``_shutdown_impl``), so the
    skill runner's single-resident eviction frees GPU VRAM before the next
    skill loads. Default is a no-op; override to drop model references and
    call ``torch.cuda.empty_cache()`` (or terminate an inference sidecar).
    """

on_warmup() -> None

Run a dummy inference to warm up the model.

Called during activate before _activate_impl. Default is a no-op; override to call the model once with dummy input.

Source code in python/rskill/src/openral_rskill/base.py

def on_warmup(self) -> None:
    """Run a dummy inference to warm up the model.

    Called during ``activate`` before ``_activate_impl``.
    Default is a no-op; override to call the model once with dummy input.
    """

shutdown() -> None

Transition any state → finalized.

Idempotent if already finalized.

Source code in python/rskill/src/openral_rskill/base.py

@final
def shutdown(self) -> None:
    """Transition any state → ``finalized``.

    Idempotent if already ``finalized``.
    """
    if self._info.state is RSkillState.FINALIZED:
        return
    log.info("rskill.shutdown", name=self.name)
    try:
        # Release GPU weights on the way down so the skill
        # runner's single-resident eviction frees VRAM before the next
        # skill loads. Guarded by ``weights_loaded`` so a shutdown before
        # configure() never invokes a subclass hook that assumes loaded
        # state.
        if self._info.weights_loaded:
            self.on_unload_weights()
            self._update(weights_loaded=False)
        self._shutdown_impl()
    except Exception as exc:  # reason: shutdown must always reach finalized; log and swallow
        log.error("skill.shutdown.error", name=self.name, exc=str(exc))
    finally:
        self._transition(RSkillState.FINALIZED)

step(world_state: WorldState) -> Action | list[Action]

Execute one inference step and return one or more Action chunks.

Records wall-clock latency and logs a warning if it exceeds latency_budget_ms.

Parameters:

Name	Type	Description	Default
world_state	WorldState	Current WorldState snapshot from the aggregator.	required

Returns:

Type	Description
Action | list[Action]	Either a single Action chunk
Action | list[Action]	(the legacy single-control-surface path used by every skill
Action | list[Action]	shipped before multi-surface action dispatch was introduced) OR
Action | list[Action]	a list of Action chunks (multi-surface action dispatch — used by skills
Action | list[Action]	whose manifest declares an action_contract.slots block;
Action | list[Action]	each slot in the manifest becomes one Action in
Action | list[Action]	the returned list, all routed by the HAL according to their
Action | list[Action]	individual control_mode).

Raises:

Type	Description
ROSRuntimeError	If the skill is not in the active state.

Source code in python/rskill/src/openral_rskill/base.py

@final
def step(self, world_state: WorldState) -> Action | list[Action]:
    """Execute one inference step and return one or more ``Action`` chunks.

    Records wall-clock latency and logs a warning if it exceeds
    ``latency_budget_ms``.

    Args:
        world_state: Current ``WorldState``
            snapshot from the aggregator.

    Returns:
        Either a single ``Action`` chunk
        (the legacy single-control-surface path used by every skill
        shipped before multi-surface action dispatch was introduced) OR
        a list of ``Action`` chunks (multi-surface action dispatch — used by skills
        whose manifest declares an ``action_contract.slots`` block;
        each slot in the manifest becomes one ``Action`` in
        the returned list, all routed by the HAL according to their
        individual ``control_mode``).

    Raises:
        ROSRuntimeError: If the skill is not in the ``active`` state.
    """
    if self._info.state is not RSkillState.ACTIVE:
        raise ROSRuntimeError(
            f"Skill '{self.name}' is in state '{self._info.state.value}'; "
            "must be 'active' to call step()."
        )
    t0 = time.perf_counter()
    with rskill_span("rskill.execute", rskill_id=self.name, role=self._info.role):
        action = self._step_impl(world_state)
    elapsed_ms = (time.perf_counter() - t0) * 1000.0
    self._update(last_inference_ms=elapsed_ms)
    budget = self._info.latency_budget_ms
    if budget is not None and elapsed_ms > budget:
        log.warning(
            "skill.step.latency_exceeded",
            name=self.name,
            elapsed_ms=round(elapsed_ms, 2),
            budget_ms=budget,
        )
    log.debug("skill.step", name=self.name, elapsed_ms=round(elapsed_ms, 2))
    return action

discover_intree_rskills() -> list[tuple[str, RSkillManifest]]

Walk the in-tree rskills/ directory and return (name, manifest) pairs.

Skipped on stderr-reported errors so a malformed local manifest does not break the listing. Sorted alphabetically by directory name. No HF Hub network call.

Returns:

Type	Description
list[tuple[str, RSkillManifest]]	List of (rskill_name, manifest) tuples, one per
list[tuple[str, RSkillManifest]]	rskills/<name>/rskill.yaml on disk.

Source code in python/rskill/src/openral_rskill/loader.py

def discover_intree_rskills() -> list[tuple[str, RSkillManifest]]:
    """Walk the in-tree ``rskills/`` directory and return ``(name, manifest)`` pairs.

    Skipped on stderr-reported errors so a malformed local manifest does
    not break the listing. Sorted alphabetically by directory name. No
    HF Hub network call.

    Returns:
        List of ``(rskill_name, manifest)`` tuples, one per
        ``rskills/<name>/rskill.yaml`` on disk.
    """
    repo_root = find_repo_root_from(Path(__file__))
    if repo_root is None:
        return []
    rskills_dir = repo_root / "rskills"
    if not rskills_dir.is_dir():
        return []

    out: list[tuple[str, RSkillManifest]] = []
    for child in sorted(rskills_dir.iterdir()):
        if not child.is_dir() or not (child / "rskill.yaml").is_file():
            continue
        try:
            manifest = load_rskill_manifest(child.name)
        except (ROSError, ValueError) as exc:
            print(f"  {child.name}: {exc}", file=sys.stderr)
            continue
        out.append((child.name, manifest))
    return out

find_repo_root_from(start: Path) -> Path | None

Locate the OpenRAL repo root by walking up from start.

Returns the first parent directory that contains both pyproject.toml and a rskills/ directory, or None if no such ancestor exists.

Source code in python/rskill/src/openral_rskill/loader.py

def find_repo_root_from(start: Path) -> Path | None:
    """Locate the OpenRAL repo root by walking up from ``start``.

    Returns the first parent directory that contains both ``pyproject.toml``
    and a ``rskills/`` directory, or ``None`` if no such ancestor exists.
    """
    here = start.resolve()
    for ancestor in (here, *here.parents):
        if (ancestor / "pyproject.toml").is_file() and (ancestor / "rskills").is_dir():
            return ancestor
    return None

gpu_allocated_mb(*, no_import: bool = False) -> float | None

Return current CUDA allocator usage in MB, or None if unavailable.

Deliberately memory_allocated (live tensors), not memory_reserved (the caching allocator's pool, which stays put by design after a free) — the question is "did the weights actually go away". THE single GPU-memory probe: eviction accounting in rskill_runner_node and the phase-timer heartbeats read this same number, so their logs never disagree about how much VRAM a swap freed.

Parameters:

Name	Type	Description	Default
no_import	bool	When True, only consult an already-imported torch (sys.modules) — importing torch just to answer would itself be the heavy operation on some paths. Default imports torch lazily, so a CPU-only host still answers None.	False

Source code in python/rskill/src/openral_rskill/_diagnostics.py

def gpu_allocated_mb(*, no_import: bool = False) -> float | None:
    """Return current CUDA allocator usage in MB, or ``None`` if unavailable.

    Deliberately ``memory_allocated`` (live tensors), not ``memory_reserved``
    (the caching allocator's pool, which stays put by design after a free) —
    the question is "did the weights actually go away". THE single
    GPU-memory probe: eviction accounting in ``rskill_runner_node`` and the
    phase-timer heartbeats read this same number, so their logs never
    disagree about how much VRAM a swap freed.

    Args:
        no_import: When True, only consult an already-imported torch
            (``sys.modules``) — importing torch just to answer would itself
            be the heavy operation on some paths. Default imports torch
            lazily, so a CPU-only host still answers ``None``.
    """
    if no_import:
        torch_mod = sys.modules.get("torch")
    else:
        try:
            import torch as torch_mod  # noqa: PLC0415
        except ImportError:
            return None
    if torch_mod is None:
        return None
    try:
        if not torch_mod.cuda.is_available():
            return None
        return float(torch_mod.cuda.memory_allocated()) / 1024 / 1024
    except Exception:
        return None

hf_download_cached_first(hf_hub_download: Any, local_not_found_exc: type[BaseException], *, repo_id: str, filename: str, revision: str | None = None, **extra: Any) -> str

Resolve an HF Hub file via the cache first, fall back to the Hub.

Every hf_hub_download(...) call without local_files_only=True HEAD-validates the cached file against the Hub, even when the file is already on disk. On a cold TLS connection that HEAD is 0.5 - 3 s per call; with N processor + state_file URIs the cost stacks into the visible portion of a 90 s policy load.

This helper tries local_files_only=True first. On cache hit (the common case for a robot bring-up against a known checkpoint) no network call happens at all. On miss — manifest pins a new revision, cache was cleared, first download — falls back to the normal call so behaviour is unchanged when no cached file exists.

Set HF_HUB_OFFLINE=1 to force offline mode for every HF call in the process, including those inside Policy.from_pretrained (which this helper does not wrap). That env-var is the broader knob when even the inner lerobot / transformers cache validation is the bottleneck.

Parameters:

Name	Type	Description	Default
hf_hub_download	Any	The imported huggingface_hub.hf_hub_download function. Injected to avoid an import in every caller.	required
local_not_found_exc	type[BaseException]	huggingface_hub.errors.LocalEntryNotFoundError. Same injection rationale.	required
repo_id	str	HF Hub repo id.	required
filename	str	File path within the repo.	required
revision	str | None	Optional git revision to pin.	None
**extra	Any	Forwarded verbatim to both hf_hub_download calls.	{}

Returns:

Type	Description
str	Absolute local path to the (now cached) file.

Source code in python/rskill/src/openral_rskill/_vla_core.py

def hf_download_cached_first(
    hf_hub_download: Any,
    local_not_found_exc: type[BaseException],
    *,
    repo_id: str,
    filename: str,
    revision: str | None = None,
    **extra: Any,
) -> str:
    """Resolve an HF Hub file via the cache first, fall back to the Hub.

    Every ``hf_hub_download(...)`` call without ``local_files_only=True``
    HEAD-validates the cached file against the Hub, even when the file
    is already on disk. On a cold TLS connection that HEAD is 0.5 - 3 s
    per call; with N processor + state_file URIs the cost stacks into
    the visible portion of a 90 s policy load.

    This helper tries ``local_files_only=True`` first. On cache hit
    (the common case for a robot bring-up against a known checkpoint)
    no network call happens at all. On miss — manifest pins a new
    revision, cache was cleared, first download — falls back to the
    normal call so behaviour is unchanged when no cached file exists.

    Set ``HF_HUB_OFFLINE=1`` to force offline mode for every HF call in
    the process, including those inside ``Policy.from_pretrained``
    (which this helper does not wrap). That env-var is the broader knob
    when even the inner lerobot / transformers cache validation is the
    bottleneck.

    Args:
        hf_hub_download: The imported ``huggingface_hub.hf_hub_download``
            function. Injected to avoid an import in every caller.
        local_not_found_exc: ``huggingface_hub.errors.LocalEntryNotFoundError``.
            Same injection rationale.
        repo_id: HF Hub repo id.
        filename: File path within the repo.
        revision: Optional git revision to pin.
        **extra: Forwarded verbatim to both ``hf_hub_download`` calls.

    Returns:
        Absolute local path to the (now cached) file.
    """
    try:
        return str(
            hf_hub_download(
                repo_id=repo_id,
                filename=filename,
                revision=revision,
                local_files_only=True,
                **extra,
            )
        )
    except local_not_found_exc:
        return str(
            hf_hub_download(
                repo_id=repo_id,
                filename=filename,
                revision=revision,
                **extra,
            )
        )

intree_embodiment_tags() -> frozenset[str]

Return the embodiment tags an in-tree rSkill manifest may declare.

SKILL_ONLY_EMBODIMENT_TAGS plus every capabilities.embodiment_tags entry of the checkout's robots/*/robot.yaml. openral_core.EmbodimentTag is an open id, so this is the typo guard the openral rskill new scaffolder and CI use. Without a checkout only SKILL_ONLY_EMBODIMENT_TAGS is returned.

Source code in python/rskill/src/openral_rskill/loader.py

def intree_embodiment_tags() -> frozenset[str]:
    """Return the embodiment tags an in-tree rSkill manifest may declare.

    ``SKILL_ONLY_EMBODIMENT_TAGS`` plus every
    ``capabilities.embodiment_tags`` entry of the checkout's
    ``robots/*/robot.yaml``. ``openral_core.EmbodimentTag`` is an open id, so
    this is the typo guard the ``openral rskill new`` scaffolder and CI use.
    Without a checkout only ``SKILL_ONLY_EMBODIMENT_TAGS`` is returned.
    """
    tags = set(SKILL_ONLY_EMBODIMENT_TAGS)
    repo_root = find_repo_root_from(Path(__file__))
    if repo_root is not None:
        for path in sorted((repo_root / "robots").glob("*/robot.yaml")):
            tags.update(RobotDescription.from_yaml(str(path)).capabilities.embodiment_tags)
    return frozenset(tags)

known_benchmark_ids() -> frozenset[str] | None

Return the benchmark ids the checkout defines, or None without one.

The benchmarks/*.yaml suite stems plus the scenes/benchmark/*.yaml scene stems (which include the task-level aloha_insertion / aloha_transfer_cube ids). openral_core.BenchmarkName is an open id; this is the set RSkillManifest.benchmarks keys are checked against.

Source code in python/rskill/src/openral_rskill/loader.py

def known_benchmark_ids() -> frozenset[str] | None:
    """Return the benchmark ids the checkout defines, or ``None`` without one.

    The ``benchmarks/*.yaml`` suite stems plus the ``scenes/benchmark/*.yaml``
    scene stems (which include the task-level ``aloha_insertion`` /
    ``aloha_transfer_cube`` ids). ``openral_core.BenchmarkName`` is an open id;
    this is the set ``RSkillManifest.benchmarks`` keys are checked against.
    """
    repo_root = find_repo_root_from(Path(__file__))
    if repo_root is None:
        return None
    return frozenset(
        p.stem for d in ("benchmarks", "scenes/benchmark") for p in (repo_root / d).glob("*.yaml")
    )

local_snapshot_dir(repo_id: str, *, revision: str | None = None, ignore_patterns: list[str] | tuple[str, ...] = ('*.md',), **extra: Any) -> str

A local directory holding repo_id's files: the directory itself, or its Hub snapshot.

The rSkill resolver hands adapters either a Hub repo id or — for a skill installed with openral rskill install, whose snapshot already carries rskill.yaml next to model.safetensors — that snapshot's directory (resolve_rskill_to_hf_with_revision). huggingface_hub.snapshot_download validates its argument as a repo id and rejects a filesystem path, so every adapter that snapshotted the resolved id broke on an installed skill: ACT, Diffusion, GR00T and the processor-sidecar fallback (found on the ACT LIBERO checkpoint, 2026-09-23). A directory is returned as is — it is already the pinned snapshot, so revision has nothing to add — and a repo id is snapshotted with ignore_patterns / extra forwarded.

Parameters:

Name	Type	Description	Default
repo_id	str	Hub repo id, or a local checkpoint directory.	required
revision	str | None	Optional git revision, forwarded for the repo-id case.	None
ignore_patterns	list[str] | tuple[str, ...]	Glob patterns snapshot_download skips.	('*.md',)
**extra	Any	Forwarded verbatim to snapshot_download.	{}

Returns:

Type	Description
str	Absolute path of a directory containing the checkpoint files.

Example

import os, tempfile d = tempfile.mkdtemp() local_snapshot_dir(d) == os.path.realpath(d) True

Source code in python/rskill/src/openral_rskill/_vla_core.py

def local_snapshot_dir(
    repo_id: str,
    *,
    revision: str | None = None,
    ignore_patterns: list[str] | tuple[str, ...] = ("*.md",),
    **extra: Any,
) -> str:
    """A local directory holding ``repo_id``'s files: the directory itself, or its Hub snapshot.

    The rSkill resolver hands adapters either a Hub repo id or — for a skill
    installed with ``openral rskill install``, whose snapshot already carries
    ``rskill.yaml`` next to ``model.safetensors`` — that snapshot's directory
    (``resolve_rskill_to_hf_with_revision``). ``huggingface_hub.snapshot_download``
    validates its argument as a repo id and rejects a filesystem path, so every
    adapter that snapshotted the resolved id broke on an installed skill: ACT,
    Diffusion, GR00T and the processor-sidecar fallback (found on the ACT
    LIBERO checkpoint, 2026-09-23). A directory is returned as is — it is
    already the pinned snapshot, so ``revision`` has nothing to add — and a
    repo id is snapshotted with ``ignore_patterns`` / ``extra`` forwarded.

    Args:
        repo_id: Hub repo id, or a local checkpoint directory.
        revision: Optional git revision, forwarded for the repo-id case.
        ignore_patterns: Glob patterns ``snapshot_download`` skips.
        **extra: Forwarded verbatim to ``snapshot_download``.

    Returns:
        Absolute path of a directory containing the checkpoint files.

    Example:
        >>> import os, tempfile
        >>> d = tempfile.mkdtemp()
        >>> local_snapshot_dir(d) == os.path.realpath(d)
        True
    """
    from pathlib import Path

    candidate = Path(repo_id)
    if candidate.is_dir():
        return str(candidate.resolve())
    from huggingface_hub import snapshot_download

    return str(
        snapshot_download(
            repo_id=repo_id, revision=revision, ignore_patterns=list(ignore_patterns), **extra
        )
    )

maybe_attach_pro_hooks(policy_name: str, skill: Any, **kwargs: Any) -> bool

Look up and invoke an OpenRAL Pro policy-attach hook, if installed.

The openral_sim.policies adapters (smolvla, act) call this instead of a per-policy try: from openral_rskill.<x>_trt import maybe_attach_<x>_trt_from_env. A hook registered under the openral.policy_attach_hooks entry-point group (name = policy_name, e.g. "smolvla" or "act") is loaded and called as hook(skill, **kwargs); its truthy/falsy return says whether it swapped the policy's inference path.

No hook installed (open-source-only install, i.e. no openral-pro-trt) is expected and explicit: a debug log, and False — the caller falls through to its own eager/host path. This is not a silent skip (CLAUDE.md §1.4): the debug log records that the lookup happened and found nothing.

Parameters:

Name	Type	Description	Default
policy_name	str	Entry-point name to look up, e.g. "smolvla", "act".	required
skill	Any	The policy/skill object the hook may mutate in place (e.g. a SmolVLAPolicy or ACTPolicy instance).	required
**kwargs	Any	Forwarded verbatim to the hook (e.g. repo_id, device, n_cameras, onnx_uri).	{}

Returns:

Type	Description
bool	True if a hook was found and reported attaching; False
bool	otherwise (no hook installed, or the hook declined — e.g. env var
bool	not set).

Example

A policy with no hook registered under it takes the absent branch on every host. (The example deliberately does not use "smolvla": on a box with openral-pro-trt installed that name resolves to a real hook, and calling it without the repo_id every real caller passes raised a TypeError from the doctest — the same environment-dependence that made mypy --strict fail on dev boxes with the pro tier present.)

from openral_rskill.runtime import NullRuntime maybe_attach_pro_hooks("_policy_with_no_hook", NullRuntime()) False

Source code in python/rskill/src/openral_rskill/backend_registry.py

def maybe_attach_pro_hooks(policy_name: str, skill: Any, **kwargs: Any) -> bool:  # noqa: ANN401  # reason: forwarded verbatim to whatever hook is registered
    """Look up and invoke an OpenRAL Pro policy-attach hook, if installed.

    The ``openral_sim.policies`` adapters (``smolvla``, ``act``) call this
    instead of a per-policy ``try: from openral_rskill.<x>_trt import
    maybe_attach_<x>_trt_from_env``. A hook
    registered under the ``openral.policy_attach_hooks`` entry-point group
    (name = *policy_name*, e.g. ``"smolvla"`` or ``"act"``) is loaded and
    called as ``hook(skill, **kwargs)``; its truthy/falsy return says whether
    it swapped the policy's inference path.

    No hook installed (open-source-only install, i.e. no ``openral-pro-trt``)
    is expected and explicit: a debug log, and ``False`` — the caller falls
    through to its own eager/host path. This is not a silent skip (CLAUDE.md
    §1.4): the debug log records that the lookup happened and found nothing.

    Args:
        policy_name: Entry-point name to look up, e.g. ``"smolvla"``,
            ``"act"``.
        skill: The policy/skill object the hook may mutate in place (e.g. a
            ``SmolVLAPolicy`` or ``ACTPolicy`` instance).
        **kwargs: Forwarded verbatim to the hook (e.g. ``repo_id``,
            ``device``, ``n_cameras``, ``onnx_uri``).

    Returns:
        ``True`` if a hook was found and reported attaching; ``False``
        otherwise (no hook installed, or the hook declined — e.g. env var
        not set).

    Example:
        A policy with no hook registered under it takes the absent branch on
        every host. (The example deliberately does *not* use ``"smolvla"``:
        on a box with ``openral-pro-trt`` installed that name resolves to a
        real hook, and calling it without the ``repo_id`` every real caller
        passes raised a ``TypeError`` from the doctest — the same
        environment-dependence that made ``mypy --strict`` fail on dev boxes
        with the pro tier present.)

        >>> from openral_rskill.runtime import NullRuntime
        >>> maybe_attach_pro_hooks("_policy_with_no_hook", NullRuntime())
        False
    """
    for ep in entry_points(group=_POLICY_ATTACH_HOOKS_GROUP):
        if ep.name == policy_name:
            hook = ep.load()
            attached = bool(hook(skill, **kwargs))
            if attached:
                log.info("pro_hooks.attached", policy=policy_name, entry_point=ep.value)
            else:
                log.debug("pro_hooks.declined", policy=policy_name, entry_point=ep.value)
            return attached

    log.debug("pro_hooks.absent", policy=policy_name, reason="openral-pro-trt not installed")
    return False

resolve_runtime_backend(kind: str) -> type[Runtime]

Resolve an inference-runtime backend class by name.

Checks the built-in backends first (pytorch, onnx, null), then the openral.runtime_backends entry-point group for privately installed backends (e.g. openral-pro-trt registering tensorrt).

Parameters:

Name	Type	Description	Default
kind	str	Backend name, e.g. "pytorch", "onnx", "null", or "tensorrt" (only resolvable with openral-pro-trt installed).	required

Returns:

Type	Description
type[Runtime]	The Runtime-conforming class
type[Runtime]	(not an instance — callers construct it themselves, e.g.
type[Runtime]	resolve_runtime_backend("pytorch")(device="cuda:0")).

Raises:

Type	Description
ROSConfigError	kind is neither a built-in backend nor a registered entry point. Names the private package to install.

Example

from openral_rskill.runtime import NullRuntime resolve_runtime_backend("null") is NullRuntime True resolve_runtime_backend("does-not-exist") # doctest: +IGNORE_EXCEPTION_DETAIL Traceback (most recent call last): ... openral_core.exceptions.ROSConfigError: ...

Source code in python/rskill/src/openral_rskill/backend_registry.py

def resolve_runtime_backend(kind: str) -> type[Runtime]:
    """Resolve an inference-runtime backend class by name.

    Checks the built-in backends first (``pytorch``, ``onnx``, ``null``),
    then the ``openral.runtime_backends`` entry-point group for privately
    installed backends (e.g. ``openral-pro-trt`` registering ``tensorrt``).

    Args:
        kind: Backend name, e.g. ``"pytorch"``, ``"onnx"``, ``"null"``, or
            ``"tensorrt"`` (only resolvable with ``openral-pro-trt``
            installed).

    Returns:
        The ``Runtime``-conforming class
        (not an instance — callers construct it themselves, e.g.
        ``resolve_runtime_backend("pytorch")(device="cuda:0")``).

    Raises:
        ROSConfigError: *kind* is neither a built-in backend nor a
            registered entry point. Names the private package to install.

    Example:
        >>> from openral_rskill.runtime import NullRuntime
        >>> resolve_runtime_backend("null") is NullRuntime
        True
        >>> resolve_runtime_backend("does-not-exist")  # doctest: +IGNORE_EXCEPTION_DETAIL
        Traceback (most recent call last):
        ...
        openral_core.exceptions.ROSConfigError: ...
    """
    builtin = _BUILTIN_RUNTIME_BACKENDS.get(kind)
    if builtin is not None:
        module_path, _, attr = builtin.partition(":")
        loaded_builtin: type[Runtime] = getattr(import_module(module_path), attr)
        return loaded_builtin

    for ep in entry_points(group=_RUNTIME_BACKENDS_GROUP):
        if ep.name == kind:
            loaded: type[Runtime] = ep.load()
            log.debug("runtime_backend.resolved", kind=kind, entry_point=ep.value)
            return loaded

    raise ROSConfigError(
        f"resolve_runtime_backend: unknown runtime backend {kind!r}. Built-in "
        f"backends are {sorted(_BUILTIN_RUNTIME_BACKENDS)!r}; 'tensorrt' and "
        "other proprietary backends require the private openral-pro-trt "
        "package, which registers itself via the "
        f"{_RUNTIME_BACKENDS_GROUP!r} entry-point group."
    )

search_hub_rskills(query: str = '', *, kind: str = '', role: str = '', embodiment: str = '', license: str = '', family: str = '', org: str = 'OpenRAL', limit: int | None = None, max_workers: int = 8) -> HubRSkillSearchResult

Search an HF Hub org for rSkills matching a free-text query + facets.

Lists every repo the org tagged rskill (one Hub call, no Hub-side search=), fetches each candidate's rskill.yaml concurrently, then matches query locally (case-insensitive, every whitespace-split token must substring-match repo id / manifest name / description / model family / kind / role / embodiment tags / Hub tags) and applies the facet filters. Repos without a valid manifest are counted in skipped and excluded from hits regardless of query or filters.

Parameters:

Name	Type	Description	Default
query	str	Free-text query; empty matches every valid manifest.	''
kind	str	Exact-match filter on manifest.kind (e.g. "vla").	''
role	str	Exact-match filter on manifest.role ("s0"/"s1"/"s2").	''
embodiment	str	Require this tag in manifest.embodiment_tags.	''
license	str	Exact-match filter on manifest.license.value.	''
family	str	Exact-match filter on manifest.model_family.	''
org	str	HF Hub org (author) to search. Defaults to "OpenRAL".	'OpenRAL'
limit	int | None	Max hits returned, applied after sorting by repo_id. None (default) returns every match.	None
max_workers	int	Thread pool size for concurrent manifest fetches.	8

Returns:

Type	Description
HubRSkillSearchResult	A HubRSkillSearchResult with hits sorted by repo_id.

Raises:

Type	Description
ImportError	If huggingface_hub is not installed.

Example

result = search_hub_rskills("aloha", kind="vla")

[h.repo_id for h in result.hits]

['OpenRAL/rskill-act-aloha-aloha_transfer_cube-fp32']

Source code in python/rskill/src/openral_rskill/hub_search.py

def search_hub_rskills(
    query: str = "",
    *,
    kind: str = "",
    role: str = "",
    embodiment: str = "",
    license: str = "",  # reason: matches the CLI's --license facet name
    family: str = "",
    org: str = "OpenRAL",
    limit: int | None = None,
    max_workers: int = 8,
) -> HubRSkillSearchResult:
    """Search an HF Hub org for rSkills matching a free-text query + facets.

    Lists every repo the org tagged ``rskill`` (one Hub call, no Hub-side
    ``search=``), fetches each candidate's ``rskill.yaml`` concurrently, then
    matches ``query`` locally (case-insensitive, every whitespace-split token
    must substring-match repo id / manifest name / description / model
    family / kind / role / embodiment tags / Hub tags) and applies the facet
    filters. Repos without a valid manifest are counted in ``skipped`` and
    excluded from ``hits`` regardless of query or filters.

    Args:
        query: Free-text query; empty matches every valid manifest.
        kind: Exact-match filter on ``manifest.kind`` (e.g. ``"vla"``).
        role: Exact-match filter on ``manifest.role`` (``"s0"``/``"s1"``/``"s2"``).
        embodiment: Require this tag in ``manifest.embodiment_tags``.
        license: Exact-match filter on ``manifest.license.value``.
        family: Exact-match filter on ``manifest.model_family``.
        org: HF Hub org (``author``) to search. Defaults to ``"OpenRAL"``.
        limit: Max hits returned, applied after sorting by ``repo_id``.
            ``None`` (default) returns every match.
        max_workers: Thread pool size for concurrent manifest fetches.

    Returns:
        A ``HubRSkillSearchResult`` with hits sorted by ``repo_id``.

    Raises:
        ImportError: If ``huggingface_hub`` is not installed.

    Example:
        >>> # result = search_hub_rskills("aloha", kind="vla")
        >>> # [h.repo_id for h in result.hits]
        >>> # ['OpenRAL/rskill-act-aloha-aloha_transfer_cube-fp32']
    """
    from huggingface_hub import HfApi  # noqa: PLC0415

    models = list(HfApi().list_models(author=org, filter="rskill", limit=None))
    inspected = len(models)

    skipped_repo_ids: list[str] = []
    candidates: list[tuple[str, RSkillManifest, tuple[str, ...]]] = []
    with ThreadPoolExecutor(
        max_workers=max_workers, thread_name_prefix="rskill_hub_search"
    ) as pool:
        for repo_id, manifest, hub_tags, _reason in pool.map(_fetch_one, models):
            if manifest is None:
                skipped_repo_ids.append(repo_id)
                continue
            candidates.append((repo_id, manifest, hub_tags))

    in_tree_names, installed_ids = _local_markers()

    hits: list[HubRSkillHit] = []
    for repo_id, manifest, hub_tags in candidates:
        if not _matches_facets(
            manifest, kind=kind, role=role, embodiment=embodiment, license_=license, family=family
        ):
            continue
        haystack = _manifest_haystack(repo_id, manifest, hub_tags)
        if not _matches_query(query, haystack):
            continue
        local: Literal["in-tree", "installed"] | None
        if repo_id in in_tree_names:
            local = "in-tree"
        elif repo_id in installed_ids:
            local = "installed"
        else:
            local = None
        hits.append(
            HubRSkillHit(repo_id=repo_id, manifest=manifest, local=local, hub_tags=hub_tags)
        )

    hits.sort(key=lambda h: h.repo_id)
    if limit is not None:
        hits = hits[:limit]

    sorted_skipped = tuple(sorted(skipped_repo_ids))
    return HubRSkillSearchResult(
        hits=hits,
        skipped=len(sorted_skipped),
        skipped_repo_ids=sorted_skipped,
        inspected=inspected,
    )

validate_skill_ref(raw: str) -> str

Validate and return a bare rSkill reference unchanged.

Accepts any non-empty string that does not carry an explicit URI scheme: a bare name ("smolvla-libero"), a path ("rskills/smolvla-libero"), or an HF repo id ("OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16"). Rejects inputs that carry a known scheme (hf://, local://, file://, etc.) so callers are never silently handed a non-rSkill URI.

Parameters:

Name	Type	Description	Default
raw	str	The user-supplied reference. Must be non-empty and carry no explicit URI scheme.	required

Returns:

Type	Description
str	The trimmed reference string unchanged.

Raises:

Type	Description
ROSConfigError	If raw is empty / whitespace-only or carries a known URI scheme.

Example

validate_skill_ref("smolvla-libero") 'smolvla-libero' validate_skill_ref("rskills/smolvla-libero") 'rskills/smolvla-libero'

Source code in python/rskill/src/openral_rskill/loader.py

def validate_skill_ref(raw: str) -> str:
    """Validate and return a bare rSkill reference unchanged.

    Accepts any non-empty string that does not carry an explicit URI scheme:
    a bare name (``"smolvla-libero"``), a path (``"rskills/smolvla-libero"``),
    or an HF repo id (``"OpenRAL/rskill-smolvla-franka_panda-libero_spatial-bf16"``). Rejects inputs
    that carry a known scheme (``hf://``, ``local://``, ``file://``, etc.)
    so callers are never silently handed a non-rSkill URI.

    Args:
        raw: The user-supplied reference. Must be non-empty and carry no
            explicit URI scheme.

    Returns:
        The trimmed reference string unchanged.

    Raises:
        ROSConfigError: If ``raw`` is empty / whitespace-only or carries
            a known URI scheme.

    Example:
        >>> validate_skill_ref("smolvla-libero")
        'smolvla-libero'
        >>> validate_skill_ref("rskills/smolvla-libero")
        'rskills/smolvla-libero'
    """
    cleaned = raw.strip()
    if not cleaned:
        raise ROSConfigError("rskill reference must be a non-empty string")
    for bad in ("hf://", "local://", "file://", "http://", "https://"):
        if cleaned.startswith(bad):
            raise ROSConfigError(
                f"rskill reference {raw!r} carries a {bad!r} scheme; pass a "
                "bare rSkill name or path (rskills/<name>) instead. "
                "The sim layer rejects raw URI schemes by design — weights "
                "must come from an rSkill manifest."
            )
    return cleaned

Eval (openral sim run)

openral sim runner — swappable (robot x scene x task x VLA) for rSkill validation.

Typical usage::

from openral_sim import SimRunner

# ``SimRunner`` consumes the runtime-composed ``SimEnvironment``
# (scene + task + VLA). The on-disk YAML is a ``SimScene`` (scene +
# task, no VLA); the CLI composes the ``SimEnvironment`` from a
# ``SimScene`` + an rSkill manifest. See ``openral sim run`` and
# ``openral_sim.cli._load_or_build_env`` for the canonical
# compose path.
env_cfg: SimEnvironment = ...  # composed by the CLI
runner = SimRunner(env_cfg)
runner.activate()
runner.run(max_ticks=env_cfg.n_episodes * ((env_cfg.task.max_steps or 1000) + 1))
for episode in runner.episode_results:
    print(episode.success, episode.steps, episode.mean_step_latency_ms)
runner.deactivate()

CLI::

openral sim run --config scenes/sim/libero_spatial.yaml \
        --rskill smolvla-libero
openral sim run --config scenes/sim/libero_spatial.yaml \
        --task libero_spatial/0 \
        --rskill smolvla-libero
openral benchmark run --suite libero_spatial \
        --rskill smolvla-libero

The sim package itself depends only on openral-core, openral-runner, and openral-rskill. Physics backends (LIBERO, MetaWorld, ...) are imported lazily by the registered adapters so installing this package never pulls heavyweight ML deps.

EpisodeResult(success: bool = False, steps: int = 0, total_reward: float = 0.0, max_step_reward: float = 0.0, mean_step_latency_ms: float = 0.0, max_step_latency_ms: float = 0.0, latency_budget_ms: float | None = None, budget_violations: int = 0, frames: list[NDArray[np.uint8]] = list(), vla_input_frames: list[NDArray[np.uint8]] = list(), joint_positions: list[NDArray[np.float32]] = list(), actions: list[NDArray[np.float32]] = list(), num_input_cameras: int = 1, metadata: dict[str, Any] = dict(), contact_stats: list[dict[str, Any]] = list()) dataclass

Outcome of one episode rollout.

Attributes:

Name	Type	Description
success	bool	Whether the task was completed (driven by task.success_key).
steps	int	Number of env.step() calls executed.
total_reward	float	Sum of per-step rewards.
max_step_reward	float	Maximum per-step reward observed during the episode. Used as the paper-faithful metric on scenes where the per-step reward is a "best-so-far" continuous signal (PushT reports coverage IoU between the T-block and its goal pose; the Diffusion Policy paper averages the per-rollout max across seeds).
mean_step_latency_ms	float	Mean policy step() latency in milliseconds.
max_step_latency_ms	float	Max policy step() latency in milliseconds.
latency_budget_ms	float | None	Latency budget from the rSkill manifest, if known.
budget_violations	int	Number of steps exceeding the latency budget.
frames	list[NDArray[uint8]]	Captured RGB frames (HWC uint8) when record_video is True; empty otherwise.
metadata	dict[str, Any]	Free-form per-run metadata propagated from openral_core.SimEnvironment.metadata.

summary() -> str

Human-readable single-line summary.

Returns:

Type	Description
str	A compact summary string suitable for CLI output.

Source code in python/sim/src/openral_sim/rollout.py

def summary(self) -> str:
    """Human-readable single-line summary.

    Returns:
        A compact summary string suitable for CLI output.
    """
    return (
        f"success={self.success} steps={self.steps} reward={self.total_reward:.3f} "
        f"mean_lat={self.mean_step_latency_ms:.1f}ms"
        f" budget_viol={self.budget_violations}"
    )

PolicyAdapter

Bases: Protocol

Uniform VLA / policy interface used by the runner.

Attributes:

Name	Type	Description
spec	VLASpec	The openral_core.VLASpec this adapter was built for.
device	str	Resolved torch / numpy device string ("cuda:0", "cpu").

close() -> None

Release any GPU memory / file handles.

Source code in python/sim/src/openral_sim/policy.py

def close(self) -> None:
    """Release any GPU memory / file handles."""

reset() -> None

Reset internal state (action queue, RNG) at the start of each episode.

Source code in python/sim/src/openral_sim/policy.py

def reset(self) -> None:
    """Reset internal state (action queue, RNG) at the start of each episode."""

step(observation: Observation, instruction: str) -> NDArray[np.float32]

Produce the next action vector for the given observation.

Parameters:

Name	Type	Description	Default
observation	Observation	Adapter-specific observation dict produced by the env.	required
instruction	str	Natural-language task string from openral_core.TaskSpec.instruction.	required

Returns:

Type	Description
NDArray[float32]	1-D float32 NumPy array of length action_dim. Adapters must
NDArray[float32]	return a flat per-step action even if their underlying VLA emits
NDArray[float32]	chunks — chunk caching belongs in the adapter.

Source code in python/sim/src/openral_sim/policy.py

def step(
    self,
    observation: Observation,
    instruction: str,
) -> NDArray[np.float32]:
    """Produce the next action vector for the given observation.

    Args:
        observation: Adapter-specific observation dict produced by the env.
        instruction: Natural-language task string from
            ``openral_core.TaskSpec.instruction``.

    Returns:
        1-D float32 NumPy array of length ``action_dim``. Adapters must
        return a flat per-step action even if their underlying VLA emits
        chunks — chunk caching belongs in the adapter.
    """

SimRollout

Bases: Protocol

Minimal gym-style env contract used by the eval runner.

Every scene factory in openral_sim.SCENES must return an object satisfying this protocol.

Attributes:

Name	Type	Description
scene	SceneSpec	Scene spec the rollout was built from.
task	TaskSpec	Task spec it is currently solving.

Optional duck-typed extensions (any of, mutually exclusive per adapter): mujoco_handles(self) -> tuple[mujoco.MjModel, mujoco.MjData] | None — MuJoCo-backed adapters expose this so openral sim run --view can open a passive mujoco.viewer window against the adapter's own MjModel / MjData.

``sim_time_ns(self) -> int | None``
    — the backend's authoritative elapsed
    simulation time in nanoseconds — the seam a sim ``/clock``
    publisher reads so the deploy-sim ROS graph runs on simulation
    time rather than wall time. Contract:

    * MuJoCo-backed adapters return ``round(MjData.time * 1e9)`` (the
      physics clock advanced by ``model.opt.timestep`` per step) via
      ``sim_time_ns_from_mujoco_handles``.
    * SAPIEN / ManiSkill-backed adapters derive elapsed control time
      from the live env's elapsed step counter and control timestep.
    * The value is **monotonic non-decreasing within a single
      episode**. It is NOT guaranteed monotonic across ``reset`` —
      backends that rewind ``MjData.time`` to ``0`` on reset (e.g.
      robocasa) restart their clock, so a consumer that needs
      cross-reset monotonicity maintains its own offset (see
      ``openral_hal.sim_attached.SimAttachedHAL.sim_time_ns``).
    * Returns ``None`` when the backend has **no sim clock** —
      clock-less backends (PushT) and sidecars whose wire protocol does
      not carry elapsed time.

    Callers MUST use ``getattr(env, "sim_time_ns", None)`` and treat
    both a missing attribute and a ``None`` return as "no sim clock"
    (fall back to wall time).

``task_success(self) -> bool | None``
    — the backend's OWN task-success predicate, read on demand
    and independent of ``step``. Robosuite/RoboCasa envs
    implement ``_check_success()`` per task ("is the cup in the
    sink"); the adapter forwards it. Contract:

    * ``True`` / ``False`` — the env's live verdict at this
      instant. It is NOT latched: a task that succeeds and is
      then undone reads ``False`` again.
    * ``None`` — this backend has no task-success predicate.
      A caller must never read ``None`` as failure.
    * Reading it is side-effect free: it must not step physics,
      latch a terminal, or influence ``StepResult``.

    The ``sim run`` / benchmark path does NOT use this — it reads
    ``StepResult.info[task.success_key]`` (see ``StepResult``).
    This extension exists for ``deploy sim``, whose continuous
    twin suppresses per-step task evaluation entirely, so the
    HAL polls the predicate for observability
    (``openral_hal.sim_attached.SimAttachedHAL.task_success``).

``viewer_render(self) -> None``
    — Adapters whose underlying engine owns the live viewer
    and needs per-step pumping (e.g. SAPIEN / ManiSkill3 envs
    constructed with ``render_mode='human'``). ``SimRunner``
    calls this once per applied step to pump the viewer; the
    adapter is responsible for promoting / opening the window
    on the first call and tearing it down in ``close``.

``enable_intrinsic_viewer(self) -> None``
    — Adapters whose engine draws its own self-managed window
    and does not need per-step pumping (e.g. ``gym_pusht`` with
    ``render_mode="human"``; the engine updates the window inside
    its own ``step`` / ``reset``). ``SimRunner`` calls this
    ONCE before the first ``reset()`` when ``--view`` is set, then
    skips both the MuJoCo viewer path AND the per-step
    ``viewer_render`` pump.

None of the above is part of the Protocol, so adapters need not
stub a method they cannot honour; callers MUST use
``getattr(env, "<name>", None)`` and treat both a missing attribute
and a ``None`` return as "unsupported".

close() -> None

Release any underlying engine resources.

Source code in python/sim/src/openral_sim/rollout.py

def close(self) -> None:
    """Release any underlying engine resources."""

render() -> NDArray[np.uint8] | None

Return an HWC uint8 RGB frame, or None when rendering is unavailable.

Source code in python/sim/src/openral_sim/rollout.py

def render(self) -> NDArray[np.uint8] | None:
    """Return an HWC uint8 RGB frame, or ``None`` when rendering is unavailable."""

reset(seed: int | None = ...) -> Observation

Reset the simulator and return the initial observation.

Source code in python/sim/src/openral_sim/rollout.py

def reset(self, seed: int | None = ...) -> Observation:
    """Reset the simulator and return the initial observation."""

step(action: NDArray[np.float32]) -> StepResult

Apply action for one timestep and return the transition.

Source code in python/sim/src/openral_sim/rollout.py

def step(self, action: NDArray[np.float32]) -> StepResult:
    """Apply ``action`` for one timestep and return the transition."""

SimRunner(env_cfg: SimEnvironment, *, view: bool = False, strict_view: bool = False, instruction_override: str | None = None, deadline_overrun_policy: DeadlineOverrunPolicy = DeadlineOverrunPolicy.WARN, recorder: RolloutRecorder | None = None)

Bases: InferenceRunnerBase

One-tick = one-env-step InferenceRunner for sim rollouts.

Drives a single (robot × scene × task × VLA) SimEnvironment for env_cfg.n_episodes episodes. Each call to tick advances by one tick of one of two flavours:

• Reset tick — emitted at the start of each episode (and once more after a terminated / truncated step). Calls env.reset and policy.reset, finalises the previous EpisodeResult if any, and returns a TickResult with action_applied=False and inference_ms == 0.0.
• Step tick — runs policy.step + env.step. Records latency, reward, terminated / truncated, and (when env_cfg.record_video is set) the rendered frame, VLA input frame, joint state, and action. When the env signals termination the next tick will be a reset tick.

_should_terminate returns True once n_episodes EpisodeResults have been emitted, so callers can pass an upper-bound max_ticks (typically n_episodes * (max_steps + 1)) and rely on the hook for the real stop condition.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated openral_core.SimEnvironment.	required
view	bool	Open a passive mujoco.viewer window during the rollout. Only valid for MuJoCo-backed scenes that expose mujoco_handles().	False
strict_view	bool	When view is True and the env adapter does not expose mujoco_handles(), raise ROSConfigError instead of warning and continuing offscreen.	False
deadline_overrun_policy	DeadlineOverrunPolicy	Forwarded to the base class. Defaults to WARN since sim rollouts are not real-time.	WARN

Attributes:

Name	Type	Description
episode_results	list[EpisodeResult]	One EpisodeResult per completed episode, populated as ticks complete.
manifest	RSkillManifest | None	The validated rSkill manifest, or None for mock policies that carry no weights.

Build the runner; defer env / policy construction to activate.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated openral_core.SimEnvironment.	required
view	bool	Open a passive mujoco.viewer window during the rollout.	False
strict_view	bool	Raise on missing viewer handles instead of warning.	False
instruction_override	str | None	An explicit --instruction CLI value that must win over a scene's per-episode obs["task"] language (for example, RoboCasa sampled object). None when the user passed nothing — the env/YAML language then takes over. See _resolve_step_instruction.	None
deadline_overrun_policy	DeadlineOverrunPolicy	Forwarded to the base class.	WARN
recorder	RolloutRecorder | None	Optional openral_dataset.RolloutRecorder. When set, per-step state / images / action plus episode boundaries are fanned out to the recorder's sinks in addition to the existing _EpisodeBuffer. The buffer drives the in-memory video / json / benchmark pipeline; the recorder drives the durable LeRobotDataset v3 path. The two are additive — the recorder is never a substitute for the buffer.	None

Source code in python/sim/src/openral_sim/sim_runner.py

def __init__(
    self,
    env_cfg: SimEnvironment,
    *,
    view: bool = False,
    strict_view: bool = False,
    instruction_override: str | None = None,
    deadline_overrun_policy: DeadlineOverrunPolicy = DeadlineOverrunPolicy.WARN,
    recorder: RolloutRecorder | None = None,
) -> None:
    """Build the runner; defer env / policy construction to ``activate``.

    Args:
        env_cfg: Validated ``openral_core.SimEnvironment``.
        view: Open a passive ``mujoco.viewer`` window during the rollout.
        strict_view: Raise on missing viewer handles instead of warning.
        instruction_override: An explicit ``--instruction`` CLI value that
            must win over a scene's per-episode ``obs["task"]`` language
            (for example, RoboCasa sampled object).
            ``None`` when the user passed nothing — the env/YAML language
            then takes over. See ``_resolve_step_instruction``.
        deadline_overrun_policy: Forwarded to the base class.
        recorder: Optional ``openral_dataset.RolloutRecorder``.
            When set, per-step state / images / action plus episode
            boundaries are fanned out to the recorder's sinks in
            addition to the existing ``_EpisodeBuffer``. The
            buffer drives the in-memory video / json / benchmark
            pipeline; the recorder drives the durable LeRobotDataset
            v3 path. The two are additive — the recorder is never
            a substitute for the buffer.
    """
    # Sim rollouts run as fast as policy + env permit; rate_hz only
    # parameterises the base class' deadline machinery. 1000 Hz keeps
    # the period at 1 ms so a single env.step never naturally
    # over-runs the deadline (deadline policy is WARN by default
    # anyway).
    super().__init__(
        rate_hz=1000.0,
        deadline_overrun_policy=deadline_overrun_policy,
        runner_name=f"sim_runner:{env_cfg.scene.id}:{env_cfg.task.id}",
        save_dir=env_cfg.save_dir,
    )
    self._env_cfg = env_cfg
    self._view = view
    self._strict_view = strict_view
    self._instruction_override = instruction_override
    self._recorder = recorder

    # Lifecycle state. Initialised properly in activate() so the runner
    # can be re-used across runs.
    self.episode_results = []
    self.manifest = None
    self._env: SimRollout | None = None
    self._policy: PolicyAdapter | None = None
    self._obs: Observation = {}
    # Scene camera name -> VLA slot, applied to every obs (see `_policy_scene_cameras`).
    self._image_rekey: dict[str, str] = {}
    self._viewer: Any = None
    # True when the scene adapter draws its own window inside env.step /
    # env.reset (e.g. gym_pusht with render_mode="human"). Suppresses
    # the lazy mujoco-viewer-open path in _reset_tick.
    self._intrinsic_viewer: bool = False
    self._frame_dt_s: float | None = None
    self._episode_idx = 0
    self._step_idx = 0
    self._needs_reset = True
    self._buf = _EpisodeBuffer()
    self._budget_ms: float | None = None
    self._run_span_ctx: Any = None
    # Tracks whether the current episode is currently "open" on the
    # recorder. The recorder requires an explicit episode_start before
    # record_frame and episode_end at the boundary; we mirror the
    # _EpisodeBuffer.has_data state here.
    self._recorder_episode_open: bool = False

activate() -> None

Validate the manifest, build env + policy, and arm the first reset tick.

make_env (MuJoCo XML compile / dataset prefetch — ~30–60 s on LIBERO / RoboCasa) and make_policy (PaliGemma 3.4B graph allocation + NF4 quantization — ~100–150 s on π0.5) are independent: both only read the immutable SimEnvironment and share no mutable state. By default they run concurrently on a 2-worker ThreadPoolExecutor (the GIL is released by MuJoCo C calls and the torch / safetensors load path), so the wall-clock for activate collapses to max(env_ms, policy_ms) instead of their sum. See GH-134.

Set OPENRAL_SIM_SEQUENTIAL_INIT=1 to force the legacy sequential path (debugging, profiling, or when interleaved logs would obscure a root cause).

Exceptions from either side propagate verbatim — no silent retry, no swallowed traceback. If the policy build raises while the env build is still running, the env future is left to finish on the worker thread (cancellation of a started future is a no-op in Python's executor model); deactivate is responsible for closing it if activate raised after a partial assignment.

Source code in python/sim/src/openral_sim/sim_runner.py

def activate(self) -> None:
    """Validate the manifest, build env + policy, and arm the first reset tick.

    ``make_env`` (MuJoCo XML compile / dataset prefetch — ~30–60 s on
    LIBERO / RoboCasa) and ``make_policy`` (PaliGemma 3.4B graph
    allocation + NF4 quantization — ~100–150 s on π0.5) are independent:
    both only read the immutable ``SimEnvironment`` and share no
    mutable state. By default they run concurrently on a 2-worker
    ``ThreadPoolExecutor`` (the GIL is released by MuJoCo C calls and
    the torch / safetensors load path), so the wall-clock for
    ``activate`` collapses to ``max(env_ms, policy_ms)`` instead
    of their sum. See GH-134.

    Set ``OPENRAL_SIM_SEQUENTIAL_INIT=1`` to force the legacy
    sequential path (debugging, profiling, or when interleaved logs
    would obscure a root cause).

    Exceptions from either side propagate verbatim — no silent retry,
    no swallowed traceback. If the policy build raises while the env
    build is still running, the env future is left to finish on the
    worker thread (cancellation of a started future is a no-op in
    Python's executor model); ``deactivate`` is responsible for
    closing it if ``activate`` raised after a partial assignment.
    """
    # Validate before the (expensive) env / policy build so misconfigs
    # fail fast without burning GPU time.
    self.manifest, self._env_cfg = _check_rskill_compatibility(self._env_cfg)
    # SAPIEN-backed scenes (simpler-env/ManiSkill3) need `render_mode`
    # ='human' at gym.make() time to build a live viewer instead of an
    # offscreen target; the SCENES factory only sees `env_cfg`, so the
    # flag is published via an env var scoped to the build window.
    # MuJoCo adapters ignore it (viewer opens lazily on first reset).
    prev_view_env = os.environ.get(_VIEW_ENV)
    if self._view:
        os.environ[_VIEW_ENV] = "1"
    # The policy sees VLA slots, like `deploy sim` / `deploy run`; the env
    # keeps rendering under the scene's own camera names and each obs is
    # rekeyed to slots in `_reset_tick` / `_step_tick`.
    policy_cameras, self._image_rekey = _policy_scene_cameras(
        list(self._env_cfg.scene.cameras), make_robot(self._env_cfg), self.manifest
    )
    policy_cfg = self._env_cfg
    if policy_cameras != list(self._env_cfg.scene.cameras):
        _log.info(
            "sim_policy_cameras_slotted",
            scene_cameras=list(self._env_cfg.scene.cameras),
            policy_cameras=policy_cameras,
        )
        policy_cfg = self._env_cfg.model_copy(
            update={"scene": self._env_cfg.scene.model_copy(update={"cameras": policy_cameras})}
        )
    try:
        self._env, self._policy = _build_env_and_policy(self._env_cfg, policy_cfg)
    finally:
        if prev_view_env is None:
            os.environ.pop(_VIEW_ENV, None)
        else:
            os.environ[_VIEW_ENV] = prev_view_env
    self._viewer = None
    self._intrinsic_viewer = False
    self._frame_dt_s = None

    # Scenes whose engine draws its own window (gym_pusht via
    # render_mode="human") expose enable_intrinsic_viewer(); for
    # those the mujoco-viewer path in _reset_tick is bypassed.
    if self._view:
        enable_fn = getattr(self._env, "enable_intrinsic_viewer", None)
        if callable(enable_fn):
            enable_fn()
            self._intrinsic_viewer = True
    self._episode_idx = 0
    self._step_idx = 0
    self._needs_reset = True
    self._buf = _EpisodeBuffer()
    self.episode_results = []
    self._budget_ms = (
        self.manifest.latency_budget.per_chunk_ms if self.manifest is not None else None
    )

    # Held open until deactivate() so child ``rskill.tick`` spans from
    # the base class nest under it.
    self._run_span_ctx = _tracer().start_as_current_span(
        "sim.run",
        attributes={
            "robot.id": self._env_cfg.robot_id,
            "scene.id": self._env_cfg.scene.id,
            "task.id": self._env_cfg.task.id,
            "vla.id": self._env_cfg.vla.id,
            "vla.weights_uri": self._env_cfg.vla.weights_uri,
            "n_episodes": self._env_cfg.n_episodes,
            "seed": self._env_cfg.seed,
        },
    )
    self._run_span_ctx.__enter__()

    super().activate()

deactivate() -> None

Finalise any trailing episode, close env + policy, idempotently.

Source code in python/sim/src/openral_sim/sim_runner.py

def deactivate(self) -> None:
    """Finalise any trailing episode, close env + policy, idempotently."""
    # Flush partial episode (e.g. when max_ticks cut us short).
    if self._buf.has_data:
        self._finalize_episode()
    # If an episode opened on the recorder never reached
    # _finalize_episode (e.g. activate() succeeded but no ticks ran),
    # close it as a failure before finalising the recorder.
    if self._recorder is not None:
        if self._recorder_episode_open:
            with contextlib.suppress(ValueError, RuntimeError):
                self._recorder.episode_end(success=False)
            self._recorder_episode_open = False
        try:
            self._recorder.finalize()
        except (ValueError, RuntimeError) as exc:
            _log.warning(
                "rollout_recorder_finalize_failed",
                error=str(exc),
                error_type=type(exc).__name__,
            )
    if self._viewer is not None:
        self._viewer.close()
        self._viewer = None
    if self._policy is not None:
        self._policy.close()
        self._policy = None
    if self._env is not None:
        self._env.close()
        self._env = None
    if self._run_span_ctx is not None:
        self._run_span_ctx.__exit__(None, None, None)
        self._run_span_ctx = None
    super().deactivate()

episode_end(*, success: bool) -> None

No-op on SimRunner: sim closes episodes inside _finalize_episode.

Argument intentionally unused — sim's success flag comes from step_result.info[task.success_key], not from the caller.

Source code in python/sim/src/openral_sim/sim_runner.py

def episode_end(self, *, success: bool) -> None:
    """No-op on SimRunner: sim closes episodes inside `_finalize_episode`.

    Argument intentionally unused — sim's success flag comes from
    ``step_result.info[task.success_key]``, not from the caller.
    """
    _ = success

episode_start(task_string: str) -> int

No-op on SimRunner: sim derives episodes from env signals.

Returns the current _episode_idx so callers expecting an integer don't crash. The recorder is opened by _reset_tick, not by this method.

Source code in python/sim/src/openral_sim/sim_runner.py

def episode_start(self, task_string: str) -> int:
    """No-op on SimRunner: sim derives episodes from env signals.

    Returns the current ``_episode_idx`` so callers expecting an
    integer don't crash. The recorder is opened by `_reset_tick`,
    not by this method.
    """
    return self._episode_idx

default_output_path(weights_uri: str, benchmark_id: str) -> str

Compute the canonical eval-JSON path from a bare rSkill reference.

A weights ref like rskills/smolvla-libero resolves to rskills/smolvla-libero/eval/<benchmark_id>.json. The caller is responsible for mkdir -p of the parent directory and for actually writing the JSON (typically via RSkillEvalResult.model_dump_json).

Parameters:

Name	Type	Description	Default
weights_uri	str	The bare rSkill reference from VLASpec.weights_uri.	required
benchmark_id	str	The benchmark suite id (the YAML filename stem) — used as the JSON filename stem so re-running the same suite overwrites the same eval JSON.	required

Returns:

Type	Description
str	A relative path string in the form <skill_dir>/eval/<id>.json.

Raises:

Type	Description
ValueError	If weights_uri starts with hf:// (cannot write to the HF Hub; only locally-resolvable rSkills are writable).

Source code in python/sim/src/openral_sim/benchmark.py

def default_output_path(weights_uri: str, benchmark_id: str) -> str:
    """Compute the canonical eval-JSON path from a bare rSkill reference.

    A weights ref like ``rskills/smolvla-libero`` resolves to
    ``rskills/smolvla-libero/eval/<benchmark_id>.json``. The caller is
    responsible for ``mkdir -p`` of the parent directory and for actually
    writing the JSON (typically via ``RSkillEvalResult.model_dump_json``).

    Args:
        weights_uri: The bare rSkill reference from ``VLASpec.weights_uri``.
        benchmark_id: The benchmark suite id (the YAML filename stem) — used
            as the JSON filename stem so re-running the same suite
            overwrites the same eval JSON.

    Returns:
        A relative path string in the form ``<skill_dir>/eval/<id>.json``.

    Raises:
        ValueError: If ``weights_uri`` starts with ``hf://`` (cannot write
            to the HF Hub; only locally-resolvable rSkills are writable).
    """
    if weights_uri.startswith("hf://"):
        raise ValueError(
            f"default_output_path expects a bare rSkill ref (e.g. rskills/<name>); "
            f"got {weights_uri!r}. The benchmark runner only writes JSONs for "
            "locally-resolvable rSkills."
        )
    return f"{weights_uri}/eval/{benchmark_id}.json"

make_env(env_cfg: SimEnvironment) -> SimRollout

Build the simulated environment described by env_cfg.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated openral_core.SimEnvironment config.	required

Returns:

Type	Description
SimRollout	A openral_sim.rollout.SimRollout ready for
SimRollout	reset / step.

Raises:

Type	Description
ROSConfigError	If env_cfg.scene.id is not registered, or its backend_options fail the backend's declared options model.

Source code in python/sim/src/openral_sim/factory.py

def make_env(env_cfg: SimEnvironment) -> SimRollout:
    """Build the simulated environment described by ``env_cfg``.

    Args:
        env_cfg: Validated ``openral_core.SimEnvironment`` config.

    Returns:
        A ``openral_sim.rollout.SimRollout`` ready for
        ``reset`` / ``step``.

    Raises:
        openral_core.exceptions.ROSConfigError: If
            ``env_cfg.scene.id`` is not registered, or its
            ``backend_options`` fail the backend's declared options model.
    """
    factory = SCENES.get(env_cfg.scene.id)
    SCENES.validate_options(env_cfg.scene.id, env_cfg.scene.backend_options)
    return factory(env_cfg)

make_policy(env_cfg: SimEnvironment) -> PolicyAdapter

Build the VLA / policy adapter described by env_cfg.vla.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated openral_core.SimEnvironment config.	required

Returns:

Type	Description
PolicyAdapter	A openral_sim.policy.PolicyAdapter ready for
PolicyAdapter	reset / step.

Raises:

Type	Description
ROSConfigError	If env_cfg.vla.id is not registered.

Source code in python/sim/src/openral_sim/factory.py

def make_policy(env_cfg: SimEnvironment) -> PolicyAdapter:
    """Build the VLA / policy adapter described by ``env_cfg.vla``.

    Args:
        env_cfg: Validated ``openral_core.SimEnvironment`` config.

    Returns:
        A ``openral_sim.policy.PolicyAdapter`` ready for
        ``reset`` / ``step``.

    Raises:
        openral_core.exceptions.ROSConfigError: If
            ``env_cfg.vla.id`` is not registered.
    """
    factory = POLICIES.get(env_cfg.vla.id)
    return factory(env_cfg)

run_benchmark(scenes: list[BenchmarkScene], *, suite_id: str, vla: VLASpec, device: str | None = None, save_dir: str | None = None, video_dir: str | None = None) -> tuple[RSkillEvalResult, list[EpisodeResult]]

Run a benchmark suite end-to-end against one rSkill.

A benchmark suite is a bare list[BenchmarkScene] plus a suite_id (typically Path("benchmarks/<id>.yaml").stem). Callers that load from disk should use openral_core.load_benchmark_suite and openral_core.raise_on_invalid_suite before calling this function; the runner does not re-validate suite invariants.

Parameters:

Name	Type	Description	Default
scenes	list[BenchmarkScene]	The list of BenchmarkScenes to evaluate. Each entry carries its own scene, task, robot, episode count, and seed offset. Suite-level invariants (uniformity of robot_id / n_episodes / seed / metadata, unique task ids, non-empty) are assumed pre-validated.	required
suite_id	str	Stable suite identifier (the YAML filename stem). Used in log lines and embedded into reproduction_cli.	required
vla	VLASpec	The single free axis — which rSkill to evaluate. Its weights_uri must be a bare rSkill reference (name, path, or HF repo ID); the strict runner rejects raw hf:// URIs.	required
device	str | None	Optional torch device override applied to every rollout ("cpu", "cuda:0", "mps", "auto"). None keeps the manifest's preferred device.	None
save_dir	str | None	Optional directory written to each SimEnvironment for adapter-side artefacts (videos, traces). Unrelated to where the final RSkillEvalResult JSON lives — that path is chosen by the caller (see default_output_path).	None
video_dir	str | None	When set, record per-step world frames and write one MP4 per episode into this directory via openral_sim._website_video.write_world_videos (named <task>[_seed<n>]_<rskill>_<success|fail>.mp4 plus a merged videos.json). Frames are written and freed after each episode so multi-hundred-episode suites don't accumulate gigabytes of RGB in memory. None disables frame capture.	None

Returns:

Type	Description
RSkillEvalResult	A pair (result, episodes) where result is a validated
list[EpisodeResult]	RSkillEvalResult ready to be written to
tuple[RSkillEvalResult, list[EpisodeResult]]	rskills/<vla>/eval/<suite_id>.json and episodes is the
tuple[RSkillEvalResult, list[EpisodeResult]]	flat per-(task, seed) list of EpisodeResult objects for
tuple[RSkillEvalResult, list[EpisodeResult]]	callers that want fine-grained data (e.g. unit tests asserting on
tuple[RSkillEvalResult, list[EpisodeResult]]	latency).

Raises:

Type	Description
ROSConfigError	Any error propagated from SimRunner — typically a missing rSkill manifest, an incompatible robot, or an unresolvable rSkill reference. The runner does not catch them; the whole suite fails so partial JSONs never reach disk.

Source code in python/sim/src/openral_sim/benchmark.py

def run_benchmark(
    scenes: list[BenchmarkScene],
    *,
    suite_id: str,
    vla: VLASpec,
    device: str | None = None,
    save_dir: str | None = None,
    video_dir: str | None = None,
) -> tuple[RSkillEvalResult, list[EpisodeResult]]:
    """Run a benchmark suite end-to-end against one rSkill.

    A benchmark suite is a bare ``list[BenchmarkScene]`` plus a
    ``suite_id`` (typically ``Path("benchmarks/<id>.yaml").stem``). Callers
    that load from disk should use ``openral_core.load_benchmark_suite``
    and ``openral_core.raise_on_invalid_suite`` before calling this
    function; the runner does not re-validate suite invariants.

    Args:
        scenes: The list of ``BenchmarkScene``s to evaluate. Each
            entry carries its own scene, task, robot, episode count, and
            seed offset. Suite-level invariants (uniformity of robot_id /
            n_episodes / seed / metadata, unique task ids, non-empty) are
            assumed pre-validated.
        suite_id: Stable suite identifier (the YAML filename stem). Used
            in log lines and embedded into ``reproduction_cli``.
        vla: The single free axis — which rSkill to evaluate. Its
            ``weights_uri`` must be a bare rSkill reference (name, path, or
            HF repo ID); the strict runner rejects raw ``hf://`` URIs.
        device: Optional torch device override applied to every rollout
            (``"cpu"``, ``"cuda:0"``, ``"mps"``, ``"auto"``). ``None``
            keeps the manifest's preferred device.
        save_dir: Optional directory written to each ``SimEnvironment``
            for adapter-side artefacts (videos, traces). Unrelated to where
            the final ``RSkillEvalResult`` JSON lives — that path is
            chosen by the caller (see ``default_output_path``).
        video_dir: When set, record per-step world frames and write one MP4
            per episode into this directory via
            ``openral_sim._website_video.write_world_videos``
            (named ``<task>[_seed<n>]_<rskill>_<success|fail>.mp4`` plus a
            merged ``videos.json``). Frames are written and freed after each
            episode so multi-hundred-episode suites don't accumulate
            gigabytes of RGB in memory. ``None`` disables frame capture.

    Returns:
        A pair ``(result, episodes)`` where ``result`` is a validated
        ``RSkillEvalResult`` ready to be written to
        ``rskills/<vla>/eval/<suite_id>.json`` and ``episodes`` is the
        flat per-(task, seed) list of ``EpisodeResult`` objects for
        callers that want fine-grained data (e.g. unit tests asserting on
        latency).

    Raises:
        openral_core.exceptions.ROSConfigError: Any error propagated
            from ``SimRunner`` — typically a missing rSkill
            manifest, an incompatible robot, or an unresolvable rSkill
            reference. The runner does not catch them; the whole suite
            fails so partial JSONs never reach disk.
    """
    from openral_core import SimEnvironment

    from openral_sim.registry import SCENES
    from openral_sim.sim_runner import SimRunner

    # Suite auto-filter: keep only the tasks the rSkill declares it
    # was trained/validated for, skip the rest with a logged summary. Unlike
    # run_benchmark_scene the suite path never gated tasks, so a task-mismatched
    # but same-embodiment rSkill used to run and silently score 0; now it is
    # skipped. Undeclared / mock / hf:// skills are unfilterable (keep all).
    manifest = _manifest_for_filter(vla)
    declared = list(manifest.evaluated_tasks) if manifest is not None else []
    kept, skipped = filter_scenes_for_skill(scenes, manifest)
    if skipped:
        _log.info(
            "benchmark_suite_task_filter",
            suite=suite_id,
            skill=vla.id,
            ran=len(kept),
            skipped=len(skipped),
            skipped_tasks=[s.task.id for s in skipped],
            declared=declared,
        )
    if not kept:
        from openral_core.exceptions import ROSCapabilityMismatch

        raise ROSCapabilityMismatch(
            f"rSkill {vla.id!r} declares evaluated_tasks={declared}, which match none "
            f"of the {len(scenes)} task(s) in suite {suite_id!r} "
            f"(e.g. {[s.task.id for s in scenes][:3]}). Nothing to run — pair the suite "
            "with an rSkill trained for its tasks."
        )
    scenes = kept

    # Suite invariants (see openral_core.raise_on_invalid_suite) guarantee
    # every BenchmarkScene shares robot_id / n_episodes / seed / metadata.
    # Read from scenes[0] for the seed sweep; pull per-scene robot_id /
    # task / max_steps from each scene inside the loop.
    first = scenes[0]
    seeds = list(range(first.seed, first.seed + first.n_episodes))

    per_task: dict[str, list[bool]] = {}
    all_episodes: list[EpisodeResult] = []

    for scene in scenes:
        task_id = scene.task.id
        # SCENES.resolve_robot is the shared binding rule (refuses a robot the
        # scene can't build). BenchmarkScene._require_task_eval_fields asserts
        # task.max_steps is set; re-narrow for mypy.
        robot_id = SCENES.resolve_robot(scene.scene.id, scene.robot_id)
        max_steps = scene.task.max_steps
        assert max_steps is not None
        per_task[task_id] = []
        for seed in seeds:
            vla_for_episode = vla.model_copy(update={"device": device}) if device else vla
            env_cfg = SimEnvironment(
                robot_id=robot_id,
                scene=scene.scene,
                task=scene.task,
                vla=vla_for_episode,
                base_pose=scene.base_pose,
                seed=seed,
                n_episodes=1,
                save_dir=save_dir,
                record_video=video_dir is not None,
            )
            runner = SimRunner(env_cfg, view=False, strict_view=False)
            try:
                runner.activate()
                runner.run(max_ticks=max_steps + 1)
                episodes = runner.episode_results
            finally:
                runner.deactivate()
            episode = episodes[0]
            if video_dir is not None:
                from openral_sim._website_video import write_world_videos

                # Suffix the seed so multi-episode sweeps of one task don't
                # overwrite each other's <task>_<rskill>_<outcome>.mp4.
                video_slug = task_id if len(seeds) == 1 else f"{task_id}_seed{seed}"
                write_world_videos(
                    [episode],
                    Path(video_dir),
                    scene=video_slug,
                    rskill=Path(str(vla.weights_uri)).name,
                    section="benchmark",
                )
                episode.frames.clear()
            per_task[task_id].append(episode.success)
            all_episodes.append(episode)
            _log.info(
                "benchmark_episode_done",
                benchmark=suite_id,
                task=task_id,
                seed=seed,
                success=episode.success,
                steps=episode.steps,
            )

    result = _aggregate_results(
        scenes,
        suite_id=suite_id,
        vla=vla,
        per_task=per_task,
        episodes=all_episodes,
    )
    return result, all_episodes

save_episode_mp4(result: EpisodeResult, path: Path, *, fps: int = 20, title: str | None = None) -> Path

Write a two-band debug MP4 for one EpisodeResult.

Parameters:

Name	Type	Description	Default
result	EpisodeResult	An EpisodeResult produced by SimRunner with record_video=True. vla_input_frames populates the top panel when present; otherwise frames (rollout/world) is used as the fallback. joint_positions populates the bottom plot.	required
path	Path	Destination .mp4 file. Parent directories are created.	required
fps	int	Playback frame rate.	20
title	str | None	Optional title text rendered above the joint plot.	None

Returns:

Type	Description
Path	The output path.

Raises:

Type	Description
ValueError	If both frames and vla_input_frames are empty (no video to render) — set record_video=True first; or if path does not have an .mp4 suffix (this helper is ffmpeg/x264/yuv420p only — see module docstring).

Source code in python/sim/src/openral_sim/_video.py

def save_episode_mp4(
    result: EpisodeResult,
    path: Path,
    *,
    fps: int = 20,
    title: str | None = None,
) -> Path:
    """Write a two-band debug MP4 for one ``EpisodeResult``.

    Args:
        result: An ``EpisodeResult`` produced by ``SimRunner`` with
            ``record_video=True``. ``vla_input_frames`` populates the top
            panel when present; otherwise ``frames`` (rollout/world) is used
            as the fallback. ``joint_positions`` populates the bottom plot.
        path: Destination ``.mp4`` file. Parent directories are created.
        fps: Playback frame rate.
        title: Optional title text rendered above the joint plot.

    Returns:
        The output path.

    Raises:
        ValueError: If both ``frames`` and ``vla_input_frames`` are empty
            (no video to render) — set ``record_video=True`` first; or if
            ``path`` does not have an ``.mp4`` suffix (this helper is
            ffmpeg/x264/yuv420p only — see module docstring).
    """
    if not result.frames and not result.vla_input_frames:
        raise ValueError(
            "EpisodeResult has no frames; set SimEnvironment.record_video=True "
            "before driving SimRunner to capture rollout + VLA-input frames"
        )
    if path.suffix.lower() != ".mp4":
        raise ValueError(
            f"save_episode_mp4 only writes MP4 (libx264/yuv420p); got path={path!s} "
            f"with suffix {path.suffix!r}. Use a '.mp4' extension — the helper is "
            "intentionally MP4-only (see module docstring)."
        )

    # Length-align the per-step series. ``frames`` and ``vla_input_frames``
    # may be 1 longer than ``actions`` because the runner records the frame
    # *before* policy.step. We trim to the shortest non-empty length to keep
    # the joint cursor in lockstep with the videos.
    n_rollout = len(result.frames)
    n_vla = len(result.vla_input_frames)
    n_state = len(result.joint_positions)
    n = max(n_rollout, n_vla, n_state, 1)

    # Multi-cam VLAs already encode their camera set into the stitched
    # ``vla_input_frames`` preview. Show that grid full-width; the rollout
    # render is often redundant or misleadingly similar to one of the cameras.
    # For policies without input frames, fall back to the rollout/world view.
    show_two = result.num_input_cameras > 1 and bool(result.vla_input_frames)

    states = _stack_padded_states(result.joint_positions, n)
    if show_two:
        top_w = _TILE_SIZE * 2
        top_seq = _resize_sequence(result.vla_input_frames, top_w, _TILE_SIZE, target_len=n)
    else:
        top_w = _TILE_SIZE * 2
        # Pick whichever stream is non-empty; prefer the rollout view.
        primary = result.frames or result.vla_input_frames
        top_seq = _resize_sequence(primary, top_w, _TILE_SIZE, target_len=n)

    plot_renderer = _JointPlotRenderer(
        states=states,
        width=_CANVAS_W,
        height=_PLOT_H,
        title=title,
    )

    path.parent.mkdir(parents=True, exist_ok=True)
    import imageio.v2 as iio

    writer = iio.get_writer(
        str(path),
        format="ffmpeg",  # type: ignore[arg-type]  # reason: imageio v2 stubs type format as Format enum but accept plugin strings at runtime
        fps=fps,
        codec="libx264",
        quality=8,
        macro_block_size=1,  # no resize; we build exact-size canvases
        pixelformat="yuv420p",
    )
    try:
        for i in range(n):
            if show_two:
                top = _annotate_panel(
                    top_seq[i], f"policy inputs ({result.num_input_cameras} cams)"
                )
            else:
                top = _annotate_panel(
                    top_seq[i],
                    "rollout / policy view",
                )  # already (H, 2W, 3) when single-panel
            plot = plot_renderer.render_at_step(i)  # (PLOT_H, CANVAS_W, 3)
            canvas = np.concatenate([top, plot], axis=0)  # (CANVAS_H, CANVAS_W, 3)
            writer.append_data(canvas)
    finally:
        writer.close()
    return path